2024-04-26T22:31:51.220 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T22:31:51.220 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675270 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/fsstress}} --owner scheduled_rishabh@teuthology --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --verbose -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675270/orig.config.yaml 2024-04-26T22:31:51.283 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T22:31:51.378 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675270 branch: wip-rishabh-testing-20240426.111959 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/fsstress}} email: null first_in_suite: false job_id: '7675270' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHJc3M70IkXgGw2vQJnONpcpSMamY/DbZ/1EOcMDU42nmo7wt39dYIvW8xLvrV0AItGfyjnFcW7z4q0xEPMTV7M= smithi080.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBESrwMuKLHQ3XUoxs3Elty6O6Mch+tz9W++j8E2HObZAZ3h6G5nldjpxc2I35ZhUZ4JNh16Bp4IkkUrmECvCLIA= smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIE/bJHaB9DV54NdPX5haBHRAqyawltpzvNPqCTlzWENECvq/SfBXWXTAOAUNiGwBXZOPQWEf8500A0yvTYOZcI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsstress.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T22:31:51.379 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T22:31:51.379 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T22:31:51.379 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T22:31:51.381 INFO:teuthology.task.internal:Checking packages... 2024-04-26T22:31:51.404 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T22:31:51.404 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T22:31:51.404 INFO:teuthology.packaging:ref: None 2024-04-26T22:31:51.404 INFO:teuthology.packaging:tag: None 2024-04-26T22:31:51.404 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T22:31:51.404 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:31:51.404 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T22:31:51.624 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T22:31:51.626 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T22:31:51.633 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T22:31:51.633 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T22:31:51.670 INFO:teuthology.task.internal:Saving configuration 2024-04-26T22:31:51.696 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T22:31:51.707 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T22:31:51.730 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675270', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 22:25:38.605595', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHJc3M70IkXgGw2vQJnONpcpSMamY/DbZ/1EOcMDU42nmo7wt39dYIvW8xLvrV0AItGfyjnFcW7z4q0xEPMTV7M='} 2024-04-26T22:31:51.754 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi080.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675270', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 22:25:38.607318', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBESrwMuKLHQ3XUoxs3Elty6O6Mch+tz9W++j8E2HObZAZ3h6G5nldjpxc2I35ZhUZ4JNh16Bp4IkkUrmECvCLIA='} 2024-04-26T22:31:51.772 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675270', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 22:25:38.606544', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIE/bJHaB9DV54NdPX5haBHRAqyawltpzvNPqCTlzWENECvq/SfBXWXTAOAUNiGwBXZOPQWEf8500A0yvTYOZcI='} 2024-04-26T22:31:51.772 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T22:31:51.780 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-26T22:31:51.780 INFO:teuthology.task.internal:roles: ubuntu@smithi080.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-26T22:31:51.780 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-26T22:31:51.780 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T22:31:51.909 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f08536d1160>, signals=[15]) 2024-04-26T22:31:51.909 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T22:31:51.918 INFO:teuthology.task.internal:Opening connections... 2024-04-26T22:31:51.918 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T22:31:51.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T22:31:52.009 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi080.front.sepia.ceph.com 2024-04-26T22:31:52.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T22:31:52.084 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-04-26T22:31:52.085 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T22:31:52.157 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T22:31:52.168 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2024-04-26T22:31:52.187 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2024-04-26T22:31:52.187 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-04-26T22:31:52.245 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-04-26T22:31:52.245 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-04-26T22:31:52.245 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-04-26T22:31:52.246 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-04-26T22:31:52.246 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-04-26T22:31:52.246 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-04-26T22:31:52.246 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T22:31:52.246 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-04-26T22:31:52.246 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-04-26T22:31:52.246 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T22:31:52.246 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-04-26T22:31:52.246 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T22:31:52.246 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T22:31:52.246 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T22:31:52.247 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-04-26T22:31:52.271 DEBUG:teuthology.orchestra.run.smithi080:> uname -m 2024-04-26T22:31:52.289 INFO:teuthology.orchestra.run.smithi080.stdout:x86_64 2024-04-26T22:31:52.289 DEBUG:teuthology.orchestra.run.smithi080:> cat /etc/os-release 2024-04-26T22:31:52.346 INFO:teuthology.orchestra.run.smithi080.stdout:NAME="CentOS Stream" 2024-04-26T22:31:52.346 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION="9" 2024-04-26T22:31:52.347 INFO:teuthology.orchestra.run.smithi080.stdout:ID="centos" 2024-04-26T22:31:52.347 INFO:teuthology.orchestra.run.smithi080.stdout:ID_LIKE="rhel fedora" 2024-04-26T22:31:52.347 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_ID="9" 2024-04-26T22:31:52.347 INFO:teuthology.orchestra.run.smithi080.stdout:PLATFORM_ID="platform:el9" 2024-04-26T22:31:52.347 INFO:teuthology.orchestra.run.smithi080.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T22:31:52.347 INFO:teuthology.orchestra.run.smithi080.stdout:ANSI_COLOR="0;31" 2024-04-26T22:31:52.347 INFO:teuthology.orchestra.run.smithi080.stdout:LOGO="fedora-logo-icon" 2024-04-26T22:31:52.347 INFO:teuthology.orchestra.run.smithi080.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T22:31:52.347 INFO:teuthology.orchestra.run.smithi080.stdout:HOME_URL="https://centos.org/" 2024-04-26T22:31:52.347 INFO:teuthology.orchestra.run.smithi080.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T22:31:52.347 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T22:31:52.347 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T22:31:52.348 INFO:teuthology.lock.ops:Updating smithi080.front.sepia.ceph.com on lock server 2024-04-26T22:31:52.363 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-04-26T22:31:52.381 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-04-26T22:31:52.381 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-04-26T22:31:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-04-26T22:31:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-04-26T22:31:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-04-26T22:31:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-04-26T22:31:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-04-26T22:31:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-04-26T22:31:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T22:31:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-04-26T22:31:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-04-26T22:31:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T22:31:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-04-26T22:31:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T22:31:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T22:31:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T22:31:52.439 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-04-26T22:31:52.462 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T22:31:52.473 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T22:31:52.507 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T22:31:52.508 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2024-04-26T22:31:52.511 DEBUG:teuthology.orchestra.run.smithi080:> test '!' -e /home/ubuntu/cephtest 2024-04-26T22:31:52.513 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-04-26T22:31:52.530 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T22:31:52.540 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T22:31:52.540 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2024-04-26T22:31:52.568 DEBUG:teuthology.orchestra.run.smithi080:> test -z $(ls -A /var/lib/ceph) 2024-04-26T22:31:52.570 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-04-26T22:31:52.596 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T22:31:52.675 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T22:31:52.692 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T22:31:52.692 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-26T22:31:52.692 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-26T22:31:52.692 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T22:31:52.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:31:52.711 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-26T22:31:52.956 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T22:31:52.956 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T22:31:52.956 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-04-26T22:31:52.974 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-437.el9.x86_64 2024-04-26T22:31:52.975 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T22:31:52.975 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T22:31:52.975 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum upgrade -y linux-firmware 2024-04-26T22:32:02.339 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 1.0 MB/s | 8.0 MB 00:07 2024-04-26T22:32:04.315 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-26T22:32:08.474 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-04-26T22:32:10.183 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-26T22:32:12.543 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 9.4 MB/s | 21 MB 00:02 2024-04-26T22:32:17.784 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-04-26T22:32:19.818 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-26T22:32:19.819 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-04-26T22:32:19.820 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-26T22:32:19.941 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T22:32:19.941 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T22:32:20.086 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T22:32:20.087 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T22:32:20.089 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi027.front.sepia.ceph.com to complete... 2024-04-26T22:33:10.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26 22:33:10 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T22:33:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:FINISHED --2024-04-26 22:33:10-- 2024-04-26T22:33:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:Total wall clock time: 51s 2024-04-26T22:33:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:Downloaded: 1 files, 341M in 50s (6.82 MB/s) 2024-04-26T22:33:10.745 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T22:33:10.745 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T22:33:12.925 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying... ######################################## 2024-04-26T22:33:13.104 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing... ######################################## 2024-04-26T22:33:13.229 INFO:teuthology.orchestra.run.smithi027.stdout:Updating / installing... 2024-04-26T22:33:21.535 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T22:33:59.399 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T22:33:59.400 DEBUG:teuthology.orchestra.run.smithi027:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T22:33:59.487 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T22:33:59.487 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T22:33:59.487 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T22:33:59.487 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T22:33:59.487 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T22:33:59.487 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T22:33:59.487 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T22:33:59.487 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T22:33:59.487 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T22:33:59.487 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T22:33:59.487 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T22:33:59.488 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T22:33:59.488 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T22:33:59.488 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T22:33:59.488 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T22:33:59.488 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T22:33:59.488 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T22:33:59.488 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T22:33:59.488 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T22:33:59.488 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T22:33:59.488 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T22:33:59.488 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T22:33:59.489 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T22:33:59.489 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T22:33:59.489 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T22:33:59.489 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T22:33:59.489 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T22:33:59.489 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T22:33:59.489 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T22:33:59.489 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T22:33:59.489 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T22:33:59.489 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T22:33:59.489 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T22:33:59.489 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T22:33:59.490 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T22:33:59.490 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T22:33:59.490 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T22:33:59.490 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T22:33:59.490 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T22:33:59.490 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T22:33:59.490 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T22:33:59.490 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T22:33:59.490 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T22:33:59.490 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T22:33:59.490 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T22:33:59.490 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T22:33:59.490 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T22:33:59.491 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T22:33:59.491 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T22:33:59.491 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T22:33:59.491 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T22:33:59.491 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T22:33:59.491 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T22:33:59.491 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T22:33:59.491 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T22:33:59.491 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T22:33:59.491 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T22:33:59.491 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T22:33:59.491 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T22:33:59.491 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T22:33:59.492 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T22:33:59.492 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T22:33:59.492 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T22:33:59.492 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T22:33:59.492 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T22:33:59.492 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T22:33:59.492 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T22:33:59.492 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T22:33:59.492 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T22:33:59.492 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T22:33:59.492 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T22:33:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T22:33:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T22:33:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T22:33:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T22:33:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T22:33:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T22:33:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T22:33:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T22:33:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T22:33:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T22:33:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T22:33:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T22:33:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T22:33:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T22:33:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T22:33:59.494 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T22:33:59.494 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T22:33:59.494 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T22:33:59.494 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T22:33:59.494 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T22:33:59.494 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T22:33:59.494 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T22:33:59.494 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T22:33:59.494 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T22:33:59.494 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T22:33:59.494 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T22:33:59.494 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T22:33:59.495 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T22:33:59.495 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T22:33:59.495 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T22:33:59.495 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T22:33:59.495 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T22:33:59.495 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T22:33:59.495 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T22:33:59.495 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T22:33:59.495 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T22:33:59.495 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T22:33:59.495 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T22:33:59.495 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T22:33:59.495 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T22:33:59.495 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T22:33:59.496 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T22:33:59.496 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T22:33:59.496 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T22:33:59.496 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T22:33:59.496 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T22:33:59.496 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T22:33:59.496 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T22:33:59.496 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T22:33:59.496 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T22:33:59.496 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T22:33:59.496 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T22:33:59.496 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T22:33:59.496 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T22:33:59.496 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T22:33:59.497 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T22:33:59.497 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T22:33:59.497 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T22:33:59.497 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T22:33:59.497 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T22:33:59.497 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T22:33:59.497 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T22:33:59.497 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T22:33:59.497 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T22:33:59.497 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T22:33:59.497 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T22:33:59.498 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T22:33:59.498 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T22:33:59.498 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T22:33:59.498 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T22:33:59.498 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T22:33:59.498 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T22:33:59.498 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T22:33:59.498 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T22:33:59.498 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T22:33:59.498 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T22:33:59.498 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T22:33:59.498 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T22:33:59.498 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T22:33:59.498 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T22:33:59.499 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T22:33:59.499 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T22:33:59.499 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T22:33:59.499 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T22:33:59.499 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T22:33:59.499 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T22:33:59.499 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T22:33:59.499 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T22:33:59.499 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T22:33:59.499 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T22:33:59.499 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T22:33:59.499 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T22:33:59.499 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T22:33:59.499 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T22:33:59.499 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T22:33:59.500 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T22:33:59.500 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T22:33:59.500 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T22:33:59.500 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T22:33:59.500 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T22:33:59.500 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T22:33:59.500 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T22:33:59.500 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T22:33:59.500 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T22:33:59.500 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T22:33:59.500 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T22:33:59.500 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T22:33:59.500 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T22:33:59.500 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T22:33:59.501 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T22:33:59.501 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T22:33:59.501 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T22:33:59.501 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T22:33:59.501 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T22:33:59.501 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T22:33:59.501 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T22:33:59.501 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T22:33:59.501 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T22:33:59.501 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T22:33:59.501 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T22:33:59.501 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T22:33:59.501 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T22:33:59.501 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T22:33:59.501 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T22:33:59.502 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T22:33:59.502 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T22:33:59.502 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T22:33:59.502 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T22:33:59.502 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T22:33:59.502 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T22:33:59.502 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T22:33:59.502 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T22:33:59.502 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T22:33:59.502 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T22:33:59.502 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T22:33:59.502 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T22:33:59.503 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T22:33:59.503 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T22:33:59.503 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T22:33:59.503 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T22:33:59.503 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T22:33:59.503 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T22:33:59.503 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T22:33:59.503 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T22:33:59.503 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T22:33:59.503 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T22:33:59.503 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T22:33:59.503 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T22:33:59.504 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T22:33:59.504 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T22:33:59.504 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T22:33:59.504 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T22:33:59.504 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T22:33:59.504 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T22:33:59.504 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T22:33:59.504 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T22:33:59.504 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T22:33:59.504 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T22:33:59.504 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T22:33:59.504 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T22:33:59.504 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T22:33:59.504 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T22:33:59.505 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T22:33:59.505 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T22:33:59.505 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T22:33:59.505 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T22:33:59.505 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T22:33:59.505 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T22:33:59.505 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T22:33:59.505 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T22:33:59.505 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T22:33:59.505 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T22:33:59.506 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T22:33:59.506 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T22:33:59.506 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T22:33:59.506 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T22:33:59.506 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T22:33:59.506 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T22:33:59.506 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T22:33:59.506 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T22:33:59.506 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T22:33:59.506 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T22:33:59.506 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T22:33:59.506 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T22:33:59.506 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T22:33:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T22:33:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T22:33:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T22:33:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T22:33:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T22:33:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T22:33:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T22:33:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T22:33:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T22:33:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T22:33:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T22:33:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T22:33:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T22:33:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T22:33:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T22:33:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T22:33:59.508 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T22:33:59.508 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T22:33:59.508 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T22:33:59.508 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T22:33:59.508 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T22:33:59.508 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T22:33:59.508 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T22:33:59.508 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T22:33:59.508 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T22:33:59.508 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T22:33:59.508 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T22:33:59.508 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T22:33:59.508 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T22:33:59.508 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T22:33:59.508 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T22:33:59.509 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T22:33:59.509 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T22:33:59.509 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T22:33:59.509 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T22:33:59.509 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T22:33:59.509 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T22:33:59.509 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T22:33:59.509 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T22:33:59.509 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T22:33:59.509 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T22:33:59.509 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T22:33:59.509 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T22:33:59.509 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T22:33:59.509 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T22:33:59.509 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T22:33:59.510 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T22:33:59.510 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T22:33:59.510 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T22:33:59.510 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T22:33:59.510 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T22:33:59.510 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T22:33:59.510 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T22:33:59.510 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T22:33:59.510 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T22:33:59.510 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T22:33:59.510 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T22:33:59.510 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T22:33:59.510 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T22:33:59.511 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T22:33:59.511 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T22:33:59.511 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T22:33:59.511 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T22:33:59.511 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T22:33:59.511 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T22:33:59.511 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T22:33:59.511 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T22:33:59.511 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T22:33:59.511 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T22:33:59.511 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T22:33:59.511 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T22:33:59.511 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T22:33:59.511 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T22:33:59.512 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T22:33:59.512 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T22:33:59.512 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T22:33:59.512 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T22:33:59.512 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T22:33:59.512 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T22:33:59.512 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T22:33:59.512 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T22:33:59.512 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T22:33:59.512 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T22:33:59.512 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T22:33:59.512 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T22:33:59.512 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T22:33:59.512 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T22:33:59.512 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T22:33:59.513 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T22:33:59.513 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T22:33:59.513 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T22:33:59.513 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T22:33:59.513 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T22:33:59.513 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T22:33:59.513 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T22:33:59.513 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T22:33:59.513 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T22:33:59.513 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T22:33:59.514 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T22:33:59.514 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T22:33:59.514 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T22:33:59.514 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T22:33:59.514 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T22:33:59.514 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T22:33:59.514 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T22:33:59.514 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T22:33:59.514 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T22:33:59.514 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T22:33:59.514 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T22:33:59.514 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T22:33:59.514 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T22:33:59.515 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T22:33:59.515 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T22:33:59.515 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T22:33:59.515 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T22:33:59.515 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T22:33:59.515 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T22:33:59.515 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T22:33:59.515 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T22:33:59.515 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T22:33:59.515 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T22:33:59.515 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T22:33:59.515 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T22:33:59.515 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T22:33:59.516 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T22:33:59.516 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T22:33:59.516 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T22:33:59.516 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T22:33:59.516 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T22:33:59.516 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T22:33:59.516 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T22:33:59.516 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T22:33:59.516 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T22:33:59.516 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T22:33:59.516 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T22:33:59.516 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T22:33:59.516 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T22:33:59.516 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T22:33:59.516 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T22:33:59.517 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T22:33:59.517 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T22:33:59.517 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T22:33:59.517 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T22:33:59.517 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T22:33:59.517 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T22:33:59.517 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T22:33:59.517 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T22:33:59.517 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T22:33:59.517 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T22:33:59.517 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T22:33:59.517 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T22:33:59.517 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T22:33:59.518 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T22:33:59.518 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T22:33:59.518 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T22:33:59.518 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T22:33:59.518 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T22:33:59.518 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T22:33:59.518 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T22:33:59.518 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T22:33:59.518 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T22:33:59.518 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T22:33:59.518 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T22:33:59.518 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T22:33:59.518 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T22:33:59.519 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T22:33:59.519 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T22:33:59.519 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T22:33:59.519 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T22:33:59.519 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T22:33:59.519 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T22:33:59.519 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T22:33:59.519 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T22:33:59.519 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T22:33:59.519 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T22:33:59.519 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T22:33:59.519 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T22:33:59.520 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T22:33:59.520 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T22:33:59.520 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T22:33:59.520 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T22:33:59.520 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T22:33:59.520 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T22:33:59.520 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T22:33:59.520 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T22:33:59.520 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T22:33:59.520 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T22:33:59.520 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T22:33:59.520 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T22:33:59.520 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T22:33:59.520 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T22:33:59.520 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T22:33:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T22:33:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T22:33:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T22:33:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T22:33:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T22:33:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T22:33:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T22:33:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T22:33:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T22:33:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T22:33:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T22:33:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T22:33:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T22:33:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T22:33:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T22:33:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T22:33:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T22:33:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T22:33:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T22:33:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T22:33:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T22:33:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T22:33:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T22:33:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T22:33:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T22:33:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T22:33:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T22:33:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T22:33:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T22:33:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T22:33:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T22:33:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T22:33:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T22:33:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T22:33:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T22:33:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T22:33:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T22:33:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T22:33:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T22:33:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T22:33:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T22:33:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T22:33:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T22:33:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T22:33:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T22:33:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T22:33:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T22:33:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T22:33:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T22:33:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T22:33:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T22:33:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T22:33:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T22:33:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T22:33:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T22:33:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T22:33:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T22:33:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T22:33:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T22:33:59.525 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T22:33:59.525 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T22:33:59.525 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T22:33:59.525 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T22:33:59.525 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T22:33:59.525 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T22:33:59.525 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T22:33:59.525 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T22:33:59.525 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T22:33:59.525 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T22:33:59.525 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T22:33:59.526 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T22:33:59.526 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T22:33:59.526 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T22:33:59.526 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T22:33:59.526 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T22:33:59.526 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T22:33:59.526 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T22:33:59.526 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T22:33:59.526 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T22:33:59.526 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T22:33:59.526 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T22:33:59.526 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T22:33:59.527 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T22:33:59.527 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T22:33:59.527 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T22:33:59.527 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T22:33:59.527 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T22:33:59.527 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T22:33:59.527 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T22:33:59.527 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T22:33:59.527 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T22:33:59.527 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T22:33:59.527 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T22:33:59.527 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T22:33:59.527 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T22:33:59.528 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T22:33:59.528 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T22:33:59.528 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T22:33:59.528 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T22:33:59.528 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T22:33:59.528 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T22:33:59.528 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T22:33:59.528 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T22:33:59.528 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T22:33:59.528 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T22:33:59.528 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T22:33:59.528 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T22:33:59.528 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T22:33:59.528 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T22:33:59.529 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T22:33:59.529 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T22:33:59.529 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T22:33:59.529 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T22:33:59.529 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T22:33:59.529 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T22:33:59.529 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T22:33:59.529 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T22:33:59.529 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T22:33:59.529 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T22:33:59.529 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T22:33:59.529 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T22:33:59.529 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T22:33:59.530 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T22:33:59.530 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T22:33:59.530 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T22:33:59.530 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T22:33:59.530 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T22:33:59.530 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T22:33:59.530 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T22:33:59.530 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T22:33:59.530 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T22:33:59.530 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T22:33:59.530 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T22:33:59.530 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T22:33:59.530 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T22:33:59.531 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T22:33:59.531 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T22:33:59.531 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T22:33:59.531 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T22:33:59.531 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T22:33:59.531 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T22:33:59.531 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T22:33:59.531 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T22:33:59.531 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T22:33:59.531 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T22:33:59.531 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T22:33:59.531 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T22:33:59.532 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T22:33:59.532 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T22:33:59.532 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T22:33:59.532 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T22:33:59.532 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T22:33:59.532 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T22:33:59.532 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T22:33:59.532 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T22:33:59.532 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T22:33:59.532 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T22:33:59.532 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T22:33:59.532 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T22:33:59.532 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T22:33:59.532 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T22:33:59.533 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T22:33:59.533 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T22:33:59.533 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T22:33:59.533 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T22:33:59.533 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T22:33:59.533 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T22:33:59.533 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T22:33:59.533 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T22:33:59.533 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T22:33:59.533 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T22:33:59.533 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T22:33:59.533 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T22:33:59.533 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T22:33:59.533 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T22:33:59.534 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T22:33:59.534 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T22:33:59.534 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T22:33:59.534 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T22:33:59.534 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T22:33:59.534 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T22:33:59.534 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T22:33:59.534 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T22:33:59.534 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T22:33:59.534 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T22:33:59.534 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T22:33:59.534 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T22:33:59.535 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T22:33:59.535 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T22:33:59.535 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T22:33:59.535 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T22:33:59.535 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T22:33:59.535 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T22:33:59.535 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T22:33:59.535 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T22:33:59.535 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T22:33:59.535 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T22:33:59.535 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T22:33:59.535 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T22:33:59.535 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T22:33:59.536 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T22:33:59.536 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T22:33:59.536 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T22:33:59.536 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T22:33:59.536 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T22:33:59.536 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T22:33:59.536 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T22:33:59.536 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T22:33:59.536 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T22:33:59.536 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T22:33:59.536 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T22:33:59.536 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T22:33:59.536 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T22:33:59.536 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T22:33:59.537 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T22:33:59.537 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T22:33:59.537 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T22:33:59.537 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T22:33:59.537 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T22:33:59.537 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T22:33:59.537 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T22:33:59.537 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T22:33:59.537 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T22:33:59.537 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T22:33:59.537 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T22:33:59.537 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T22:33:59.537 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T22:33:59.537 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T22:33:59.538 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T22:33:59.538 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T22:33:59.538 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T22:33:59.538 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T22:33:59.538 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T22:33:59.538 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T22:33:59.538 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T22:33:59.538 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T22:33:59.538 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T22:33:59.538 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T22:33:59.538 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T22:33:59.538 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T22:33:59.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T22:33:59.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T22:33:59.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T22:33:59.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T22:33:59.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T22:33:59.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T22:33:59.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T22:33:59.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T22:33:59.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T22:33:59.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T22:33:59.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T22:33:59.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T22:33:59.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T22:33:59.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T22:33:59.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T22:33:59.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T22:33:59.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T22:33:59.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T22:33:59.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T22:33:59.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T22:33:59.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T22:33:59.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T22:33:59.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T22:33:59.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T22:33:59.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T22:33:59.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T22:33:59.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T22:33:59.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T22:33:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T22:33:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T22:33:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T22:33:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T22:33:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T22:33:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T22:33:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T22:33:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T22:33:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T22:33:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T22:33:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T22:33:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T22:33:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T22:33:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T22:33:59.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T22:33:59.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T22:33:59.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T22:33:59.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T22:33:59.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T22:33:59.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T22:33:59.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T22:33:59.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T22:33:59.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T22:33:59.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T22:33:59.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T22:33:59.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T22:33:59.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T22:33:59.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T22:33:59.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T22:33:59.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T22:33:59.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T22:33:59.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T22:33:59.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T22:33:59.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T22:33:59.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T22:33:59.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T22:33:59.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T22:33:59.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T22:33:59.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T22:33:59.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T22:33:59.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T22:33:59.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T22:33:59.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T22:33:59.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T22:33:59.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T22:33:59.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T22:33:59.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T22:33:59.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T22:33:59.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T22:33:59.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T22:33:59.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T22:33:59.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T22:33:59.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T22:33:59.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T22:33:59.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T22:33:59.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T22:33:59.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T22:33:59.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T22:33:59.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T22:33:59.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T22:33:59.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T22:33:59.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T22:33:59.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T22:33:59.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T22:33:59.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T22:33:59.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T22:33:59.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T22:33:59.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T22:33:59.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T22:33:59.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T22:33:59.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T22:33:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T22:33:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T22:33:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T22:33:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T22:33:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T22:33:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T22:33:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T22:33:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T22:33:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T22:33:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T22:33:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T22:33:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T22:33:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T22:33:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T22:33:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T22:33:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T22:33:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T22:33:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T22:33:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T22:33:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T22:33:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T22:33:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T22:33:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T22:33:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T22:33:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T22:33:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T22:33:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T22:33:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T22:33:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T22:33:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T22:33:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T22:33:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T22:33:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T22:33:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T22:33:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T22:33:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T22:33:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T22:33:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T22:33:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T22:33:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T22:33:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T22:33:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T22:33:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T22:33:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T22:33:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T22:33:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T22:33:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T22:33:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T22:33:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T22:33:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T22:33:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T22:33:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T22:33:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T22:33:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T22:33:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T22:33:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T22:33:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T22:33:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T22:33:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T22:33:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T22:33:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T22:33:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T22:33:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T22:33:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T22:33:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T22:33:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T22:33:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T22:33:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T22:33:59.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T22:33:59.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T22:33:59.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T22:33:59.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T22:33:59.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T22:33:59.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T22:33:59.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T22:33:59.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T22:33:59.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T22:33:59.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T22:33:59.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T22:33:59.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T22:33:59.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T22:33:59.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T22:33:59.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T22:33:59.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T22:33:59.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T22:33:59.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T22:33:59.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T22:33:59.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T22:33:59.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T22:33:59.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T22:33:59.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T22:33:59.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T22:33:59.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T22:33:59.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T22:33:59.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T22:33:59.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T22:33:59.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T22:33:59.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T22:33:59.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T22:33:59.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T22:33:59.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T22:33:59.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T22:33:59.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T22:33:59.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T22:33:59.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T22:33:59.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T22:33:59.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T22:33:59.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T22:33:59.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T22:33:59.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T22:33:59.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T22:33:59.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T22:33:59.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T22:33:59.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T22:33:59.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T22:33:59.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T22:33:59.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T22:33:59.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T22:33:59.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T22:33:59.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T22:33:59.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T22:33:59.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T22:33:59.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T22:33:59.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T22:33:59.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T22:33:59.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T22:33:59.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T22:33:59.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T22:33:59.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T22:33:59.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T22:33:59.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T22:33:59.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T22:33:59.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T22:33:59.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T22:33:59.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T22:33:59.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T22:33:59.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T22:33:59.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T22:33:59.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T22:33:59.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T22:33:59.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T22:33:59.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T22:33:59.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T22:33:59.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T22:33:59.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T22:33:59.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T22:33:59.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T22:33:59.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T22:33:59.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T22:33:59.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T22:33:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T22:33:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T22:33:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T22:33:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T22:33:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T22:33:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T22:33:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T22:33:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T22:33:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T22:33:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T22:33:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T22:33:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T22:33:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T22:33:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T22:33:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T22:33:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T22:33:59.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T22:33:59.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T22:33:59.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T22:33:59.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T22:33:59.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T22:33:59.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T22:33:59.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T22:33:59.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T22:33:59.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T22:33:59.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T22:33:59.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T22:33:59.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T22:33:59.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T22:33:59.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T22:33:59.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T22:33:59.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T22:33:59.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T22:33:59.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T22:33:59.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T22:33:59.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T22:33:59.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T22:33:59.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T22:33:59.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T22:33:59.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T22:33:59.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T22:33:59.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T22:33:59.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T22:33:59.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T22:33:59.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T22:33:59.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T22:33:59.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T22:33:59.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T22:33:59.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T22:33:59.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T22:33:59.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T22:33:59.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T22:33:59.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T22:33:59.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T22:33:59.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T22:33:59.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T22:33:59.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T22:33:59.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T22:33:59.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T22:33:59.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T22:33:59.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T22:33:59.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T22:33:59.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T22:33:59.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T22:33:59.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T22:33:59.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T22:33:59.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T22:33:59.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T22:33:59.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T22:33:59.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T22:33:59.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T22:33:59.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T22:33:59.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T22:33:59.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T22:33:59.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T22:33:59.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T22:33:59.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T22:33:59.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T22:33:59.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T22:33:59.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T22:33:59.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T22:33:59.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T22:33:59.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T22:33:59.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T22:33:59.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T22:33:59.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T22:33:59.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T22:33:59.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T22:33:59.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T22:33:59.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T22:33:59.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T22:33:59.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T22:33:59.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T22:33:59.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T22:33:59.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T22:33:59.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T22:33:59.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T22:33:59.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T22:33:59.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T22:33:59.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T22:33:59.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T22:33:59.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T22:33:59.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T22:33:59.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T22:33:59.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T22:33:59.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T22:33:59.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T22:33:59.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T22:33:59.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T22:33:59.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T22:33:59.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T22:33:59.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T22:33:59.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T22:33:59.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T22:33:59.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T22:33:59.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T22:33:59.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T22:33:59.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T22:33:59.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T22:33:59.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T22:33:59.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T22:33:59.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T22:33:59.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T22:33:59.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T22:33:59.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T22:33:59.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T22:33:59.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T22:33:59.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T22:33:59.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T22:33:59.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T22:33:59.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T22:33:59.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T22:33:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T22:33:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T22:33:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T22:33:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T22:33:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T22:33:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T22:33:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T22:33:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T22:33:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T22:33:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T22:33:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T22:33:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T22:33:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T22:33:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T22:33:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T22:33:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T22:33:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T22:33:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T22:33:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T22:33:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T22:33:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T22:33:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T22:33:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T22:33:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T22:33:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T22:33:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T22:33:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T22:33:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T22:33:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T22:33:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T22:33:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T22:33:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T22:33:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T22:33:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T22:33:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T22:33:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T22:33:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T22:33:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T22:33:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T22:33:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T22:33:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T22:33:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T22:33:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T22:33:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T22:33:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T22:33:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T22:33:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T22:33:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T22:33:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T22:33:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T22:33:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T22:33:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T22:33:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T22:33:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T22:33:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T22:33:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T22:33:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T22:33:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T22:33:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T22:33:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T22:33:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T22:33:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T22:33:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T22:33:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T22:33:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T22:33:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T22:33:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T22:33:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T22:33:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T22:33:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T22:33:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T22:33:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T22:33:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T22:33:59.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T22:33:59.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T22:33:59.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T22:33:59.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T22:33:59.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T22:33:59.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T22:33:59.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T22:33:59.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T22:33:59.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T22:33:59.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T22:33:59.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T22:33:59.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T22:33:59.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T22:33:59.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T22:33:59.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T22:33:59.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T22:33:59.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T22:33:59.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T22:33:59.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T22:33:59.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T22:33:59.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T22:33:59.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T22:33:59.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T22:33:59.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T22:33:59.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T22:33:59.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T22:33:59.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T22:33:59.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T22:33:59.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T22:33:59.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T22:33:59.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T22:33:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T22:33:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T22:33:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T22:33:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T22:33:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T22:33:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T22:33:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T22:33:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T22:33:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T22:33:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T22:33:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T22:33:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T22:33:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T22:33:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T22:33:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T22:33:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T22:33:59.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T22:33:59.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T22:33:59.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T22:33:59.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T22:33:59.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T22:33:59.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T22:33:59.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T22:33:59.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T22:33:59.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T22:33:59.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T22:33:59.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T22:33:59.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T22:33:59.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T22:33:59.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T22:33:59.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T22:33:59.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T22:33:59.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T22:33:59.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T22:33:59.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T22:33:59.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T22:33:59.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T22:33:59.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T22:33:59.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T22:33:59.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T22:33:59.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T22:33:59.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T22:33:59.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T22:33:59.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T22:33:59.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T22:33:59.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T22:33:59.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T22:33:59.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T22:33:59.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T22:33:59.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T22:33:59.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T22:33:59.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T22:33:59.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T22:33:59.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T22:33:59.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T22:33:59.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T22:33:59.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T22:33:59.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T22:33:59.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T22:33:59.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T22:33:59.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T22:33:59.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T22:33:59.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T22:33:59.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T22:33:59.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T22:33:59.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T22:33:59.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T22:33:59.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T22:33:59.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T22:33:59.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T22:33:59.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T22:33:59.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T22:33:59.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T22:33:59.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T22:33:59.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T22:33:59.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T22:33:59.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T22:33:59.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T22:33:59.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T22:33:59.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T22:33:59.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T22:33:59.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T22:33:59.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T22:33:59.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T22:33:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T22:33:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T22:33:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T22:33:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T22:33:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T22:33:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T22:33:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T22:33:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T22:33:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T22:33:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T22:33:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T22:33:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T22:33:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T22:33:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T22:33:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T22:33:59.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T22:33:59.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T22:33:59.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T22:33:59.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T22:33:59.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T22:33:59.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T22:33:59.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T22:33:59.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T22:33:59.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T22:33:59.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T22:33:59.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T22:33:59.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T22:33:59.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T22:33:59.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T22:33:59.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T22:33:59.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T22:33:59.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T22:33:59.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T22:33:59.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T22:33:59.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T22:33:59.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T22:33:59.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T22:33:59.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T22:33:59.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T22:33:59.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T22:33:59.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T22:33:59.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T22:33:59.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T22:33:59.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T22:33:59.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T22:33:59.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T22:33:59.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T22:33:59.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T22:33:59.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T22:33:59.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T22:33:59.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T22:33:59.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T22:33:59.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T22:33:59.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T22:33:59.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T22:33:59.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T22:33:59.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T22:33:59.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T22:33:59.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T22:33:59.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T22:33:59.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T22:33:59.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T22:33:59.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T22:33:59.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T22:33:59.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T22:33:59.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T22:33:59.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T22:33:59.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T22:33:59.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T22:33:59.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T22:33:59.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T22:33:59.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T22:33:59.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T22:33:59.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T22:33:59.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T22:33:59.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T22:33:59.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T22:33:59.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T22:33:59.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T22:33:59.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T22:33:59.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T22:33:59.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T22:33:59.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T22:33:59.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T22:33:59.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T22:33:59.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T22:33:59.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T22:33:59.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T22:33:59.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T22:33:59.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T22:33:59.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T22:33:59.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T22:33:59.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T22:33:59.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T22:33:59.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T22:33:59.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T22:33:59.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T22:33:59.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T22:33:59.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T22:33:59.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T22:33:59.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T22:33:59.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T22:33:59.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T22:33:59.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T22:33:59.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T22:33:59.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T22:33:59.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T22:33:59.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T22:33:59.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T22:33:59.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T22:33:59.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T22:33:59.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T22:33:59.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T22:33:59.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T22:33:59.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T22:33:59.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T22:33:59.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T22:33:59.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T22:33:59.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T22:33:59.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T22:33:59.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T22:33:59.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T22:33:59.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T22:33:59.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T22:33:59.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T22:33:59.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T22:33:59.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T22:33:59.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T22:33:59.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T22:33:59.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T22:33:59.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T22:33:59.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T22:33:59.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T22:33:59.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T22:33:59.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T22:33:59.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T22:33:59.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T22:33:59.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T22:33:59.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T22:33:59.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T22:33:59.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T22:33:59.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T22:33:59.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T22:33:59.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T22:33:59.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T22:33:59.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T22:33:59.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T22:33:59.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T22:33:59.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T22:33:59.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T22:33:59.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T22:33:59.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T22:33:59.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T22:33:59.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T22:33:59.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T22:33:59.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T22:33:59.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T22:33:59.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T22:33:59.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T22:33:59.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T22:33:59.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T22:33:59.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T22:33:59.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T22:33:59.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T22:33:59.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T22:33:59.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T22:33:59.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T22:33:59.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T22:33:59.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T22:33:59.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T22:33:59.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T22:33:59.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T22:33:59.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T22:33:59.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T22:33:59.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T22:33:59.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T22:33:59.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T22:33:59.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T22:33:59.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T22:33:59.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T22:33:59.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T22:33:59.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T22:33:59.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T22:33:59.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T22:33:59.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T22:33:59.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T22:33:59.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T22:33:59.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T22:33:59.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T22:33:59.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T22:33:59.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T22:33:59.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T22:33:59.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T22:33:59.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T22:33:59.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T22:33:59.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T22:33:59.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T22:33:59.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T22:33:59.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T22:33:59.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T22:33:59.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T22:33:59.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T22:33:59.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T22:33:59.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T22:33:59.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T22:33:59.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T22:33:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T22:33:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T22:33:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T22:33:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T22:33:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T22:33:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T22:33:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T22:33:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T22:33:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T22:33:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T22:33:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T22:33:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T22:33:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T22:33:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T22:33:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T22:33:59.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T22:33:59.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T22:33:59.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T22:33:59.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T22:33:59.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T22:33:59.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T22:33:59.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T22:33:59.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T22:33:59.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T22:33:59.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T22:33:59.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T22:33:59.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T22:33:59.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T22:33:59.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T22:33:59.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T22:33:59.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T22:33:59.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T22:33:59.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T22:33:59.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T22:33:59.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T22:33:59.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T22:33:59.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T22:33:59.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T22:33:59.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T22:33:59.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T22:33:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T22:33:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T22:33:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T22:33:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T22:33:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T22:33:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T22:33:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T22:33:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T22:33:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T22:33:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T22:33:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T22:33:59.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T22:33:59.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T22:33:59.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T22:33:59.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T22:33:59.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T22:33:59.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T22:33:59.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T22:33:59.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T22:33:59.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T22:33:59.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T22:33:59.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T22:33:59.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T22:33:59.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T22:33:59.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T22:33:59.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T22:33:59.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T22:33:59.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T22:33:59.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T22:33:59.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T22:33:59.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T22:33:59.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T22:33:59.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T22:33:59.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T22:33:59.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T22:33:59.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T22:33:59.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T22:33:59.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T22:33:59.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T22:33:59.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T22:33:59.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T22:33:59.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T22:33:59.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T22:33:59.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T22:33:59.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T22:33:59.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T22:33:59.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T22:33:59.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T22:33:59.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T22:33:59.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T22:33:59.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T22:33:59.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T22:33:59.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T22:33:59.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T22:33:59.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T22:33:59.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T22:33:59.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T22:33:59.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T22:33:59.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T22:33:59.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T22:33:59.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T22:33:59.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T22:33:59.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T22:33:59.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T22:33:59.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T22:33:59.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T22:33:59.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T22:33:59.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T22:33:59.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T22:33:59.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T22:33:59.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T22:33:59.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T22:33:59.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T22:33:59.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T22:33:59.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T22:33:59.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T22:33:59.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T22:33:59.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T22:33:59.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T22:33:59.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T22:33:59.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T22:33:59.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T22:33:59.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T22:33:59.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T22:33:59.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T22:33:59.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T22:33:59.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T22:33:59.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T22:33:59.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T22:33:59.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T22:33:59.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T22:33:59.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T22:33:59.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T22:33:59.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T22:33:59.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T22:33:59.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T22:33:59.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T22:33:59.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T22:33:59.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T22:33:59.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T22:33:59.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T22:33:59.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T22:33:59.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T22:33:59.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T22:33:59.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T22:33:59.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T22:33:59.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T22:33:59.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T22:33:59.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T22:33:59.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T22:33:59.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T22:33:59.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T22:33:59.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T22:33:59.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T22:33:59.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T22:33:59.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T22:33:59.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T22:33:59.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T22:33:59.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T22:33:59.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T22:33:59.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T22:33:59.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T22:33:59.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T22:33:59.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T22:33:59.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T22:33:59.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T22:33:59.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T22:33:59.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T22:33:59.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T22:33:59.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T22:33:59.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T22:33:59.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T22:33:59.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T22:33:59.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T22:33:59.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T22:33:59.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T22:33:59.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T22:33:59.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T22:33:59.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T22:33:59.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T22:33:59.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T22:33:59.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T22:33:59.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T22:33:59.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T22:33:59.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T22:33:59.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T22:33:59.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T22:33:59.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T22:33:59.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T22:33:59.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T22:33:59.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T22:33:59.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T22:33:59.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T22:33:59.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T22:33:59.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T22:33:59.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T22:33:59.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T22:33:59.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T22:33:59.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T22:33:59.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T22:33:59.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T22:33:59.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T22:33:59.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T22:33:59.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T22:33:59.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T22:33:59.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T22:33:59.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T22:33:59.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T22:33:59.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T22:33:59.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T22:33:59.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T22:33:59.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T22:33:59.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T22:33:59.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T22:33:59.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T22:33:59.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T22:33:59.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T22:33:59.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T22:33:59.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T22:33:59.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T22:33:59.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T22:33:59.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T22:33:59.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T22:33:59.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T22:33:59.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T22:33:59.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T22:33:59.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T22:33:59.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T22:33:59.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T22:33:59.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T22:33:59.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T22:33:59.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T22:33:59.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T22:33:59.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T22:33:59.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T22:33:59.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T22:33:59.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T22:33:59.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T22:33:59.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T22:33:59.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T22:33:59.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T22:33:59.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T22:33:59.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T22:33:59.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T22:33:59.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T22:33:59.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T22:33:59.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T22:33:59.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T22:33:59.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T22:33:59.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T22:33:59.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T22:33:59.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T22:33:59.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T22:33:59.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T22:33:59.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T22:33:59.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T22:33:59.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T22:33:59.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T22:33:59.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T22:33:59.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T22:33:59.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T22:33:59.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T22:33:59.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T22:33:59.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T22:33:59.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T22:33:59.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T22:33:59.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T22:33:59.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T22:33:59.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T22:33:59.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T22:33:59.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T22:33:59.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T22:33:59.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T22:33:59.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T22:33:59.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T22:33:59.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T22:33:59.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T22:33:59.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T22:33:59.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T22:33:59.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T22:33:59.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T22:33:59.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T22:33:59.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T22:33:59.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T22:33:59.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T22:33:59.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T22:33:59.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T22:33:59.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T22:33:59.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T22:33:59.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T22:33:59.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T22:33:59.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T22:33:59.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T22:33:59.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T22:33:59.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T22:33:59.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T22:33:59.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T22:33:59.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T22:33:59.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T22:33:59.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T22:33:59.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T22:33:59.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T22:33:59.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T22:33:59.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T22:33:59.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T22:33:59.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T22:33:59.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T22:33:59.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T22:33:59.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T22:33:59.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T22:33:59.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T22:33:59.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T22:33:59.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T22:33:59.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T22:33:59.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T22:33:59.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T22:33:59.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T22:33:59.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T22:33:59.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T22:33:59.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T22:33:59.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T22:33:59.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T22:33:59.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T22:33:59.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T22:33:59.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T22:33:59.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T22:33:59.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T22:33:59.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T22:33:59.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T22:33:59.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T22:33:59.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T22:33:59.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T22:33:59.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T22:33:59.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T22:33:59.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T22:33:59.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T22:33:59.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T22:33:59.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T22:33:59.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T22:33:59.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T22:33:59.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T22:33:59.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T22:33:59.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T22:33:59.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T22:33:59.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T22:33:59.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T22:33:59.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T22:33:59.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T22:33:59.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T22:33:59.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T22:33:59.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T22:33:59.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T22:33:59.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T22:33:59.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T22:33:59.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T22:33:59.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T22:33:59.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T22:33:59.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T22:33:59.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T22:33:59.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T22:33:59.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T22:33:59.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T22:33:59.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T22:33:59.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T22:33:59.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T22:33:59.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T22:33:59.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T22:33:59.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T22:33:59.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T22:33:59.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T22:33:59.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T22:33:59.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T22:33:59.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T22:33:59.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T22:33:59.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T22:33:59.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T22:33:59.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T22:33:59.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T22:33:59.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T22:33:59.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T22:33:59.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T22:33:59.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T22:33:59.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T22:33:59.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T22:33:59.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T22:33:59.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T22:33:59.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T22:33:59.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T22:33:59.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T22:33:59.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T22:33:59.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T22:33:59.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T22:33:59.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T22:33:59.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T22:33:59.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T22:33:59.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T22:33:59.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T22:33:59.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T22:33:59.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T22:33:59.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T22:33:59.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T22:33:59.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T22:33:59.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T22:33:59.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T22:33:59.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T22:33:59.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T22:33:59.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T22:33:59.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T22:33:59.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T22:33:59.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T22:33:59.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T22:33:59.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T22:33:59.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T22:33:59.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T22:33:59.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T22:33:59.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T22:33:59.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T22:33:59.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T22:33:59.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T22:33:59.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T22:33:59.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T22:33:59.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T22:33:59.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T22:33:59.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T22:33:59.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T22:33:59.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T22:33:59.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T22:33:59.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T22:33:59.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T22:33:59.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T22:33:59.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T22:33:59.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T22:33:59.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T22:33:59.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T22:33:59.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T22:33:59.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T22:33:59.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T22:33:59.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T22:33:59.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T22:33:59.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T22:33:59.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T22:33:59.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T22:33:59.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T22:33:59.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T22:33:59.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T22:33:59.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T22:33:59.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T22:33:59.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T22:33:59.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T22:33:59.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T22:33:59.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T22:33:59.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T22:33:59.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T22:33:59.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T22:33:59.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T22:33:59.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T22:33:59.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T22:33:59.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T22:33:59.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T22:33:59.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T22:33:59.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T22:33:59.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T22:33:59.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T22:33:59.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T22:33:59.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T22:33:59.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T22:33:59.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T22:33:59.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T22:33:59.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T22:33:59.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T22:33:59.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T22:33:59.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T22:33:59.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T22:33:59.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T22:33:59.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T22:33:59.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T22:33:59.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T22:33:59.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T22:33:59.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T22:33:59.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T22:33:59.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T22:33:59.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T22:33:59.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T22:33:59.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T22:33:59.640 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T22:33:59.640 DEBUG:teuthology.orchestra.run.smithi027:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T22:33:59.664 INFO:teuthology.orchestra.run.smithi027.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T22:33:59.664 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T22:33:59.664 INFO:teuthology.orchestra.run.smithi027.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T22:33:59.665 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-04-26T22:33:59.665 INFO:teuthology.orchestra.run.smithi027.stdout:for file in vmlinuz System.map config; do 2024-04-26T22:33:59.665 INFO:teuthology.orchestra.run.smithi027.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T22:33:59.665 INFO:teuthology.orchestra.run.smithi027.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T22:33:59.665 INFO:teuthology.orchestra.run.smithi027.stdout: fi 2024-04-26T22:33:59.665 INFO:teuthology.orchestra.run.smithi027.stdout:done 2024-04-26T22:33:59.665 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T22:33:59.665 INFO:teuthology.orchestra.run.smithi027.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T22:33:59.665 INFO:teuthology.orchestra.run.smithi027.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T22:33:59.665 INFO:teuthology.orchestra.run.smithi027.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T22:33:59.665 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-04-26T22:33:59.665 INFO:teuthology.orchestra.run.smithi027.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T22:33:59.665 INFO:teuthology.orchestra.run.smithi027.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T22:33:59.665 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-04-26T22:33:59.665 INFO:teuthology.orchestra.run.smithi027.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T22:33:59.665 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T22:33:59.666 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T22:33:59.666 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-04-26T22:33:59.667 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2024-04-26T22:33:59.749 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2024-04-26T22:33:59.750 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2024-04-26T22:33:59.750 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.06 2024-04-26T22:33:59.750 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 68.el9 2024-04-26T22:33:59.750 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2024-04-26T22:33:59.750 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T22:33:59.750 INFO:teuthology.orchestra.run.smithi027.stdout:Group : Unspecified 2024-04-26T22:33:59.750 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 8264254 2024-04-26T22:33:59.750 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2024-04-26T22:33:59.750 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T22:33:59.750 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T22:33:59.750 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T22:33:59.750 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T22:33:59.750 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : builder@centos.org 2024-04-26T22:33:59.750 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2024-04-26T22:33:59.751 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T22:33:59.751 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2024-04-26T22:33:59.751 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2024-04-26T22:33:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:33:59.751 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T22:33:59.751 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T22:33:59.751 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T22:33:59.751 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2024-04-26T22:33:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:33:59.751 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T22:33:59.752 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T22:33:59.752 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T22:33:59.752 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T22:34:00.207 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2024-04-26T22:34:01.650 INFO:teuthology.orchestra.run.smithi027.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T22:34:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:done 2024-04-26T22:34:01.668 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-04-26T22:34:01.687 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.hFjvA7BqCk 2024-04-26T22:34:01.688 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.hFjvA7BqCk 2024-04-26T22:34:01.753 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.hFjvA7BqCk 2024-04-26T22:34:01.891 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.hFjvA7BqCk is 6KB 2024-04-26T22:34:01.940 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.hFjvA7BqCk 2024-04-26T22:34:01.954 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T22:34:02.022 INFO:teuthology.orchestra.run.smithi027.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T22:34:02.022 INFO:teuthology.orchestra.run.smithi027.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T22:34:02.022 INFO:teuthology.orchestra.run.smithi027.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T22:34:02.023 INFO:teuthology.orchestra.run.smithi027.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T22:34:02.024 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T22:34:02.190 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2024-04-26T22:34:32.193 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-26T22:34:32.194 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-26T22:34:32.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T22:35:06.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-04-26T22:35:15.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-26T22:35:15.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T22:35:18.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-04-26T22:35:30.719 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-26T22:35:30.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T22:35:33.794 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-04-26T22:35:48.796 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-26T22:35:48.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T22:36:48.799 DEBUG:teuthology.orchestra.remote:timed out 2024-04-26T22:37:06.800 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-26T22:37:06.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T22:37:07.218 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-04-26T22:37:07.892 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-26T22:37:07.893 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T22:37:07.893 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-04-26T22:37:07.949 INFO:teuthology.orchestra.run.smithi027.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T22:37:07.949 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T22:37:07.950 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T22:37:07.950 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T22:37:08.951 DEBUG:teuthology.parallel:result is None 2024-04-26T22:37:08.952 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T22:37:08.962 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T22:37:08.963 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T22:37:08.972 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T22:37:08.976 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T22:37:09.025 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T22:37:09.032 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T22:37:09.059 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T22:37:09.060 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T22:37:09.066 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T22:37:09.068 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T22:37:09.228 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T22:37:09.236 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T22:37:09.237 DEBUG:teuthology.orchestra.run.smithi080:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T22:37:09.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:37:09.254 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T22:37:09.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:37:09.270 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T22:37:09.276 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T22:37:09.297 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T22:37:09.330 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T22:37:09.348 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T22:37:09.351 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T22:37:09.368 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T22:37:09.404 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T22:37:09.465 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T22:37:09.470 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T22:37:09.478 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T22:37:09.478 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T22:37:09.484 DEBUG:teuthology.orchestra.run.smithi080:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T22:37:09.486 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T22:37:09.566 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T22:37:09.575 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T22:37:09.576 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T22:37:09.581 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T22:37:09.584 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T22:37:09.631 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T22:37:09.718 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T22:37:09.822 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T22:37:09.907 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T22:37:09.997 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:37:09.997 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T22:37:10.082 DEBUG:teuthology.orchestra.run.smithi080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T22:37:10.129 DEBUG:teuthology.orchestra.run.smithi080:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T22:37:10.182 DEBUG:teuthology.orchestra.run.smithi080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T22:37:10.268 DEBUG:teuthology.orchestra.run.smithi080:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T22:37:10.310 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-26T22:37:10.310 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T22:37:10.377 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T22:37:10.422 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T22:37:10.473 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T22:37:10.557 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T22:37:10.600 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T22:37:10.600 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T22:37:10.666 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2024-04-26T22:37:10.674 DEBUG:teuthology.orchestra.run.smithi080:> sudo service rsyslog restart 2024-04-26T22:37:10.676 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-04-26T22:37:10.717 INFO:teuthology.orchestra.run.smithi080.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T22:37:10.742 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T22:37:10.787 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T22:37:11.205 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T22:37:11.213 INFO:teuthology.task.internal:Starting timer... 2024-04-26T22:37:11.214 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T22:37:11.258 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T22:37:11.292 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T22:37:11.292 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2024-04-26T22:37:11.399 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2024-04-26T22:37:11.403 DEBUG:teuthology.orchestra.run.smithi080:> sudo service auditd rotate 2024-04-26T22:37:11.478 INFO:teuthology.orchestra.run.smithi080.stdout:Rotating logs: 2024-04-26T22:37:11.480 DEBUG:teuthology.orchestra.run.smithi094:> sudo service auditd rotate 2024-04-26T22:37:11.560 INFO:teuthology.orchestra.run.smithi094.stdout:Rotating logs: 2024-04-26T22:37:11.563 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T22:37:11.563 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2024-04-26T22:37:11.642 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2024-04-26T22:37:11.643 DEBUG:teuthology.orchestra.run.smithi080:> /usr/sbin/getenforce 2024-04-26T22:37:11.662 INFO:teuthology.orchestra.run.smithi080.stdout:Permissive 2024-04-26T22:37:11.662 DEBUG:teuthology.orchestra.run.smithi094:> /usr/sbin/getenforce 2024-04-26T22:37:11.689 INFO:teuthology.orchestra.run.smithi094.stdout:Permissive 2024-04-26T22:37:11.690 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi080.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi094.front.sepia.ceph.com': 'permissive'} 2024-04-26T22:37:11.690 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T22:37:11.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:37:11.767 DEBUG:teuthology.orchestra.run.smithi080:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T22:37:11.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:37:11.795 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T22:37:11.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:37:11.822 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T22:37:11.822 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2024-04-26T22:37:11.901 DEBUG:teuthology.orchestra.run.smithi080:> sudo /usr/sbin/setenforce permissive 2024-04-26T22:37:11.927 DEBUG:teuthology.orchestra.run.smithi094:> sudo /usr/sbin/setenforce permissive 2024-04-26T22:37:11.952 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T22:37:11.961 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T22:37:11.966 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T22:37:12.088 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T22:37:12.102 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T22:37:12.103 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com,smithi080.front.sepia.ceph.com,smithi094.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T22:42:33.015 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi080.front.sepia.ceph.com'), Remote(name='ubuntu@smithi094.front.sepia.ceph.com')] 2024-04-26T22:42:33.016 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-26T22:42:33.017 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T22:42:33.134 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-04-26T22:42:33.292 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-26T22:42:33.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-26T22:42:33.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T22:42:33.366 DEBUG:teuthology.orchestra.run.smithi080:> true 2024-04-26T22:42:33.441 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-26T22:42:33.441 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-26T22:42:33.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T22:42:33.514 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-04-26T22:42:33.590 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-26T22:42:33.591 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T22:42:33.602 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T22:42:33.602 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T22:42:33.602 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T22:42:33.610 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T22:42:33.610 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T22:42:33.612 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T22:42:33.612 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T22:42:33.638 INFO:teuthology.orchestra.run.smithi080.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T22:42:33.651 INFO:teuthology.orchestra.run.smithi080.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T22:42:33.660 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T22:42:33.673 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: ntpd: command not found 2024-04-26T22:42:33.673 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T22:42:33.683 INFO:teuthology.orchestra.run.smithi080.stdout:506 Cannot talk to daemon 2024-04-26T22:42:33.696 INFO:teuthology.orchestra.run.smithi080.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T22:42:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: ntpd: command not found 2024-04-26T22:42:33.704 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T22:42:33.708 INFO:teuthology.orchestra.run.smithi094.stdout:506 Cannot talk to daemon 2024-04-26T22:42:33.709 INFO:teuthology.orchestra.run.smithi080.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T22:42:33.721 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T22:42:33.734 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T22:42:33.750 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T22:42:33.769 INFO:teuthology.orchestra.run.smithi080.stderr:bash: line 1: ntpq: command not found 2024-04-26T22:42:33.773 INFO:teuthology.orchestra.run.smithi080.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T22:42:33.773 INFO:teuthology.orchestra.run.smithi080.stdout:=============================================================================== 2024-04-26T22:42:33.773 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:42:33.773 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:42:33.773 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:42:33.773 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:42:33.794 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-04-26T22:42:33.797 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T22:42:33.797 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-04-26T22:42:33.797 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:42:33.797 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:42:33.797 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:42:33.797 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:42:33.825 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2024-04-26T22:42:33.860 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2024-04-26T22:42:33.904 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T22:42:33.951 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T22:42:34.094 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-04-26T22:42:34.103 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T22:42:34.103 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-04-26T22:42:34.103 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:42:34.103 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:42:34.103 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:42:34.103 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:42:34.104 INFO:teuthology.run_tasks:Running task install... 2024-04-26T22:42:34.113 DEBUG:teuthology.task.install:project ceph 2024-04-26T22:42:34.114 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T22:42:34.114 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T22:42:34.114 INFO:teuthology.task.install:Using flavor: default 2024-04-26T22:42:34.119 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T22:42:34.119 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T22:42:34.120 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T22:42:34.120 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:42:34.122 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T22:42:34.122 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:42:34.124 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T22:42:34.124 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:42:34.275 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T22:42:34.275 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T22:42:34.279 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T22:42:34.279 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T22:42:34.280 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T22:42:34.280 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T22:42:34.392 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T22:42:34.392 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-26T22:42:34.392 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T22:42:34.397 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T22:42:34.397 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T22:42:34.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T22:42:34.418 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T22:42:34.419 DEBUG:teuthology.orchestra.run.smithi080:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T22:42:34.421 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T22:42:34.422 DEBUG:teuthology.orchestra.run.smithi094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T22:42:34.452 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T22:42:34.452 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:42:34.452 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T22:42:34.485 DEBUG:teuthology.orchestra.run.smithi080:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T22:42:34.486 DEBUG:teuthology.orchestra.run.smithi094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T22:42:34.551 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T22:42:34.551 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T22:42:34.558 DEBUG:teuthology.orchestra.run.smithi094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T22:42:34.559 DEBUG:teuthology.orchestra.run.smithi080:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T22:42:34.620 INFO:teuthology.orchestra.run.smithi094.stdout:check_obsoletes = 1 2024-04-26T22:42:34.622 INFO:teuthology.orchestra.run.smithi080.stdout:check_obsoletes = 1 2024-04-26T22:42:34.622 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum clean all 2024-04-26T22:42:34.623 DEBUG:teuthology.orchestra.run.smithi080:> sudo yum clean all 2024-04-26T22:42:34.686 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T22:42:34.802 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T22:42:34.845 INFO:teuthology.orchestra.run.smithi094.stdout:48 files removed 2024-04-26T22:42:34.847 INFO:teuthology.orchestra.run.smithi080.stdout:48 files removed 2024-04-26T22:42:34.872 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T22:42:34.874 DEBUG:teuthology.orchestra.run.smithi080:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T22:42:34.887 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2024-04-26T22:42:34.891 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2024-04-26T22:42:35.222 INFO:teuthology.orchestra.run.smithi027.stdout:48 files removed 2024-04-26T22:42:35.261 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T22:42:35.544 INFO:teuthology.orchestra.run.smithi080.stdout:ceph packages for x86_64 203 kB/s | 84 kB 00:00 2024-04-26T22:42:35.595 INFO:teuthology.orchestra.run.smithi094.stdout:ceph packages for x86_64 181 kB/s | 84 kB 00:00 2024-04-26T22:42:35.954 INFO:teuthology.orchestra.run.smithi094.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-26T22:42:35.995 INFO:teuthology.orchestra.run.smithi080.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-04-26T22:42:36.199 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 174 kB/s | 84 kB 00:00 2024-04-26T22:42:36.305 INFO:teuthology.orchestra.run.smithi094.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-04-26T22:42:36.387 INFO:teuthology.orchestra.run.smithi080.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-04-26T22:42:36.791 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-04-26T22:42:36.855 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-26T22:42:37.291 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-04-26T22:42:38.006 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-26T22:42:38.699 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-26T22:42:40.049 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-26T22:42:44.417 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.2 MB 00:02 2024-04-26T22:42:44.544 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - BaseOS 1.0 MB/s | 8.0 MB 00:08 2024-04-26T22:42:44.897 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.2 MB 00:00 2024-04-26T22:42:45.970 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-26T22:42:46.971 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 16 kB 00:00 2024-04-26T22:42:47.012 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-26T22:42:47.364 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-26T22:42:51.637 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.2 MB 00:01 2024-04-26T22:42:52.520 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-26T22:42:53.129 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 16 kB 00:00 2024-04-26T22:42:54.198 INFO:teuthology.orchestra.run.smithi080.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-26T22:42:54.654 INFO:teuthology.orchestra.run.smithi094.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T22:42:54.654 INFO:teuthology.orchestra.run.smithi094.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T22:42:54.665 INFO:teuthology.orchestra.run.smithi094.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T22:42:54.666 INFO:teuthology.orchestra.run.smithi094.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T22:42:54.666 INFO:teuthology.orchestra.run.smithi094.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T22:42:54.667 INFO:teuthology.orchestra.run.smithi094.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T22:42:54.669 INFO:teuthology.orchestra.run.smithi094.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T22:42:54.671 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T22:42:54.672 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T22:42:54.731 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-26T22:42:54.751 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-26T22:42:54.751 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-04-26T22:42:54.751 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-26T22:42:54.751 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-26T22:42:54.751 INFO:teuthology.orchestra.run.smithi094.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T22:42:54.751 INFO:teuthology.orchestra.run.smithi094.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T22:42:54.751 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T22:42:54.751 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T22:42:54.752 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T22:42:54.752 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T22:42:54.752 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T22:42:54.752 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T22:42:54.752 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T22:42:54.752 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T22:42:54.752 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T22:42:54.752 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T22:42:54.752 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T22:42:54.752 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T22:42:54.752 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T22:42:54.752 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T22:42:54.752 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T22:42:54.752 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T22:42:54.753 INFO:teuthology.orchestra.run.smithi094.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T22:42:54.753 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T22:42:54.753 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T22:42:54.753 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T22:42:54.753 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T22:42:54.753 INFO:teuthology.orchestra.run.smithi094.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T22:42:54.753 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T22:42:54.753 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T22:42:54.753 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T22:42:54.754 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T22:42:54.754 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T22:42:54.754 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T22:42:54.754 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T22:42:54.754 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T22:42:54.757 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T22:42:54.757 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T22:42:54.757 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T22:42:54.757 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T22:42:54.758 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T22:42:54.758 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrading: 2024-04-26T22:42:54.758 INFO:teuthology.orchestra.run.smithi094.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T22:42:54.758 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T22:42:54.758 INFO:teuthology.orchestra.run.smithi094.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T22:42:54.758 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T22:42:54.758 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-04-26T22:42:54.758 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T22:42:54.758 INFO:teuthology.orchestra.run.smithi094.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T22:42:54.758 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T22:42:54.758 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T22:42:54.758 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T22:42:54.759 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T22:42:54.759 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T22:42:54.759 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T22:42:54.759 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T22:42:54.759 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T22:42:54.759 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T22:42:54.759 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T22:42:54.759 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T22:42:54.759 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T22:42:54.759 INFO:teuthology.orchestra.run.smithi094.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T22:42:54.759 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T22:42:54.759 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T22:42:54.759 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T22:42:54.759 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T22:42:54.759 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T22:42:54.760 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T22:42:54.760 INFO:teuthology.orchestra.run.smithi094.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T22:42:54.760 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T22:42:54.760 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T22:42:54.760 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T22:42:54.760 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T22:42:54.760 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T22:42:54.760 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T22:42:54.760 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T22:42:54.760 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T22:42:54.760 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T22:42:54.760 INFO:teuthology.orchestra.run.smithi094.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T22:42:54.760 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T22:42:54.760 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T22:42:54.760 INFO:teuthology.orchestra.run.smithi094.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T22:42:54.760 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T22:42:54.761 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T22:42:54.761 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T22:42:54.761 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T22:42:54.761 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T22:42:54.761 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T22:42:54.761 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T22:42:54.761 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T22:42:54.761 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T22:42:54.761 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T22:42:54.761 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T22:42:54.761 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T22:42:54.761 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T22:42:54.761 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T22:42:54.761 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T22:42:54.761 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T22:42:54.762 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T22:42:54.762 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T22:42:54.762 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T22:42:54.762 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T22:42:54.762 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T22:42:54.762 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T22:42:54.762 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T22:42:54.762 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T22:42:54.762 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T22:42:54.762 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T22:42:54.762 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T22:42:54.763 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T22:42:54.763 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T22:42:54.763 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T22:42:54.763 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T22:42:54.763 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T22:42:54.763 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T22:42:54.763 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T22:42:54.763 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T22:42:54.763 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T22:42:54.763 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T22:42:54.763 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T22:42:54.763 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T22:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T22:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T22:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T22:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T22:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T22:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T22:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T22:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T22:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T22:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T22:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T22:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T22:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T22:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T22:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T22:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T22:42:54.765 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T22:42:54.765 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T22:42:54.765 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T22:42:54.765 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T22:42:54.765 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T22:42:54.765 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T22:42:54.765 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T22:42:54.765 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T22:42:54.765 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T22:42:54.765 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T22:42:54.765 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T22:42:54.765 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T22:42:54.765 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T22:42:54.765 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T22:42:54.765 INFO:teuthology.orchestra.run.smithi094.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T22:42:54.765 INFO:teuthology.orchestra.run.smithi094.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T22:42:54.766 INFO:teuthology.orchestra.run.smithi094.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T22:42:54.766 INFO:teuthology.orchestra.run.smithi094.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T22:42:54.766 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T22:42:54.766 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T22:42:54.766 INFO:teuthology.orchestra.run.smithi094.stdout:Installing weak dependencies: 2024-04-26T22:42:54.766 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T22:42:54.766 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:42:54.766 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-26T22:42:54.766 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-26T22:42:54.766 INFO:teuthology.orchestra.run.smithi094.stdout:Install 148 Packages 2024-04-26T22:42:54.766 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrade 4 Packages 2024-04-26T22:42:54.766 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:42:54.767 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 222 M 2024-04-26T22:42:54.767 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-26T22:42:55.546 INFO:teuthology.orchestra.run.smithi094.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-04-26T22:42:55.972 INFO:teuthology.orchestra.run.smithi094.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 2.0 MB/s | 889 kB 00:00 2024-04-26T22:42:56.047 INFO:teuthology.orchestra.run.smithi094.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.9 MB/s | 145 kB 00:00 2024-04-26T22:42:56.122 INFO:teuthology.orchestra.run.smithi094.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 6.7 MB/s | 5.4 MB 00:00 2024-04-26T22:42:56.248 INFO:teuthology.orchestra.run.smithi094.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 12 MB/s | 1.5 MB 00:00 2024-04-26T22:42:56.598 INFO:teuthology.orchestra.run.smithi094.stdout:(6/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 13 MB/s | 4.7 MB 00:00 2024-04-26T22:42:56.690 INFO:teuthology.orchestra.run.smithi094.stdout:(7/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 3.7 MB/s | 2.4 MB 00:00 2024-04-26T22:42:57.791 INFO:teuthology.orchestra.run.smithi094.stdout:(8/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 14 MB/s | 17 MB 00:01 2024-04-26T22:42:57.866 INFO:teuthology.orchestra.run.smithi094.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 335 kB/s | 25 kB 00:00 2024-04-26T22:42:58.492 INFO:teuthology.orchestra.run.smithi094.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 5.9 MB/s | 11 MB 00:01 2024-04-26T22:42:58.592 INFO:teuthology.orchestra.run.smithi094.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.2 MB/s | 225 kB 00:00 2024-04-26T22:42:58.692 INFO:teuthology.orchestra.run.smithi094.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 270 kB/s | 27 kB 00:00 2024-04-26T22:42:58.918 INFO:teuthology.orchestra.run.smithi094.stdout:(13/152): ceph-common-19.0.0-3306.gbc612ffb.el9 5.9 MB/s | 21 MB 00:03 2024-04-26T22:42:58.943 INFO:teuthology.orchestra.run.smithi094.stdout:(14/152): libcephfs-devel-19.0.0-3306.gbc612ffb 127 kB/s | 32 kB 00:00 2024-04-26T22:42:59.001 INFO:teuthology.orchestra.run.smithi094.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.7 MB/s | 163 kB 00:00 2024-04-26T22:42:59.068 INFO:teuthology.orchestra.run.smithi094.stdout:(16/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 4.8 MB/s | 747 kB 00:00 2024-04-26T22:42:59.102 INFO:teuthology.orchestra.run.smithi094.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 1.2 MB/s | 127 kB 00:00 2024-04-26T22:42:59.178 INFO:teuthology.orchestra.run.smithi094.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 4.5 MB/s | 501 kB 00:00 2024-04-26T22:42:59.270 INFO:teuthology.orchestra.run.smithi094.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 492 kB/s | 45 kB 00:00 2024-04-26T22:42:59.290 INFO:teuthology.orchestra.run.smithi080.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-26T22:42:59.353 INFO:teuthology.orchestra.run.smithi094.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.6 MB/s | 135 kB 00:00 2024-04-26T22:42:59.437 INFO:teuthology.orchestra.run.smithi094.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.9 MB/s | 161 kB 00:00 2024-04-26T22:42:59.537 INFO:teuthology.orchestra.run.smithi094.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.1 MB/s | 321 kB 00:00 2024-04-26T22:42:59.629 INFO:teuthology.orchestra.run.smithi094.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.2 MB/s | 298 kB 00:00 2024-04-26T22:42:59.721 INFO:teuthology.orchestra.run.smithi094.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.1 MB/s | 99 kB 00:00 2024-04-26T22:42:59.797 INFO:teuthology.orchestra.run.smithi094.stdout:(25/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 7.7 MB/s | 5.4 MB 00:00 2024-04-26T22:42:59.822 INFO:teuthology.orchestra.run.smithi094.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 850 kB/s | 85 kB 00:00 2024-04-26T22:42:59.914 INFO:teuthology.orchestra.run.smithi094.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.8 MB/s | 171 kB 00:00 2024-04-26T22:43:00.031 INFO:teuthology.orchestra.run.smithi094.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 293 kB/s | 34 kB 00:00 2024-04-26T22:43:00.131 INFO:teuthology.orchestra.run.smithi094.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.4 MB/s | 142 kB 00:00 2024-04-26T22:43:00.240 INFO:teuthology.orchestra.run.smithi094.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 7.0 MB/s | 3.1 MB 00:00 2024-04-26T22:43:01.082 INFO:teuthology.orchestra.run.smithi094.stdout:(31/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 15 MB/s | 49 MB 00:03 2024-04-26T22:43:01.149 INFO:teuthology.orchestra.run.smithi094.stdout:(32/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 3.5 MB/s | 3.5 MB 00:01 2024-04-26T22:43:01.174 INFO:teuthology.orchestra.run.smithi094.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.6 MB/s | 249 kB 00:00 2024-04-26T22:43:01.208 INFO:teuthology.orchestra.run.smithi094.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 841 kB/s | 49 kB 00:00 2024-04-26T22:43:01.225 INFO:teuthology.orchestra.run.smithi094.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 327 kB/s | 16 kB 00:00 2024-04-26T22:43:01.309 INFO:teuthology.orchestra.run.smithi094.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 6.9 MB/s | 7.4 MB 00:01 2024-04-26T22:43:01.334 INFO:teuthology.orchestra.run.smithi094.stdout:(37/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.1 MB/s | 273 kB 00:00 2024-04-26T22:43:01.368 INFO:teuthology.orchestra.run.smithi094.stdout:(38/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 5.2 MB/s | 762 kB 00:00 2024-04-26T22:43:01.393 INFO:teuthology.orchestra.run.smithi094.stdout:(39/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 217 kB/s | 18 kB 00:00 2024-04-26T22:43:01.440 INFO:teuthology.orchestra.run.smithi080.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T22:43:01.441 INFO:teuthology.orchestra.run.smithi080.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T22:43:01.451 INFO:teuthology.orchestra.run.smithi080.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T22:43:01.452 INFO:teuthology.orchestra.run.smithi080.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T22:43:01.453 INFO:teuthology.orchestra.run.smithi080.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T22:43:01.454 INFO:teuthology.orchestra.run.smithi080.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T22:43:01.455 INFO:teuthology.orchestra.run.smithi080.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T22:43:01.457 INFO:teuthology.orchestra.run.smithi080.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T22:43:01.458 INFO:teuthology.orchestra.run.smithi080.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T22:43:01.485 INFO:teuthology.orchestra.run.smithi094.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 534 kB/s | 80 kB 00:00 2024-04-26T22:43:01.518 INFO:teuthology.orchestra.run.smithi080.stdout:Dependencies resolved. 2024-04-26T22:43:01.528 INFO:teuthology.orchestra.run.smithi080.stdout:======================================================================================= 2024-04-26T22:43:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: Package Arch Version Repository Size 2024-04-26T22:43:01.528 INFO:teuthology.orchestra.run.smithi080.stdout:======================================================================================= 2024-04-26T22:43:01.528 INFO:teuthology.orchestra.run.smithi080.stdout:Installing: 2024-04-26T22:43:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T22:43:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T22:43:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T22:43:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T22:43:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T22:43:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T22:43:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T22:43:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T22:43:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T22:43:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T22:43:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T22:43:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T22:43:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T22:43:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T22:43:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T22:43:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T22:43:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T22:43:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T22:43:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T22:43:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T22:43:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T22:43:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T22:43:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T22:43:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T22:43:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T22:43:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T22:43:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T22:43:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T22:43:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T22:43:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T22:43:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T22:43:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T22:43:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T22:43:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T22:43:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T22:43:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T22:43:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T22:43:01.531 INFO:teuthology.orchestra.run.smithi080.stdout:Upgrading: 2024-04-26T22:43:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T22:43:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T22:43:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T22:43:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T22:43:01.531 INFO:teuthology.orchestra.run.smithi080.stdout:Installing dependencies: 2024-04-26T22:43:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T22:43:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T22:43:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T22:43:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T22:43:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T22:43:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T22:43:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T22:43:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T22:43:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T22:43:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T22:43:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T22:43:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T22:43:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T22:43:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T22:43:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T22:43:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T22:43:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T22:43:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T22:43:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T22:43:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T22:43:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T22:43:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T22:43:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T22:43:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T22:43:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T22:43:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T22:43:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T22:43:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T22:43:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T22:43:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T22:43:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T22:43:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T22:43:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T22:43:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T22:43:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T22:43:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T22:43:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T22:43:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T22:43:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T22:43:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T22:43:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T22:43:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T22:43:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T22:43:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T22:43:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T22:43:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T22:43:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T22:43:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T22:43:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T22:43:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T22:43:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T22:43:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T22:43:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T22:43:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T22:43:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T22:43:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T22:43:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T22:43:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T22:43:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T22:43:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T22:43:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T22:43:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T22:43:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T22:43:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T22:43:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T22:43:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T22:43:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T22:43:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T22:43:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T22:43:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T22:43:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T22:43:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T22:43:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T22:43:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T22:43:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T22:43:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T22:43:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T22:43:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T22:43:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T22:43:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T22:43:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T22:43:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T22:43:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T22:43:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T22:43:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T22:43:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T22:43:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T22:43:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T22:43:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T22:43:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T22:43:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T22:43:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T22:43:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T22:43:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T22:43:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T22:43:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T22:43:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T22:43:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T22:43:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T22:43:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T22:43:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T22:43:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T22:43:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T22:43:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T22:43:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T22:43:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T22:43:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T22:43:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T22:43:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T22:43:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T22:43:01.539 INFO:teuthology.orchestra.run.smithi080.stdout:Installing weak dependencies: 2024-04-26T22:43:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T22:43:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-26T22:43:01.539 INFO:teuthology.orchestra.run.smithi080.stdout:Transaction Summary 2024-04-26T22:43:01.540 INFO:teuthology.orchestra.run.smithi080.stdout:======================================================================================= 2024-04-26T22:43:01.540 INFO:teuthology.orchestra.run.smithi080.stdout:Install 148 Packages 2024-04-26T22:43:01.540 INFO:teuthology.orchestra.run.smithi080.stdout:Upgrade 4 Packages 2024-04-26T22:43:01.540 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-26T22:43:01.540 INFO:teuthology.orchestra.run.smithi080.stdout:Total download size: 222 M 2024-04-26T22:43:01.540 INFO:teuthology.orchestra.run.smithi080.stdout:Downloading Packages: 2024-04-26T22:43:01.610 INFO:teuthology.orchestra.run.smithi094.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 900 kB/s | 195 kB 00:00 2024-04-26T22:43:01.636 INFO:teuthology.orchestra.run.smithi094.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-04-26T22:43:01.669 INFO:teuthology.orchestra.run.smithi094.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 803 kB 00:00 2024-04-26T22:43:01.694 INFO:teuthology.orchestra.run.smithi094.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-26T22:43:01.728 INFO:teuthology.orchestra.run.smithi094.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-26T22:43:01.753 INFO:teuthology.orchestra.run.smithi094.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-26T22:43:01.804 INFO:teuthology.orchestra.run.smithi094.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 6.3 MB/s | 1.2 MB 00:00 2024-04-26T22:43:01.829 INFO:teuthology.orchestra.run.smithi094.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-26T22:43:01.896 INFO:teuthology.orchestra.run.smithi094.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 4.7 MB/s | 689 kB 00:00 2024-04-26T22:43:01.922 INFO:teuthology.orchestra.run.smithi094.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-26T22:43:01.955 INFO:teuthology.orchestra.run.smithi094.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 6.1 MB/s | 943 kB 00:00 2024-04-26T22:43:01.981 INFO:teuthology.orchestra.run.smithi094.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 256 kB/s | 21 kB 00:00 2024-04-26T22:43:02.023 INFO:teuthology.orchestra.run.smithi094.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 487 kB/s | 33 kB 00:00 2024-04-26T22:43:02.048 INFO:teuthology.orchestra.run.smithi094.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-04-26T22:43:02.090 INFO:teuthology.orchestra.run.smithi094.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-04-26T22:43:02.115 INFO:teuthology.orchestra.run.smithi094.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-04-26T22:43:02.174 INFO:teuthology.orchestra.run.smithi094.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 776 kB/s | 45 kB 00:00 2024-04-26T22:43:02.208 INFO:teuthology.orchestra.run.smithi094.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-26T22:43:02.258 INFO:teuthology.orchestra.run.smithi094.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-04-26T22:43:02.308 INFO:teuthology.orchestra.run.smithi094.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-04-26T22:43:02.350 INFO:teuthology.orchestra.run.smithi094.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-04-26T22:43:02.374 INFO:teuthology.orchestra.run.smithi080.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 41 kB/s | 6.6 kB 00:00 2024-04-26T22:43:02.376 INFO:teuthology.orchestra.run.smithi094.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 707 kB/s | 47 kB 00:00 2024-04-26T22:43:02.443 INFO:teuthology.orchestra.run.smithi094.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-26T22:43:02.468 INFO:teuthology.orchestra.run.smithi094.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-04-26T22:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 603 kB/s | 35 kB 00:00 2024-04-26T22:43:02.641 INFO:teuthology.orchestra.run.smithi080.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 3.3 MB/s | 889 kB 00:00 2024-04-26T22:43:02.691 INFO:teuthology.orchestra.run.smithi080.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.8 MB/s | 145 kB 00:00 2024-04-26T22:43:02.738 INFO:teuthology.orchestra.run.smithi094.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-04-26T22:43:02.802 INFO:teuthology.orchestra.run.smithi094.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-26T22:43:02.911 INFO:teuthology.orchestra.run.smithi094.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-04-26T22:43:02.953 INFO:teuthology.orchestra.run.smithi094.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-04-26T22:43:03.008 INFO:teuthology.orchestra.run.smithi080.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 7.5 MB/s | 2.4 MB 00:00 2024-04-26T22:43:03.011 INFO:teuthology.orchestra.run.smithi094.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-04-26T22:43:03.112 INFO:teuthology.orchestra.run.smithi094.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-04-26T22:43:03.179 INFO:teuthology.orchestra.run.smithi094.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-26T22:43:03.208 INFO:teuthology.orchestra.run.smithi080.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 7.6 MB/s | 1.5 MB 00:00 2024-04-26T22:43:03.263 INFO:teuthology.orchestra.run.smithi094.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-04-26T22:43:03.338 INFO:teuthology.orchestra.run.smithi094.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-26T22:43:03.413 INFO:teuthology.orchestra.run.smithi094.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-26T22:43:03.463 INFO:teuthology.orchestra.run.smithi094.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-04-26T22:43:03.564 INFO:teuthology.orchestra.run.smithi094.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-04-26T22:43:03.589 INFO:teuthology.orchestra.run.smithi094.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-04-26T22:43:03.639 INFO:teuthology.orchestra.run.smithi094.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-04-26T22:43:03.665 INFO:teuthology.orchestra.run.smithi094.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-04-26T22:43:03.667 INFO:teuthology.orchestra.run.smithi080.stdout:(6/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 3.7 MB/s | 5.4 MB 00:01 2024-04-26T22:43:03.740 INFO:teuthology.orchestra.run.smithi094.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-26T22:43:03.776 INFO:teuthology.orchestra.run.smithi080.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 8.3 MB/s | 4.7 MB 00:00 2024-04-26T22:43:03.815 INFO:teuthology.orchestra.run.smithi094.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-04-26T22:43:03.891 INFO:teuthology.orchestra.run.smithi094.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-04-26T22:43:03.916 INFO:teuthology.orchestra.run.smithi094.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 767 kB/s | 77 kB 00:00 2024-04-26T22:43:03.966 INFO:teuthology.orchestra.run.smithi094.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 558 kB/s | 42 kB 00:00 2024-04-26T22:43:04.017 INFO:teuthology.orchestra.run.smithi094.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-26T22:43:04.059 INFO:teuthology.orchestra.run.smithi094.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-26T22:43:04.117 INFO:teuthology.orchestra.run.smithi094.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-04-26T22:43:04.143 INFO:teuthology.orchestra.run.smithi094.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 610 kB/s | 51 kB 00:00 2024-04-26T22:43:04.193 INFO:teuthology.orchestra.run.smithi094.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-04-26T22:43:04.252 INFO:teuthology.orchestra.run.smithi094.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 714 kB/s | 42 kB 00:00 2024-04-26T22:43:04.301 INFO:teuthology.orchestra.run.smithi080.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 10 MB/s | 21 MB 00:02 2024-04-26T22:43:04.310 INFO:teuthology.orchestra.run.smithi094.stdout:(92/152): librdkafka-1.6.1-102.el9.x86_64.rpm 310 kB/s | 662 kB 00:02 2024-04-26T22:43:04.336 INFO:teuthology.orchestra.run.smithi094.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 527 kB/s | 44 kB 00:00 2024-04-26T22:43:04.359 INFO:teuthology.orchestra.run.smithi080.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 434 kB/s | 25 kB 00:00 2024-04-26T22:43:04.378 INFO:teuthology.orchestra.run.smithi094.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 860 kB/s | 36 kB 00:00 2024-04-26T22:43:04.403 INFO:teuthology.orchestra.run.smithi094.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-04-26T22:43:04.445 INFO:teuthology.orchestra.run.smithi094.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 707 kB/s | 47 kB 00:00 2024-04-26T22:43:04.495 INFO:teuthology.orchestra.run.smithi094.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-04-26T22:43:04.512 INFO:teuthology.orchestra.run.smithi094.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 861 kB/s | 58 kB 00:00 2024-04-26T22:43:04.579 INFO:teuthology.orchestra.run.smithi094.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-26T22:43:04.605 INFO:teuthology.orchestra.run.smithi094.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 692 kB/s | 64 kB 00:00 2024-04-26T22:43:04.664 INFO:teuthology.orchestra.run.smithi094.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 534 kB/s | 45 kB 00:00 2024-04-26T22:43:04.785 INFO:teuthology.orchestra.run.smithi080.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 10 MB/s | 11 MB 00:01 2024-04-26T22:43:04.822 INFO:teuthology.orchestra.run.smithi094.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-26T22:43:04.876 INFO:teuthology.orchestra.run.smithi080.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.4 MB/s | 225 kB 00:00 2024-04-26T22:43:04.943 INFO:teuthology.orchestra.run.smithi080.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 406 kB/s | 27 kB 00:00 2024-04-26T22:43:04.947 INFO:teuthology.orchestra.run.smithi094.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 65 kB/s | 22 kB 00:00 2024-04-26T22:43:05.019 INFO:teuthology.orchestra.run.smithi080.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 423 kB/s | 32 kB 00:00 2024-04-26T22:43:05.073 INFO:teuthology.orchestra.run.smithi094.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-04-26T22:43:05.123 INFO:teuthology.orchestra.run.smithi094.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-04-26T22:43:05.177 INFO:teuthology.orchestra.run.smithi080.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 4.6 MB/s | 747 kB 00:00 2024-04-26T22:43:05.190 INFO:teuthology.orchestra.run.smithi094.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-26T22:43:05.232 INFO:teuthology.orchestra.run.smithi094.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 122 kB/s | 19 kB 00:00 2024-04-26T22:43:05.257 INFO:teuthology.orchestra.run.smithi094.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 387 kB/s | 26 kB 00:00 2024-04-26T22:43:05.261 INFO:teuthology.orchestra.run.smithi080.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.9 MB/s | 163 kB 00:00 2024-04-26T22:43:05.324 INFO:teuthology.orchestra.run.smithi094.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-26T22:43:05.336 INFO:teuthology.orchestra.run.smithi080.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.7 MB/s | 127 kB 00:00 2024-04-26T22:43:05.350 INFO:teuthology.orchestra.run.smithi094.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 413 kB/s | 49 kB 00:00 2024-04-26T22:43:05.409 INFO:teuthology.orchestra.run.smithi094.stdout:(111/152): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-26T22:43:05.434 INFO:teuthology.orchestra.run.smithi094.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-26T22:43:05.484 INFO:teuthology.orchestra.run.smithi094.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-04-26T22:43:05.511 INFO:teuthology.orchestra.run.smithi080.stdout:(17/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 9.1 MB/s | 17 MB 00:01 2024-04-26T22:43:05.518 INFO:teuthology.orchestra.run.smithi094.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-04-26T22:43:05.543 INFO:teuthology.orchestra.run.smithi094.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-04-26T22:43:05.554 INFO:teuthology.orchestra.run.smithi080.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 2.3 MB/s | 501 kB 00:00 2024-04-26T22:43:05.577 INFO:teuthology.orchestra.run.smithi094.stdout:(116/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 839 kB 00:00 2024-04-26T22:43:05.611 INFO:teuthology.orchestra.run.smithi094.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 162 kB/s | 15 kB 00:00 2024-04-26T22:43:05.629 INFO:teuthology.orchestra.run.smithi080.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 602 kB/s | 45 kB 00:00 2024-04-26T22:43:05.636 INFO:teuthology.orchestra.run.smithi094.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-26T22:43:05.661 INFO:teuthology.orchestra.run.smithi094.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-26T22:43:05.686 INFO:teuthology.orchestra.run.smithi094.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-26T22:43:05.712 INFO:teuthology.orchestra.run.smithi094.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 467 kB/s | 35 kB 00:00 2024-04-26T22:43:05.713 INFO:teuthology.orchestra.run.smithi080.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.6 MB/s | 135 kB 00:00 2024-04-26T22:43:05.737 INFO:teuthology.orchestra.run.smithi094.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-26T22:43:05.771 INFO:teuthology.orchestra.run.smithi094.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-26T22:43:05.796 INFO:teuthology.orchestra.run.smithi094.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-04-26T22:43:05.796 INFO:teuthology.orchestra.run.smithi080.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.9 MB/s | 161 kB 00:00 2024-04-26T22:43:05.821 INFO:teuthology.orchestra.run.smithi094.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-04-26T22:43:05.864 INFO:teuthology.orchestra.run.smithi094.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-26T22:43:05.880 INFO:teuthology.orchestra.run.smithi094.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 275 kB/s | 23 kB 00:00 2024-04-26T22:43:05.906 INFO:teuthology.orchestra.run.smithi094.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 231 kB/s | 19 kB 00:00 2024-04-26T22:43:05.913 INFO:teuthology.orchestra.run.smithi080.stdout:(22/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 13 MB/s | 5.4 MB 00:00 2024-04-26T22:43:05.923 INFO:teuthology.orchestra.run.smithi094.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 334 kB/s | 20 kB 00:00 2024-04-26T22:43:05.938 INFO:teuthology.orchestra.run.smithi080.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 2.2 MB/s | 321 kB 00:00 2024-04-26T22:43:05.964 INFO:teuthology.orchestra.run.smithi094.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 793 kB/s | 46 kB 00:00 2024-04-26T22:43:05.980 INFO:teuthology.orchestra.run.smithi080.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 4.3 MB/s | 298 kB 00:00 2024-04-26T22:43:05.990 INFO:teuthology.orchestra.run.smithi094.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-04-26T22:43:06.006 INFO:teuthology.orchestra.run.smithi080.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.5 MB/s | 99 kB 00:00 2024-04-26T22:43:06.024 INFO:teuthology.orchestra.run.smithi094.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-04-26T22:43:06.049 INFO:teuthology.orchestra.run.smithi094.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 686 kB/s | 58 kB 00:00 2024-04-26T22:43:06.056 INFO:teuthology.orchestra.run.smithi080.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-26T22:43:06.074 INFO:teuthology.orchestra.run.smithi094.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-26T22:43:06.100 INFO:teuthology.orchestra.run.smithi094.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 216 kB/s | 16 kB 00:00 2024-04-26T22:43:06.125 INFO:teuthology.orchestra.run.smithi094.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-26T22:43:06.148 INFO:teuthology.orchestra.run.smithi080.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.8 MB/s | 171 kB 00:00 2024-04-26T22:43:06.150 INFO:teuthology.orchestra.run.smithi094.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 372 kB/s | 28 kB 00:00 2024-04-26T22:43:06.175 INFO:teuthology.orchestra.run.smithi094.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 406 kB/s | 31 kB 00:00 2024-04-26T22:43:06.201 INFO:teuthology.orchestra.run.smithi094.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-26T22:43:06.226 INFO:teuthology.orchestra.run.smithi094.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2024-04-26T22:43:06.231 INFO:teuthology.orchestra.run.smithi080.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 411 kB/s | 34 kB 00:00 2024-04-26T22:43:06.251 INFO:teuthology.orchestra.run.smithi094.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 472 kB/s | 36 kB 00:00 2024-04-26T22:43:06.277 INFO:teuthology.orchestra.run.smithi094.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 673 kB/s | 51 kB 00:00 2024-04-26T22:43:06.290 INFO:teuthology.orchestra.run.smithi080.stdout:(29/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 11 MB/s | 3.1 MB 00:00 2024-04-26T22:43:06.302 INFO:teuthology.orchestra.run.smithi094.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-26T22:43:06.315 INFO:teuthology.orchestra.run.smithi080.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.7 MB/s | 142 kB 00:00 2024-04-26T22:43:06.328 INFO:teuthology.orchestra.run.smithi094.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-26T22:43:06.353 INFO:teuthology.orchestra.run.smithi094.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-04-26T22:43:06.356 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-26T22:43:06.370 INFO:teuthology.orchestra.run.smithi094.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 297 kB/s | 20 kB 00:00 2024-04-26T22:43:06.395 INFO:teuthology.orchestra.run.smithi094.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-26T22:43:06.471 INFO:teuthology.orchestra.run.smithi094.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-26T22:43:06.749 INFO:teuthology.orchestra.run.smithi080.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 7.7 MB/s | 3.5 MB 00:00 2024-04-26T22:43:06.762 INFO:teuthology.orchestra.run.smithi094.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.6 MB/s | 1.3 MB 00:00 2024-04-26T22:43:06.841 INFO:teuthology.orchestra.run.smithi080.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.7 MB/s | 249 kB 00:00 2024-04-26T22:43:06.941 INFO:teuthology.orchestra.run.smithi080.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 494 kB/s | 49 kB 00:00 2024-04-26T22:43:07.005 INFO:teuthology.orchestra.run.smithi094.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.9 MB/s | 2.4 MB 00:00 2024-04-26T22:43:07.034 INFO:teuthology.orchestra.run.smithi080.stdout:(34/152): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-04-26T22:43:07.067 INFO:teuthology.orchestra.run.smithi080.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 131 kB/s | 16 kB 00:00 2024-04-26T22:43:07.159 INFO:teuthology.orchestra.run.smithi080.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.1 MB/s | 273 kB 00:00 2024-04-26T22:43:07.171 INFO:teuthology.orchestra.run.smithi094.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 4.0 MB/s | 3.1 MB 00:00 2024-04-26T22:43:07.226 INFO:teuthology.orchestra.run.smithi080.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 274 kB/s | 18 kB 00:00 2024-04-26T22:43:07.318 INFO:teuthology.orchestra.run.smithi080.stdout:(38/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 3.0 MB/s | 762 kB 00:00 2024-04-26T22:43:07.338 INFO:teuthology.orchestra.run.smithi094.stdout:(152/152): librados2-19.0.0-3306.gbc612ffb.el9. 3.5 MB/s | 3.4 MB 00:00 2024-04-26T22:43:07.343 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-26T22:43:07.343 INFO:teuthology.orchestra.run.smithi094.stdout:Total 18 MB/s | 222 MB 00:12 2024-04-26T22:43:07.343 INFO:teuthology.orchestra.run.smithi080.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 688 kB/s | 80 kB 00:00 2024-04-26T22:43:07.410 INFO:teuthology.orchestra.run.smithi080.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-26T22:43:07.460 INFO:teuthology.orchestra.run.smithi080.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-26T22:43:07.519 INFO:teuthology.orchestra.run.smithi080.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 803 kB 00:00 2024-04-26T22:43:07.552 INFO:teuthology.orchestra.run.smithi080.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-04-26T22:43:07.578 INFO:teuthology.orchestra.run.smithi080.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-26T22:43:07.603 INFO:teuthology.orchestra.run.smithi080.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-26T22:43:07.629 INFO:teuthology.orchestra.run.smithi080.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-26T22:43:07.654 INFO:teuthology.orchestra.run.smithi080.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-26T22:43:07.879 INFO:teuthology.orchestra.run.smithi080.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 689 kB 00:00 2024-04-26T22:43:07.930 INFO:teuthology.orchestra.run.smithi080.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-26T22:43:07.963 INFO:teuthology.orchestra.run.smithi080.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-04-26T22:43:07.988 INFO:teuthology.orchestra.run.smithi080.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 368 kB/s | 21 kB 00:00 2024-04-26T22:43:08.022 INFO:teuthology.orchestra.run.smithi080.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-26T22:43:08.047 INFO:teuthology.orchestra.run.smithi080.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-04-26T22:43:08.106 INFO:teuthology.orchestra.run.smithi080.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-04-26T22:43:08.165 INFO:teuthology.orchestra.run.smithi080.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-26T22:43:08.223 INFO:teuthology.orchestra.run.smithi080.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-26T22:43:08.433 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-26T22:43:08.574 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-26T22:43:08.575 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-26T22:43:08.649 INFO:teuthology.orchestra.run.smithi080.stdout:(57/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 11 MB/s | 49 MB 00:04 2024-04-26T22:43:08.674 INFO:teuthology.orchestra.run.smithi080.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 100 kB/s | 45 kB 00:00 2024-04-26T22:43:08.732 INFO:teuthology.orchestra.run.smithi080.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.2 MB/s | 3.0 MB 00:00 2024-04-26T22:43:08.791 INFO:teuthology.orchestra.run.smithi080.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 437 kB/s | 25 kB 00:00 2024-04-26T22:43:08.816 INFO:teuthology.orchestra.run.smithi080.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-04-26T22:43:08.850 INFO:teuthology.orchestra.run.smithi080.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-04-26T22:43:08.875 INFO:teuthology.orchestra.run.smithi080.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-26T22:43:08.909 INFO:teuthology.orchestra.run.smithi080.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-26T22:43:08.934 INFO:teuthology.orchestra.run.smithi080.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 600 kB/s | 35 kB 00:00 2024-04-26T22:43:09.001 INFO:teuthology.orchestra.run.smithi080.stdout:(66/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.8 MB/s | 662 kB 00:00 2024-04-26T22:43:09.210 INFO:teuthology.orchestra.run.smithi080.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-26T22:43:09.381 INFO:teuthology.orchestra.run.smithi080.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 10 MB/s | 4.7 MB 00:00 2024-04-26T22:43:09.410 INFO:teuthology.orchestra.run.smithi080.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 4.4 MB/s | 911 kB 00:00 2024-04-26T22:43:09.477 INFO:teuthology.orchestra.run.smithi080.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.2 MB/s | 4.4 MB 00:00 2024-04-26T22:43:09.502 INFO:teuthology.orchestra.run.smithi080.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-26T22:43:09.561 INFO:teuthology.orchestra.run.smithi080.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-26T22:43:09.603 INFO:teuthology.orchestra.run.smithi080.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.0 MB/s | 1.0 MB 00:00 2024-04-26T22:43:09.670 INFO:teuthology.orchestra.run.smithi080.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-26T22:43:09.737 INFO:teuthology.orchestra.run.smithi080.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-26T22:43:09.787 INFO:teuthology.orchestra.run.smithi080.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-26T22:43:09.838 INFO:teuthology.orchestra.run.smithi080.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-04-26T22:43:09.896 INFO:teuthology.orchestra.run.smithi080.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-26T22:43:09.971 INFO:teuthology.orchestra.run.smithi080.stdout:(79/152): postgresql-server-13.14-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-04-26T22:43:09.997 INFO:teuthology.orchestra.run.smithi080.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 346 kB/s | 35 kB 00:00 2024-04-26T22:43:10.080 INFO:teuthology.orchestra.run.smithi080.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-26T22:43:10.106 INFO:teuthology.orchestra.run.smithi080.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-26T22:43:10.148 INFO:teuthology.orchestra.run.smithi080.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-04-26T22:43:10.173 INFO:teuthology.orchestra.run.smithi080.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-26T22:43:10.206 INFO:teuthology.orchestra.run.smithi080.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 724 kB/s | 42 kB 00:00 2024-04-26T22:43:10.248 INFO:teuthology.orchestra.run.smithi080.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-26T22:43:10.274 INFO:teuthology.orchestra.run.smithi080.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-26T22:43:10.324 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-26T22:43:10.325 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-26T22:43:10.357 INFO:teuthology.orchestra.run.smithi080.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-04-26T22:43:10.383 INFO:teuthology.orchestra.run.smithi080.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 469 kB/s | 51 kB 00:00 2024-04-26T22:43:10.408 INFO:teuthology.orchestra.run.smithi080.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.7 MB/s | 279 kB 00:00 2024-04-26T22:43:10.433 INFO:teuthology.orchestra.run.smithi080.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-04-26T22:43:10.467 INFO:teuthology.orchestra.run.smithi080.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 711 kB/s | 42 kB 00:00 2024-04-26T22:43:10.492 INFO:teuthology.orchestra.run.smithi080.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 755 kB/s | 44 kB 00:00 2024-04-26T22:43:10.526 INFO:teuthology.orchestra.run.smithi080.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-26T22:43:10.551 INFO:teuthology.orchestra.run.smithi080.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 611 kB/s | 36 kB 00:00 2024-04-26T22:43:10.584 INFO:teuthology.orchestra.run.smithi080.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-04-26T22:43:10.618 INFO:teuthology.orchestra.run.smithi080.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-26T22:43:10.643 INFO:teuthology.orchestra.run.smithi080.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 985 kB/s | 58 kB 00:00 2024-04-26T22:43:10.685 INFO:teuthology.orchestra.run.smithi080.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-26T22:43:10.711 INFO:teuthology.orchestra.run.smithi080.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 953 kB/s | 64 kB 00:00 2024-04-26T22:43:10.745 INFO:teuthology.orchestra.run.smithi080.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 762 kB/s | 45 kB 00:00 2024-04-26T22:43:11.137 INFO:teuthology.orchestra.run.smithi080.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-26T22:43:11.379 INFO:teuthology.orchestra.run.smithi080.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 33 kB/s | 22 kB 00:00 2024-04-26T22:43:11.629 INFO:teuthology.orchestra.run.smithi080.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-26T22:43:11.704 INFO:teuthology.orchestra.run.smithi080.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-04-26T22:43:11.813 INFO:teuthology.orchestra.run.smithi080.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 712 kB/s | 309 kB 00:00 2024-04-26T22:43:11.896 INFO:teuthology.orchestra.run.smithi080.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-04-26T22:43:11.971 INFO:teuthology.orchestra.run.smithi080.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 650 kB/s | 49 kB 00:00 2024-04-26T22:43:12.047 INFO:teuthology.orchestra.run.smithi080.stdout:(109/152): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-26T22:43:12.072 INFO:teuthology.orchestra.run.smithi080.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 671 kB/s | 67 kB 00:00 2024-04-26T22:43:12.122 INFO:teuthology.orchestra.run.smithi080.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-26T22:43:12.206 INFO:teuthology.orchestra.run.smithi080.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-04-26T22:43:12.248 INFO:teuthology.orchestra.run.smithi080.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-04-26T22:43:12.273 INFO:teuthology.orchestra.run.smithi080.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 650 kB/s | 43 kB 00:00 2024-04-26T22:43:12.315 INFO:teuthology.orchestra.run.smithi080.stdout:(115/152): protobuf-compiler-3.14.0-13.el9.x86_ 550 kB/s | 863 kB 00:01 2024-04-26T22:43:12.332 INFO:teuthology.orchestra.run.smithi080.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 384 kB/s | 32 kB 00:00 2024-04-26T22:43:12.349 INFO:teuthology.orchestra.run.smithi080.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 198 kB/s | 15 kB 00:00 2024-04-26T22:43:12.416 INFO:teuthology.orchestra.run.smithi080.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-26T22:43:12.441 INFO:teuthology.orchestra.run.smithi080.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-26T22:43:12.483 INFO:teuthology.orchestra.run.smithi080.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 526 kB/s | 35 kB 00:00 2024-04-26T22:43:12.525 INFO:teuthology.orchestra.run.smithi080.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-26T22:43:12.575 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-26T22:43:12.592 INFO:teuthology.orchestra.run.smithi080.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-04-26T22:43:12.617 INFO:teuthology.orchestra.run.smithi080.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-26T22:43:12.668 INFO:teuthology.orchestra.run.smithi080.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-26T22:43:12.693 INFO:teuthology.orchestra.run.smithi080.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-26T22:43:12.718 INFO:teuthology.orchestra.run.smithi080.stdout:(126/152): python3-cheroot-8.6.0-4.el9.noarch.r 427 kB/s | 172 kB 00:00 2024-04-26T22:43:12.744 INFO:teuthology.orchestra.run.smithi080.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-04-26T22:43:12.769 INFO:teuthology.orchestra.run.smithi080.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-04-26T22:43:12.794 INFO:teuthology.orchestra.run.smithi080.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-04-26T22:43:12.836 INFO:teuthology.orchestra.run.smithi080.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-26T22:43:12.862 INFO:teuthology.orchestra.run.smithi080.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 501 kB/s | 46 kB 00:00 2024-04-26T22:43:12.887 INFO:teuthology.orchestra.run.smithi080.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 852 kB/s | 79 kB 00:00 2024-04-26T22:43:12.912 INFO:teuthology.orchestra.run.smithi080.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 764 kB/s | 58 kB 00:00 2024-04-26T22:43:12.946 INFO:teuthology.orchestra.run.smithi080.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-26T22:43:12.971 INFO:teuthology.orchestra.run.smithi080.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-26T22:43:12.996 INFO:teuthology.orchestra.run.smithi080.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-04-26T22:43:13.022 INFO:teuthology.orchestra.run.smithi080.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-04-26T22:43:13.055 INFO:teuthology.orchestra.run.smithi080.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-04-26T22:43:13.080 INFO:teuthology.orchestra.run.smithi080.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-26T22:43:13.122 INFO:teuthology.orchestra.run.smithi080.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-04-26T22:43:13.164 INFO:teuthology.orchestra.run.smithi080.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-04-26T22:43:13.190 INFO:teuthology.orchestra.run.smithi080.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 468 kB/s | 51 kB 00:00 2024-04-26T22:43:13.215 INFO:teuthology.orchestra.run.smithi080.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-26T22:43:13.248 INFO:teuthology.orchestra.run.smithi080.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-26T22:43:13.274 INFO:teuthology.orchestra.run.smithi080.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-26T22:43:13.299 INFO:teuthology.orchestra.run.smithi080.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-04-26T22:43:13.374 INFO:teuthology.orchestra.run.smithi080.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-26T22:43:13.400 INFO:teuthology.orchestra.run.smithi080.stdout:(148/152): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-26T22:43:13.608 INFO:teuthology.orchestra.run.smithi080.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 1.3 MB 00:00 2024-04-26T22:43:13.700 INFO:teuthology.orchestra.run.smithi080.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-04-26T22:43:13.759 INFO:teuthology.orchestra.run.smithi080.stdout:(151/152): librados2-19.0.0-3306.gbc612ffb.el9. 7.4 MB/s | 3.4 MB 00:00 2024-04-26T22:43:13.809 INFO:teuthology.orchestra.run.smithi080.stdout:(152/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 7.2 MB/s | 3.1 MB 00:00 2024-04-26T22:43:13.815 INFO:teuthology.orchestra.run.smithi080.stdout:-------------------------------------------------------------------------------- 2024-04-26T22:43:13.815 INFO:teuthology.orchestra.run.smithi080.stdout:Total 18 MB/s | 222 MB 00:12 2024-04-26T22:43:14.937 INFO:teuthology.orchestra.run.smithi080.stdout:Running transaction check 2024-04-26T22:43:15.080 INFO:teuthology.orchestra.run.smithi080.stdout:Transaction check succeeded. 2024-04-26T22:43:15.080 INFO:teuthology.orchestra.run.smithi080.stdout:Running transaction test 2024-04-26T22:43:15.103 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T22:43:15.105 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T22:43:15.126 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T22:43:15.127 INFO:teuthology.orchestra.run.smithi027.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T22:43:15.129 INFO:teuthology.orchestra.run.smithi027.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T22:43:15.130 INFO:teuthology.orchestra.run.smithi027.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T22:43:15.132 INFO:teuthology.orchestra.run.smithi027.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T22:43:15.136 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T22:43:15.137 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T22:43:15.212 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-26T22:43:15.233 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-04-26T22:43:15.233 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-04-26T22:43:15.233 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-04-26T22:43:15.233 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-26T22:43:15.233 INFO:teuthology.orchestra.run.smithi027.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T22:43:15.233 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T22:43:15.233 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T22:43:15.234 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T22:43:15.234 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T22:43:15.234 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T22:43:15.234 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T22:43:15.234 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T22:43:15.234 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T22:43:15.234 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T22:43:15.234 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T22:43:15.234 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T22:43:15.234 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T22:43:15.234 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T22:43:15.234 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T22:43:15.234 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T22:43:15.235 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T22:43:15.235 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T22:43:15.235 INFO:teuthology.orchestra.run.smithi027.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T22:43:15.235 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T22:43:15.235 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T22:43:15.235 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T22:43:15.235 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T22:43:15.235 INFO:teuthology.orchestra.run.smithi027.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T22:43:15.235 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T22:43:15.235 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T22:43:15.235 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T22:43:15.235 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T22:43:15.235 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T22:43:15.235 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T22:43:15.236 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T22:43:15.236 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T22:43:15.236 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T22:43:15.236 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T22:43:15.236 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T22:43:15.236 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T22:43:15.236 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T22:43:15.236 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2024-04-26T22:43:15.236 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T22:43:15.236 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T22:43:15.236 INFO:teuthology.orchestra.run.smithi027.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T22:43:15.236 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T22:43:15.236 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-04-26T22:43:15.236 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T22:43:15.237 INFO:teuthology.orchestra.run.smithi027.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T22:43:15.237 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T22:43:15.237 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T22:43:15.237 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T22:43:15.237 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T22:43:15.237 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T22:43:15.237 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T22:43:15.237 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T22:43:15.237 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T22:43:15.237 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T22:43:15.237 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T22:43:15.237 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T22:43:15.237 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T22:43:15.237 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T22:43:15.237 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T22:43:15.238 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T22:43:15.238 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T22:43:15.238 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T22:43:15.238 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T22:43:15.238 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T22:43:15.238 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T22:43:15.238 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T22:43:15.238 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T22:43:15.238 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T22:43:15.238 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T22:43:15.238 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T22:43:15.238 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T22:43:15.238 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T22:43:15.238 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T22:43:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T22:43:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T22:43:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T22:43:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T22:43:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T22:43:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T22:43:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T22:43:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T22:43:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T22:43:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T22:43:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T22:43:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T22:43:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T22:43:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T22:43:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T22:43:15.240 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T22:43:15.240 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T22:43:15.240 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T22:43:15.240 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T22:43:15.240 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T22:43:15.240 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T22:43:15.240 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T22:43:15.240 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T22:43:15.240 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T22:43:15.240 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T22:43:15.240 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T22:43:15.240 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T22:43:15.240 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T22:43:15.240 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T22:43:15.241 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T22:43:15.241 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T22:43:15.241 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T22:43:15.241 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T22:43:15.241 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T22:43:15.241 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T22:43:15.241 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T22:43:15.241 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T22:43:15.241 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T22:43:15.241 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T22:43:15.241 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T22:43:15.241 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T22:43:15.241 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T22:43:15.241 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T22:43:15.242 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T22:43:15.242 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T22:43:15.242 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T22:43:15.242 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T22:43:15.242 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T22:43:15.242 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T22:43:15.242 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T22:43:15.242 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T22:43:15.242 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T22:43:15.242 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T22:43:15.242 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T22:43:15.242 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T22:43:15.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T22:43:15.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T22:43:15.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T22:43:15.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T22:43:15.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T22:43:15.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T22:43:15.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T22:43:15.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T22:43:15.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T22:43:15.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T22:43:15.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T22:43:15.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T22:43:15.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T22:43:15.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T22:43:15.244 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T22:43:15.244 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T22:43:15.244 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T22:43:15.244 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T22:43:15.244 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T22:43:15.244 INFO:teuthology.orchestra.run.smithi027.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T22:43:15.244 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T22:43:15.244 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T22:43:15.244 INFO:teuthology.orchestra.run.smithi027.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T22:43:15.244 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T22:43:15.244 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T22:43:15.244 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2024-04-26T22:43:15.244 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T22:43:15.244 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:43:15.244 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-26T22:43:15.245 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-04-26T22:43:15.245 INFO:teuthology.orchestra.run.smithi027.stdout:Install 148 Packages 2024-04-26T22:43:15.245 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 4 Packages 2024-04-26T22:43:15.245 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:43:15.245 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 222 M 2024-04-26T22:43:15.245 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-26T22:43:16.119 INFO:teuthology.orchestra.run.smithi027.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-26T22:43:16.454 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-26T22:43:16.478 INFO:teuthology.orchestra.run.smithi027.stdout:(2/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 7.6 MB/s | 5.4 MB 00:00 2024-04-26T22:43:16.559 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T22:43:16.561 INFO:teuthology.orchestra.run.smithi027.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.7 MB/s | 145 kB 00:00 2024-04-26T22:43:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T22:43:16.651 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T22:43:16.685 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T22:43:16.711 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T22:43:16.745 INFO:teuthology.orchestra.run.smithi027.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 13 MB/s | 2.4 MB 00:00 2024-04-26T22:43:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T22:43:16.787 INFO:teuthology.orchestra.run.smithi027.stdout:(5/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.3 MB/s | 889 kB 00:00 2024-04-26T22:43:16.833 INFO:teuthology.orchestra.run.smithi080.stdout:Transaction test succeeded. 2024-04-26T22:43:16.833 INFO:teuthology.orchestra.run.smithi080.stdout:Running transaction 2024-04-26T22:43:16.879 INFO:teuthology.orchestra.run.smithi027.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 11 MB/s | 1.5 MB 00:00 2024-04-26T22:43:17.142 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T22:43:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T22:43:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T22:43:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T22:43:17.437 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T22:43:17.476 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T22:43:17.622 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T22:43:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T22:43:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T22:43:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T22:43:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T22:43:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T22:43:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T22:43:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T22:43:17.937 INFO:teuthology.orchestra.run.smithi027.stdout:(7/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 16 MB/s | 17 MB 00:01 2024-04-26T22:43:18.121 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T22:43:18.154 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T22:43:18.270 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T22:43:18.356 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T22:43:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T22:43:18.422 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T22:43:18.486 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T22:43:18.504 INFO:teuthology.orchestra.run.smithi027.stdout:(8/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 2.7 MB/s | 4.7 MB 00:01 2024-04-26T22:43:18.521 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T22:43:18.564 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T22:43:18.596 INFO:teuthology.orchestra.run.smithi027.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 277 kB/s | 25 kB 00:00 2024-04-26T22:43:18.597 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T22:43:18.625 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T22:43:18.694 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T22:43:18.748 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T22:43:18.787 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T22:43:18.788 INFO:teuthology.orchestra.run.smithi027.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 12 MB/s | 11 MB 00:00 2024-04-26T22:43:18.839 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T22:43:18.871 INFO:teuthology.orchestra.run.smithi027.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.7 MB/s | 225 kB 00:00 2024-04-26T22:43:18.915 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T22:43:18.930 INFO:teuthology.orchestra.run.smithi027.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 472 kB/s | 27 kB 00:00 2024-04-26T22:43:18.958 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T22:43:19.002 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T22:43:19.031 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T22:43:19.099 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T22:43:19.145 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T22:43:19.219 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T22:43:19.230 INFO:teuthology.orchestra.run.smithi027.stdout:(13/152): ceph-common-19.0.0-3306.gbc612ffb.el9 6.1 MB/s | 21 MB 00:03 2024-04-26T22:43:19.256 INFO:teuthology.orchestra.run.smithi027.stdout:(14/152): libcephfs-devel-19.0.0-3306.gbc612ffb 98 kB/s | 32 kB 00:00 2024-04-26T22:43:19.314 INFO:teuthology.orchestra.run.smithi027.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.8 MB/s | 163 kB 00:00 2024-04-26T22:43:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T22:43:19.381 INFO:teuthology.orchestra.run.smithi027.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.9 MB/s | 127 kB 00:00 2024-04-26T22:43:19.406 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T22:43:19.423 INFO:teuthology.orchestra.run.smithi027.stdout:(17/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 3.8 MB/s | 747 kB 00:00 2024-04-26T22:43:19.490 INFO:teuthology.orchestra.run.smithi027.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 4.6 MB/s | 501 kB 00:00 2024-04-26T22:43:19.586 INFO:teuthology.orchestra.run.smithi027.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 478 kB/s | 45 kB 00:00 2024-04-26T22:43:19.674 INFO:teuthology.orchestra.run.smithi027.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.5 MB/s | 135 kB 00:00 2024-04-26T22:43:19.749 INFO:teuthology.orchestra.run.smithi027.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.1 MB/s | 161 kB 00:00 2024-04-26T22:43:19.833 INFO:teuthology.orchestra.run.smithi027.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.8 MB/s | 321 kB 00:00 2024-04-26T22:43:19.916 INFO:teuthology.orchestra.run.smithi027.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.5 MB/s | 298 kB 00:00 2024-04-26T22:43:19.952 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T22:43:19.984 INFO:teuthology.orchestra.run.smithi027.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.5 MB/s | 99 kB 00:00 2024-04-26T22:43:20.038 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T22:43:20.074 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T22:43:20.118 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T22:43:20.126 INFO:teuthology.orchestra.run.smithi027.stdout:(25/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 7.7 MB/s | 5.4 MB 00:00 2024-04-26T22:43:20.151 INFO:teuthology.orchestra.run.smithi027.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 514 kB/s | 85 kB 00:00 2024-04-26T22:43:20.155 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T22:43:20.197 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T22:43:20.226 INFO:teuthology.orchestra.run.smithi027.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.2 MB/s | 171 kB 00:00 2024-04-26T22:43:20.332 INFO:teuthology.orchestra.run.smithi027.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 326 kB/s | 34 kB 00:00 2024-04-26T22:43:20.360 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T22:43:20.417 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T22:43:20.427 INFO:teuthology.orchestra.run.smithi027.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.5 MB/s | 142 kB 00:00 2024-04-26T22:43:20.472 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T22:43:20.545 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T22:43:20.577 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T22:43:20.614 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T22:43:20.644 INFO:teuthology.orchestra.run.smithi027.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 6.0 MB/s | 3.1 MB 00:00 2024-04-26T22:43:20.659 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T22:43:20.696 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T22:43:20.739 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T22:43:20.781 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T22:43:20.886 INFO:teuthology.orchestra.run.smithi027.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 7.7 MB/s | 3.5 MB 00:00 2024-04-26T22:43:20.914 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T22:43:20.971 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T22:43:21.001 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T22:43:21.003 INFO:teuthology.orchestra.run.smithi027.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.1 MB/s | 249 kB 00:00 2024-04-26T22:43:21.042 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T22:43:21.078 INFO:teuthology.orchestra.run.smithi027.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 665 kB/s | 49 kB 00:00 2024-04-26T22:43:21.145 INFO:teuthology.orchestra.run.smithi027.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 251 kB/s | 16 kB 00:00 2024-04-26T22:43:21.146 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T22:43:21.184 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T22:43:21.225 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T22:43:21.229 INFO:teuthology.orchestra.run.smithi027.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 3.2 MB/s | 273 kB 00:00 2024-04-26T22:43:21.250 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T22:43:21.275 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T22:43:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T22:43:21.387 INFO:teuthology.orchestra.run.smithi027.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 4.7 MB/s | 762 kB 00:00 2024-04-26T22:43:21.462 INFO:teuthology.orchestra.run.smithi027.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 246 kB/s | 18 kB 00:00 2024-04-26T22:43:21.477 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T22:43:21.517 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T22:43:21.545 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T22:43:21.579 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T22:43:21.619 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T22:43:21.621 INFO:teuthology.orchestra.run.smithi027.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 511 kB/s | 80 kB 00:00 2024-04-26T22:43:21.747 INFO:teuthology.orchestra.run.smithi027.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-04-26T22:43:21.848 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T22:43:22.014 INFO:teuthology.orchestra.run.smithi027.stdout:(40/152): ceph-mgr-diskprediction-local-19.0.0- 5.4 MB/s | 7.4 MB 00:01 2024-04-26T22:43:22.032 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T22:43:22.047 INFO:teuthology.orchestra.run.smithi027.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 653 kB/s | 195 kB 00:00 2024-04-26T22:43:22.175 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T22:43:22.208 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T22:43:22.250 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T22:43:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T22:43:22.374 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T22:43:22.422 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T22:43:22.454 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T22:43:22.501 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T22:43:22.600 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T22:43:22.606 INFO:teuthology.orchestra.run.smithi027.stdout:(42/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 12 MB/s | 49 MB 00:04 2024-04-26T22:43:22.647 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T22:43:22.664 INFO:teuthology.orchestra.run.smithi027.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 2.0 MB/s | 1.2 MB 00:00 2024-04-26T22:43:22.698 INFO:teuthology.orchestra.run.smithi027.stdout:(44/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 371 kB/s | 253 kB 00:00 2024-04-26T22:43:22.723 INFO:teuthology.orchestra.run.smithi027.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-26T22:43:22.773 INFO:teuthology.orchestra.run.smithi027.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.3 MB/s | 215 kB 00:00 2024-04-26T22:43:22.782 INFO:teuthology.orchestra.run.smithi080.stdout: Preparing : 1/1 2024-04-26T22:43:22.798 INFO:teuthology.orchestra.run.smithi027.stdout:(47/152): python3-ply-3.11-14.el9.noarch.rpm 557 kB/s | 106 kB 00:00 2024-04-26T22:43:22.824 INFO:teuthology.orchestra.run.smithi027.stdout:(48/152): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-04-26T22:43:22.870 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T22:43:22.974 INFO:teuthology.orchestra.run.smithi027.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 716 kB/s | 107 kB 00:00 2024-04-26T22:43:23.014 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T22:43:23.024 INFO:teuthology.orchestra.run.smithi027.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 439 kB/s | 21 kB 00:00 2024-04-26T22:43:23.141 INFO:teuthology.orchestra.run.smithi027.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-04-26T22:43:23.197 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T22:43:23.208 INFO:teuthology.orchestra.run.smithi027.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 2.3 MB/s | 943 kB 00:00 2024-04-26T22:43:23.234 INFO:teuthology.orchestra.run.smithi027.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 359 kB/s | 33 kB 00:00 2024-04-26T22:43:23.238 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T22:43:23.264 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T22:43:23.267 INFO:teuthology.orchestra.run.smithi027.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 1.3 MB/s | 313 kB 00:00 2024-04-26T22:43:23.307 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T22:43:23.318 INFO:teuthology.orchestra.run.smithi027.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 220 kB/s | 18 kB 00:00 2024-04-26T22:43:23.343 INFO:teuthology.orchestra.run.smithi027.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 151 kB/s | 11 kB 00:00 2024-04-26T22:43:23.356 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T22:43:23.403 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T22:43:23.410 INFO:teuthology.orchestra.run.smithi027.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 689 kB/s | 45 kB 00:00 2024-04-26T22:43:23.435 INFO:teuthology.orchestra.run.smithi027.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-26T22:43:23.437 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T22:43:23.534 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T22:43:23.560 INFO:teuthology.orchestra.run.smithi027.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.5 MB/s | 3.0 MB 00:00 2024-04-26T22:43:23.587 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T22:43:23.642 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T22:43:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T22:43:23.686 INFO:teuthology.orchestra.run.smithi027.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 2.3 MB/s | 582 kB 00:00 2024-04-26T22:43:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T22:43:23.711 INFO:teuthology.orchestra.run.smithi027.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 171 kB/s | 25 kB 00:00 2024-04-26T22:43:23.744 INFO:teuthology.orchestra.run.smithi027.stdout:(62/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 662 kB 00:00 2024-04-26T22:43:23.764 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T22:43:23.770 INFO:teuthology.orchestra.run.smithi027.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 573 kB/s | 47 kB 00:00 2024-04-26T22:43:23.828 INFO:teuthology.orchestra.run.smithi027.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 615 kB/s | 35 kB 00:00 2024-04-26T22:43:23.883 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T22:43:23.885 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T22:43:23.895 INFO:teuthology.orchestra.run.smithi027.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-04-26T22:43:23.904 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T22:43:23.924 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T22:43:23.929 INFO:teuthology.orchestra.run.smithi027.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.3 MB/s | 292 kB 00:00 2024-04-26T22:43:23.981 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T22:43:24.014 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T22:43:24.053 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T22:43:24.199 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T22:43:24.238 INFO:teuthology.orchestra.run.smithi027.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-04-26T22:43:24.250 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T22:43:24.295 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T22:43:24.354 INFO:teuthology.orchestra.run.smithi027.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-04-26T22:43:24.399 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T22:43:24.421 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T22:43:24.438 INFO:teuthology.orchestra.run.smithi027.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 4.5 MB/s | 911 kB 00:00 2024-04-26T22:43:24.467 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T22:43:24.472 INFO:teuthology.orchestra.run.smithi027.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-26T22:43:24.489 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T22:43:24.514 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T22:43:24.606 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T22:43:24.614 INFO:teuthology.orchestra.run.smithi027.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.2 MB/s | 4.4 MB 00:00 2024-04-26T22:43:24.689 INFO:teuthology.orchestra.run.smithi027.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 87 kB 00:00 2024-04-26T22:43:24.706 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T22:43:24.720 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T22:43:24.739 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T22:43:24.814 INFO:teuthology.orchestra.run.smithi027.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.0 MB/s | 1.0 MB 00:00 2024-04-26T22:43:24.846 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T22:43:24.914 INFO:teuthology.orchestra.run.smithi027.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-04-26T22:43:24.941 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T22:43:24.971 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T22:43:25.005 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T22:43:25.040 INFO:teuthology.orchestra.run.smithi027.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.0 MB/s | 248 kB 00:00 2024-04-26T22:43:25.072 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T22:43:25.105 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T22:43:25.107 INFO:teuthology.orchestra.run.smithi027.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 390 kB/s | 26 kB 00:00 2024-04-26T22:43:25.150 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T22:43:25.181 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T22:43:25.210 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T22:43:25.232 INFO:teuthology.orchestra.run.smithi027.stdout:(77/152): postgresql-server-13.14-1.el9.x86_64. 7.3 MB/s | 5.8 MB 00:00 2024-04-26T22:43:25.257 INFO:teuthology.orchestra.run.smithi027.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 128 kB/s | 19 kB 00:00 2024-04-26T22:43:25.280 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T22:43:25.324 INFO:teuthology.orchestra.run.smithi027.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 530 kB/s | 35 kB 00:00 2024-04-26T22:43:25.332 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T22:43:25.363 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T22:43:25.366 INFO:teuthology.orchestra.run.smithi027.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-04-26T22:43:25.407 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T22:43:25.458 INFO:teuthology.orchestra.run.smithi027.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-26T22:43:25.484 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T22:43:25.518 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T22:43:25.561 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T22:43:25.567 INFO:teuthology.orchestra.run.smithi027.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-04-26T22:43:25.590 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T22:43:25.660 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T22:43:25.684 INFO:teuthology.orchestra.run.smithi027.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 6.0 MB/s | 6.0 MB 00:00 2024-04-26T22:43:25.713 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T22:43:25.742 INFO:teuthology.orchestra.run.smithi027.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 443 kB/s | 77 kB 00:00 2024-04-26T22:43:25.759 INFO:teuthology.orchestra.run.smithi027.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 570 kB/s | 42 kB 00:00 2024-04-26T22:43:25.787 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T22:43:25.885 INFO:teuthology.orchestra.run.smithi027.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 1.9 MB/s | 269 kB 00:00 2024-04-26T22:43:25.910 INFO:teuthology.orchestra.run.smithi027.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-04-26T22:43:25.923 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T22:43:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T22:43:25.974 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T22:43:26.018 INFO:teuthology.orchestra.run.smithi027.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.4 MB/s | 5.1 MB 00:00 2024-04-26T22:43:26.020 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T22:43:26.048 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T22:43:26.077 INFO:teuthology.orchestra.run.smithi027.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 307 kB/s | 51 kB 00:00 2024-04-26T22:43:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T22:43:26.102 INFO:teuthology.orchestra.run.smithi027.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 650 kB/s | 54 kB 00:00 2024-04-26T22:43:26.136 INFO:teuthology.orchestra.run.smithi027.stdout:(91/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.1 MB/s | 279 kB 00:00 2024-04-26T22:43:26.186 INFO:teuthology.orchestra.run.smithi027.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 504 kB/s | 42 kB 00:00 2024-04-26T22:43:26.211 INFO:teuthology.orchestra.run.smithi027.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 595 kB/s | 44 kB 00:00 2024-04-26T22:43:26.287 INFO:teuthology.orchestra.run.smithi027.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 485 kB/s | 36 kB 00:00 2024-04-26T22:43:26.354 INFO:teuthology.orchestra.run.smithi027.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.3 MB/s | 221 kB 00:00 2024-04-26T22:43:26.379 INFO:teuthology.orchestra.run.smithi027.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 522 kB/s | 47 kB 00:00 2024-04-26T22:43:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T22:43:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T22:43:26.454 INFO:teuthology.orchestra.run.smithi027.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 781 kB/s | 58 kB 00:00 2024-04-26T22:43:26.500 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T22:43:26.513 INFO:teuthology.orchestra.run.smithi027.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.9 MB/s | 305 kB 00:00 2024-04-26T22:43:26.519 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T22:43:26.546 INFO:teuthology.orchestra.run.smithi027.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 94 kB 00:00 2024-04-26T22:43:26.550 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T22:43:26.585 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T22:43:26.586 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T22:43:26.615 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T22:43:26.638 INFO:teuthology.orchestra.run.smithi027.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 512 kB/s | 64 kB 00:00 2024-04-26T22:43:26.649 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T22:43:26.664 INFO:teuthology.orchestra.run.smithi027.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 386 kB/s | 45 kB 00:00 2024-04-26T22:43:26.687 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T22:43:26.729 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T22:43:26.878 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T22:43:26.931 INFO:teuthology.orchestra.run.smithi027.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-04-26T22:43:26.932 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T22:43:26.987 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T22:43:27.059 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T22:43:27.063 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T22:43:27.090 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T22:43:27.091 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T22:43:27.128 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T22:43:27.152 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T22:43:27.174 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T22:43:27.175 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T22:43:27.219 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T22:43:27.262 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T22:43:27.273 INFO:teuthology.orchestra.run.smithi027.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-04-26T22:43:27.305 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T22:43:27.366 INFO:teuthology.orchestra.run.smithi027.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 3.3 MB/s | 309 kB 00:00 2024-04-26T22:43:27.398 INFO:teuthology.orchestra.run.smithi027.stdout:(105/152): lua-devel-5.4.4-4.el9.x86_64.rpm 29 kB/s | 22 kB 00:00 2024-04-26T22:43:27.423 INFO:teuthology.orchestra.run.smithi027.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 343 kB/s | 19 kB 00:00 2024-04-26T22:43:27.439 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T22:43:27.493 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T22:43:27.523 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T22:43:27.565 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T22:43:27.590 INFO:teuthology.orchestra.run.smithi027.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 164 kB/s | 26 kB 00:00 2024-04-26T22:43:27.658 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T22:43:27.699 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T22:43:27.699 INFO:teuthology.orchestra.run.smithi027.stdout:(108/152): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-26T22:43:27.740 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T22:43:27.743 INFO:teuthology.orchestra.run.smithi027.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 342 kB/s | 49 kB 00:00 2024-04-26T22:43:27.765 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T22:43:27.775 INFO:teuthology.orchestra.run.smithi027.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 913 kB/s | 67 kB 00:00 2024-04-26T22:43:27.789 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T22:43:27.825 INFO:teuthology.orchestra.run.smithi027.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.9 MB/s | 151 kB 00:00 2024-04-26T22:43:27.862 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T22:43:27.990 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T22:43:28.000 INFO:teuthology.orchestra.run.smithi027.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 839 kB 00:00 2024-04-26T22:43:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T22:43:28.042 INFO:teuthology.orchestra.run.smithi027.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-04-26T22:43:28.058 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T22:43:28.092 INFO:teuthology.orchestra.run.smithi027.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 509 kB/s | 43 kB 00:00 2024-04-26T22:43:28.093 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T22:43:28.131 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T22:43:28.143 INFO:teuthology.orchestra.run.smithi027.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 324 kB/s | 32 kB 00:00 2024-04-26T22:43:28.163 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T22:43:28.168 INFO:teuthology.orchestra.run.smithi027.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 202 kB/s | 15 kB 00:00 2024-04-26T22:43:28.201 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T22:43:28.218 INFO:teuthology.orchestra.run.smithi027.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.3 MB/s | 172 kB 00:00 2024-04-26T22:43:28.236 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T22:43:28.252 INFO:teuthology.orchestra.run.smithi027.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-26T22:43:28.269 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T22:43:28.285 INFO:teuthology.orchestra.run.smithi027.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.8 MB/s | 303 kB 00:00 2024-04-26T22:43:28.302 INFO:teuthology.orchestra.run.smithi027.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 764 kB/s | 35 kB 00:00 2024-04-26T22:43:28.311 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T22:43:28.336 INFO:teuthology.orchestra.run.smithi027.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 4.1 MB/s | 206 kB 00:00 2024-04-26T22:43:28.345 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T22:43:28.433 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T22:43:28.461 INFO:teuthology.orchestra.run.smithi027.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-04-26T22:43:28.476 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T22:43:28.507 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T22:43:28.520 INFO:teuthology.orchestra.run.smithi027.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 814 kB/s | 144 kB 00:00 2024-04-26T22:43:28.546 INFO:teuthology.orchestra.run.smithi027.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 131 kB/s | 11 kB 00:00 2024-04-26T22:43:28.554 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T22:43:28.588 INFO:teuthology.orchestra.run.smithi027.stdout:(125/152): protobuf-compiler-3.14.0-13.el9.x86_ 449 kB/s | 863 kB 00:01 2024-04-26T22:43:28.613 INFO:teuthology.orchestra.run.smithi027.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 193 kB/s | 18 kB 00:00 2024-04-26T22:43:28.638 INFO:teuthology.orchestra.run.smithi027.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 261 kB/s | 23 kB 00:00 2024-04-26T22:43:28.663 INFO:teuthology.orchestra.run.smithi027.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 398 kB/s | 19 kB 00:00 2024-04-26T22:43:28.688 INFO:teuthology.orchestra.run.smithi027.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 266 kB/s | 20 kB 00:00 2024-04-26T22:43:28.805 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T22:43:28.859 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T22:43:28.869 INFO:teuthology.orchestra.run.smithi027.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-04-26T22:43:28.897 INFO:teuthology.orchestra.run.smithi027.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 223 kB/s | 46 kB 00:00 2024-04-26T22:43:28.922 INFO:teuthology.orchestra.run.smithi027.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 340 kB/s | 79 kB 00:00 2024-04-26T22:43:28.948 INFO:teuthology.orchestra.run.smithi027.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 752 kB/s | 58 kB 00:00 2024-04-26T22:43:28.998 INFO:teuthology.orchestra.run.smithi027.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-04-26T22:43:29.023 INFO:teuthology.orchestra.run.smithi027.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 167 kB/s | 16 kB 00:00 2024-04-26T22:43:29.049 INFO:teuthology.orchestra.run.smithi027.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 901 kB/s | 90 kB 00:00 2024-04-26T22:43:29.074 INFO:teuthology.orchestra.run.smithi027.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 386 kB/s | 28 kB 00:00 2024-04-26T22:43:29.099 INFO:teuthology.orchestra.run.smithi027.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 632 kB/s | 31 kB 00:00 2024-04-26T22:43:29.133 INFO:teuthology.orchestra.run.smithi027.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-26T22:43:29.158 INFO:teuthology.orchestra.run.smithi027.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 720 kB/s | 59 kB 00:00 2024-04-26T22:43:29.183 INFO:teuthology.orchestra.run.smithi027.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 442 kB/s | 36 kB 00:00 2024-04-26T22:43:29.209 INFO:teuthology.orchestra.run.smithi027.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-26T22:43:29.267 INFO:teuthology.orchestra.run.smithi027.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-26T22:43:29.292 INFO:teuthology.orchestra.run.smithi027.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 839 kB/s | 90 kB 00:00 2024-04-26T22:43:29.326 INFO:teuthology.orchestra.run.smithi027.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.7 MB/s | 427 kB 00:00 2024-04-26T22:43:29.368 INFO:teuthology.orchestra.run.smithi027.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 287 kB/s | 20 kB 00:00 2024-04-26T22:43:29.410 INFO:teuthology.orchestra.run.smithi027.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-26T22:43:29.469 INFO:teuthology.orchestra.run.smithi027.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-26T22:43:29.555 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T22:43:29.570 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T22:43:29.590 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T22:43:29.623 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T22:43:29.657 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T22:43:29.757 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T22:43:29.785 INFO:teuthology.orchestra.run.smithi027.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.3 MB/s | 1.3 MB 00:00 2024-04-26T22:43:29.815 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T22:43:29.861 INFO:teuthology.orchestra.run.smithi027.stdout:(150/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 7.1 MB/s | 3.1 MB 00:00 2024-04-26T22:43:29.901 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T22:43:29.951 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T22:43:29.952 INFO:teuthology.orchestra.run.smithi027.stdout:(151/152): librados2-19.0.0-3306.gbc612ffb.el9. 6.0 MB/s | 3.4 MB 00:00 2024-04-26T22:43:30.128 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T22:43:30.131 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T22:43:30.157 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T22:43:30.185 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T22:43:30.203 INFO:teuthology.orchestra.run.smithi027.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.8 MB/s | 2.4 MB 00:00 2024-04-26T22:43:30.211 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-26T22:43:30.212 INFO:teuthology.orchestra.run.smithi027.stdout:Total 15 MB/s | 222 MB 00:14 2024-04-26T22:43:30.234 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T22:43:30.274 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T22:43:30.274 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T22:43:30.274 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T22:43:30.274 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:43:30.329 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T22:43:30.866 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T22:43:30.988 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T22:43:32.193 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T22:43:32.278 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T22:43:32.306 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T22:43:32.340 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T22:43:32.653 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T22:43:32.695 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T22:43:32.785 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T22:43:32.819 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T22:43:32.855 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T22:43:33.188 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-26T22:43:33.333 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T22:43:33.356 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T22:43:33.438 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T22:43:33.450 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T22:43:33.456 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-26T22:43:33.456 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-26T22:43:35.845 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T22:43:35.888 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T22:43:36.246 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-26T22:43:36.246 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-26T22:43:36.454 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T22:43:36.500 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T22:43:36.575 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T22:43:36.575 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T22:43:36.575 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T22:43:36.575 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-26T22:43:36.610 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T22:43:41.924 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T22:43:41.924 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /sys 2024-04-26T22:43:41.924 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /proc 2024-04-26T22:43:41.924 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /mnt 2024-04-26T22:43:41.924 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /var/tmp 2024-04-26T22:43:41.924 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /home 2024-04-26T22:43:41.925 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /root 2024-04-26T22:43:41.925 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /tmp 2024-04-26T22:43:41.925 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:43:42.099 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T22:43:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T22:43:42.116 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:42.116 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T22:43:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:43:43.022 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-26T22:43:43.169 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T22:43:43.322 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T22:43:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T22:43:43.606 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T22:43:43.629 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T22:43:43.629 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:43.629 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T22:43:43.629 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T22:43:43.629 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T22:43:43.629 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:43:43.670 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T22:43:43.703 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T22:43:43.838 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T22:43:43.854 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T22:43:43.876 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T22:43:43.876 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:43.876 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T22:43:43.877 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T22:43:43.877 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T22:43:43.877 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:43:44.313 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T22:43:44.316 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T22:43:44.336 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T22:43:44.336 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:44.336 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T22:43:44.336 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T22:43:44.336 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T22:43:44.336 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:43:44.411 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T22:43:44.443 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T22:43:44.492 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T22:43:44.547 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T22:43:44.595 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T22:43:44.646 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T22:43:44.747 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T22:43:44.792 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T22:43:44.798 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T22:43:44.847 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T22:43:44.876 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T22:43:44.965 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T22:43:45.095 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T22:43:45.140 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T22:43:45.172 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T22:43:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T22:43:45.228 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T22:43:45.293 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T22:43:45.324 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T22:43:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T22:43:45.542 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T22:43:45.645 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T22:43:45.666 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T22:43:45.699 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T22:43:45.755 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T22:43:45.776 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T22:43:45.807 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T22:43:45.818 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T22:43:45.886 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T22:43:46.213 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T22:43:46.278 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T22:43:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T22:43:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T22:43:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T22:43:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T22:43:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T22:43:46.833 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T22:43:46.875 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T22:43:46.876 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T22:43:46.944 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T22:43:46.995 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T22:43:47.032 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T22:43:47.143 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T22:43:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T22:43:47.274 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T22:43:47.351 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T22:43:47.473 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T22:43:47.515 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T22:43:47.570 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T22:43:47.614 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T22:43:47.771 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T22:43:47.834 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T22:43:47.838 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T22:43:47.860 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T22:43:47.953 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T22:43:47.961 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T22:43:48.131 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T22:43:48.157 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T22:43:48.157 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /sys 2024-04-26T22:43:48.157 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /proc 2024-04-26T22:43:48.157 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /mnt 2024-04-26T22:43:48.157 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /var/tmp 2024-04-26T22:43:48.157 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /home 2024-04-26T22:43:48.158 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /root 2024-04-26T22:43:48.158 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /tmp 2024-04-26T22:43:48.158 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-26T22:43:48.162 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T22:43:48.169 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T22:43:48.185 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T22:43:48.185 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:48.186 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T22:43:48.186 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T22:43:48.186 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T22:43:48.186 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:43:48.230 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T22:43:48.237 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T22:43:48.240 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T22:43:48.330 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T22:43:48.338 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T22:43:48.346 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T22:43:48.346 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:48.346 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T22:43:48.346 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-26T22:43:48.395 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T22:43:48.977 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T22:43:49.089 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T22:43:49.148 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T22:43:49.200 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T22:43:49.251 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T22:43:49.318 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T22:43:49.328 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T22:43:49.349 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T22:43:49.372 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T22:43:49.372 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:49.372 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T22:43:49.372 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T22:43:49.372 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T22:43:49.373 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:43:49.417 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T22:43:49.439 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T22:43:49.439 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:49.439 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T22:43:49.439 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:43:49.484 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T22:43:49.507 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T22:43:49.507 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:49.507 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T22:43:49.507 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T22:43:49.507 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T22:43:49.507 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:43:49.566 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T22:43:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T22:43:49.705 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T22:43:49.796 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T22:43:49.820 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T22:43:49.826 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T22:43:49.838 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T22:43:49.842 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T22:43:49.842 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:49.842 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T22:43:49.843 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T22:43:49.843 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T22:43:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:43:49.851 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T22:43:49.852 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:49.852 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T22:43:49.852 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T22:43:49.852 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T22:43:49.852 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-26T22:43:49.910 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T22:43:49.984 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T22:43:50.030 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T22:43:50.077 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T22:43:50.097 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T22:43:50.099 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T22:43:50.100 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:50.100 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T22:43:50.100 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T22:43:50.100 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T22:43:50.100 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-26T22:43:50.170 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T22:43:50.393 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T22:43:50.460 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T22:43:50.497 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T22:43:50.536 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T22:43:50.558 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T22:43:50.558 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T22:43:50.558 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:50.559 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T22:43:50.559 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T22:43:50.559 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T22:43:50.559 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-26T22:43:50.666 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T22:43:50.701 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T22:43:50.715 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T22:43:50.746 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T22:43:50.770 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T22:43:50.798 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T22:43:50.837 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T22:43:50.870 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T22:43:50.954 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T22:43:51.003 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T22:43:51.071 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T22:43:51.073 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T22:43:51.161 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T22:43:51.266 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T22:43:51.300 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T22:43:51.318 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T22:43:51.354 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T22:43:51.368 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T22:43:51.385 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T22:43:51.413 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T22:43:51.442 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T22:43:51.512 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T22:43:51.575 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T22:43:51.624 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T22:43:51.679 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T22:43:51.727 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T22:43:51.812 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T22:43:51.840 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T22:43:51.868 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T22:43:51.870 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T22:43:51.902 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T22:43:51.972 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T22:43:52.002 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T22:43:52.028 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T22:43:52.100 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T22:43:52.175 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T22:43:52.253 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T22:43:52.411 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T22:43:52.479 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T22:43:53.030 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T22:43:53.062 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T22:43:53.511 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T22:43:53.569 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T22:43:53.619 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T22:43:53.742 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T22:43:53.808 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T22:43:53.917 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T22:43:53.994 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T22:43:54.027 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T22:43:54.054 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T22:43:54.159 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T22:43:54.256 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T22:43:54.284 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T22:43:54.333 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T22:43:54.356 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T22:43:54.379 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T22:43:54.379 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:54.379 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T22:43:54.379 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T22:43:54.379 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T22:43:54.380 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-26T22:43:54.391 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T22:43:54.423 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T22:43:54.434 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T22:43:54.440 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T22:43:54.482 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T22:43:54.523 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T22:43:54.531 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T22:43:54.555 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T22:43:54.581 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T22:43:54.582 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T22:43:54.763 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T22:43:54.811 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T22:43:54.841 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T22:43:55.282 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T22:43:55.438 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T22:43:55.461 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T22:43:55.504 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T22:43:55.505 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T22:43:55.524 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T22:43:55.543 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T22:43:55.566 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T22:43:55.566 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:55.566 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T22:43:55.566 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T22:43:55.567 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T22:43:55.567 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-26T22:43:55.611 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T22:43:55.632 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T22:43:55.632 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:55.632 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T22:43:55.632 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-26T22:43:55.677 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T22:43:55.700 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T22:43:55.700 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:55.700 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T22:43:55.700 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T22:43:55.701 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T22:43:55.701 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-26T22:43:56.011 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T22:43:56.035 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T22:43:56.035 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:43:56.035 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T22:43:56.035 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T22:43:56.035 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T22:43:56.035 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-26T22:43:56.344 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T22:43:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T22:43:56.458 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T22:43:56.547 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T22:43:56.580 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T22:43:56.580 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T22:43:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T22:43:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T22:43:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T22:43:56.776 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T22:43:56.854 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T22:43:57.192 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T22:43:57.323 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T22:43:57.365 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T22:43:57.411 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T22:43:57.899 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T22:43:58.096 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T22:43:58.254 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T22:43:58.287 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T22:43:58.354 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T22:43:58.962 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T22:43:58.997 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T22:43:59.091 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T22:43:59.116 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T22:44:00.599 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T22:44:00.640 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T22:44:00.681 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T22:44:00.713 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T22:44:00.740 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T22:44:00.920 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T22:44:00.961 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T22:44:00.992 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T22:44:01.599 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T22:44:01.631 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T22:44:01.651 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T22:44:01.959 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T22:44:01.997 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T22:44:02.494 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T22:44:02.578 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T22:44:02.619 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T22:44:02.680 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T22:44:02.701 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T22:44:02.725 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T22:44:02.732 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T22:44:02.733 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T22:44:02.817 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T22:44:02.889 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T22:44:02.889 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T22:44:02.889 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T22:44:02.889 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:44:02.897 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T22:44:02.897 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T22:44:02.934 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T22:44:02.980 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T22:44:03.051 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T22:44:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T22:44:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T22:44:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T22:44:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T22:44:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T22:44:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T22:44:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T22:44:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T22:44:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T22:44:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T22:44:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T22:44:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T22:44:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T22:44:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T22:44:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T22:44:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T22:44:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T22:44:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T22:44:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T22:44:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T22:44:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T22:44:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T22:44:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T22:44:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T22:44:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T22:44:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T22:44:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T22:44:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T22:44:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T22:44:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T22:44:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T22:44:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T22:44:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T22:44:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T22:44:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T22:44:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T22:44:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T22:44:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T22:44:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T22:44:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T22:44:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T22:44:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T22:44:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T22:44:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T22:44:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T22:44:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T22:44:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T22:44:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T22:44:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T22:44:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T22:44:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T22:44:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T22:44:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T22:44:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T22:44:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T22:44:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T22:44:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T22:44:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T22:44:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T22:44:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T22:44:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T22:44:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T22:44:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T22:44:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T22:44:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T22:44:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T22:44:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T22:44:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T22:44:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T22:44:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T22:44:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T22:44:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T22:44:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T22:44:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T22:44:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T22:44:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T22:44:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T22:44:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T22:44:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T22:44:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T22:44:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T22:44:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T22:44:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T22:44:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T22:44:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T22:44:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T22:44:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T22:44:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T22:44:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T22:44:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T22:44:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T22:44:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T22:44:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T22:44:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T22:44:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T22:44:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T22:44:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T22:44:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T22:44:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T22:44:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T22:44:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T22:44:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T22:44:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T22:44:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T22:44:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T22:44:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T22:44:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T22:44:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T22:44:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T22:44:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T22:44:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T22:44:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T22:44:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T22:44:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T22:44:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T22:44:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T22:44:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T22:44:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T22:44:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T22:44:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T22:44:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T22:44:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T22:44:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T22:44:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T22:44:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T22:44:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T22:44:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T22:44:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T22:44:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T22:44:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T22:44:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T22:44:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T22:44:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T22:44:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T22:44:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T22:44:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T22:44:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T22:44:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T22:44:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T22:44:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T22:44:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T22:44:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T22:44:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T22:44:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T22:44:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T22:44:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T22:44:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T22:44:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T22:44:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T22:44:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T22:44:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T22:44:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T22:44:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T22:44:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T22:44:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T22:44:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T22:44:05.119 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T22:44:05.120 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:44:05.120 INFO:teuthology.orchestra.run.smithi094.stdout:Upgraded: 2024-04-26T22:44:05.120 INFO:teuthology.orchestra.run.smithi094.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.120 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.120 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T22:44:05.120 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T22:44:05.120 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-26T22:44:05.120 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T22:44:05.120 INFO:teuthology.orchestra.run.smithi094.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T22:44:05.120 INFO:teuthology.orchestra.run.smithi094.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T22:44:05.120 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T22:44:05.120 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.120 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.121 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.121 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.121 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:05.121 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.121 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.121 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.121 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:05.121 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:05.121 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:05.121 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:05.121 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:05.121 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.121 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.121 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:05.122 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.122 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.122 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.122 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:05.122 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:05.122 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.122 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.122 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:05.122 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T22:44:05.122 INFO:teuthology.orchestra.run.smithi094.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T22:44:05.122 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T22:44:05.122 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T22:44:05.123 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T22:44:05.123 INFO:teuthology.orchestra.run.smithi094.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T22:44:05.123 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T22:44:05.123 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T22:44:05.123 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T22:44:05.123 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T22:44:05.123 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T22:44:05.123 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.123 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.123 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.123 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T22:44:05.123 INFO:teuthology.orchestra.run.smithi094.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T22:44:05.123 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T22:44:05.123 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T22:44:05.123 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T22:44:05.124 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.124 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.124 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T22:44:05.124 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.124 INFO:teuthology.orchestra.run.smithi094.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T22:44:05.124 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T22:44:05.124 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T22:44:05.124 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T22:44:05.124 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T22:44:05.124 INFO:teuthology.orchestra.run.smithi094.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T22:44:05.124 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T22:44:05.124 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T22:44:05.124 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T22:44:05.125 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T22:44:05.125 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T22:44:05.125 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T22:44:05.125 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T22:44:05.125 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T22:44:05.125 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T22:44:05.125 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T22:44:05.125 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T22:44:05.125 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T22:44:05.125 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T22:44:05.125 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T22:44:05.125 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T22:44:05.125 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T22:44:05.125 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T22:44:05.126 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.126 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.126 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.126 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T22:44:05.126 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T22:44:05.126 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T22:44:05.126 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T22:44:05.126 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T22:44:05.126 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T22:44:05.126 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T22:44:05.126 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T22:44:05.126 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T22:44:05.126 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T22:44:05.127 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T22:44:05.127 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T22:44:05.127 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T22:44:05.127 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T22:44:05.127 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T22:44:05.127 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T22:44:05.127 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T22:44:05.127 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T22:44:05.127 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T22:44:05.128 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T22:44:05.128 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T22:44:05.128 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T22:44:05.128 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T22:44:05.128 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T22:44:05.128 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T22:44:05.128 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T22:44:05.128 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T22:44:05.128 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T22:44:05.128 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T22:44:05.128 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T22:44:05.128 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T22:44:05.128 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T22:44:05.129 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T22:44:05.129 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T22:44:05.133 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T22:44:05.133 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T22:44:05.133 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T22:44:05.133 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T22:44:05.134 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T22:44:05.134 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T22:44:05.134 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.134 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.134 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T22:44:05.134 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T22:44:05.134 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T22:44:05.134 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.134 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T22:44:05.135 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T22:44:05.135 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T22:44:05.135 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T22:44:05.135 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T22:44:05.135 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T22:44:05.135 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T22:44:05.135 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T22:44:05.135 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T22:44:05.135 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T22:44:05.135 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T22:44:05.135 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T22:44:05.135 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T22:44:05.135 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T22:44:05.135 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T22:44:05.136 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.136 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.136 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:05.136 INFO:teuthology.orchestra.run.smithi094.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T22:44:05.136 INFO:teuthology.orchestra.run.smithi094.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T22:44:05.136 INFO:teuthology.orchestra.run.smithi094.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T22:44:05.136 INFO:teuthology.orchestra.run.smithi094.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T22:44:05.136 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T22:44:05.136 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T22:44:05.137 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T22:44:05.137 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:44:05.137 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-26T22:44:05.544 DEBUG:teuthology.parallel:result is None 2024-04-26T22:44:08.914 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T22:44:08.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T22:44:08.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T22:44:08.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T22:44:08.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T22:44:08.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T22:44:08.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T22:44:08.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T22:44:08.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T22:44:08.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T22:44:08.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T22:44:08.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T22:44:08.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T22:44:08.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T22:44:08.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T22:44:08.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T22:44:08.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T22:44:08.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T22:44:08.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T22:44:08.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T22:44:08.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T22:44:08.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T22:44:08.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T22:44:08.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T22:44:08.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T22:44:08.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T22:44:08.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T22:44:08.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T22:44:08.928 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T22:44:08.928 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T22:44:08.928 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T22:44:08.928 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T22:44:08.929 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T22:44:08.929 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T22:44:08.929 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T22:44:08.929 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T22:44:08.929 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T22:44:08.929 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T22:44:08.929 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T22:44:08.929 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T22:44:08.929 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T22:44:08.929 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T22:44:08.929 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T22:44:08.929 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T22:44:08.929 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T22:44:08.930 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T22:44:08.930 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T22:44:08.930 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T22:44:08.930 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T22:44:08.930 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T22:44:08.930 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T22:44:08.930 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T22:44:08.930 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T22:44:08.930 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T22:44:08.930 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T22:44:08.930 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T22:44:08.930 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T22:44:08.930 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T22:44:08.931 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T22:44:08.931 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T22:44:08.931 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T22:44:08.931 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T22:44:08.931 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T22:44:08.931 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T22:44:08.931 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T22:44:08.931 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T22:44:08.931 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T22:44:08.931 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T22:44:08.931 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T22:44:08.931 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T22:44:08.931 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T22:44:08.932 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T22:44:08.932 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T22:44:08.932 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T22:44:08.932 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T22:44:08.932 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T22:44:08.932 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T22:44:08.932 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T22:44:08.932 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T22:44:08.932 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T22:44:08.932 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T22:44:08.932 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T22:44:08.932 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T22:44:08.932 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T22:44:08.932 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T22:44:08.933 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T22:44:08.933 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T22:44:08.933 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T22:44:08.933 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T22:44:08.933 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T22:44:08.933 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T22:44:08.933 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T22:44:08.933 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T22:44:08.933 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T22:44:08.933 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T22:44:08.933 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T22:44:08.933 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T22:44:08.933 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T22:44:08.934 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T22:44:08.934 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T22:44:08.934 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T22:44:08.934 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T22:44:08.934 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T22:44:08.934 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T22:44:08.934 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T22:44:08.934 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T22:44:08.934 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T22:44:08.934 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T22:44:08.934 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T22:44:08.934 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T22:44:08.934 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T22:44:08.934 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T22:44:08.935 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T22:44:08.935 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T22:44:08.935 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T22:44:08.935 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T22:44:08.935 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T22:44:08.935 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T22:44:08.935 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T22:44:08.935 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T22:44:08.935 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T22:44:08.935 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T22:44:08.935 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T22:44:08.935 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T22:44:08.935 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T22:44:08.936 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T22:44:08.936 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T22:44:08.936 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T22:44:08.936 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T22:44:08.936 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T22:44:08.936 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T22:44:08.936 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T22:44:08.936 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T22:44:08.936 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T22:44:08.936 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T22:44:08.936 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T22:44:08.936 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T22:44:08.936 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T22:44:08.936 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T22:44:08.937 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T22:44:08.937 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T22:44:08.937 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T22:44:08.937 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T22:44:08.937 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T22:44:08.937 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T22:44:08.937 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T22:44:08.937 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T22:44:08.937 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T22:44:08.937 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T22:44:08.937 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T22:44:08.937 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T22:44:08.937 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T22:44:08.937 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T22:44:08.938 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T22:44:08.938 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T22:44:08.938 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T22:44:11.084 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T22:44:11.085 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-26T22:44:11.085 INFO:teuthology.orchestra.run.smithi080.stdout:Upgraded: 2024-04-26T22:44:11.085 INFO:teuthology.orchestra.run.smithi080.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.085 INFO:teuthology.orchestra.run.smithi080.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.085 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T22:44:11.085 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T22:44:11.085 INFO:teuthology.orchestra.run.smithi080.stdout:Installed: 2024-04-26T22:44:11.085 INFO:teuthology.orchestra.run.smithi080.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T22:44:11.085 INFO:teuthology.orchestra.run.smithi080.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T22:44:11.085 INFO:teuthology.orchestra.run.smithi080.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T22:44:11.085 INFO:teuthology.orchestra.run.smithi080.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T22:44:11.085 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.085 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T22:44:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T22:44:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T22:44:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T22:44:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T22:44:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T22:44:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T22:44:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T22:44:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T22:44:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T22:44:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T22:44:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T22:44:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T22:44:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T22:44:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T22:44:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T22:44:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T22:44:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T22:44:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T22:44:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T22:44:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T22:44:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T22:44:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T22:44:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T22:44:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T22:44:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T22:44:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T22:44:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T22:44:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T22:44:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T22:44:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T22:44:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T22:44:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T22:44:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T22:44:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T22:44:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T22:44:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T22:44:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T22:44:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T22:44:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T22:44:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T22:44:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T22:44:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T22:44:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T22:44:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T22:44:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T22:44:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T22:44:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T22:44:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T22:44:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T22:44:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T22:44:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T22:44:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T22:44:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T22:44:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T22:44:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T22:44:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T22:44:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T22:44:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T22:44:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T22:44:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T22:44:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T22:44:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T22:44:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T22:44:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T22:44:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T22:44:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T22:44:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T22:44:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T22:44:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T22:44:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T22:44:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T22:44:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T22:44:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T22:44:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T22:44:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T22:44:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T22:44:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T22:44:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T22:44:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T22:44:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T22:44:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T22:44:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T22:44:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T22:44:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T22:44:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T22:44:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T22:44:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T22:44:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T22:44:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T22:44:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T22:44:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T22:44:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T22:44:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T22:44:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T22:44:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T22:44:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T22:44:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T22:44:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T22:44:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T22:44:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T22:44:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T22:44:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T22:44:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T22:44:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T22:44:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-26T22:44:11.096 INFO:teuthology.orchestra.run.smithi080.stdout:Complete! 2024-04-26T22:44:11.529 DEBUG:teuthology.parallel:result is None 2024-04-26T22:44:16.672 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T22:44:16.672 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2024-04-26T22:44:16.672 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2024-04-26T22:44:16.672 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2024-04-26T22:44:16.672 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2024-04-26T22:44:16.672 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2024-04-26T22:44:16.672 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /root 2024-04-26T22:44:16.672 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2024-04-26T22:44:16.672 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:44:16.899 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T22:44:16.950 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T22:44:16.950 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:44:16.950 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T22:44:16.950 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:44:18.643 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T22:44:18.755 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T22:44:18.755 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:44:18.755 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T22:44:18.755 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T22:44:18.755 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T22:44:18.755 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:44:19.026 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T22:44:19.102 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T22:44:19.102 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:44:19.103 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T22:44:19.103 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T22:44:19.103 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T22:44:19.103 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:44:19.627 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T22:44:19.700 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T22:44:19.700 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:44:19.700 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T22:44:19.700 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T22:44:19.700 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T22:44:19.700 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:44:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T22:44:19.899 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T22:44:19.974 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T22:44:20.220 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T22:44:20.282 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T22:44:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T22:44:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T22:44:20.669 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T22:44:20.729 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T22:44:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T22:44:20.830 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T22:44:21.464 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T22:44:21.500 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T22:44:21.592 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T22:44:21.717 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T22:44:21.751 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T22:44:23.102 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T22:44:23.145 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T22:44:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T22:44:24.345 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T22:44:24.500 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T22:44:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T22:44:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T22:44:24.819 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T22:44:24.819 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:44:24.819 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T22:44:24.819 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T22:44:24.819 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T22:44:24.819 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:44:24.881 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T22:44:24.921 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T22:44:25.058 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T22:44:25.141 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T22:44:26.201 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T22:44:26.234 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T22:44:26.318 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T22:44:26.318 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:44:26.318 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T22:44:26.318 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T22:44:26.318 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T22:44:26.319 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:44:26.376 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T22:44:26.471 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T22:44:26.471 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:44:26.471 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T22:44:26.471 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:44:26.552 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T22:44:26.658 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T22:44:26.658 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:44:26.658 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T22:44:26.658 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T22:44:26.658 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T22:44:26.658 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:44:27.078 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T22:44:27.160 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T22:44:27.161 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:44:27.161 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T22:44:27.161 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T22:44:27.161 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T22:44:27.161 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:44:32.484 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T22:44:32.538 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T22:44:32.613 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T22:44:32.655 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T22:44:32.703 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T22:44:33.039 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T22:44:33.097 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T22:44:33.125 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T22:44:34.059 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T22:44:34.106 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T22:44:34.161 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T22:44:35.615 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T22:44:35.722 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T22:44:35.790 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T22:44:35.971 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T22:44:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T22:44:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T22:44:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T22:44:36.210 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T22:44:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T22:44:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T22:44:36.501 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T22:44:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T22:44:36.818 INFO:teuthology.orchestra.run.smithi027.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T22:44:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:44:42.741 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T22:44:42.741 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T22:44:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T22:44:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T22:44:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T22:44:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T22:44:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T22:44:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T22:44:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T22:44:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T22:44:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T22:44:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T22:44:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T22:44:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T22:44:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T22:44:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T22:44:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T22:44:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T22:44:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T22:44:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T22:44:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T22:44:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T22:44:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T22:44:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T22:44:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T22:44:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T22:44:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T22:44:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T22:44:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T22:44:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T22:44:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T22:44:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T22:44:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T22:44:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T22:44:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T22:44:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T22:44:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T22:44:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T22:44:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T22:44:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T22:44:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T22:44:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T22:44:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T22:44:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T22:44:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T22:44:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T22:44:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T22:44:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T22:44:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T22:44:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T22:44:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T22:44:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T22:44:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T22:44:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T22:44:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T22:44:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T22:44:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T22:44:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T22:44:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T22:44:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T22:44:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T22:44:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T22:44:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T22:44:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T22:44:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T22:44:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T22:44:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T22:44:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T22:44:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T22:44:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T22:44:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T22:44:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T22:44:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T22:44:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T22:44:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T22:44:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T22:44:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T22:44:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T22:44:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T22:44:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T22:44:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T22:44:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T22:44:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T22:44:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T22:44:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T22:44:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T22:44:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T22:44:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T22:44:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T22:44:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T22:44:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T22:44:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T22:44:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T22:44:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T22:44:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T22:44:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T22:44:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T22:44:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T22:44:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T22:44:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T22:44:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T22:44:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T22:44:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T22:44:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T22:44:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T22:44:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T22:44:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T22:44:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T22:44:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T22:44:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T22:44:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T22:44:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T22:44:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T22:44:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T22:44:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T22:44:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T22:44:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T22:44:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T22:44:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T22:44:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T22:44:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T22:44:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T22:44:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T22:44:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T22:44:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T22:44:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T22:44:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T22:44:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T22:44:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T22:44:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T22:44:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T22:44:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T22:44:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T22:44:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T22:44:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T22:44:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T22:44:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T22:44:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T22:44:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T22:44:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T22:44:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T22:44:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T22:44:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T22:44:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T22:44:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T22:44:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T22:44:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T22:44:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T22:44:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T22:44:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T22:44:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T22:44:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T22:44:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T22:44:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T22:44:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T22:44:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T22:44:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:44:45.612 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2024-04-26T22:44:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T22:44:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T22:44:45.612 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-26T22:44:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T22:44:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T22:44:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T22:44:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T22:44:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:44:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T22:44:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T22:44:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T22:44:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T22:44:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T22:44:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T22:44:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T22:44:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T22:44:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T22:44:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T22:44:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T22:44:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T22:44:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T22:44:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T22:44:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T22:44:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T22:44:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T22:44:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T22:44:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T22:44:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T22:44:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T22:44:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T22:44:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T22:44:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T22:44:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T22:44:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T22:44:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T22:44:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T22:44:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T22:44:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T22:44:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T22:44:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T22:44:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T22:44:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T22:44:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T22:44:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T22:44:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T22:44:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T22:44:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T22:44:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T22:44:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T22:44:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T22:44:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T22:44:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T22:44:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T22:44:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T22:44:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T22:44:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T22:44:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T22:44:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T22:44:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T22:44:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T22:44:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T22:44:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T22:44:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T22:44:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T22:44:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T22:44:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T22:44:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T22:44:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T22:44:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T22:44:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T22:44:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T22:44:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T22:44:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T22:44:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T22:44:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T22:44:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T22:44:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T22:44:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T22:44:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T22:44:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T22:44:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T22:44:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T22:44:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T22:44:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T22:44:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T22:44:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T22:44:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T22:44:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T22:44:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T22:44:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T22:44:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T22:44:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T22:44:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T22:44:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T22:44:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T22:44:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T22:44:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T22:44:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T22:44:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T22:44:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T22:44:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T22:44:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T22:44:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T22:44:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T22:44:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T22:44:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T22:44:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:44:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T22:44:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T22:44:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T22:44:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T22:44:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T22:44:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T22:44:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T22:44:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:44:45.624 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-26T22:44:46.217 DEBUG:teuthology.parallel:result is None 2024-04-26T22:44:46.218 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:44:46.435 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T22:44:46.491 INFO:teuthology.orchestra.run.smithi027.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T22:44:46.491 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T22:44:46.491 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T22:44:46.493 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:44:46.711 DEBUG:teuthology.orchestra.run.smithi080:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T22:44:46.736 INFO:teuthology.orchestra.run.smithi080.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T22:44:46.736 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T22:44:46.736 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T22:44:46.738 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:44:46.872 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T22:44:46.898 INFO:teuthology.orchestra.run.smithi094.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T22:44:46.899 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T22:44:46.899 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T22:44:46.900 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T22:44:46.901 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:44:46.901 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T22:44:46.990 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-26T22:44:46.990 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T22:44:47.020 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T22:44:47.020 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T22:44:47.051 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T22:44:47.051 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:44:47.051 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T22:44:47.123 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T22:44:47.231 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-26T22:44:47.231 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T22:44:47.258 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T22:44:47.326 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T22:44:47.326 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T22:44:47.355 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T22:44:47.423 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T22:44:47.423 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:44:47.423 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T22:44:47.491 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T22:44:47.606 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-26T22:44:47.606 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T22:44:47.631 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T22:44:47.694 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T22:44:47.695 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T22:44:47.722 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T22:44:47.790 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T22:44:47.790 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:44:47.790 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T22:44:47.857 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T22:44:47.968 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-26T22:44:47.968 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T22:44:47.993 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T22:44:48.056 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T22:44:48.056 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T22:44:48.084 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T22:44:48.151 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T22:44:48.256 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-26T22:44:48.256 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:44:48.256 INFO:tasks.cephadm:Cluster fsid is 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:44:48.257 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T22:44:48.257 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.27', 'mon.b': '172.21.15.80', 'mon.c': '172.21.15.94'} 2024-04-26T22:44:48.257 INFO:tasks.cephadm:First mon is mon.a on smithi027 2024-04-26T22:44:48.257 INFO:tasks.cephadm:First mgr is x 2024-04-26T22:44:48.257 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T22:44:48.257 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname $(hostname -s) 2024-04-26T22:44:48.331 DEBUG:teuthology.orchestra.run.smithi080:> sudo hostname $(hostname -s) 2024-04-26T22:44:48.355 DEBUG:teuthology.orchestra.run.smithi094:> sudo hostname $(hostname -s) 2024-04-26T22:44:48.383 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-26T22:44:48.383 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:44:48.616 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78918/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3306-gbc612ffb', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3306.gbc612ffb'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-26 13:02:42.256491', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240426.111959', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-26T22:44:48.755 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240426.111959, sha1 bc612ffb9c1d5bc27536bc917a705262a36d4387 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:44:48.757 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm 2024-04-26T22:44:48.757 DEBUG:teuthology.orchestra.run.smithi027:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T22:44:49.237 INFO:teuthology.orchestra.run.smithi027.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 22:44 /home/ubuntu/cephtest/cephadm 2024-04-26T22:44:49.237 DEBUG:teuthology.orchestra.run.smithi080:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T22:44:49.677 INFO:teuthology.orchestra.run.smithi080.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 22:44 /home/ubuntu/cephtest/cephadm 2024-04-26T22:44:49.677 DEBUG:teuthology.orchestra.run.smithi094:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T22:44:50.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 22:44 /home/ubuntu/cephtest/cephadm 2024-04-26T22:44:50.277 DEBUG:teuthology.orchestra.run.smithi027:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T22:44:50.328 DEBUG:teuthology.orchestra.run.smithi080:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T22:44:50.347 DEBUG:teuthology.orchestra.run.smithi094:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T22:44:50.380 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 on all hosts... 2024-04-26T22:44:50.380 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T22:44:50.384 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T22:44:50.389 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T22:44:51.910 INFO:teuthology.orchestra.run.smithi094.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T22:44:51.945 INFO:teuthology.orchestra.run.smithi080.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T22:44:52.665 INFO:teuthology.orchestra.run.smithi027.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T22:45:23.461 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T22:45:23.461 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T22:45:23.461 INFO:teuthology.orchestra.run.smithi094.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T22:45:23.462 INFO:teuthology.orchestra.run.smithi094.stdout: "repo_digests": [ 2024-04-26T22:45:23.462 INFO:teuthology.orchestra.run.smithi094.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T22:45:23.462 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-26T22:45:23.462 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T22:45:27.088 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-26T22:45:27.088 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T22:45:27.088 INFO:teuthology.orchestra.run.smithi080.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T22:45:27.088 INFO:teuthology.orchestra.run.smithi080.stdout: "repo_digests": [ 2024-04-26T22:45:27.088 INFO:teuthology.orchestra.run.smithi080.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T22:45:27.088 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-26T22:45:27.088 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-26T22:45:30.511 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T22:45:30.512 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T22:45:30.512 INFO:teuthology.orchestra.run.smithi027.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T22:45:30.512 INFO:teuthology.orchestra.run.smithi027.stdout: "repo_digests": [ 2024-04-26T22:45:30.512 INFO:teuthology.orchestra.run.smithi027.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T22:45:30.512 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T22:45:30.512 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T22:45:30.580 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph 2024-04-26T22:45:30.678 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /etc/ceph 2024-04-26T22:45:30.710 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph 2024-04-26T22:45:30.741 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /etc/ceph 2024-04-26T22:45:30.837 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 777 /etc/ceph 2024-04-26T22:45:30.864 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /etc/ceph 2024-04-26T22:45:30.892 INFO:tasks.cephadm:Writing seed config... 2024-04-26T22:45:30.893 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T22:45:30.894 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T22:45:30.894 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T22:45:30.894 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T22:45:30.894 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T22:45:30.894 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-26T22:45:30.894 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T22:45:30.894 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T22:45:30.894 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T22:45:30.894 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-26T22:45:30.894 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-26T22:45:30.894 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-26T22:45:30.894 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-26T22:45:30.894 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T22:45:30.895 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T22:45:30.895 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T22:45:30.895 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T22:45:30.895 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-26T22:45:30.895 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T22:45:30.895 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T22:45:30.895 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T22:45:30.895 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-26T22:45:30.895 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-26T22:45:30.895 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T22:45:30.895 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T22:45:30.895 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T22:45:30.895 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T22:45:30.895 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T22:45:30.895 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T22:45:30.896 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T22:45:30.896 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T22:45:30.896 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T22:45:30.896 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-26T22:45:30.896 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T22:45:30.896 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-26T22:45:30.896 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-26T22:45:30.896 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T22:45:30.896 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T22:45:30.896 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-26T22:45:30.896 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T22:45:30.896 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T22:45:30.896 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T22:45:30.896 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T22:45:30.896 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T22:45:30.897 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T22:45:30.897 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-26T22:45:30.897 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-26T22:45:30.897 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:45:30.897 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T22:45:30.959 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 95ebdda6-041e-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T22:45:30.959 DEBUG:teuthology.orchestra.run.smithi027:mon.a> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mon.a.service 2024-04-26T22:45:31.007 DEBUG:teuthology.orchestra.run.smithi027:mgr.x> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mgr.x.service 2024-04-26T22:45:31.055 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T22:45:31.055 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 -v bootstrap --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.27 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:45:31.348 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-26T22:45:31.348 INFO:teuthology.orchestra.run.smithi027.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387', '-v', 'bootstrap', '--fsid', '95ebdda6-041e-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.27', '--skip-admin-label'] 2024-04-26T22:45:31.349 INFO:teuthology.orchestra.run.smithi027.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T22:45:31.349 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying podman|docker is present... 2024-04-26T22:45:31.407 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T22:45:31.407 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying lvm2 is present... 2024-04-26T22:45:31.407 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying time synchronization is in place... 2024-04-26T22:45:31.423 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T22:45:31.423 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T22:45:31.437 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T22:45:31.437 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-04-26T22:45:31.456 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout enabled 2024-04-26T22:45:31.473 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout active 2024-04-26T22:45:31.473 INFO:teuthology.orchestra.run.smithi027.stdout:Unit chronyd.service is enabled and running 2024-04-26T22:45:31.473 INFO:teuthology.orchestra.run.smithi027.stdout:Repeating the final host check... 2024-04-26T22:45:31.537 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T22:45:31.538 INFO:teuthology.orchestra.run.smithi027.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T22:45:31.538 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl is present 2024-04-26T22:45:31.538 INFO:teuthology.orchestra.run.smithi027.stdout:lvcreate is present 2024-04-26T22:45:31.554 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T22:45:31.554 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T22:45:31.570 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T22:45:31.571 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-04-26T22:45:31.590 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout enabled 2024-04-26T22:45:31.607 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout active 2024-04-26T22:45:31.607 INFO:teuthology.orchestra.run.smithi027.stdout:Unit chronyd.service is enabled and running 2024-04-26T22:45:31.607 INFO:teuthology.orchestra.run.smithi027.stdout:Host looks OK 2024-04-26T22:45:31.607 INFO:teuthology.orchestra.run.smithi027.stdout:Cluster fsid: 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:45:31.607 INFO:teuthology.orchestra.run.smithi027.stdout:Acquiring lock 140534394482640 on /run/cephadm/95ebdda6-041e-11ef-bc93-c7b262605968.lock 2024-04-26T22:45:31.608 INFO:teuthology.orchestra.run.smithi027.stdout:Lock 140534394482640 acquired on /run/cephadm/95ebdda6-041e-11ef-bc93-c7b262605968.lock 2024-04-26T22:45:31.608 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying IP 172.21.15.27 port 3300 ... 2024-04-26T22:45:31.609 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying IP 172.21.15.27 port 6789 ... 2024-04-26T22:45:31.609 INFO:teuthology.orchestra.run.smithi027.stdout:Base mon IP(s) is [172.21.15.27:3300, 172.21.15.27:6789], mon addrv is [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-04-26T22:45:31.618 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.27 metric 100 2024-04-26T22:45:31.618 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.27 metric 100 2024-04-26T22:45:31.626 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T22:45:31.626 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-26T22:45:31.626 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-26T22:45:31.634 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T22:45:31.634 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T22:45:31.634 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T22:45:31.634 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-26T22:45:31.634 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2024-04-26T22:45:31.634 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T22:45:31.635 INFO:teuthology.orchestra.run.smithi027.stdout:Mon IP `172.21.15.27` is in CIDR network `172.21.0.0/20` 2024-04-26T22:45:31.635 INFO:teuthology.orchestra.run.smithi027.stdout:Mon IP `172.21.15.27` is in CIDR network `172.21.0.0/20` 2024-04-26T22:45:31.635 INFO:teuthology.orchestra.run.smithi027.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T22:45:31.636 INFO:teuthology.orchestra.run.smithi027.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T22:45:31.637 INFO:teuthology.orchestra.run.smithi027.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T22:45:32.316 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T22:45:32.316 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T22:45:32.316 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T22:45:32.317 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying blob sha256:f340d36a00bce3feed0ad509aeafc9092c1c625bf553b8a613c380a79149b87e 2024-04-26T22:45:32.317 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-26T22:45:32.317 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying config sha256:55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T22:45:32.317 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T22:45:33.589 INFO:teuthology.orchestra.run.smithi027.stdout:ceph: stdout ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T22:45:33.589 INFO:teuthology.orchestra.run.smithi027.stdout:Ceph version: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T22:45:33.589 INFO:teuthology.orchestra.run.smithi027.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T22:45:34.859 INFO:teuthology.orchestra.run.smithi027.stdout:stat: stdout 167 167 2024-04-26T22:45:34.860 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial keys... 2024-04-26T22:45:44.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQCTLixmTV0BARAAaeuWUqfL9DPH85b1p2i45w== 2024-04-26T22:45:45.494 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQCYLixmeZPUMxAA9+7/fJwYvM7O/kWUP1yBbA== 2024-04-26T22:45:46.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQCaLixmIpDrARAAGItt/ccSPsnUBbWD6024rw== 2024-04-26T22:45:46.741 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial monmap... 2024-04-26T22:45:47.955 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T22:45:47.956 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-26T22:45:47.956 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:45:47.956 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T22:45:47.957 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool for a [v2:172.21.15.27:3300,v1:172.21.15.27:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T22:45:47.957 INFO:teuthology.orchestra.run.smithi027.stdout:setting min_mon_release = quincy 2024-04-26T22:45:47.957 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: set fsid to 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:45:47.957 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T22:45:47.957 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:45:47.957 INFO:teuthology.orchestra.run.smithi027.stdout:Creating mon... 2024-04-26T22:45:49.983 INFO:teuthology.orchestra.run.smithi027.stdout:create mon.a on 2024-04-26T22:45:50.339 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T22:45:50.589 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T22:45:50.907 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-95ebdda6-041e-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-95ebdda6-041e-11ef-bc93-c7b262605968.target. 2024-04-26T22:45:50.907 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-95ebdda6-041e-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-95ebdda6-041e-11ef-bc93-c7b262605968.target. 2024-04-26T22:45:51.311 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mon.a 2024-04-26T22:45:51.311 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-26T22:45:51.623 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-95ebdda6-041e-11ef-bc93-c7b262605968.target.wants/ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-95ebdda6-041e-11ef-bc93-c7b262605968@.service. 2024-04-26T22:45:52.311 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:52 smithi027 podman[27643]: 2024-04-26 22:45:52.047722902 +0000 UTC m=+0.137722424 container create c82b0743f8db8a5dac7f2df8295fb6dc2ecccf4c5682f7d1f4fc7758cc0023ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-26T22:45:52.312 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:52 smithi027 podman[27643]: 2024-04-26 22:45:52.309878739 +0000 UTC m=+0.399878304 container init c82b0743f8db8a5dac7f2df8295fb6dc2ecccf4c5682f7d1f4fc7758cc0023ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True) 2024-04-26T22:45:52.434 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T22:45:52.434 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-04-26T22:45:52.451 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T22:45:52.452 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-04-26T22:45:52.452 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-04-26T22:45:52.452 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T22:45:52.453 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mon to start... 2024-04-26T22:45:52.454 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mon... 2024-04-26T22:45:52.607 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:52 smithi027 podman[27643]: 2024-04-26 22:45:52.321650232 +0000 UTC m=+0.411649757 container start c82b0743f8db8a5dac7f2df8295fb6dc2ecccf4c5682f7d1f4fc7758cc0023ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T22:45:52.607 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:52 smithi027 bash[27643]: c82b0743f8db8a5dac7f2df8295fb6dc2ecccf4c5682f7d1f4fc7758cc0023ca 2024-04-26T22:45:52.607 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:52 smithi027 systemd[1]: Started Ceph mon.a for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:45:52.607 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:52 smithi027 ceph-mon[27661]: mkfs 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:45:52.607 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:52 smithi027 ceph-mon[27661]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T22:45:53.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:53 smithi027 ceph-mon[27661]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T22:45:53.561 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:53 smithi027 ceph-mon[27661]: monmap epoch 1 2024-04-26T22:45:53.561 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:53 smithi027 ceph-mon[27661]: fsid 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:45:53.561 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:53 smithi027 ceph-mon[27661]: last_changed 2024-04-26T22:45:47.264621+0000 2024-04-26T22:45:53.561 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:53 smithi027 ceph-mon[27661]: created 2024-04-26T22:45:47.264621+0000 2024-04-26T22:45:53.561 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:53 smithi027 ceph-mon[27661]: min_mon_release 19 (squid) 2024-04-26T22:45:53.561 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:53 smithi027 ceph-mon[27661]: election_strategy: 1 2024-04-26T22:45:53.561 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:53 smithi027 ceph-mon[27661]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T22:45:53.561 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:53 smithi027 ceph-mon[27661]: fsmap 2024-04-26T22:45:53.561 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:53 smithi027 ceph-mon[27661]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T22:45:53.561 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:53 smithi027 ceph-mon[27661]: mgrmap e1: no daemons active 2024-04-26T22:45:53.561 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:53 smithi027 ceph-mon[27661]: from='client.? 172.21.15.27:0/3726185794' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-26T22:45:53.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T22:45:53.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout id: 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:45:53.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T22:45:53.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T22:45:53.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout services: 2024-04-26T22:45:53.853 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.845932s) 2024-04-26T22:45:53.853 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T22:45:53.853 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T22:45:53.857 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T22:45:53.857 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout data: 2024-04-26T22:45:53.858 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T22:45:53.858 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T22:45:53.858 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T22:45:53.858 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T22:45:53.858 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T22:45:53.858 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.151+0000 7ff0f1200700 1 Processor -- start 2024-04-26T22:45:53.858 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.152+0000 7ff0f1200700 1 -- start start 2024-04-26T22:45:53.858 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.152+0000 7ff0f1200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0ec107710 0x7ff0ec107b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:45:53.858 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.152+0000 7ff0f1200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec07d660 con 0x7ff0ec107710 2024-04-26T22:45:53.858 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.154+0000 7ff0e9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0ec107710 0x7ff0ec107b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:45:53.858 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.154+0000 7ff0e9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0ec107710 0x7ff0ec107b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40712/0 (socket says 172.21.15.27:40712) 2024-04-26T22:45:53.858 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.154+0000 7ff0e9e00700 1 -- 172.21.15.27:0/167222724 learned_addr learned my addr 172.21.15.27:0/167222724 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:45:53.859 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.154+0000 7ff0e9e00700 1 -- 172.21.15.27:0/167222724 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec108070 con 0x7ff0ec107710 2024-04-26T22:45:53.859 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.155+0000 7ff0e9e00700 1 --2- 172.21.15.27:0/167222724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0ec107710 0x7ff0ec107b30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0098a0 tx=0x7ff0dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caba3ff6e89542d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:45:53.859 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.155+0000 7ff0e9400700 1 -- 172.21.15.27:0/167222724 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0dc004160 con 0x7ff0ec107710 2024-04-26T22:45:53.859 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.155+0000 7ff0e9400700 1 -- 172.21.15.27:0/167222724 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0dc004300 con 0x7ff0ec107710 2024-04-26T22:45:53.859 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.155+0000 7ff0e9400700 1 -- 172.21.15.27:0/167222724 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0dc0036a0 con 0x7ff0ec107710 2024-04-26T22:45:53.859 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.156+0000 7ff0f1200700 1 -- 172.21.15.27:0/167222724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0ec107710 msgr2=0x7ff0ec107b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:45:53.859 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.156+0000 7ff0f1200700 1 --2- 172.21.15.27:0/167222724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0ec107710 0x7ff0ec107b30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0098a0 tx=0x7ff0dc009bb0 comp rx=0 tx=0).stop 2024-04-26T22:45:53.859 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.156+0000 7ff0f1200700 1 -- 172.21.15.27:0/167222724 shutdown_connections 2024-04-26T22:45:53.859 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.156+0000 7ff0f1200700 1 --2- 172.21.15.27:0/167222724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0ec107710 0x7ff0ec107b30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:45:53.859 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.156+0000 7ff0f1200700 1 -- 172.21.15.27:0/167222724 >> 172.21.15.27:0/167222724 conn(0x7ff0ec1032b0 msgr2=0x7ff0ec1056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:45:53.859 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.156+0000 7ff0f1200700 1 -- 172.21.15.27:0/167222724 shutdown_connections 2024-04-26T22:45:53.859 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.156+0000 7ff0f1200700 1 -- 172.21.15.27:0/167222724 wait complete. 2024-04-26T22:45:53.859 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.157+0000 7ff0f1200700 1 Processor -- start 2024-04-26T22:45:53.859 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.157+0000 7ff0f1200700 1 -- start start 2024-04-26T22:45:53.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.158+0000 7ff0f1200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0ec107710 0x7ff0ec19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:45:53.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.158+0000 7ff0f1200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec1085a0 con 0x7ff0ec107710 2024-04-26T22:45:53.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.158+0000 7ff0e9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0ec107710 0x7ff0ec19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:45:53.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.158+0000 7ff0e9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0ec107710 0x7ff0ec19e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40718/0 (socket says 172.21.15.27:40718) 2024-04-26T22:45:53.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.158+0000 7ff0e9e00700 1 -- 172.21.15.27:0/3726185794 learned_addr learned my addr 172.21.15.27:0/3726185794 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:45:53.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.159+0000 7ff0e9e00700 1 -- 172.21.15.27:0/3726185794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec19ec30 con 0x7ff0ec107710 2024-04-26T22:45:53.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.159+0000 7ff0e9e00700 1 --2- 172.21.15.27:0/3726185794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0ec107710 0x7ff0ec19e6f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009fb0 tx=0x7ff0dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:45:53.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.159+0000 7ff0e2a00700 1 -- 172.21.15.27:0/3726185794 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0dc003de0 con 0x7ff0ec107710 2024-04-26T22:45:53.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.159+0000 7ff0e2a00700 1 -- 172.21.15.27:0/3726185794 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0dc011630 con 0x7ff0ec107710 2024-04-26T22:45:53.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.159+0000 7ff0e2a00700 1 -- 172.21.15.27:0/3726185794 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0dc011930 con 0x7ff0ec107710 2024-04-26T22:45:53.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.159+0000 7ff0f1200700 1 -- 172.21.15.27:0/3726185794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ec19eec0 con 0x7ff0ec107710 2024-04-26T22:45:53.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.159+0000 7ff0f1200700 1 -- 172.21.15.27:0/3726185794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ec1a1bb0 con 0x7ff0ec107710 2024-04-26T22:45:53.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.160+0000 7ff0e2a00700 1 -- 172.21.15.27:0/3726185794 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff0dc024480 con 0x7ff0ec107710 2024-04-26T22:45:53.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.160+0000 7ff0e2a00700 1 -- 172.21.15.27:0/3726185794 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff0dc01a7e0 con 0x7ff0ec107710 2024-04-26T22:45:53.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.160+0000 7ff0f1200700 1 -- 172.21.15.27:0/3726185794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ec064a80 con 0x7ff0ec107710 2024-04-26T22:45:53.862 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.164+0000 7ff0e2a00700 1 -- 172.21.15.27:0/3726185794 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ff0dc01a9e0 con 0x7ff0ec107710 2024-04-26T22:45:53.862 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.246+0000 7ff0f1200700 1 -- 172.21.15.27:0/3726185794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7ff0ec1a20f0 con 0x7ff0ec107710 2024-04-26T22:45:53.862 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.247+0000 7ff0e2a00700 1 -- 172.21.15.27:0/3726185794 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7ff0dc01abc0 con 0x7ff0ec107710 2024-04-26T22:45:53.862 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.249+0000 7ff0f1200700 1 -- 172.21.15.27:0/3726185794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0ec107710 msgr2=0x7ff0ec19e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:45:53.862 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.249+0000 7ff0f1200700 1 --2- 172.21.15.27:0/3726185794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0ec107710 0x7ff0ec19e6f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009fb0 tx=0x7ff0dc004750 comp rx=0 tx=0).stop 2024-04-26T22:45:53.862 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.249+0000 7ff0f1200700 1 -- 172.21.15.27:0/3726185794 shutdown_connections 2024-04-26T22:45:53.862 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.249+0000 7ff0f1200700 1 --2- 172.21.15.27:0/3726185794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0ec107710 0x7ff0ec19e6f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:45:53.862 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.249+0000 7ff0f1200700 1 -- 172.21.15.27:0/3726185794 >> 172.21.15.27:0/3726185794 conn(0x7ff0ec1032b0 msgr2=0x7ff0ec194d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:45:53.863 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.249+0000 7ff0f1200700 1 -- 172.21.15.27:0/3726185794 shutdown_connections 2024-04-26T22:45:53.863 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:53.249+0000 7ff0f1200700 1 -- 172.21.15.27:0/3726185794 wait complete. 2024-04-26T22:45:53.863 INFO:teuthology.orchestra.run.smithi027.stdout:mon is available 2024-04-26T22:45:53.863 INFO:teuthology.orchestra.run.smithi027.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T22:45:55.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:55 smithi027 ceph-mon[27661]: from='client.? 172.21.15.27:0/3687744400' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T22:45:55.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:55 smithi027 ceph-mon[27661]: from='client.? 172.21.15.27:0/3687744400' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-26T22:45:56.583 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T22:45:56.589 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T22:45:56.589 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout fsid = 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:45:56.589 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T22:45:56.589 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-04-26T22:45:56.590 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T22:45:56.590 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T22:45:56.590 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T22:45:56.590 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T22:45:56.590 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T22:45:56.590 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T22:45:56.590 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T22:45:56.590 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T22:45:56.590 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T22:45:56.590 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T22:45:56.590 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T22:45:56.590 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T22:45:56.590 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T22:45:56.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T22:45:56.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.617+0000 7f241e400700 1 Processor -- start 2024-04-26T22:45:56.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.617+0000 7f241e400700 1 -- start start 2024-04-26T22:45:56.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.617+0000 7f241e400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f241810b080 0x7f241810b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:45:56.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.618+0000 7f241e400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f241810ba70 con 0x7f241810b080 2024-04-26T22:45:56.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.618+0000 7f2417400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f241810b080 0x7f241810b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:45:56.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.618+0000 7f2417400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f241810b080 0x7f241810b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40732/0 (socket says 172.21.15.27:40732) 2024-04-26T22:45:56.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.618+0000 7f2417400700 1 -- 172.21.15.27:0/1577471851 learned_addr learned my addr 172.21.15.27:0/1577471851 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:45:56.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.619+0000 7f2417400700 1 -- 172.21.15.27:0/1577471851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241810c220 con 0x7f241810b080 2024-04-26T22:45:56.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.619+0000 7f2417400700 1 --2- 172.21.15.27:0/1577471851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f241810b080 0x7f241810b4a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f24080098a0 tx=0x7f2408009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d318cfe8bbcf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:45:56.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.619+0000 7f2416000700 1 -- 172.21.15.27:0/1577471851 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2408004160 con 0x7f241810b080 2024-04-26T22:45:56.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.619+0000 7f2416000700 1 -- 172.21.15.27:0/1577471851 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2408004300 con 0x7f241810b080 2024-04-26T22:45:56.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.619+0000 7f2416000700 1 -- 172.21.15.27:0/1577471851 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24080036a0 con 0x7f241810b080 2024-04-26T22:45:56.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.620+0000 7f241e400700 1 -- 172.21.15.27:0/1577471851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f241810b080 msgr2=0x7f241810b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:45:56.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.620+0000 7f241e400700 1 --2- 172.21.15.27:0/1577471851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f241810b080 0x7f241810b4a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f24080098a0 tx=0x7f2408009bb0 comp rx=0 tx=0).stop 2024-04-26T22:45:56.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.620+0000 7f241e400700 1 -- 172.21.15.27:0/1577471851 shutdown_connections 2024-04-26T22:45:56.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.620+0000 7f241e400700 1 --2- 172.21.15.27:0/1577471851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f241810b080 0x7f241810b4a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:45:56.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.620+0000 7f241e400700 1 -- 172.21.15.27:0/1577471851 >> 172.21.15.27:0/1577471851 conn(0x7f241807e3e0 msgr2=0x7f241807e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:45:56.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.620+0000 7f241e400700 1 -- 172.21.15.27:0/1577471851 shutdown_connections 2024-04-26T22:45:56.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.620+0000 7f241e400700 1 -- 172.21.15.27:0/1577471851 wait complete. 2024-04-26T22:45:56.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.621+0000 7f241e400700 1 Processor -- start 2024-04-26T22:45:56.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.621+0000 7f241e400700 1 -- start start 2024-04-26T22:45:56.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.622+0000 7f241e400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f241810b080 0x7f241819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:45:56.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.622+0000 7f241e400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f241810c750 con 0x7f241810b080 2024-04-26T22:45:56.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.622+0000 7f2417400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f241810b080 0x7f241819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:45:56.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.622+0000 7f2417400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f241810b080 0x7f241819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40738/0 (socket says 172.21.15.27:40738) 2024-04-26T22:45:56.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.622+0000 7f2417400700 1 -- 172.21.15.27:0/3687744400 learned_addr learned my addr 172.21.15.27:0/3687744400 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:45:56.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.622+0000 7f2417400700 1 -- 172.21.15.27:0/3687744400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241819ec80 con 0x7f241810b080 2024-04-26T22:45:56.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.623+0000 7f2417400700 1 --2- 172.21.15.27:0/3687744400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f241810b080 0x7f241819e740 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2408000c00 tx=0x7f2408004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:45:56.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.623+0000 7f240fe00700 1 -- 172.21.15.27:0/3687744400 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2408003db0 con 0x7f241810b080 2024-04-26T22:45:56.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.623+0000 7f240fe00700 1 -- 172.21.15.27:0/3687744400 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2408011630 con 0x7f241810b080 2024-04-26T22:45:56.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.623+0000 7f240fe00700 1 -- 172.21.15.27:0/3687744400 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2408011950 con 0x7f241810b080 2024-04-26T22:45:56.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.623+0000 7f241e400700 1 -- 172.21.15.27:0/3687744400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241819ef10 con 0x7f241810b080 2024-04-26T22:45:56.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.623+0000 7f241e400700 1 -- 172.21.15.27:0/3687744400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24181a1c00 con 0x7f241810b080 2024-04-26T22:45:56.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.624+0000 7f240fe00700 1 -- 172.21.15.27:0/3687744400 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f240801a460 con 0x7f241810b080 2024-04-26T22:45:56.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.624+0000 7f240fe00700 1 -- 172.21.15.27:0/3687744400 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f240801b780 con 0x7f241810b080 2024-04-26T22:45:56.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.624+0000 7f241e400700 1 -- 172.21.15.27:0/3687744400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2418064a80 con 0x7f241810b080 2024-04-26T22:45:56.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.627+0000 7f240fe00700 1 -- 172.21.15.27:0/3687744400 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2408031020 con 0x7f241810b080 2024-04-26T22:45:56.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.705+0000 7f241e400700 1 -- 172.21.15.27:0/3687744400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f241802e6e0 con 0x7f241810b080 2024-04-26T22:45:56.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.715+0000 7f240fe00700 1 -- 172.21.15.27:0/3687744400 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f240801b9d0 con 0x7f241810b080 2024-04-26T22:45:56.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.716+0000 7f241e400700 1 -- 172.21.15.27:0/3687744400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f241810b080 msgr2=0x7f241819e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:45:56.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.716+0000 7f241e400700 1 --2- 172.21.15.27:0/3687744400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f241810b080 0x7f241819e740 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2408000c00 tx=0x7f2408004750 comp rx=0 tx=0).stop 2024-04-26T22:45:56.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.717+0000 7f241e400700 1 -- 172.21.15.27:0/3687744400 shutdown_connections 2024-04-26T22:45:56.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.717+0000 7f241e400700 1 --2- 172.21.15.27:0/3687744400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f241810b080 0x7f241819e740 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:45:56.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.717+0000 7f241e400700 1 -- 172.21.15.27:0/3687744400 >> 172.21.15.27:0/3687744400 conn(0x7f241807e3e0 msgr2=0x7f24181952d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:45:56.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.717+0000 7f241e400700 1 -- 172.21.15.27:0/3687744400 shutdown_connections 2024-04-26T22:45:56.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:55.717+0000 7f241e400700 1 -- 172.21.15.27:0/3687744400 wait complete. 2024-04-26T22:45:56.594 INFO:teuthology.orchestra.run.smithi027.stdout:Generating new minimal ceph.conf... 2024-04-26T22:45:57.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:57 smithi027 ceph-mon[27661]: from='client.? 172.21.15.27:0/1665615515' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:45:57.952 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.164+0000 7fcd55400700 1 Processor -- start 2024-04-26T22:45:57.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.165+0000 7fcd55400700 1 -- start start 2024-04-26T22:45:57.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.165+0000 7fcd55400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd5010b080 0x7fcd5010b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:45:57.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.165+0000 7fcd55400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5010ba70 con 0x7fcd5010b080 2024-04-26T22:45:57.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.166+0000 7fcd4e800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd5010b080 0x7fcd5010b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:45:57.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.166+0000 7fcd4e800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd5010b080 0x7fcd5010b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40744/0 (socket says 172.21.15.27:40744) 2024-04-26T22:45:57.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.166+0000 7fcd4e800700 1 -- 172.21.15.27:0/1419910093 learned_addr learned my addr 172.21.15.27:0/1419910093 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:45:57.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.166+0000 7fcd4e800700 1 -- 172.21.15.27:0/1419910093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5010c220 con 0x7fcd5010b080 2024-04-26T22:45:57.956 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.167+0000 7fcd4e800700 1 --2- 172.21.15.27:0/1419910093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd5010b080 0x7fcd5010b4a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcd400098a0 tx=0x7fcd40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99841ea2fbe1722 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:45:57.957 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.167+0000 7fcd4d400700 1 -- 172.21.15.27:0/1419910093 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd40004160 con 0x7fcd5010b080 2024-04-26T22:45:57.957 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.167+0000 7fcd4d400700 1 -- 172.21.15.27:0/1419910093 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcd40004300 con 0x7fcd5010b080 2024-04-26T22:45:57.957 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.167+0000 7fcd4d400700 1 -- 172.21.15.27:0/1419910093 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd40011af0 con 0x7fcd5010b080 2024-04-26T22:45:57.957 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.168+0000 7fcd55400700 1 -- 172.21.15.27:0/1419910093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd5010b080 msgr2=0x7fcd5010b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:45:57.957 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.168+0000 7fcd55400700 1 --2- 172.21.15.27:0/1419910093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd5010b080 0x7fcd5010b4a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcd400098a0 tx=0x7fcd40009bb0 comp rx=0 tx=0).stop 2024-04-26T22:45:57.957 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.168+0000 7fcd55400700 1 -- 172.21.15.27:0/1419910093 shutdown_connections 2024-04-26T22:45:57.957 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.168+0000 7fcd55400700 1 --2- 172.21.15.27:0/1419910093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd5010b080 0x7fcd5010b4a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:45:57.957 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.168+0000 7fcd55400700 1 -- 172.21.15.27:0/1419910093 >> 172.21.15.27:0/1419910093 conn(0x7fcd5007e3e0 msgr2=0x7fcd5007e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:45:57.957 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.168+0000 7fcd55400700 1 -- 172.21.15.27:0/1419910093 shutdown_connections 2024-04-26T22:45:57.957 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.168+0000 7fcd55400700 1 -- 172.21.15.27:0/1419910093 wait complete. 2024-04-26T22:45:57.957 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.169+0000 7fcd55400700 1 Processor -- start 2024-04-26T22:45:57.957 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.169+0000 7fcd55400700 1 -- start start 2024-04-26T22:45:57.957 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.170+0000 7fcd55400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd5010b080 0x7fcd5019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:45:57.958 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.170+0000 7fcd55400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5010f5d0 con 0x7fcd5010b080 2024-04-26T22:45:57.958 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.170+0000 7fcd4e800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd5010b080 0x7fcd5019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:45:57.958 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.170+0000 7fcd4e800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd5010b080 0x7fcd5019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40754/0 (socket says 172.21.15.27:40754) 2024-04-26T22:45:57.958 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.170+0000 7fcd4e800700 1 -- 172.21.15.27:0/1665615515 learned_addr learned my addr 172.21.15.27:0/1665615515 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:45:57.958 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.170+0000 7fcd4e800700 1 -- 172.21.15.27:0/1665615515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5019f4c0 con 0x7fcd5010b080 2024-04-26T22:45:57.958 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.171+0000 7fcd4e800700 1 --2- 172.21.15.27:0/1665615515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd5010b080 0x7fcd5019ef80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcd400117b0 tx=0x7fcd40011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:45:57.958 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.171+0000 7fcd47400700 1 -- 172.21.15.27:0/1665615515 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd40003af0 con 0x7fcd5010b080 2024-04-26T22:45:57.958 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.171+0000 7fcd47400700 1 -- 172.21.15.27:0/1665615515 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcd40020070 con 0x7fcd5010b080 2024-04-26T22:45:57.958 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.171+0000 7fcd47400700 1 -- 172.21.15.27:0/1665615515 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd40019630 con 0x7fcd5010b080 2024-04-26T22:45:57.958 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.171+0000 7fcd55400700 1 -- 172.21.15.27:0/1665615515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5019f750 con 0x7fcd5010b080 2024-04-26T22:45:57.958 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.171+0000 7fcd55400700 1 -- 172.21.15.27:0/1665615515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd501a2440 con 0x7fcd5010b080 2024-04-26T22:45:57.958 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.172+0000 7fcd47400700 1 -- 172.21.15.27:0/1665615515 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcd40019c30 con 0x7fcd5010b080 2024-04-26T22:45:57.958 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.172+0000 7fcd47400700 1 -- 172.21.15.27:0/1665615515 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcd4000b720 con 0x7fcd5010b080 2024-04-26T22:45:57.958 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.173+0000 7fcd55400700 1 -- 172.21.15.27:0/1665615515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd50064a80 con 0x7fcd5010b080 2024-04-26T22:45:57.958 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.176+0000 7fcd47400700 1 -- 172.21.15.27:0/1665615515 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fcd4001a9c0 con 0x7fcd5010b080 2024-04-26T22:45:57.959 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.255+0000 7fcd55400700 1 -- 172.21.15.27:0/1665615515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fcd501a2990 con 0x7fcd5010b080 2024-04-26T22:45:57.959 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.256+0000 7fcd47400700 1 -- 172.21.15.27:0/1665615515 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fcd4001a430 con 0x7fcd5010b080 2024-04-26T22:45:57.959 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.257+0000 7fcd55400700 1 -- 172.21.15.27:0/1665615515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd5010b080 msgr2=0x7fcd5019ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:45:57.959 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.257+0000 7fcd55400700 1 --2- 172.21.15.27:0/1665615515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd5010b080 0x7fcd5019ef80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcd400117b0 tx=0x7fcd40011b70 comp rx=0 tx=0).stop 2024-04-26T22:45:57.959 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.257+0000 7fcd55400700 1 -- 172.21.15.27:0/1665615515 shutdown_connections 2024-04-26T22:45:57.960 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.257+0000 7fcd55400700 1 --2- 172.21.15.27:0/1665615515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd5010b080 0x7fcd5019ef80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:45:57.960 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.257+0000 7fcd55400700 1 -- 172.21.15.27:0/1665615515 >> 172.21.15.27:0/1665615515 conn(0x7fcd5007e3e0 msgr2=0x7fcd50108dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:45:57.960 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.258+0000 7fcd55400700 1 -- 172.21.15.27:0/1665615515 shutdown_connections 2024-04-26T22:45:57.960 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:45:57.258+0000 7fcd55400700 1 -- 172.21.15.27:0/1665615515 wait complete. 2024-04-26T22:45:57.960 INFO:teuthology.orchestra.run.smithi027.stdout:Restarting the monitor... 2024-04-26T22:45:58.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:58 smithi027 systemd[1]: Stopping Ceph mon.a for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T22:45:58.767 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:58 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a[27657]: 2024-04-26T22:45:58.469+0000 7fc113e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:45:58.767 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:58 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a[27657]: 2024-04-26T22:45:58.469+0000 7fc113e00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T22:45:58.768 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:58 smithi027 podman[27902]: 2024-04-26 22:45:58.490797355 +0000 UTC m=+0.154940966 container died c82b0743f8db8a5dac7f2df8295fb6dc2ecccf4c5682f7d1f4fc7758cc0023ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T22:45:59.037 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:58 smithi027 podman[27902]: 2024-04-26 22:45:58.861700281 +0000 UTC m=+0.525843921 container cleanup c82b0743f8db8a5dac7f2df8295fb6dc2ecccf4c5682f7d1f4fc7758cc0023ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T22:45:59.037 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:58 smithi027 bash[27902]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a 2024-04-26T22:45:59.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:59 smithi027 podman[27916]: 2024-04-26 22:45:59.037278997 +0000 UTC m=+0.530328243 container remove c82b0743f8db8a5dac7f2df8295fb6dc2ecccf4c5682f7d1f4fc7758cc0023ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T22:45:59.828 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:59 smithi027 systemd[1]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T22:45:59.828 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:59 smithi027 systemd[1]: Stopped Ceph mon.a for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:45:59.828 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:59 smithi027 systemd[1]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mon.a.service: Consumed 2.035s CPU time. 2024-04-26T22:45:59.828 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:59 smithi027 systemd[1]: Starting Ceph mon.a for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T22:46:00.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:45:59 smithi027 podman[28006]: 2024-04-26 22:45:59.888804621 +0000 UTC m=+0.039920932 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:46:00.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:00 smithi027 podman[28006]: 2024-04-26 22:46:00.012429319 +0000 UTC m=+0.163545627 container create a43cb86680a7d01387b3d36eef60300d0e3d9f320cd25e47a15be9876a9c39a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T22:46:01.052 INFO:teuthology.orchestra.run.smithi027.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T22:46:01.204 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:00 smithi027 podman[28006]: 2024-04-26 22:46:00.951050727 +0000 UTC m=+1.102167113 container init a43cb86680a7d01387b3d36eef60300d0e3d9f320cd25e47a15be9876a9c39a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-04-26T22:46:01.204 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:00 smithi027 podman[28006]: 2024-04-26 22:46:00.962934526 +0000 UTC m=+1.114050878 container start a43cb86680a7d01387b3d36eef60300d0e3d9f320cd25e47a15be9876a9c39a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T22:46:01.204 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T22:46:01.204 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T22:46:01.204 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: pidfile_write: ignore empty --pid-file 2024-04-26T22:46:01.204 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: load: jerasure load: lrc 2024-04-26T22:46:01.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: RocksDB version: 7.9.2 2024-04-26T22:46:01.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Git sha 0 2024-04-26T22:46:01.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T22:46:01.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: DB SUMMARY 2024-04-26T22:46:01.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: DB Session ID: 2KVEEDE1XZ8RTSQVG982 2024-04-26T22:46:01.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: CURRENT file: CURRENT 2024-04-26T22:46:01.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T22:46:01.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-26T22:46:01.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T22:46:01.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105098 ; 2024-04-26T22:46:01.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.error_if_exists: 0 2024-04-26T22:46:01.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.create_if_missing: 0 2024-04-26T22:46:01.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.paranoid_checks: 1 2024-04-26T22:46:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T22:46:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T22:46:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T22:46:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.env: 0x56035f0bdcc0 2024-04-26T22:46:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T22:46:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.info_log: 0x560380503dc0 2024-04-26T22:46:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T22:46:01.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.statistics: (nil) 2024-04-26T22:46:01.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.use_fsync: 0 2024-04-26T22:46:01.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_log_file_size: 0 2024-04-26T22:46:01.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T22:46:01.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T22:46:01.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T22:46:01.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T22:46:01.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.allow_fallocate: 1 2024-04-26T22:46:01.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T22:46:01.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T22:46:01.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.use_direct_reads: 0 2024-04-26T22:46:01.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T22:46:01.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T22:46:01.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.db_log_dir: 2024-04-26T22:46:01.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.wal_dir: 2024-04-26T22:46:01.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T22:46:01.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T22:46:01.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T22:46:01.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T22:46:01.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T22:46:01.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T22:46:01.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T22:46:01.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T22:46:01.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.write_buffer_manager: 0x5603804d50e0 2024-04-26T22:46:01.209 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T22:46:01.209 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T22:46:01.209 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T22:46:01.209 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T22:46:01.209 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T22:46:01.209 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T22:46:01.209 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T22:46:01.209 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T22:46:01.209 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.unordered_write: 0 2024-04-26T22:46:01.209 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T22:46:01.209 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T22:46:01.209 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T22:46:01.209 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T22:46:01.210 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.row_cache: None 2024-04-26T22:46:01.210 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.wal_filter: None 2024-04-26T22:46:01.210 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T22:46:01.210 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T22:46:01.210 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.two_write_queues: 0 2024-04-26T22:46:01.210 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T22:46:01.210 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.wal_compression: 0 2024-04-26T22:46:01.210 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.atomic_flush: 0 2024-04-26T22:46:01.210 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T22:46:01.210 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T22:46:01.210 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T22:46:01.210 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.log_readahead_size: 0 2024-04-26T22:46:01.210 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T22:46:01.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T22:46:01.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T22:46:01.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T22:46:01.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T22:46:01.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T22:46:01.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T22:46:01.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_background_jobs: 2 2024-04-26T22:46:01.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_background_compactions: -1 2024-04-26T22:46:01.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_subcompactions: 1 2024-04-26T22:46:01.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T22:46:01.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T22:46:01.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T22:46:01.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T22:46:01.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T22:46:01.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T22:46:01.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T22:46:01.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T22:46:01.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_open_files: -1 2024-04-26T22:46:01.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T22:46:01.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T22:46:01.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T22:46:01.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T22:46:01.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_background_flushes: -1 2024-04-26T22:46:01.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Compression algorithms supported: 2024-04-26T22:46:01.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T22:46:01.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: kZSTD supported: 0 2024-04-26T22:46:01.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: kXpressCompression supported: 0 2024-04-26T22:46:01.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T22:46:01.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: kZlibCompression supported: 1 2024-04-26T22:46:01.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: kSnappyCompression supported: 1 2024-04-26T22:46:01.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: kLZ4Compression supported: 1 2024-04-26T22:46:01.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: kBZip2Compression supported: 0 2024-04-26T22:46:01.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T22:46:01.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T22:46:01.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-26T22:46:01.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T22:46:01.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T22:46:01.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.merge_operator: 2024-04-26T22:46:01.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compaction_filter: None 2024-04-26T22:46:01.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T22:46:01.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T22:46:01.214 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T22:46:01.214 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T22:46:01.214 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560380503f80) 2024-04-26T22:46:01.214 INFO:journalctl@ceph.mon.a.smithi027.stdout: cache_index_and_filter_blocks: 1 2024-04-26T22:46:01.214 INFO:journalctl@ceph.mon.a.smithi027.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T22:46:01.214 INFO:journalctl@ceph.mon.a.smithi027.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T22:46:01.214 INFO:journalctl@ceph.mon.a.smithi027.stdout: pin_top_level_index_and_filter: 1 2024-04-26T22:46:01.214 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_type: 0 2024-04-26T22:46:01.214 INFO:journalctl@ceph.mon.a.smithi027.stdout: data_block_index_type: 0 2024-04-26T22:46:01.214 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_shortening: 1 2024-04-26T22:46:01.214 INFO:journalctl@ceph.mon.a.smithi027.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T22:46:01.214 INFO:journalctl@ceph.mon.a.smithi027.stdout: checksum: 4 2024-04-26T22:46:01.214 INFO:journalctl@ceph.mon.a.smithi027.stdout: no_block_cache: 0 2024-04-26T22:46:01.214 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache: 0x5603805071f0 2024-04-26T22:46:01.215 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_name: BinnedLRUCache 2024-04-26T22:46:01.215 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_options: 2024-04-26T22:46:01.215 INFO:journalctl@ceph.mon.a.smithi027.stdout: capacity : 536870912 2024-04-26T22:46:01.215 INFO:journalctl@ceph.mon.a.smithi027.stdout: num_shard_bits : 4 2024-04-26T22:46:01.215 INFO:journalctl@ceph.mon.a.smithi027.stdout: strict_capacity_limit : 0 2024-04-26T22:46:01.215 INFO:journalctl@ceph.mon.a.smithi027.stdout: high_pri_pool_ratio: 0.000 2024-04-26T22:46:01.215 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_compressed: (nil) 2024-04-26T22:46:01.215 INFO:journalctl@ceph.mon.a.smithi027.stdout: persistent_cache: (nil) 2024-04-26T22:46:01.215 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_size: 4096 2024-04-26T22:46:01.215 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_size_deviation: 10 2024-04-26T22:46:01.215 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_restart_interval: 16 2024-04-26T22:46:01.215 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_block_restart_interval: 1 2024-04-26T22:46:01.215 INFO:journalctl@ceph.mon.a.smithi027.stdout: metadata_block_size: 4096 2024-04-26T22:46:01.215 INFO:journalctl@ceph.mon.a.smithi027.stdout: partition_filters: 0 2024-04-26T22:46:01.215 INFO:journalctl@ceph.mon.a.smithi027.stdout: use_delta_encoding: 1 2024-04-26T22:46:01.216 INFO:journalctl@ceph.mon.a.smithi027.stdout: filter_policy: bloomfilter 2024-04-26T22:46:01.216 INFO:journalctl@ceph.mon.a.smithi027.stdout: whole_key_filtering: 1 2024-04-26T22:46:01.216 INFO:journalctl@ceph.mon.a.smithi027.stdout: verify_compression: 0 2024-04-26T22:46:01.216 INFO:journalctl@ceph.mon.a.smithi027.stdout: read_amp_bytes_per_bit: 0 2024-04-26T22:46:01.216 INFO:journalctl@ceph.mon.a.smithi027.stdout: format_version: 5 2024-04-26T22:46:01.216 INFO:journalctl@ceph.mon.a.smithi027.stdout: enable_index_compression: 1 2024-04-26T22:46:01.216 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_align: 0 2024-04-26T22:46:01.216 INFO:journalctl@ceph.mon.a.smithi027.stdout: max_auto_readahead_size: 262144 2024-04-26T22:46:01.216 INFO:journalctl@ceph.mon.a.smithi027.stdout: prepopulate_block_cache: 0 2024-04-26T22:46:01.216 INFO:journalctl@ceph.mon.a.smithi027.stdout: initial_auto_readahead_size: 8192 2024-04-26T22:46:01.216 INFO:journalctl@ceph.mon.a.smithi027.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T22:46:01.217 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T22:46:01.217 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T22:46:01.217 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compression: NoCompression 2024-04-26T22:46:01.218 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T22:46:01.218 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T22:46:01.218 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T22:46:01.218 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.num_levels: 7 2024-04-26T22:46:01.218 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T22:46:01.218 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T22:46:01.218 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T22:46:01.218 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T22:46:01.218 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T22:46:01.218 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T22:46:01.218 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T22:46:01.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T22:46:01.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T22:46:01.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T22:46:01.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T22:46:01.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T22:46:01.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T22:46:01.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T22:46:01.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T22:46:01.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T22:46:01.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T22:46:01.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T22:46:01.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T22:46:01.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T22:46:01.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T22:46:01.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T22:46:01.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T22:46:01.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T22:46:01.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T22:46:01.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T22:46:01.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T22:46:01.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T22:46:01.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T22:46:01.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T22:46:01.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T22:46:01.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T22:46:01.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T22:46:01.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T22:46:01.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T22:46:01.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T22:46:01.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T22:46:01.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T22:46:01.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T22:46:01.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T22:46:01.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T22:46:01.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T22:46:01.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T22:46:01.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T22:46:01.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T22:46:01.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T22:46:01.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T22:46:01.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T22:46:01.222 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T22:46:01.222 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T22:46:01.222 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T22:46:01.222 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T22:46:01.222 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T22:46:01.222 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T22:46:01.222 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.inplace_update_support: 0 2024-04-26T22:46:01.222 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T22:46:01.222 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T22:46:01.222 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T22:46:01.222 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T22:46:01.222 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.bloom_locality: 0 2024-04-26T22:46:01.222 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.max_successive_merges: 0 2024-04-26T22:46:01.222 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T22:46:01.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T22:46:01.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T22:46:01.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T22:46:01.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.ttl: 2592000 2024-04-26T22:46:01.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T22:46:01.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T22:46:01.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T22:46:01.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.enable_blob_files: false 2024-04-26T22:46:01.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.min_blob_size: 0 2024-04-26T22:46:01.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T22:46:01.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T22:46:01.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T22:46:01.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T22:46:01.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T22:46:01.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T22:46:01.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T22:46:01.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T22:46:01.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-26T22:46:01.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-26T22:46:01.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 049f4e9a-052b-42cc-a829-14ab0e3cae8c 2024-04-26T22:46:01.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714171561023088, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-26T22:46:01.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-26T22:46:01.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714171561024991, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100257, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98142, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14161, "raw_average_key_size": 49, "raw_value_size": 90442, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714171561, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "049f4e9a-052b-42cc-a829-14ab0e3cae8c", "db_session_id": "2KVEEDE1XZ8RTSQVG982", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-26T22:46:01.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714171561025189, "job": 1, "event": "recovery_finished"} 2024-04-26T22:46:01.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-26T22:46:01.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T22:46:01.225 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560380624000 2024-04-26T22:46:01.225 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: DB pointer 0x560380610000 2024-04-26T22:46:01.225 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T22:46:01.225 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T22:46:01.225 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** DB Stats ** 2024-04-26T22:46:01.225 INFO:journalctl@ceph.mon.a.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T22:46:01.225 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T22:46:01.225 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T22:46:01.225 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T22:46:01.226 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T22:46:01.226 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T22:46:01.226 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T22:46:01.226 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-04-26T22:46:01.226 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** Compaction Stats [default] ** 2024-04-26T22:46:01.226 INFO:journalctl@ceph.mon.a.smithi027.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T22:46:01.226 INFO:journalctl@ceph.mon.a.smithi027.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T22:46:01.227 INFO:journalctl@ceph.mon.a.smithi027.stdout: L0 2/0 99.77 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 90.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T22:46:01.227 INFO:journalctl@ceph.mon.a.smithi027.stdout: Sum 2/0 99.77 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 90.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T22:46:01.227 INFO:journalctl@ceph.mon.a.smithi027.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 90.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T22:46:01.227 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-04-26T22:46:01.227 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** Compaction Stats [default] ** 2024-04-26T22:46:01.227 INFO:journalctl@ceph.mon.a.smithi027.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T22:46:01.227 INFO:journalctl@ceph.mon.a.smithi027.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T22:46:01.227 INFO:journalctl@ceph.mon.a.smithi027.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 90.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T22:46:01.227 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-04-26T22:46:01.227 INFO:journalctl@ceph.mon.a.smithi027.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T22:46:01.227 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-04-26T22:46:01.228 INFO:journalctl@ceph.mon.a.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T22:46:01.228 INFO:journalctl@ceph.mon.a.smithi027.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T22:46:01.228 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T22:46:01.228 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T22:46:01.228 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T22:46:01.228 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T22:46:01.228 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative compaction: 0.00 GB write, 11.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T22:46:01.228 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval compaction: 0.00 GB write, 11.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T22:46:01.228 INFO:journalctl@ceph.mon.a.smithi027.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T22:46:01.228 INFO:journalctl@ceph.mon.a.smithi027.stdout: Block cache BinnedLRUCache@0x5603805071f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-26T22:46:01.228 INFO:journalctl@ceph.mon.a.smithi027.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-26T22:46:01.228 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-04-26T22:46:01.228 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T22:46:01.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: starting mon.a rank 0 at public addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] at bind addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:46:01.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: mon.a@-1(???) e1 preinit fsid 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:46:01.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: mon.a@-1(???).mds e1 new map 2024-04-26T22:46:01.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: mon.a@-1(???).mds e1 print_map 2024-04-26T22:46:01.229 INFO:journalctl@ceph.mon.a.smithi027.stdout: e1 2024-04-26T22:46:01.229 INFO:journalctl@ceph.mon.a.smithi027.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T22:46:01.229 INFO:journalctl@ceph.mon.a.smithi027.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T22:46:01.229 INFO:journalctl@ceph.mon.a.smithi027.stdout: legacy client fscid: -1 2024-04-26T22:46:01.229 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-04-26T22:46:01.229 INFO:journalctl@ceph.mon.a.smithi027.stdout: No filesystems configured 2024-04-26T22:46:01.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T22:46:01.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T22:46:01.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T22:46:01.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T22:46:01.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T22:46:01.230 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 bash[28006]: a43cb86680a7d01387b3d36eef60300d0e3d9f320cd25e47a15be9876a9c39a3 2024-04-26T22:46:01.230 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 systemd[1]: Started Ceph mon.a for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:46:01.230 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T22:46:01.230 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: monmap epoch 1 2024-04-26T22:46:01.230 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: fsid 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:46:01.230 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: last_changed 2024-04-26T22:45:47.264621+0000 2024-04-26T22:46:01.230 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: created 2024-04-26T22:45:47.264621+0000 2024-04-26T22:46:01.230 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: min_mon_release 19 (squid) 2024-04-26T22:46:01.230 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: election_strategy: 1 2024-04-26T22:46:01.230 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T22:46:01.230 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: fsmap 2024-04-26T22:46:01.230 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T22:46:01.230 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:01 smithi027 ceph-mon[28034]: mgrmap e1: no daemons active 2024-04-26T22:46:02.611 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.754+0000 7f4a50200700 1 Processor -- start 2024-04-26T22:46:02.611 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.755+0000 7f4a50200700 1 -- start start 2024-04-26T22:46:02.611 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.755+0000 7f4a50200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a4807f730 0x7f4a4807db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:02.611 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.755+0000 7f4a50200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4807e160 con 0x7f4a4807f730 2024-04-26T22:46:02.611 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.756+0000 7f4a4d800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a4807f730 0x7f4a4807db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:02.611 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.756+0000 7f4a4d800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a4807f730 0x7f4a4807db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33528/0 (socket says 172.21.15.27:33528) 2024-04-26T22:46:02.611 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.756+0000 7f4a4d800700 1 -- 172.21.15.27:0/2420095597 learned_addr learned my addr 172.21.15.27:0/2420095597 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:02.611 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.757+0000 7f4a4d800700 1 -- 172.21.15.27:0/2420095597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4807e2e0 con 0x7f4a4807f730 2024-04-26T22:46:02.611 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.757+0000 7f4a4d800700 1 --2- 172.21.15.27:0/2420095597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a4807f730 0x7f4a4807db90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0098a0 tx=0x7f4a3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b55248b78505083 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:02.611 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.757+0000 7f4a47e00700 1 -- 172.21.15.27:0/2420095597 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a3c004160 con 0x7f4a4807f730 2024-04-26T22:46:02.611 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.758+0000 7f4a47e00700 1 -- 172.21.15.27:0/2420095597 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4a3c004300 con 0x7f4a4807f730 2024-04-26T22:46:02.611 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.758+0000 7f4a47e00700 1 -- 172.21.15.27:0/2420095597 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a3c011af0 con 0x7f4a4807f730 2024-04-26T22:46:02.612 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.758+0000 7f4a50200700 1 -- 172.21.15.27:0/2420095597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a4807f730 msgr2=0x7f4a4807db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:02.612 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.758+0000 7f4a50200700 1 --2- 172.21.15.27:0/2420095597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a4807f730 0x7f4a4807db90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0098a0 tx=0x7f4a3c009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:02.612 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.758+0000 7f4a50200700 1 -- 172.21.15.27:0/2420095597 shutdown_connections 2024-04-26T22:46:02.612 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.758+0000 7f4a50200700 1 --2- 172.21.15.27:0/2420095597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a4807f730 0x7f4a4807db90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:02.612 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.758+0000 7f4a50200700 1 -- 172.21.15.27:0/2420095597 >> 172.21.15.27:0/2420095597 conn(0x7f4a48104460 msgr2=0x7f4a48106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:02.612 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.758+0000 7f4a50200700 1 -- 172.21.15.27:0/2420095597 shutdown_connections 2024-04-26T22:46:02.612 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.759+0000 7f4a50200700 1 -- 172.21.15.27:0/2420095597 wait complete. 2024-04-26T22:46:02.612 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.760+0000 7f4a50200700 1 Processor -- start 2024-04-26T22:46:02.612 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.760+0000 7f4a50200700 1 -- start start 2024-04-26T22:46:02.612 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.760+0000 7f4a50200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a4807f730 0x7f4a4819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:02.612 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.760+0000 7f4a50200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4810f3c0 con 0x7f4a4807f730 2024-04-26T22:46:02.612 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.760+0000 7f4a4d800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a4807f730 0x7f4a4819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:02.612 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.760+0000 7f4a4d800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a4807f730 0x7f4a4819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33538/0 (socket says 172.21.15.27:33538) 2024-04-26T22:46:02.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.760+0000 7f4a4d800700 1 -- 172.21.15.27:0/3823238929 learned_addr learned my addr 172.21.15.27:0/3823238929 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:02.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.761+0000 7f4a4d800700 1 -- 172.21.15.27:0/3823238929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4819f480 con 0x7f4a4807f730 2024-04-26T22:46:02.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.761+0000 7f4a4d800700 1 --2- 172.21.15.27:0/3823238929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a4807f730 0x7f4a4819ef40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c000c00 tx=0x7f4a3c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:02.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.761+0000 7f4a46000700 1 -- 172.21.15.27:0/3823238929 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a3c003b20 con 0x7f4a4807f730 2024-04-26T22:46:02.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.761+0000 7f4a50200700 1 -- 172.21.15.27:0/3823238929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4819f710 con 0x7f4a4807f730 2024-04-26T22:46:02.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.762+0000 7f4a50200700 1 -- 172.21.15.27:0/3823238929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a481a2400 con 0x7f4a4807f730 2024-04-26T22:46:02.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.762+0000 7f4a46000700 1 -- 172.21.15.27:0/3823238929 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4a3c0042a0 con 0x7f4a4807f730 2024-04-26T22:46:02.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.762+0000 7f4a46000700 1 -- 172.21.15.27:0/3823238929 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a3c0245a0 con 0x7f4a4807f730 2024-04-26T22:46:02.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.762+0000 7f4a46000700 1 -- 172.21.15.27:0/3823238929 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4a3c022020 con 0x7f4a4807f730 2024-04-26T22:46:02.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.762+0000 7f4a46000700 1 -- 172.21.15.27:0/3823238929 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4a3c0193c0 con 0x7f4a4807f730 2024-04-26T22:46:02.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.763+0000 7f4a50200700 1 -- 172.21.15.27:0/3823238929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a28005360 con 0x7f4a4807f730 2024-04-26T22:46:02.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.768+0000 7f4a46000700 1 -- 172.21.15.27:0/3823238929 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f4a3c02dde0 con 0x7f4a4807f730 2024-04-26T22:46:02.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.876+0000 7f4a50200700 1 -- 172.21.15.27:0/3823238929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f4a28005650 con 0x7f4a4807f730 2024-04-26T22:46:02.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.879+0000 7f4a46000700 1 -- 172.21.15.27:0/3823238929 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f4a3c032030 con 0x7f4a4807f730 2024-04-26T22:46:02.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.880+0000 7f4a50200700 1 -- 172.21.15.27:0/3823238929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a4807f730 msgr2=0x7f4a4819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:02.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.880+0000 7f4a50200700 1 --2- 172.21.15.27:0/3823238929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a4807f730 0x7f4a4819ef40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c000c00 tx=0x7f4a3c011b70 comp rx=0 tx=0).stop 2024-04-26T22:46:02.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.880+0000 7f4a50200700 1 -- 172.21.15.27:0/3823238929 shutdown_connections 2024-04-26T22:46:02.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.880+0000 7f4a50200700 1 --2- 172.21.15.27:0/3823238929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a4807f730 0x7f4a4819ef40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:02.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.880+0000 7f4a50200700 1 -- 172.21.15.27:0/3823238929 >> 172.21.15.27:0/3823238929 conn(0x7f4a48104460 msgr2=0x7f4a48104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:02.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.880+0000 7f4a50200700 1 -- 172.21.15.27:0/3823238929 shutdown_connections 2024-04-26T22:46:02.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:01.881+0000 7f4a50200700 1 -- 172.21.15.27:0/3823238929 wait complete. 2024-04-26T22:46:02.614 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T22:46:02.630 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:46:02.630 INFO:teuthology.orchestra.run.smithi027.stdout:Creating mgr... 2024-04-26T22:46:02.631 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-26T22:46:02.631 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-26T22:46:03.055 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mgr.x 2024-04-26T22:46:03.056 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-26T22:46:03.317 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-95ebdda6-041e-11ef-bc93-c7b262605968.target.wants/ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-95ebdda6-041e-11ef-bc93-c7b262605968@.service. 2024-04-26T22:46:03.362 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:02 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3823238929' entity='client.admin' 2024-04-26T22:46:03.615 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:03 smithi027 systemd[1]: Starting Ceph mgr.x for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T22:46:04.021 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:03 smithi027 podman[28248]: 2024-04-26 22:46:03.654135428 +0000 UTC m=+0.036887187 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:46:04.022 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:03 smithi027 podman[28248]: 2024-04-26 22:46:03.78341413 +0000 UTC m=+0.166165928 container create a588d685001bb42ecf77c547ae172f4f9a8147f7dfb7cee8fe7317cf0b7f20a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T22:46:04.022 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:04 smithi027 podman[28248]: 2024-04-26 22:46:04.020318488 +0000 UTC m=+0.403070306 container init a588d685001bb42ecf77c547ae172f4f9a8147f7dfb7cee8fe7317cf0b7f20a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True) 2024-04-26T22:46:04.214 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T22:46:04.214 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-04-26T22:46:04.235 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T22:46:04.235 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-04-26T22:46:04.235 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-04-26T22:46:04.235 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T22:46:04.261 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T22:46:04.262 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-04-26T22:46:04.274 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:04 smithi027 podman[28248]: 2024-04-26 22:46:04.030990864 +0000 UTC m=+0.413742672 container start a588d685001bb42ecf77c547ae172f4f9a8147f7dfb7cee8fe7317cf0b7f20a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T22:46:04.274 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:04 smithi027 bash[28248]: a588d685001bb42ecf77c547ae172f4f9a8147f7dfb7cee8fe7317cf0b7f20a2 2024-04-26T22:46:04.274 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:04 smithi027 systemd[1]: Started Ceph mgr.x for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:46:04.282 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T22:46:04.282 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-04-26T22:46:04.283 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-04-26T22:46:04.283 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-26T22:46:04.283 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr to start... 2024-04-26T22:46:04.283 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr... 2024-04-26T22:46:04.547 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:04 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:04.272+0000 7f255b672200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T22:46:04.813 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:04 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:04.546+0000 7f255b672200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:46:04.814 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:04 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:04.686+0000 7f255b672200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T22:46:05.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:05 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2278887198' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T22:46:05.615 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:05 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:05.113+0000 7f255b672200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T22:46:05.615 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:05 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:05.231+0000 7f255b672200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T22:46:05.661 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T22:46:05.661 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-04-26T22:46:05.661 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "95ebdda6-041e-11ef-bc93-c7b262605968", 2024-04-26T22:46:05.661 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T22:46:05.661 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T22:46:05.664 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T22:46:05.664 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T22:46:05.664 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:05.664 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T22:46:05.664 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T22:46:05.664 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-04-26T22:46:05.664 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T22:46:05.664 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T22:46:05.664 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T22:46:05.664 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T22:46:05.664 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-26T22:46:05.665 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T22:46:05.665 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T22:46:05.665 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T22:46:05.665 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T22:46:05.665 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:05.665 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T22:46:05.665 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T22:46:05.665 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T22:46:05.665 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T22:46:05.665 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T22:46:05.675 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T22:46:05.675 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T22:46:05.675 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T22:46:05.675 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:05.675 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T22:46:05.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T22:46:05.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T22:46:05.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T22:46:05.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T22:46:05.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T22:46:05.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T22:46:05.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T22:46:05.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T22:46:05.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:05.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T22:46:05.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T22:46:05.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T22:46:05.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T22:46:05.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:05.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T22:46:05.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T22:46:05.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T22:46:05.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T22:46:05.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T22:46:05.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T22:46:05.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T22:46:05.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T22:46:05.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T22:46:05.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:05.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T22:46:05.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T22:46:05.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T22:45:52.404809+0000", 2024-04-26T22:46:05.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T22:46:05.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:05.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T22:46:05.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-04-26T22:46:05.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.864+0000 7f75b1c00700 1 Processor -- start 2024-04-26T22:46:05.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.864+0000 7f75b1c00700 1 -- start start 2024-04-26T22:46:05.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.864+0000 7f75b1c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac109920 0x7f75ac10bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:05.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.864+0000 7f75b1c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac07cdf0 con 0x7f75ac109920 2024-04-26T22:46:05.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.865+0000 7f75aaa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac109920 0x7f75ac10bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:05.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.865+0000 7f75aaa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac109920 0x7f75ac10bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33558/0 (socket says 172.21.15.27:33558) 2024-04-26T22:46:05.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.865+0000 7f75aaa00700 1 -- 172.21.15.27:0/3277314924 learned_addr learned my addr 172.21.15.27:0/3277314924 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:05.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.866+0000 7f75aaa00700 1 -- 172.21.15.27:0/3277314924 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ac10c250 con 0x7f75ac109920 2024-04-26T22:46:05.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.866+0000 7f75aaa00700 1 --2- 172.21.15.27:0/3277314924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac109920 0x7f75ac10bd10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f759c0098a0 tx=0x7f759c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82359c89129babb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:05.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.867+0000 7f75a9600700 1 -- 172.21.15.27:0/3277314924 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f759c004160 con 0x7f75ac109920 2024-04-26T22:46:05.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.867+0000 7f75a9600700 1 -- 172.21.15.27:0/3277314924 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f759c004300 con 0x7f75ac109920 2024-04-26T22:46:05.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.867+0000 7f75a9600700 1 -- 172.21.15.27:0/3277314924 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f759c011af0 con 0x7f75ac109920 2024-04-26T22:46:05.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.867+0000 7f75b1c00700 1 -- 172.21.15.27:0/3277314924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac109920 msgr2=0x7f75ac10bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:05.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.867+0000 7f75b1c00700 1 --2- 172.21.15.27:0/3277314924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac109920 0x7f75ac10bd10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f759c0098a0 tx=0x7f759c009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:05.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.868+0000 7f75b1c00700 1 -- 172.21.15.27:0/3277314924 shutdown_connections 2024-04-26T22:46:05.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.868+0000 7f75b1c00700 1 --2- 172.21.15.27:0/3277314924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac109920 0x7f75ac10bd10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:05.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.868+0000 7f75b1c00700 1 -- 172.21.15.27:0/3277314924 >> 172.21.15.27:0/3277314924 conn(0x7f75ac103290 msgr2=0x7f75ac105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:05.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.868+0000 7f75b1c00700 1 -- 172.21.15.27:0/3277314924 shutdown_connections 2024-04-26T22:46:05.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.868+0000 7f75b1c00700 1 -- 172.21.15.27:0/3277314924 wait complete. 2024-04-26T22:46:05.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.869+0000 7f75b1c00700 1 Processor -- start 2024-04-26T22:46:05.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.869+0000 7f75b1c00700 1 -- start start 2024-04-26T22:46:05.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.870+0000 7f75b1c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac109920 0x7f75ac19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:05.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.870+0000 7f75b1c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac10d1b0 con 0x7f75ac109920 2024-04-26T22:46:05.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.870+0000 7f75aaa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac109920 0x7f75ac19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:05.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.870+0000 7f75aaa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac109920 0x7f75ac19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33574/0 (socket says 172.21.15.27:33574) 2024-04-26T22:46:05.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.870+0000 7f75aaa00700 1 -- 172.21.15.27:0/2278887198 learned_addr learned my addr 172.21.15.27:0/2278887198 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:05.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.871+0000 7f75aaa00700 1 -- 172.21.15.27:0/2278887198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ac19f510 con 0x7f75ac109920 2024-04-26T22:46:05.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.871+0000 7f75aaa00700 1 --2- 172.21.15.27:0/2278887198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac109920 0x7f75ac19efd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f759c011ac0 tx=0x7f759c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:05.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.871+0000 7f75a3400700 1 -- 172.21.15.27:0/2278887198 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f759c003970 con 0x7f75ac109920 2024-04-26T22:46:05.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.871+0000 7f75a3400700 1 -- 172.21.15.27:0/2278887198 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f759c003b10 con 0x7f75ac109920 2024-04-26T22:46:05.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.871+0000 7f75a3400700 1 -- 172.21.15.27:0/2278887198 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f759c024610 con 0x7f75ac109920 2024-04-26T22:46:05.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.871+0000 7f75b1c00700 1 -- 172.21.15.27:0/2278887198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ac19f7a0 con 0x7f75ac109920 2024-04-26T22:46:05.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.871+0000 7f75b1c00700 1 -- 172.21.15.27:0/2278887198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ac1a2490 con 0x7f75ac109920 2024-04-26T22:46:05.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.872+0000 7f75a3400700 1 -- 172.21.15.27:0/2278887198 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f759c024c20 con 0x7f75ac109920 2024-04-26T22:46:05.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.872+0000 7f75a3400700 1 -- 172.21.15.27:0/2278887198 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f759c019700 con 0x7f75ac109920 2024-04-26T22:46:05.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.872+0000 7f75b1c00700 1 -- 172.21.15.27:0/2278887198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75ac064a80 con 0x7f75ac109920 2024-04-26T22:46:05.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.875+0000 7f75a3400700 1 -- 172.21.15.27:0/2278887198 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f759c01a9e0 con 0x7f75ac109920 2024-04-26T22:46:05.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.955+0000 7f75b1c00700 1 -- 172.21.15.27:0/2278887198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f75ac1a29e0 con 0x7f75ac109920 2024-04-26T22:46:05.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.956+0000 7f75a3400700 1 -- 172.21.15.27:0/2278887198 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f759c01a430 con 0x7f75ac109920 2024-04-26T22:46:05.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.957+0000 7f75b1c00700 1 -- 172.21.15.27:0/2278887198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac109920 msgr2=0x7f75ac19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:05.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.957+0000 7f75b1c00700 1 --2- 172.21.15.27:0/2278887198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac109920 0x7f75ac19efd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f759c011ac0 tx=0x7f759c011ba0 comp rx=0 tx=0).stop 2024-04-26T22:46:05.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.957+0000 7f75b1c00700 1 -- 172.21.15.27:0/2278887198 shutdown_connections 2024-04-26T22:46:05.683 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.957+0000 7f75b1c00700 1 --2- 172.21.15.27:0/2278887198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac109920 0x7f75ac19efd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:05.683 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.957+0000 7f75b1c00700 1 -- 172.21.15.27:0/2278887198 >> 172.21.15.27:0/2278887198 conn(0x7f75ac103290 msgr2=0x7f75ac103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:05.683 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.958+0000 7f75b1c00700 1 -- 172.21.15.27:0/2278887198 shutdown_connections 2024-04-26T22:46:05.683 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:04.958+0000 7f75b1c00700 1 -- 172.21.15.27:0/2278887198 wait complete. 2024-04-26T22:46:05.683 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (1/15)... 2024-04-26T22:46:07.341 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:06 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:06.883+0000 7f255b672200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T22:46:07.341 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:07 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:07.118+0000 7f255b672200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T22:46:07.341 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:07 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:07.340+0000 7f255b672200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T22:46:08.025 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:07 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:07.733+0000 7f255b672200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T22:46:08.025 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:07 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:07.858+0000 7f255b672200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T22:46:08.362 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:08 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:08.127+0000 7f255b672200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T22:46:08.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:08 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2036928217' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T22:46:08.615 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:08 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:08.389+0000 7f255b672200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:46:09.174 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T22:46:09.175 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-04-26T22:46:09.175 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "95ebdda6-041e-11ef-bc93-c7b262605968", 2024-04-26T22:46:09.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T22:46:09.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T22:46:09.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T22:46:09.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T22:46:09.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:09.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T22:46:09.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T22:46:09.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-04-26T22:46:09.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T22:46:09.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T22:46:09.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T22:46:09.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T22:46:09.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-26T22:46:09.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T22:46:09.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T22:46:09.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T22:46:09.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T22:46:09.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:09.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T22:46:09.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T22:46:09.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T22:46:09.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T22:46:09.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T22:46:09.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T22:46:09.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T22:46:09.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T22:46:09.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:09.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T22:46:09.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T22:46:09.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T22:46:09.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T22:46:09.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T22:46:09.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T22:46:09.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T22:46:09.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T22:46:09.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T22:46:09.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:09.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T22:46:09.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T22:46:09.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T22:46:09.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T22:46:09.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:09.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T22:46:09.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T22:46:09.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T22:46:09.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T22:46:09.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T22:46:09.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T22:46:09.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T22:46:09.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T22:46:09.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T22:46:09.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:09.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T22:46:09.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T22:46:09.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T22:45:52.404809+0000", 2024-04-26T22:46:09.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T22:46:09.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:09.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T22:46:09.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-04-26T22:46:09.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.355+0000 7f1f96600700 1 Processor -- start 2024-04-26T22:46:09.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.355+0000 7f1f96600700 1 -- start start 2024-04-26T22:46:09.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.356+0000 7f1f96600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90108e00 0x7f1f90109220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:09.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.356+0000 7f1f96600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1f90109760 con 0x7f1f90108e00 2024-04-26T22:46:09.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.356+0000 7f1f8f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90108e00 0x7f1f90109220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:09.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.356+0000 7f1f8f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90108e00 0x7f1f90109220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44842/0 (socket says 172.21.15.27:44842) 2024-04-26T22:46:09.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.356+0000 7f1f8f400700 1 -- 172.21.15.27:0/1664420953 learned_addr learned my addr 172.21.15.27:0/1664420953 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:09.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.357+0000 7f1f8f400700 1 -- 172.21.15.27:0/1664420953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f90109f60 con 0x7f1f90108e00 2024-04-26T22:46:09.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.357+0000 7f1f8f400700 1 --2- 172.21.15.27:0/1664420953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90108e00 0x7f1f90109220 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f800098a0 tx=0x7f1f80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3caa71e4c18ee36 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:09.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.357+0000 7f1f8e000700 1 -- 172.21.15.27:0/1664420953 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f80004160 con 0x7f1f90108e00 2024-04-26T22:46:09.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.358+0000 7f1f8e000700 1 -- 172.21.15.27:0/1664420953 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1f80004300 con 0x7f1f90108e00 2024-04-26T22:46:09.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.358+0000 7f1f8e000700 1 -- 172.21.15.27:0/1664420953 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f80011af0 con 0x7f1f90108e00 2024-04-26T22:46:09.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.358+0000 7f1f96600700 1 -- 172.21.15.27:0/1664420953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90108e00 msgr2=0x7f1f90109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:09.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.358+0000 7f1f96600700 1 --2- 172.21.15.27:0/1664420953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90108e00 0x7f1f90109220 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f800098a0 tx=0x7f1f80009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:09.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.358+0000 7f1f96600700 1 -- 172.21.15.27:0/1664420953 shutdown_connections 2024-04-26T22:46:09.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.358+0000 7f1f96600700 1 --2- 172.21.15.27:0/1664420953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90108e00 0x7f1f90109220 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:09.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.359+0000 7f1f96600700 1 -- 172.21.15.27:0/1664420953 >> 172.21.15.27:0/1664420953 conn(0x7f1f90104380 msgr2=0x7f1f901067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:09.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.359+0000 7f1f96600700 1 -- 172.21.15.27:0/1664420953 shutdown_connections 2024-04-26T22:46:09.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.359+0000 7f1f96600700 1 -- 172.21.15.27:0/1664420953 wait complete. 2024-04-26T22:46:09.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.360+0000 7f1f96600700 1 Processor -- start 2024-04-26T22:46:09.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.360+0000 7f1f96600700 1 -- start start 2024-04-26T22:46:09.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.361+0000 7f1f96600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90108e00 0x7f1f9019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:09.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.361+0000 7f1f96600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9010d3a0 con 0x7f1f90108e00 2024-04-26T22:46:09.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.361+0000 7f1f8f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90108e00 0x7f1f9019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:09.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.361+0000 7f1f8f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90108e00 0x7f1f9019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44850/0 (socket says 172.21.15.27:44850) 2024-04-26T22:46:09.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.361+0000 7f1f8f400700 1 -- 172.21.15.27:0/2036928217 learned_addr learned my addr 172.21.15.27:0/2036928217 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:09.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.361+0000 7f1f8f400700 1 -- 172.21.15.27:0/2036928217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9019f4e0 con 0x7f1f90108e00 2024-04-26T22:46:09.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.362+0000 7f1f8f400700 1 --2- 172.21.15.27:0/2036928217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90108e00 0x7f1f9019efa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f80011ac0 tx=0x7f1f80011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:09.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.362+0000 7f1f87e00700 1 -- 172.21.15.27:0/2036928217 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f80003970 con 0x7f1f90108e00 2024-04-26T22:46:09.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.362+0000 7f1f87e00700 1 -- 172.21.15.27:0/2036928217 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1f80003b10 con 0x7f1f90108e00 2024-04-26T22:46:09.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.362+0000 7f1f96600700 1 -- 172.21.15.27:0/2036928217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9019f770 con 0x7f1f90108e00 2024-04-26T22:46:09.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.362+0000 7f1f87e00700 1 -- 172.21.15.27:0/2036928217 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f80024610 con 0x7f1f90108e00 2024-04-26T22:46:09.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.362+0000 7f1f96600700 1 -- 172.21.15.27:0/2036928217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f901a2460 con 0x7f1f90108e00 2024-04-26T22:46:09.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.363+0000 7f1f87e00700 1 -- 172.21.15.27:0/2036928217 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1f80024c20 con 0x7f1f90108e00 2024-04-26T22:46:09.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.363+0000 7f1f87e00700 1 -- 172.21.15.27:0/2036928217 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1f80019700 con 0x7f1f90108e00 2024-04-26T22:46:09.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.364+0000 7f1f96600700 1 -- 172.21.15.27:0/2036928217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f90064a80 con 0x7f1f90108e00 2024-04-26T22:46:09.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.366+0000 7f1f87e00700 1 -- 172.21.15.27:0/2036928217 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1f80043ec0 con 0x7f1f90108e00 2024-04-26T22:46:09.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.448+0000 7f1f96600700 1 -- 172.21.15.27:0/2036928217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1f901a2630 con 0x7f1f90108e00 2024-04-26T22:46:09.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.449+0000 7f1f87e00700 1 -- 172.21.15.27:0/2036928217 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f1f80032030 con 0x7f1f90108e00 2024-04-26T22:46:09.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.450+0000 7f1f96600700 1 -- 172.21.15.27:0/2036928217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90108e00 msgr2=0x7f1f9019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:09.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.450+0000 7f1f96600700 1 --2- 172.21.15.27:0/2036928217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90108e00 0x7f1f9019efa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f80011ac0 tx=0x7f1f80011ba0 comp rx=0 tx=0).stop 2024-04-26T22:46:09.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.450+0000 7f1f96600700 1 -- 172.21.15.27:0/2036928217 shutdown_connections 2024-04-26T22:46:09.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.450+0000 7f1f96600700 1 --2- 172.21.15.27:0/2036928217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90108e00 0x7f1f9019efa0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:09.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.450+0000 7f1f96600700 1 -- 172.21.15.27:0/2036928217 >> 172.21.15.27:0/2036928217 conn(0x7f1f90104380 msgr2=0x7f1f90104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:09.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.450+0000 7f1f96600700 1 -- 172.21.15.27:0/2036928217 shutdown_connections 2024-04-26T22:46:09.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:08.450+0000 7f1f96600700 1 -- 172.21.15.27:0/2036928217 wait complete. 2024-04-26T22:46:09.193 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (2/15)... 2024-04-26T22:46:09.614 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:09 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:09.241+0000 7f255b672200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T22:46:10.114 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:09 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:09.806+0000 7f255b672200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T22:46:10.115 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:09 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:09.938+0000 7f255b672200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T22:46:10.614 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:10 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:10.210+0000 7f255b672200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T22:46:11.345 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:10 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:10.881+0000 7f255b672200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T22:46:11.346 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:11 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:11.007+0000 7f255b672200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T22:46:11.614 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:11 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:11.346+0000 7f255b672200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T22:46:11.615 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:11 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:11.547+0000 7f255b672200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T22:46:11.911 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:11 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:11.681+0000 7f255b672200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T22:46:11.911 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:11 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:11.840+0000 7f255b672200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T22:46:12.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:11 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3022795212' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T22:46:12.513 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:12 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:12.232+0000 7f255b672200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T22:46:12.522 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T22:46:12.522 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-04-26T22:46:12.522 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "95ebdda6-041e-11ef-bc93-c7b262605968", 2024-04-26T22:46:12.522 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T22:46:12.522 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T22:46:12.522 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T22:46:12.522 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T22:46:12.522 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:12.522 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T22:46:12.523 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T22:46:12.523 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-04-26T22:46:12.523 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T22:46:12.523 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T22:46:12.523 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T22:46:12.523 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T22:46:12.523 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-26T22:46:12.523 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T22:46:12.523 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T22:46:12.523 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T22:46:12.523 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T22:46:12.523 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:12.523 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T22:46:12.523 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T22:46:12.523 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T22:46:12.523 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T22:46:12.524 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T22:46:12.524 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T22:46:12.524 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T22:46:12.524 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T22:46:12.524 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:12.524 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T22:46:12.524 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T22:46:12.531 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T22:46:12.531 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T22:46:12.531 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T22:46:12.531 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T22:46:12.531 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T22:46:12.531 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T22:46:12.531 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T22:46:12.531 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:12.531 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T22:46:12.531 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T22:46:12.531 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T22:46:12.531 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T22:46:12.531 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:12.532 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T22:46:12.532 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T22:46:12.532 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T22:46:12.532 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T22:46:12.532 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T22:46:12.532 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T22:46:12.532 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T22:46:12.532 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T22:46:12.532 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T22:46:12.533 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:12.533 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T22:46:12.533 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T22:46:12.533 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T22:45:52.404809+0000", 2024-04-26T22:46:12.533 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T22:46:12.533 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:12.533 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T22:46:12.533 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-04-26T22:46:12.533 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.818+0000 7f16a4000700 1 Processor -- start 2024-04-26T22:46:12.533 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.818+0000 7f16a4000700 1 -- start start 2024-04-26T22:46:12.533 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.818+0000 7f16a4000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c107710 0x7f169c107b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:12.533 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.818+0000 7f16a4000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f169c07cdf0 con 0x7f169c107710 2024-04-26T22:46:12.533 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.819+0000 7f16a1600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c107710 0x7f169c107b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:12.533 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.819+0000 7f16a1600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c107710 0x7f169c107b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44854/0 (socket says 172.21.15.27:44854) 2024-04-26T22:46:12.534 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.819+0000 7f16a1600700 1 -- 172.21.15.27:0/2593596137 learned_addr learned my addr 172.21.15.27:0/2593596137 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:12.534 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.819+0000 7f16a1600700 1 -- 172.21.15.27:0/2593596137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169c108070 con 0x7f169c107710 2024-04-26T22:46:12.534 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.820+0000 7f16a1600700 1 --2- 172.21.15.27:0/2593596137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c107710 0x7f169c107b30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f16900098a0 tx=0x7f1690009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3f3c60a62f9344 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:12.534 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.820+0000 7f169be00700 1 -- 172.21.15.27:0/2593596137 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1690004160 con 0x7f169c107710 2024-04-26T22:46:12.534 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.820+0000 7f169be00700 1 -- 172.21.15.27:0/2593596137 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1690004300 con 0x7f169c107710 2024-04-26T22:46:12.534 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.820+0000 7f169be00700 1 -- 172.21.15.27:0/2593596137 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1690011af0 con 0x7f169c107710 2024-04-26T22:46:12.534 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.821+0000 7f16a4000700 1 -- 172.21.15.27:0/2593596137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c107710 msgr2=0x7f169c107b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:12.534 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.821+0000 7f16a4000700 1 --2- 172.21.15.27:0/2593596137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c107710 0x7f169c107b30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f16900098a0 tx=0x7f1690009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:12.534 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.821+0000 7f16a4000700 1 -- 172.21.15.27:0/2593596137 shutdown_connections 2024-04-26T22:46:12.535 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.821+0000 7f16a4000700 1 --2- 172.21.15.27:0/2593596137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c107710 0x7f169c107b30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:12.535 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.821+0000 7f16a4000700 1 -- 172.21.15.27:0/2593596137 >> 172.21.15.27:0/2593596137 conn(0x7f169c103290 msgr2=0x7f169c105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:12.535 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.821+0000 7f16a4000700 1 -- 172.21.15.27:0/2593596137 shutdown_connections 2024-04-26T22:46:12.535 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.821+0000 7f16a4000700 1 -- 172.21.15.27:0/2593596137 wait complete. 2024-04-26T22:46:12.535 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.822+0000 7f16a4000700 1 Processor -- start 2024-04-26T22:46:12.535 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.822+0000 7f16a4000700 1 -- start start 2024-04-26T22:46:12.535 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.822+0000 7f16a4000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c107710 0x7f169c10d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:12.535 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.822+0000 7f16a4000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f169c1090a0 con 0x7f169c107710 2024-04-26T22:46:12.535 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.823+0000 7f16a1600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c107710 0x7f169c10d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:12.535 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.823+0000 7f16a1600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c107710 0x7f169c10d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44860/0 (socket says 172.21.15.27:44860) 2024-04-26T22:46:12.535 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.823+0000 7f16a1600700 1 -- 172.21.15.27:0/3022795212 learned_addr learned my addr 172.21.15.27:0/3022795212 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:12.535 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.823+0000 7f16a1600700 1 -- 172.21.15.27:0/3022795212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169c10d6f0 con 0x7f169c107710 2024-04-26T22:46:12.535 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.823+0000 7f16a1600700 1 --2- 172.21.15.27:0/3022795212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c107710 0x7f169c10d1b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1690011ac0 tx=0x7f1690011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:12.535 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.824+0000 7f169a000700 1 -- 172.21.15.27:0/3022795212 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1690003970 con 0x7f169c107710 2024-04-26T22:46:12.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.824+0000 7f169a000700 1 -- 172.21.15.27:0/3022795212 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1690003b10 con 0x7f169c107710 2024-04-26T22:46:12.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.824+0000 7f169a000700 1 -- 172.21.15.27:0/3022795212 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1690024610 con 0x7f169c107710 2024-04-26T22:46:12.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.824+0000 7f16a4000700 1 -- 172.21.15.27:0/3022795212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169c10d980 con 0x7f169c107710 2024-04-26T22:46:12.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.824+0000 7f16a4000700 1 -- 172.21.15.27:0/3022795212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f169c109f40 con 0x7f169c107710 2024-04-26T22:46:12.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.824+0000 7f169a000700 1 -- 172.21.15.27:0/3022795212 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1690024c20 con 0x7f169c107710 2024-04-26T22:46:12.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.824+0000 7f169a000700 1 -- 172.21.15.27:0/3022795212 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1690019700 con 0x7f169c107710 2024-04-26T22:46:12.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.825+0000 7f16a4000700 1 -- 172.21.15.27:0/3022795212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f169c10db50 con 0x7f169c107710 2024-04-26T22:46:12.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.828+0000 7f169a000700 1 -- 172.21.15.27:0/3022795212 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f169c10db50 con 0x7f169c107710 2024-04-26T22:46:12.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.908+0000 7f16a4000700 1 -- 172.21.15.27:0/3022795212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f169c1985d0 con 0x7f169c107710 2024-04-26T22:46:12.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.909+0000 7f169a000700 1 -- 172.21.15.27:0/3022795212 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f169001a9e0 con 0x7f169c107710 2024-04-26T22:46:12.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.911+0000 7f16a4000700 1 -- 172.21.15.27:0/3022795212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c107710 msgr2=0x7f169c10d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:12.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.911+0000 7f16a4000700 1 --2- 172.21.15.27:0/3022795212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c107710 0x7f169c10d1b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1690011ac0 tx=0x7f1690011ba0 comp rx=0 tx=0).stop 2024-04-26T22:46:12.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.911+0000 7f16a4000700 1 -- 172.21.15.27:0/3022795212 shutdown_connections 2024-04-26T22:46:12.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.911+0000 7f16a4000700 1 --2- 172.21.15.27:0/3022795212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c107710 0x7f169c10d1b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:12.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.911+0000 7f16a4000700 1 -- 172.21.15.27:0/3022795212 >> 172.21.15.27:0/3022795212 conn(0x7f169c103290 msgr2=0x7f169c195b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:12.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.911+0000 7f16a4000700 1 -- 172.21.15.27:0/3022795212 shutdown_connections 2024-04-26T22:46:12.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:11.912+0000 7f16a4000700 1 -- 172.21.15.27:0/3022795212 wait complete. 2024-04-26T22:46:12.537 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (3/15)... 2024-04-26T22:46:13.313 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:13 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:13.018+0000 7f255b672200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T22:46:13.615 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:13 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:13.312+0000 7f255b672200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T22:46:13.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:13 smithi027 ceph-mon[28034]: Activating manager daemon x 2024-04-26T22:46:13.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:13 smithi027 ceph-mon[28034]: mgrmap e2: x(active, starting, since 0.0135299s) 2024-04-26T22:46:13.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:13 smithi027 ceph-mon[28034]: from='mgr.14100 172.21.15.27:0/798749729' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T22:46:13.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:13 smithi027 ceph-mon[28034]: from='mgr.14100 172.21.15.27:0/798749729' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T22:46:13.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:13 smithi027 ceph-mon[28034]: from='mgr.14100 172.21.15.27:0/798749729' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T22:46:13.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:13 smithi027 ceph-mon[28034]: from='mgr.14100 172.21.15.27:0/798749729' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:46:13.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:13 smithi027 ceph-mon[28034]: from='mgr.14100 172.21.15.27:0/798749729' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T22:46:13.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:13 smithi027 ceph-mon[28034]: Manager daemon x is now available 2024-04-26T22:46:13.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:13 smithi027 ceph-mon[28034]: from='mgr.14100 172.21.15.27:0/798749729' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T22:46:13.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:13 smithi027 ceph-mon[28034]: from='mgr.14100 172.21.15.27:0/798749729' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T22:46:13.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:13 smithi027 ceph-mon[28034]: from='mgr.14100 172.21.15.27:0/798749729' entity='mgr.x' 2024-04-26T22:46:14.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:14 smithi027 ceph-mon[28034]: from='mgr.14100 172.21.15.27:0/798749729' entity='mgr.x' 2024-04-26T22:46:14.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:14 smithi027 ceph-mon[28034]: from='mgr.14100 172.21.15.27:0/798749729' entity='mgr.x' 2024-04-26T22:46:14.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:14 smithi027 ceph-mon[28034]: mgrmap e3: x(active, since 1.02151s) 2024-04-26T22:46:16.198 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T22:46:16.198 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-04-26T22:46:16.198 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "95ebdda6-041e-11ef-bc93-c7b262605968", 2024-04-26T22:46:16.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T22:46:16.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T22:46:16.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T22:46:16.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T22:46:16.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:16.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T22:46:16.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T22:46:16.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-04-26T22:46:16.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T22:46:16.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T22:46:16.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T22:46:16.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T22:46:16.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-26T22:46:16.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T22:46:16.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T22:46:16.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T22:46:16.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T22:46:16.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:16.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T22:46:16.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T22:46:16.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T22:46:16.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T22:46:16.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T22:46:16.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T22:46:16.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T22:46:16.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T22:46:16.211 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:16.211 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T22:46:16.211 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T22:46:16.211 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T22:46:16.211 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T22:46:16.211 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T22:46:16.211 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T22:46:16.211 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T22:46:16.211 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T22:46:16.211 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T22:46:16.211 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:16.211 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T22:46:16.212 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T22:46:16.212 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T22:46:16.212 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T22:46:16.212 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:16.212 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T22:46:16.212 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T22:46:16.212 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T22:46:16.212 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T22:46:16.212 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T22:46:16.212 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T22:46:16.212 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T22:46:16.212 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T22:46:16.212 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T22:46:16.213 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:16.213 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T22:46:16.213 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T22:46:16.213 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T22:45:52.404809+0000", 2024-04-26T22:46:16.213 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T22:46:16.213 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T22:46:16.213 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T22:46:16.213 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-04-26T22:46:16.213 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.133+0000 7fefdb200700 1 Processor -- start 2024-04-26T22:46:16.213 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.133+0000 7fefdb200700 1 -- start start 2024-04-26T22:46:16.213 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.134+0000 7fefdb200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefdc07f730 0x7fefdc07db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:16.214 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.134+0000 7fefdb200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc07e160 con 0x7fefdc07f730 2024-04-26T22:46:16.214 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.134+0000 7fefd9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefdc07f730 0x7fefdc07db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:16.214 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.134+0000 7fefd9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefdc07f730 0x7fefdc07db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44946/0 (socket says 172.21.15.27:44946) 2024-04-26T22:46:16.214 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.134+0000 7fefd9e00700 1 -- 172.21.15.27:0/1773733330 learned_addr learned my addr 172.21.15.27:0/1773733330 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:16.214 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.135+0000 7fefd9e00700 1 -- 172.21.15.27:0/1773733330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefdc07e2e0 con 0x7fefdc07f730 2024-04-26T22:46:16.214 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.135+0000 7fefd9e00700 1 --2- 172.21.15.27:0/1773733330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefdc07f730 0x7fefdc07db90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0098a0 tx=0x7fefcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb6ede6e69fb49c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:16.214 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.136+0000 7fefd8a00700 1 -- 172.21.15.27:0/1773733330 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefcc004160 con 0x7fefdc07f730 2024-04-26T22:46:16.214 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.136+0000 7fefd8a00700 1 -- 172.21.15.27:0/1773733330 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fefcc004300 con 0x7fefdc07f730 2024-04-26T22:46:16.214 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.136+0000 7fefd8a00700 1 -- 172.21.15.27:0/1773733330 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefcc011af0 con 0x7fefdc07f730 2024-04-26T22:46:16.214 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.136+0000 7fefdb200700 1 -- 172.21.15.27:0/1773733330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefdc07f730 msgr2=0x7fefdc07db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:16.214 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.136+0000 7fefdb200700 1 --2- 172.21.15.27:0/1773733330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefdc07f730 0x7fefdc07db90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0098a0 tx=0x7fefcc009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:16.215 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.136+0000 7fefdb200700 1 -- 172.21.15.27:0/1773733330 shutdown_connections 2024-04-26T22:46:16.215 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.136+0000 7fefdb200700 1 --2- 172.21.15.27:0/1773733330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefdc07f730 0x7fefdc07db90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:16.215 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.136+0000 7fefdb200700 1 -- 172.21.15.27:0/1773733330 >> 172.21.15.27:0/1773733330 conn(0x7fefdc104190 msgr2=0x7fefdc1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:16.215 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.137+0000 7fefdb200700 1 -- 172.21.15.27:0/1773733330 shutdown_connections 2024-04-26T22:46:16.215 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.137+0000 7fefdb200700 1 -- 172.21.15.27:0/1773733330 wait complete. 2024-04-26T22:46:16.215 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.138+0000 7fefdb200700 1 Processor -- start 2024-04-26T22:46:16.215 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.138+0000 7fefdb200700 1 -- start start 2024-04-26T22:46:16.215 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.138+0000 7fefdb200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefdc07f730 0x7fefdc1a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:16.215 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.138+0000 7fefdb200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc10adc0 con 0x7fefdc07f730 2024-04-26T22:46:16.215 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.139+0000 7fefd9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefdc07f730 0x7fefdc1a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:16.215 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.139+0000 7fefd9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefdc07f730 0x7fefdc1a31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44962/0 (socket says 172.21.15.27:44962) 2024-04-26T22:46:16.215 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.139+0000 7fefd9e00700 1 -- 172.21.15.27:0/3799085977 learned_addr learned my addr 172.21.15.27:0/3799085977 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:16.215 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.139+0000 7fefd9e00700 1 -- 172.21.15.27:0/3799085977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefdc1a36f0 con 0x7fefdc07f730 2024-04-26T22:46:16.216 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.139+0000 7fefd9e00700 1 --2- 172.21.15.27:0/3799085977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefdc07f730 0x7fefdc1a31b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009fb0 tx=0x7fefcc011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:16.216 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.140+0000 7fefd2a00700 1 -- 172.21.15.27:0/3799085977 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefcc0036a0 con 0x7fefdc07f730 2024-04-26T22:46:16.216 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.140+0000 7fefd2a00700 1 -- 172.21.15.27:0/3799085977 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fefcc01a430 con 0x7fefdc07f730 2024-04-26T22:46:16.216 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.140+0000 7fefd2a00700 1 -- 172.21.15.27:0/3799085977 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefcc019550 con 0x7fefdc07f730 2024-04-26T22:46:16.216 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.140+0000 7fefdb200700 1 -- 172.21.15.27:0/3799085977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefdc1a3980 con 0x7fefdc07f730 2024-04-26T22:46:16.216 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.140+0000 7fefdb200700 1 -- 172.21.15.27:0/3799085977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefdc1a6670 con 0x7fefdc07f730 2024-04-26T22:46:16.216 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.141+0000 7fefd2a00700 1 -- 172.21.15.27:0/3799085977 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7fefcc003840 con 0x7fefdc07f730 2024-04-26T22:46:16.216 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.141+0000 7fefdb200700 1 -- 172.21.15.27:0/3799085977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefdc064a80 con 0x7fefdc07f730 2024-04-26T22:46:16.216 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.141+0000 7fefd2a00700 1 --2- 172.21.15.27:0/3799085977 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7fefc0039430 0x7fefc003b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:16.216 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.142+0000 7fefd2a00700 1 -- 172.21.15.27:0/3799085977 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fefcc04d640 con 0x7fefdc07f730 2024-04-26T22:46:16.216 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.142+0000 7fefd9400700 1 --2- 172.21.15.27:0/3799085977 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7fefc0039430 0x7fefc003b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:16.216 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.142+0000 7fefd9400700 1 --2- 172.21.15.27:0/3799085977 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7fefc0039430 0x7fefc003b8f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fefc8006fd0 tx=0x7fefc8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:16.217 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.148+0000 7fefd2a00700 1 -- 172.21.15.27:0/3799085977 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fefcc003b30 con 0x7fefdc07f730 2024-04-26T22:46:16.217 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.379+0000 7fefd2a00700 1 -- 172.21.15.27:0/3799085977 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fefcc019800 con 0x7fefdc07f730 2024-04-26T22:46:16.217 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.473+0000 7fefdb200700 1 -- 172.21.15.27:0/3799085977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fefdc1a6840 con 0x7fefdc07f730 2024-04-26T22:46:16.217 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.475+0000 7fefd2a00700 1 -- 172.21.15.27:0/3799085977 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fefdc1a6840 con 0x7fefdc07f730 2024-04-26T22:46:16.217 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.477+0000 7fefdb200700 1 -- 172.21.15.27:0/3799085977 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7fefc0039430 msgr2=0x7fefc003b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:16.217 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.477+0000 7fefdb200700 1 --2- 172.21.15.27:0/3799085977 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7fefc0039430 0x7fefc003b8f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fefc8006fd0 tx=0x7fefc8006e40 comp rx=0 tx=0).stop 2024-04-26T22:46:16.217 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.478+0000 7fefdb200700 1 -- 172.21.15.27:0/3799085977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefdc07f730 msgr2=0x7fefdc1a31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:16.217 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.478+0000 7fefdb200700 1 --2- 172.21.15.27:0/3799085977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefdc07f730 0x7fefdc1a31b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009fb0 tx=0x7fefcc011940 comp rx=0 tx=0).stop 2024-04-26T22:46:16.217 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.478+0000 7fefdb200700 1 -- 172.21.15.27:0/3799085977 shutdown_connections 2024-04-26T22:46:16.217 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.478+0000 7fefdb200700 1 --2- 172.21.15.27:0/3799085977 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7fefc0039430 0x7fefc003b8f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:16.217 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.478+0000 7fefdb200700 1 --2- 172.21.15.27:0/3799085977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefdc07f730 0x7fefdc1a31b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:16.217 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.478+0000 7fefdb200700 1 -- 172.21.15.27:0/3799085977 >> 172.21.15.27:0/3799085977 conn(0x7fefdc104190 msgr2=0x7fefdc1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:16.218 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.478+0000 7fefdb200700 1 -- 172.21.15.27:0/3799085977 shutdown_connections 2024-04-26T22:46:16.218 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:15.478+0000 7fefdb200700 1 -- 172.21.15.27:0/3799085977 wait complete. 2024-04-26T22:46:16.218 INFO:teuthology.orchestra.run.smithi027.stdout:mgr is available 2024-04-26T22:46:16.403 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:16 smithi027 ceph-mon[28034]: mgrmap e4: x(active, since 2s) 2024-04-26T22:46:16.403 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:16 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3799085977' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T22:46:17.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:17 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1534538972' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T22:46:17.800 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T22:46:17.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T22:46:17.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout fsid = 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:46:17.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T22:46:17.802 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T22:46:17.802 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T22:46:17.802 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T22:46:17.802 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T22:46:17.804 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T22:46:17.804 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T22:46:17.805 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T22:46:17.805 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T22:46:17.805 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T22:46:17.805 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T22:46:17.805 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T22:46:17.805 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T22:46:17.805 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T22:46:17.805 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T22:46:17.805 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.840+0000 7f36c0e00700 1 Processor -- start 2024-04-26T22:46:17.805 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.841+0000 7f36c0e00700 1 -- start start 2024-04-26T22:46:17.805 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.841+0000 7f36c0e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc082e50 0x7f36bc083270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:17.805 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.841+0000 7f36c0e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc083840 con 0x7f36bc082e50 2024-04-26T22:46:17.805 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.841+0000 7f36b9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc082e50 0x7f36bc083270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:17.805 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.842+0000 7f36b9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc082e50 0x7f36bc083270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44970/0 (socket says 172.21.15.27:44970) 2024-04-26T22:46:17.805 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.842+0000 7f36b9e00700 1 -- 172.21.15.27:0/2364406823 learned_addr learned my addr 172.21.15.27:0/2364406823 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:17.806 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.842+0000 7f36b9e00700 1 -- 172.21.15.27:0/2364406823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc110620 con 0x7f36bc082e50 2024-04-26T22:46:17.806 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.842+0000 7f36b9e00700 1 --2- 172.21.15.27:0/2364406823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc082e50 0x7f36bc083270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0098a0 tx=0x7f36ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f881149962de51e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:17.806 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.843+0000 7f36b8a00700 1 -- 172.21.15.27:0/2364406823 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36ac004160 con 0x7f36bc082e50 2024-04-26T22:46:17.806 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.843+0000 7f36b8a00700 1 -- 172.21.15.27:0/2364406823 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f36ac004300 con 0x7f36bc082e50 2024-04-26T22:46:17.810 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.843+0000 7f36b8a00700 1 -- 172.21.15.27:0/2364406823 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36ac011af0 con 0x7f36bc082e50 2024-04-26T22:46:17.810 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.843+0000 7f36c0e00700 1 -- 172.21.15.27:0/2364406823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc082e50 msgr2=0x7f36bc083270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:17.810 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.843+0000 7f36c0e00700 1 --2- 172.21.15.27:0/2364406823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc082e50 0x7f36bc083270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0098a0 tx=0x7f36ac009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:17.810 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.843+0000 7f36c0e00700 1 -- 172.21.15.27:0/2364406823 shutdown_connections 2024-04-26T22:46:17.810 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.843+0000 7f36c0e00700 1 --2- 172.21.15.27:0/2364406823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc082e50 0x7f36bc083270 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:17.811 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.844+0000 7f36c0e00700 1 -- 172.21.15.27:0/2364406823 >> 172.21.15.27:0/2364406823 conn(0x7f36bc07e430 msgr2=0x7f36bc080850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:17.811 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.844+0000 7f36c0e00700 1 -- 172.21.15.27:0/2364406823 shutdown_connections 2024-04-26T22:46:17.811 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.844+0000 7f36c0e00700 1 -- 172.21.15.27:0/2364406823 wait complete. 2024-04-26T22:46:17.811 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.845+0000 7f36c0e00700 1 Processor -- start 2024-04-26T22:46:17.811 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.845+0000 7f36c0e00700 1 -- start start 2024-04-26T22:46:17.811 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.845+0000 7f36c0e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc082e50 0x7f36bc1a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:17.811 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.845+0000 7f36c0e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc1139d0 con 0x7f36bc082e50 2024-04-26T22:46:17.811 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.845+0000 7f36b9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc082e50 0x7f36bc1a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:17.811 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.845+0000 7f36b9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc082e50 0x7f36bc1a33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44978/0 (socket says 172.21.15.27:44978) 2024-04-26T22:46:17.811 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.845+0000 7f36b9e00700 1 -- 172.21.15.27:0/1534538972 learned_addr learned my addr 172.21.15.27:0/1534538972 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:17.811 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.846+0000 7f36b9e00700 1 -- 172.21.15.27:0/1534538972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc1a3910 con 0x7f36bc082e50 2024-04-26T22:46:17.811 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.846+0000 7f36b9e00700 1 --2- 172.21.15.27:0/1534538972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc082e50 0x7f36bc1a33d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0117b0 tx=0x7f36ac011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:17.811 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.846+0000 7f36b2a00700 1 -- 172.21.15.27:0/1534538972 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36ac003af0 con 0x7f36bc082e50 2024-04-26T22:46:17.812 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.846+0000 7f36b2a00700 1 -- 172.21.15.27:0/1534538972 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f36ac020070 con 0x7f36bc082e50 2024-04-26T22:46:17.812 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.846+0000 7f36c0e00700 1 -- 172.21.15.27:0/1534538972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36bc1a3ba0 con 0x7f36bc082e50 2024-04-26T22:46:17.812 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.846+0000 7f36b2a00700 1 -- 172.21.15.27:0/1534538972 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36ac019630 con 0x7f36bc082e50 2024-04-26T22:46:17.812 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.847+0000 7f36c0e00700 1 -- 172.21.15.27:0/1534538972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36bc1a6890 con 0x7f36bc082e50 2024-04-26T22:46:17.812 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.848+0000 7f36b2a00700 1 -- 172.21.15.27:0/1534538972 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f36ac01a430 con 0x7f36bc082e50 2024-04-26T22:46:17.812 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.848+0000 7f36c0e00700 1 -- 172.21.15.27:0/1534538972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36bc064a80 con 0x7f36bc082e50 2024-04-26T22:46:17.812 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.848+0000 7f36b2a00700 1 --2- 172.21.15.27:0/1534538972 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f36a0039600 0x7f36a003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:17.812 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.848+0000 7f36b2a00700 1 -- 172.21.15.27:0/1534538972 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f36ac04da90 con 0x7f36bc082e50 2024-04-26T22:46:17.812 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.849+0000 7f36b9400700 1 --2- 172.21.15.27:0/1534538972 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f36a0039600 0x7f36a003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:17.812 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.849+0000 7f36b9400700 1 --2- 172.21.15.27:0/1534538972 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f36a0039600 0x7f36a003bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f36a8006fd0 tx=0x7f36a8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:17.812 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:16.854+0000 7f36b2a00700 1 -- 172.21.15.27:0/1534538972 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f36ac00e260 con 0x7f36bc082e50 2024-04-26T22:46:17.812 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:17.083+0000 7f36c0e00700 1 -- 172.21.15.27:0/1534538972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f36bc1a6a60 con 0x7f36bc082e50 2024-04-26T22:46:17.812 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:17.086+0000 7f36b2a00700 1 -- 172.21.15.27:0/1534538972 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f36bc1a6a60 con 0x7f36bc082e50 2024-04-26T22:46:17.812 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:17.089+0000 7f36c0e00700 1 -- 172.21.15.27:0/1534538972 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f36a0039600 msgr2=0x7f36a003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:17.813 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:17.089+0000 7f36c0e00700 1 --2- 172.21.15.27:0/1534538972 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f36a0039600 0x7f36a003bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f36a8006fd0 tx=0x7f36a8006e40 comp rx=0 tx=0).stop 2024-04-26T22:46:17.813 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:17.089+0000 7f36c0e00700 1 -- 172.21.15.27:0/1534538972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc082e50 msgr2=0x7f36bc1a33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:17.813 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:17.089+0000 7f36c0e00700 1 --2- 172.21.15.27:0/1534538972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc082e50 0x7f36bc1a33d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0117b0 tx=0x7f36ac011b70 comp rx=0 tx=0).stop 2024-04-26T22:46:17.813 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:17.090+0000 7f36c0e00700 1 -- 172.21.15.27:0/1534538972 shutdown_connections 2024-04-26T22:46:17.813 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:17.090+0000 7f36c0e00700 1 --2- 172.21.15.27:0/1534538972 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f36a0039600 0x7f36a003bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:17.813 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:17.090+0000 7f36c0e00700 1 --2- 172.21.15.27:0/1534538972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc082e50 0x7f36bc1a33d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:17.813 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:17.090+0000 7f36c0e00700 1 -- 172.21.15.27:0/1534538972 >> 172.21.15.27:0/1534538972 conn(0x7f36bc07e430 msgr2=0x7f36bc07f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:17.813 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:17.090+0000 7f36c0e00700 1 -- 172.21.15.27:0/1534538972 shutdown_connections 2024-04-26T22:46:17.813 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:17.090+0000 7f36c0e00700 1 -- 172.21.15.27:0/1534538972 wait complete. 2024-04-26T22:46:17.814 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling cephadm module... 2024-04-26T22:46:19.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:18 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2946381574' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T22:46:19.115 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:18 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: ignoring --setuser ceph since I am not root 2024-04-26T22:46:19.115 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:18 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: ignoring --setgroup ceph since I am not root 2024-04-26T22:46:19.391 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:19 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:19.124+0000 7f8007be9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T22:46:19.391 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:19 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:19.391+0000 7f8007be9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:46:19.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.455+0000 7f3749400700 1 Processor -- start 2024-04-26T22:46:19.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.455+0000 7f3749400700 1 -- start start 2024-04-26T22:46:19.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.456+0000 7f3749400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3744108e00 0x7f3744109220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:19.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.456+0000 7f3749400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3744109760 con 0x7f3744108e00 2024-04-26T22:46:19.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.456+0000 7f3742800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3744108e00 0x7f3744109220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:19.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.456+0000 7f3742800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3744108e00 0x7f3744109220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36122/0 (socket says 172.21.15.27:36122) 2024-04-26T22:46:19.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.456+0000 7f3742800700 1 -- 172.21.15.27:0/844951457 learned_addr learned my addr 172.21.15.27:0/844951457 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:19.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.457+0000 7f3742800700 1 -- 172.21.15.27:0/844951457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3744109f60 con 0x7f3744108e00 2024-04-26T22:46:19.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.457+0000 7f3742800700 1 --2- 172.21.15.27:0/844951457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3744108e00 0x7f3744109220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f37340098a0 tx=0x7f3734009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffba78145777b75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:19.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.458+0000 7f3741400700 1 -- 172.21.15.27:0/844951457 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3734004160 con 0x7f3744108e00 2024-04-26T22:46:19.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.458+0000 7f3741400700 1 -- 172.21.15.27:0/844951457 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3734004300 con 0x7f3744108e00 2024-04-26T22:46:19.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.458+0000 7f3741400700 1 -- 172.21.15.27:0/844951457 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3734011af0 con 0x7f3744108e00 2024-04-26T22:46:19.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.458+0000 7f3749400700 1 -- 172.21.15.27:0/844951457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3744108e00 msgr2=0x7f3744109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:19.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.458+0000 7f3749400700 1 --2- 172.21.15.27:0/844951457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3744108e00 0x7f3744109220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f37340098a0 tx=0x7f3734009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:19.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.458+0000 7f3749400700 1 -- 172.21.15.27:0/844951457 shutdown_connections 2024-04-26T22:46:19.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.458+0000 7f3749400700 1 --2- 172.21.15.27:0/844951457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3744108e00 0x7f3744109220 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:19.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.458+0000 7f3749400700 1 -- 172.21.15.27:0/844951457 >> 172.21.15.27:0/844951457 conn(0x7f37441043e0 msgr2=0x7f3744106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:19.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.459+0000 7f3749400700 1 -- 172.21.15.27:0/844951457 shutdown_connections 2024-04-26T22:46:19.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.459+0000 7f3749400700 1 -- 172.21.15.27:0/844951457 wait complete. 2024-04-26T22:46:19.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.460+0000 7f3749400700 1 Processor -- start 2024-04-26T22:46:19.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.460+0000 7f3749400700 1 -- start start 2024-04-26T22:46:19.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.460+0000 7f3749400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3744108e00 0x7f374419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:19.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.460+0000 7f3749400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f374410d3a0 con 0x7f3744108e00 2024-04-26T22:46:19.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.460+0000 7f3742800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3744108e00 0x7f374419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:19.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.460+0000 7f3742800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3744108e00 0x7f374419ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36124/0 (socket says 172.21.15.27:36124) 2024-04-26T22:46:19.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.460+0000 7f3742800700 1 -- 172.21.15.27:0/2946381574 learned_addr learned my addr 172.21.15.27:0/2946381574 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:19.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.461+0000 7f3742800700 1 -- 172.21.15.27:0/2946381574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374419f450 con 0x7f3744108e00 2024-04-26T22:46:19.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.461+0000 7f3742800700 1 --2- 172.21.15.27:0/2946381574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3744108e00 0x7f374419ef10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3734011ac0 tx=0x7f3734011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:19.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.461+0000 7f373b400700 1 -- 172.21.15.27:0/2946381574 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3734003970 con 0x7f3744108e00 2024-04-26T22:46:19.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.461+0000 7f373b400700 1 -- 172.21.15.27:0/2946381574 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3734003b10 con 0x7f3744108e00 2024-04-26T22:46:19.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.461+0000 7f373b400700 1 -- 172.21.15.27:0/2946381574 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3734024610 con 0x7f3744108e00 2024-04-26T22:46:19.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.461+0000 7f3749400700 1 -- 172.21.15.27:0/2946381574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374419f6e0 con 0x7f3744108e00 2024-04-26T22:46:19.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.462+0000 7f3749400700 1 -- 172.21.15.27:0/2946381574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37441a23d0 con 0x7f3744108e00 2024-04-26T22:46:19.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.463+0000 7f3749400700 1 -- 172.21.15.27:0/2946381574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3744064a80 con 0x7f3744108e00 2024-04-26T22:46:19.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.463+0000 7f373b400700 1 -- 172.21.15.27:0/2946381574 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f373401a9e0 con 0x7f3744108e00 2024-04-26T22:46:19.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.464+0000 7f373b400700 1 --2- 172.21.15.27:0/2946381574 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f37280395b0 0x7f372803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:19.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.464+0000 7f373b400700 1 -- 172.21.15.27:0/2946381574 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f373404d8b0 con 0x7f3744108e00 2024-04-26T22:46:19.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.464+0000 7f3741e00700 1 --2- 172.21.15.27:0/2946381574 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f37280395b0 0x7f372803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:19.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.465+0000 7f3741e00700 1 --2- 172.21.15.27:0/2946381574 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f37280395b0 0x7f372803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3730006fd0 tx=0x7f3730006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:19.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.469+0000 7f373b400700 1 -- 172.21.15.27:0/2946381574 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f373400b3e0 con 0x7f3744108e00 2024-04-26T22:46:19.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.765+0000 7f3749400700 1 -- 172.21.15.27:0/2946381574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f37441a2640 con 0x7f3744108e00 2024-04-26T22:46:19.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.819+0000 7f373b400700 1 -- 172.21.15.27:0/2946381574 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f373404d2d0 con 0x7f3744108e00 2024-04-26T22:46:19.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.823+0000 7f373b400700 1 -- 172.21.15.27:0/2946381574 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f373404fcf0 con 0x7f3744108e00 2024-04-26T22:46:19.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.823+0000 7f3749400700 1 -- 172.21.15.27:0/2946381574 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f37280395b0 msgr2=0x7f372803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:19.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.823+0000 7f3749400700 1 --2- 172.21.15.27:0/2946381574 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f37280395b0 0x7f372803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3730006fd0 tx=0x7f3730006e40 comp rx=0 tx=0).stop 2024-04-26T22:46:19.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.823+0000 7f3749400700 1 -- 172.21.15.27:0/2946381574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3744108e00 msgr2=0x7f374419ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:19.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.823+0000 7f3749400700 1 --2- 172.21.15.27:0/2946381574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3744108e00 0x7f374419ef10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3734011ac0 tx=0x7f3734011ba0 comp rx=0 tx=0).stop 2024-04-26T22:46:19.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.825+0000 7f3749400700 1 -- 172.21.15.27:0/2946381574 shutdown_connections 2024-04-26T22:46:19.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.825+0000 7f3749400700 1 --2- 172.21.15.27:0/2946381574 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f37280395b0 0x7f372803ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:19.544 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.825+0000 7f3749400700 1 --2- 172.21.15.27:0/2946381574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3744108e00 0x7f374419ef10 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3734011ac0 tx=0x7f3734011ba0 comp rx=0 tx=0).stop 2024-04-26T22:46:19.544 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.825+0000 7f3749400700 1 -- 172.21.15.27:0/2946381574 >> 172.21.15.27:0/2946381574 conn(0x7f37441043e0 msgr2=0x7f3744104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:19.544 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.825+0000 7f3749400700 1 -- 172.21.15.27:0/2946381574 shutdown_connections 2024-04-26T22:46:19.544 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:18.825+0000 7f3749400700 1 -- 172.21.15.27:0/2946381574 wait complete. 2024-04-26T22:46:19.692 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:19 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:19.530+0000 7f8007be9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T22:46:19.951 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:19 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2946381574' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T22:46:19.951 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:19 smithi027 ceph-mon[28034]: mgrmap e5: x(active, since 5s) 2024-04-26T22:46:20.202 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:19 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:19.951+0000 7f8007be9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T22:46:20.202 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:20 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:20.071+0000 7f8007be9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T22:46:21.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:20 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/4090719283' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T22:46:21.281 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-04-26T22:46:21.281 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T22:46:21.282 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T22:46:21.283 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T22:46:21.283 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T22:46:21.283 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-04-26T22:46:21.283 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.195+0000 7f8272c00700 1 Processor -- start 2024-04-26T22:46:21.283 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.195+0000 7f8272c00700 1 -- start start 2024-04-26T22:46:21.283 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.195+0000 7f8272c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f826c107d30 0x7f826c108150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:21.283 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.196+0000 7f8272c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f826c108720 con 0x7f826c107d30 2024-04-26T22:46:21.283 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.196+0000 7f826be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f826c107d30 0x7f826c108150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:21.286 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.196+0000 7f826be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f826c107d30 0x7f826c108150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36148/0 (socket says 172.21.15.27:36148) 2024-04-26T22:46:21.287 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.196+0000 7f826be00700 1 -- 172.21.15.27:0/2719264600 learned_addr learned my addr 172.21.15.27:0/2719264600 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:21.287 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.196+0000 7f826be00700 1 -- 172.21.15.27:0/2719264600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c108f30 con 0x7f826c107d30 2024-04-26T22:46:21.287 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.197+0000 7f826be00700 1 --2- 172.21.15.27:0/2719264600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f826c107d30 0x7f826c108150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f825c00bb40 tx=0x7f825c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b3e9695473e6fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:21.287 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.197+0000 7f826aa00700 1 -- 172.21.15.27:0/2719264600 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f825c00dbe0 con 0x7f826c107d30 2024-04-26T22:46:21.287 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.197+0000 7f826aa00700 1 -- 172.21.15.27:0/2719264600 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f825c0044c0 con 0x7f826c107d30 2024-04-26T22:46:21.287 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.197+0000 7f826aa00700 1 -- 172.21.15.27:0/2719264600 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f825c010470 con 0x7f826c107d30 2024-04-26T22:46:21.287 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.198+0000 7f8272c00700 1 -- 172.21.15.27:0/2719264600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f826c107d30 msgr2=0x7f826c108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:21.287 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.198+0000 7f8272c00700 1 --2- 172.21.15.27:0/2719264600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f826c107d30 0x7f826c108150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f825c00bb40 tx=0x7f825c00be50 comp rx=0 tx=0).stop 2024-04-26T22:46:21.287 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.198+0000 7f8272c00700 1 -- 172.21.15.27:0/2719264600 shutdown_connections 2024-04-26T22:46:21.287 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.198+0000 7f8272c00700 1 --2- 172.21.15.27:0/2719264600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f826c107d30 0x7f826c108150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:21.287 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.198+0000 7f8272c00700 1 -- 172.21.15.27:0/2719264600 >> 172.21.15.27:0/2719264600 conn(0x7f826c103290 msgr2=0x7f826c105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:21.287 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.198+0000 7f8272c00700 1 -- 172.21.15.27:0/2719264600 shutdown_connections 2024-04-26T22:46:21.287 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.198+0000 7f8272c00700 1 -- 172.21.15.27:0/2719264600 wait complete. 2024-04-26T22:46:21.287 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.199+0000 7f8272c00700 1 Processor -- start 2024-04-26T22:46:21.288 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.199+0000 7f8272c00700 1 -- start start 2024-04-26T22:46:21.288 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.200+0000 7f8272c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f826c107d30 0x7f826c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:21.288 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.200+0000 7f8272c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f826c07bf90 con 0x7f826c107d30 2024-04-26T22:46:21.288 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.200+0000 7f826be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f826c107d30 0x7f826c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:21.288 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.200+0000 7f826be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f826c107d30 0x7f826c19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36156/0 (socket says 172.21.15.27:36156) 2024-04-26T22:46:21.288 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.200+0000 7f826be00700 1 -- 172.21.15.27:0/4090719283 learned_addr learned my addr 172.21.15.27:0/4090719283 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:21.288 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.200+0000 7f826be00700 1 -- 172.21.15.27:0/4090719283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c19f4c0 con 0x7f826c107d30 2024-04-26T22:46:21.288 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.201+0000 7f826be00700 1 --2- 172.21.15.27:0/4090719283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f826c107d30 0x7f826c19ef80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f825c00b340 tx=0x7f825c003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:21.288 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.201+0000 7f8268c00700 1 -- 172.21.15.27:0/4090719283 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f825c010720 con 0x7f826c107d30 2024-04-26T22:46:21.288 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.201+0000 7f8268c00700 1 -- 172.21.15.27:0/4090719283 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f825c01d070 con 0x7f826c107d30 2024-04-26T22:46:21.288 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.201+0000 7f8268c00700 1 -- 172.21.15.27:0/4090719283 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f825c021ba0 con 0x7f826c107d30 2024-04-26T22:46:21.288 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.201+0000 7f8272c00700 1 -- 172.21.15.27:0/4090719283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826c19f750 con 0x7f826c107d30 2024-04-26T22:46:21.288 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.201+0000 7f8272c00700 1 -- 172.21.15.27:0/4090719283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826c1a2440 con 0x7f826c107d30 2024-04-26T22:46:21.288 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.202+0000 7f8272c00700 1 -- 172.21.15.27:0/4090719283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f826c064a80 con 0x7f826c107d30 2024-04-26T22:46:21.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.203+0000 7f8268c00700 1 -- 172.21.15.27:0/4090719283 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f825c01f030 con 0x7f826c107d30 2024-04-26T22:46:21.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.203+0000 7f8268c00700 1 --2- 172.21.15.27:0/4090719283 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f8250039600 0x7f825003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:21.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.203+0000 7f8268c00700 1 -- 172.21.15.27:0/4090719283 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f825c04d620 con 0x7f826c107d30 2024-04-26T22:46:21.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.203+0000 7f826b400700 1 -- 172.21.15.27:0/4090719283 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f8250039600 msgr2=0x7f825003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1521074971 2024-04-26T22:46:21.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.203+0000 7f826b400700 1 --2- 172.21.15.27:0/4090719283 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f8250039600 0x7f825003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T22:46:21.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.211+0000 7f8268c00700 1 -- 172.21.15.27:0/4090719283 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f825c015070 con 0x7f826c107d30 2024-04-26T22:46:21.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.404+0000 7f826b400700 1 -- 172.21.15.27:0/4090719283 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f8250039600 msgr2=0x7f825003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1521074971 2024-04-26T22:46:21.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.404+0000 7f826b400700 1 --2- 172.21.15.27:0/4090719283 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f8250039600 0x7f825003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T22:46:21.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.498+0000 7f8272c00700 1 -- 172.21.15.27:0/4090719283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f826c0008d0 con 0x7f826c107d30 2024-04-26T22:46:21.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.499+0000 7f8268c00700 1 -- 172.21.15.27:0/4090719283 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f825c04d380 con 0x7f826c107d30 2024-04-26T22:46:21.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.502+0000 7f8272c00700 1 -- 172.21.15.27:0/4090719283 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f8250039600 msgr2=0x7f825003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:46:21.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.502+0000 7f8272c00700 1 --2- 172.21.15.27:0/4090719283 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f8250039600 0x7f825003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:21.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.502+0000 7f8272c00700 1 -- 172.21.15.27:0/4090719283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f826c107d30 msgr2=0x7f826c19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:21.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.502+0000 7f8272c00700 1 --2- 172.21.15.27:0/4090719283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f826c107d30 0x7f826c19ef80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f825c00b340 tx=0x7f825c003dc0 comp rx=0 tx=0).stop 2024-04-26T22:46:21.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.502+0000 7f8272c00700 1 -- 172.21.15.27:0/4090719283 shutdown_connections 2024-04-26T22:46:21.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.502+0000 7f8272c00700 1 --2- 172.21.15.27:0/4090719283 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f8250039600 0x7f825003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:21.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.502+0000 7f8272c00700 1 --2- 172.21.15.27:0/4090719283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f826c107d30 0x7f826c19ef80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:21.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.502+0000 7f8272c00700 1 -- 172.21.15.27:0/4090719283 >> 172.21.15.27:0/4090719283 conn(0x7f826c103290 msgr2=0x7f826c191700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:21.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.503+0000 7f8272c00700 1 -- 172.21.15.27:0/4090719283 shutdown_connections 2024-04-26T22:46:21.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:20.503+0000 7f8272c00700 1 -- 172.21.15.27:0/4090719283 wait complete. 2024-04-26T22:46:21.292 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for the mgr to restart... 2024-04-26T22:46:21.293 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr epoch 5... 2024-04-26T22:46:21.949 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:21 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:21.718+0000 7f8007be9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T22:46:22.364 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:21 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:21.959+0000 7f8007be9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T22:46:22.365 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:22 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:22.182+0000 7f8007be9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T22:46:22.864 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:22 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:22.577+0000 7f8007be9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T22:46:22.865 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:22 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:22.705+0000 7f8007be9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T22:46:23.230 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:22 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:22.967+0000 7f8007be9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T22:46:23.231 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:23 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:23.229+0000 7f8007be9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:46:24.364 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:24 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:24.052+0000 7f8007be9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T22:46:24.984 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:24 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:24.623+0000 7f8007be9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T22:46:24.984 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:24 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:24.756+0000 7f8007be9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T22:46:25.364 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:25 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:25.031+0000 7f8007be9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T22:46:26.114 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:25 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:25.706+0000 7f8007be9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T22:46:26.115 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:25 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:25.834+0000 7f8007be9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T22:46:26.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:26 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:26.169+0000 7f8007be9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T22:46:26.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:26 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:26.352+0000 7f8007be9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T22:46:26.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:26 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:26.483+0000 7f8007be9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T22:46:26.864 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:26 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:26.640+0000 7f8007be9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T22:46:27.364 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:27 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:27.022+0000 7f8007be9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T22:46:28.104 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:27 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:27.809+0000 7f8007be9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T22:46:28.364 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:28 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:28.103+0000 7f8007be9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T22:46:28.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:28 smithi027 ceph-mon[28034]: Active manager daemon x restarted 2024-04-26T22:46:28.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:28 smithi027 ceph-mon[28034]: Activating manager daemon x 2024-04-26T22:46:28.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:28 smithi027 ceph-mon[28034]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T22:46:28.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:28 smithi027 ceph-mon[28034]: mgrmap e6: x(active, starting, since 0.00935165s) 2024-04-26T22:46:28.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:28 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:46:28.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:28 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T22:46:28.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:28 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T22:46:28.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:28 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T22:46:28.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:28 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T22:46:28.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:28 smithi027 ceph-mon[28034]: Manager daemon x is now available 2024-04-26T22:46:28.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:28 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' 2024-04-26T22:46:29.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:29 smithi027 ceph-mon[28034]: Found migration_current of "None". Setting to last migration. 2024-04-26T22:46:29.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:29 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' 2024-04-26T22:46:29.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:29 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:46:29.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:29 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:46:29.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:29 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T22:46:29.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:29 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T22:46:29.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:29 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' 2024-04-26T22:46:29.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:29 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' 2024-04-26T22:46:29.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:29 smithi027 ceph-mon[28034]: mgrmap e7: x(active, since 1.01759s) 2024-04-26T22:46:29.859 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-04-26T22:46:29.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T22:46:29.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T22:46:29.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-04-26T22:46:29.861 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.971+0000 7f6000a00700 1 Processor -- start 2024-04-26T22:46:29.861 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.971+0000 7f6000a00700 1 -- start start 2024-04-26T22:46:29.861 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.972+0000 7f6000a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ffc10b020 0x7f5ffc10b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:29.861 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.972+0000 7f6000a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc10ba10 con 0x7f5ffc10b020 2024-04-26T22:46:29.861 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.972+0000 7f5ff9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ffc10b020 0x7f5ffc10b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:29.862 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.972+0000 7f5ff9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ffc10b020 0x7f5ffc10b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36168/0 (socket says 172.21.15.27:36168) 2024-04-26T22:46:29.862 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.972+0000 7f5ff9e00700 1 -- 172.21.15.27:0/4169732080 learned_addr learned my addr 172.21.15.27:0/4169732080 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:29.863 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.973+0000 7f5ff9e00700 1 -- 172.21.15.27:0/4169732080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc10c220 con 0x7f5ffc10b020 2024-04-26T22:46:29.863 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.973+0000 7f5ff9e00700 1 --2- 172.21.15.27:0/4169732080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ffc10b020 0x7f5ffc10b440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0098a0 tx=0x7f5fec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c3af1a0f2d4d9a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:29.863 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.973+0000 7f5ff8a00700 1 -- 172.21.15.27:0/4169732080 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fec004160 con 0x7f5ffc10b020 2024-04-26T22:46:29.863 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.973+0000 7f5ff8a00700 1 -- 172.21.15.27:0/4169732080 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5fec004300 con 0x7f5ffc10b020 2024-04-26T22:46:29.863 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.973+0000 7f5ff8a00700 1 -- 172.21.15.27:0/4169732080 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fec011af0 con 0x7f5ffc10b020 2024-04-26T22:46:29.864 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.974+0000 7f6000a00700 1 -- 172.21.15.27:0/4169732080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ffc10b020 msgr2=0x7f5ffc10b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:29.864 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.974+0000 7f6000a00700 1 --2- 172.21.15.27:0/4169732080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ffc10b020 0x7f5ffc10b440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0098a0 tx=0x7f5fec009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:29.864 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.974+0000 7f6000a00700 1 -- 172.21.15.27:0/4169732080 shutdown_connections 2024-04-26T22:46:29.864 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.974+0000 7f6000a00700 1 --2- 172.21.15.27:0/4169732080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ffc10b020 0x7f5ffc10b440 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:29.864 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.974+0000 7f6000a00700 1 -- 172.21.15.27:0/4169732080 >> 172.21.15.27:0/4169732080 conn(0x7f5ffc07e430 msgr2=0x7f5ffc07e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:29.864 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.974+0000 7f6000a00700 1 -- 172.21.15.27:0/4169732080 shutdown_connections 2024-04-26T22:46:29.865 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.975+0000 7f6000a00700 1 -- 172.21.15.27:0/4169732080 wait complete. 2024-04-26T22:46:29.865 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.975+0000 7f6000a00700 1 Processor -- start 2024-04-26T22:46:29.865 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.976+0000 7f6000a00700 1 -- start start 2024-04-26T22:46:29.865 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.976+0000 7f6000a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ffc10b020 0x7f5ffc19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:29.867 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.976+0000 7f6000a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc10f5d0 con 0x7f5ffc10b020 2024-04-26T22:46:29.867 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.976+0000 7f5ff9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ffc10b020 0x7f5ffc19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:29.867 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.976+0000 7f5ff9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ffc10b020 0x7f5ffc19efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36172/0 (socket says 172.21.15.27:36172) 2024-04-26T22:46:29.867 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.976+0000 7f5ff9e00700 1 -- 172.21.15.27:0/1472488540 learned_addr learned my addr 172.21.15.27:0/1472488540 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:29.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.977+0000 7f5ff9e00700 1 -- 172.21.15.27:0/1472488540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc19f4f0 con 0x7f5ffc10b020 2024-04-26T22:46:29.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.977+0000 7f5ff9e00700 1 --2- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ffc10b020 0x7f5ffc19efb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0117b0 tx=0x7f5fec011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:29.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.977+0000 7f5ff2a00700 1 -- 172.21.15.27:0/1472488540 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fec003af0 con 0x7f5ffc10b020 2024-04-26T22:46:29.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.977+0000 7f5ff2a00700 1 -- 172.21.15.27:0/1472488540 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5fec020070 con 0x7f5ffc10b020 2024-04-26T22:46:29.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.977+0000 7f6000a00700 1 -- 172.21.15.27:0/1472488540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ffc19f780 con 0x7f5ffc10b020 2024-04-26T22:46:29.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.977+0000 7f5ff2a00700 1 -- 172.21.15.27:0/1472488540 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fec019630 con 0x7f5ffc10b020 2024-04-26T22:46:29.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.977+0000 7f6000a00700 1 -- 172.21.15.27:0/1472488540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ffc1a2470 con 0x7f5ffc10b020 2024-04-26T22:46:29.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.979+0000 7f5ff2a00700 1 -- 172.21.15.27:0/1472488540 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f5fec01a430 con 0x7f5ffc10b020 2024-04-26T22:46:29.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.979+0000 7f5ff2a00700 1 --2- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f5fe0039600 0x7f5fe003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:29.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.979+0000 7f5ff2a00700 1 -- 172.21.15.27:0/1472488540 --> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5fe003c1d0 con 0x7f5fe0039600 2024-04-26T22:46:29.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.979+0000 7f5ff2a00700 1 -- 172.21.15.27:0/1472488540 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5fec04db00 con 0x7f5ffc10b020 2024-04-26T22:46:29.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.979+0000 7f5ff9400700 1 -- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f5fe0039600 msgr2=0x7f5fe003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1521074971 2024-04-26T22:46:29.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:21.979+0000 7f5ff9400700 1 --2- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f5fe0039600 0x7f5fe003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T22:46:29.869 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:22.180+0000 7f5ff9400700 1 -- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f5fe0039600 msgr2=0x7f5fe003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1521074971 2024-04-26T22:46:29.869 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:22.180+0000 7f5ff9400700 1 --2- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f5fe0039600 0x7f5fe003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T22:46:29.871 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:22.581+0000 7f5ff9400700 1 -- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f5fe0039600 msgr2=0x7f5fe003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1521074971 2024-04-26T22:46:29.871 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:22.581+0000 7f5ff9400700 1 --2- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f5fe0039600 0x7f5fe003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T22:46:29.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:23.381+0000 7f5ff9400700 1 -- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f5fe0039600 msgr2=0x7f5fe003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1521074971 2024-04-26T22:46:29.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:23.381+0000 7f5ff9400700 1 --2- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f5fe0039600 0x7f5fe003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T22:46:29.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:24.983+0000 7f5ff9400700 1 -- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f5fe0039600 msgr2=0x7f5fe003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1521074971 2024-04-26T22:46:29.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:24.983+0000 7f5ff9400700 1 --2- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f5fe0039600 0x7f5fe003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T22:46:29.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:28.113+0000 7f5ff2a00700 1 -- 172.21.15.27:0/1472488540 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f5fec0198a0 con 0x7f5ffc10b020 2024-04-26T22:46:29.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:28.113+0000 7f5ff2a00700 1 -- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f5fe0039600 msgr2=0x7f5fe003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:46:29.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:28.113+0000 7f5ff2a00700 1 --2- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f5fe0039600 0x7f5fe003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:29.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.118+0000 7f5ff2a00700 1 -- 172.21.15.27:0/1472488540 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f5fec04f660 con 0x7f5ffc10b020 2024-04-26T22:46:29.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.118+0000 7f5ff2a00700 1 --2- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5fe003cef0 0x7f5fe003f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:29.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.118+0000 7f5ff2a00700 1 -- 172.21.15.27:0/1472488540 --> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5fec01cec0 con 0x7f5fe003cef0 2024-04-26T22:46:29.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.120+0000 7f5ff9400700 1 --2- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5fe003cef0 0x7f5fe003f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:29.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.121+0000 7f5ff9400700 1 --2- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5fe003cef0 0x7f5fe003f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4003a10 tx=0x7f5fe40092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:29.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.123+0000 7f5ff2a00700 1 -- 172.21.15.27:0/1472488540 <== mgr.14120 v2:172.21.15.27:6800/1661629783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f5fec01cec0 con 0x7f5fe003cef0 2024-04-26T22:46:29.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.132+0000 7f6000a00700 1 -- 172.21.15.27:0/1472488540 --> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5ffc064a80 con 0x7f5fe003cef0 2024-04-26T22:46:29.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.133+0000 7f5ff2a00700 1 -- 172.21.15.27:0/1472488540 <== mgr.14120 v2:172.21.15.27:6800/1661629783 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f5ffc064a80 con 0x7f5fe003cef0 2024-04-26T22:46:29.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.133+0000 7f6000a00700 1 -- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5fe003cef0 msgr2=0x7f5fe003f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:29.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.133+0000 7f6000a00700 1 --2- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5fe003cef0 0x7f5fe003f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4003a10 tx=0x7f5fe40092b0 comp rx=0 tx=0).stop 2024-04-26T22:46:29.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.133+0000 7f6000a00700 1 -- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ffc10b020 msgr2=0x7f5ffc19efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:29.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.133+0000 7f6000a00700 1 --2- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ffc10b020 0x7f5ffc19efb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0117b0 tx=0x7f5fec011b70 comp rx=0 tx=0).stop 2024-04-26T22:46:29.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.134+0000 7f6000a00700 1 -- 172.21.15.27:0/1472488540 shutdown_connections 2024-04-26T22:46:29.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.134+0000 7f6000a00700 1 --2- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5fe003cef0 0x7f5fe003f2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:29.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.134+0000 7f6000a00700 1 --2- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ffc10b020 0x7f5ffc19efb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:29.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.134+0000 7f6000a00700 1 --2- 172.21.15.27:0/1472488540 >> [v2:172.21.15.27:6800/1521074971,v1:172.21.15.27:6801/1521074971] conn(0x7f5fe0039600 0x7f5fe003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:29.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.134+0000 7f6000a00700 1 -- 172.21.15.27:0/1472488540 >> 172.21.15.27:0/1472488540 conn(0x7f5ffc07e430 msgr2=0x7f5ffc108500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:29.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.134+0000 7f6000a00700 1 -- 172.21.15.27:0/1472488540 shutdown_connections 2024-04-26T22:46:29.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:29.134+0000 7f6000a00700 1 -- 172.21.15.27:0/1472488540 wait complete. 2024-04-26T22:46:29.874 INFO:teuthology.orchestra.run.smithi027.stdout:mgr epoch 5 is available 2024-04-26T22:46:29.874 INFO:teuthology.orchestra.run.smithi027.stdout:Setting orchestrator backend to cephadm... 2024-04-26T22:46:30.482 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:30 smithi027 ceph-mon[28034]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T22:46:30.482 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:30 smithi027 ceph-mon[28034]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T22:46:30.482 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:30 smithi027 ceph-mon[28034]: [26/Apr/2024:22:46:29] ENGINE Bus STARTING 2024-04-26T22:46:30.482 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:30 smithi027 ceph-mon[28034]: [26/Apr/2024:22:46:29] ENGINE Serving on https://172.21.15.27:7150 2024-04-26T22:46:30.482 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:30 smithi027 ceph-mon[28034]: [26/Apr/2024:22:46:29] ENGINE Serving on http://172.21.15.27:8765 2024-04-26T22:46:30.482 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:30 smithi027 ceph-mon[28034]: [26/Apr/2024:22:46:29] ENGINE Bus STARTED 2024-04-26T22:46:30.482 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:30 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:46:31.469 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:31 smithi027 ceph-mon[28034]: mgrmap e8: x(active, since 2s) 2024-04-26T22:46:31.470 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:31 smithi027 ceph-mon[28034]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:46:31.470 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:31 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' 2024-04-26T22:46:31.470 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:31 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:46:31.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.531+0000 7f3f3d200700 1 Processor -- start 2024-04-26T22:46:31.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.532+0000 7f3f3d200700 1 -- start start 2024-04-26T22:46:31.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.532+0000 7f3f3d200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b1a0 0x7f3f3810b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:31.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.532+0000 7f3f3d200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3810bb90 con 0x7f3f3810b1a0 2024-04-26T22:46:31.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.532+0000 7f3f37400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b1a0 0x7f3f3810b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:31.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.533+0000 7f3f37400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b1a0 0x7f3f3810b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36596/0 (socket says 172.21.15.27:36596) 2024-04-26T22:46:31.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.533+0000 7f3f37400700 1 -- 172.21.15.27:0/422148332 learned_addr learned my addr 172.21.15.27:0/422148332 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:31.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.533+0000 7f3f37400700 1 -- 172.21.15.27:0/422148332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3810bfd0 con 0x7f3f3810b1a0 2024-04-26T22:46:31.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.533+0000 7f3f37400700 1 --2- 172.21.15.27:0/422148332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b1a0 0x7f3f3810b5c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3f280098a0 tx=0x7f3f28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1eab627094c9cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:31.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.534+0000 7f3f36a00700 1 -- 172.21.15.27:0/422148332 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f28004160 con 0x7f3f3810b1a0 2024-04-26T22:46:31.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.534+0000 7f3f36a00700 1 -- 172.21.15.27:0/422148332 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3f28004300 con 0x7f3f3810b1a0 2024-04-26T22:46:31.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.534+0000 7f3f36a00700 1 -- 172.21.15.27:0/422148332 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f28011af0 con 0x7f3f3810b1a0 2024-04-26T22:46:31.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.534+0000 7f3f3d200700 1 -- 172.21.15.27:0/422148332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b1a0 msgr2=0x7f3f3810b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:31.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.534+0000 7f3f3d200700 1 --2- 172.21.15.27:0/422148332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b1a0 0x7f3f3810b5c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3f280098a0 tx=0x7f3f28009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:31.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.534+0000 7f3f3d200700 1 -- 172.21.15.27:0/422148332 shutdown_connections 2024-04-26T22:46:31.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.534+0000 7f3f3d200700 1 --2- 172.21.15.27:0/422148332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b1a0 0x7f3f3810b5c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:31.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.534+0000 7f3f3d200700 1 -- 172.21.15.27:0/422148332 >> 172.21.15.27:0/422148332 conn(0x7f3f3807e1b0 msgr2=0x7f3f3807e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:31.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.535+0000 7f3f3d200700 1 -- 172.21.15.27:0/422148332 shutdown_connections 2024-04-26T22:46:31.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.535+0000 7f3f3d200700 1 -- 172.21.15.27:0/422148332 wait complete. 2024-04-26T22:46:31.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.536+0000 7f3f3d200700 1 Processor -- start 2024-04-26T22:46:31.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.536+0000 7f3f3d200700 1 -- start start 2024-04-26T22:46:31.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.536+0000 7f3f3d200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b1a0 0x7f3f3819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:31.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.536+0000 7f3f3d200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3810f380 con 0x7f3f3810b1a0 2024-04-26T22:46:31.494 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.536+0000 7f3f37400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b1a0 0x7f3f3819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:31.494 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.536+0000 7f3f37400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b1a0 0x7f3f3819ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36600/0 (socket says 172.21.15.27:36600) 2024-04-26T22:46:31.494 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.536+0000 7f3f37400700 1 -- 172.21.15.27:0/173679941 learned_addr learned my addr 172.21.15.27:0/173679941 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:31.494 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.537+0000 7f3f37400700 1 -- 172.21.15.27:0/173679941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3819f260 con 0x7f3f3810b1a0 2024-04-26T22:46:31.494 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.537+0000 7f3f37400700 1 --2- 172.21.15.27:0/173679941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b1a0 0x7f3f3819ed20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3f28011ac0 tx=0x7f3f28011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:31.495 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.537+0000 7f3f34c00700 1 -- 172.21.15.27:0/173679941 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f28003b20 con 0x7f3f3810b1a0 2024-04-26T22:46:31.495 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.537+0000 7f3f34c00700 1 -- 172.21.15.27:0/173679941 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3f2801a8c0 con 0x7f3f3810b1a0 2024-04-26T22:46:31.495 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.537+0000 7f3f34c00700 1 -- 172.21.15.27:0/173679941 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f280248d0 con 0x7f3f3810b1a0 2024-04-26T22:46:31.495 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.537+0000 7f3f3d200700 1 -- 172.21.15.27:0/173679941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3819f4f0 con 0x7f3f3810b1a0 2024-04-26T22:46:31.495 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.538+0000 7f3f3d200700 1 -- 172.21.15.27:0/173679941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f381a21d0 con 0x7f3f3810b1a0 2024-04-26T22:46:31.495 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.539+0000 7f3f3d200700 1 -- 172.21.15.27:0/173679941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f3810f500 con 0x7f3f3810b1a0 2024-04-26T22:46:31.496 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.539+0000 7f3f34c00700 1 -- 172.21.15.27:0/173679941 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3f28022020 con 0x7f3f3810b1a0 2024-04-26T22:46:31.496 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.539+0000 7f3f34c00700 1 --2- 172.21.15.27:0/173679941 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f3f1c039600 0x7f3f1c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:31.496 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.540+0000 7f3f34c00700 1 -- 172.21.15.27:0/173679941 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3f2804d8e0 con 0x7f3f3810b1a0 2024-04-26T22:46:31.496 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.540+0000 7f3f2fe00700 1 --2- 172.21.15.27:0/173679941 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f3f1c039600 0x7f3f1c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:31.496 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.541+0000 7f3f2fe00700 1 --2- 172.21.15.27:0/173679941 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f3f1c039600 0x7f3f1c03bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3f20006fd0 tx=0x7f3f20006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:31.496 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.546+0000 7f3f34c00700 1 -- 172.21.15.27:0/173679941 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3f2801ab20 con 0x7f3f3810b1a0 2024-04-26T22:46:31.497 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.787+0000 7f3f3d200700 1 -- 172.21.15.27:0/173679941 --> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f3f381093e0 con 0x7f3f1c039600 2024-04-26T22:46:31.497 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.803+0000 7f3f34c00700 1 -- 172.21.15.27:0/173679941 <== mgr.14120 v2:172.21.15.27:6800/1661629783 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3f381093e0 con 0x7f3f1c039600 2024-04-26T22:46:31.497 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.806+0000 7f3f3d200700 1 -- 172.21.15.27:0/173679941 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f3f1c039600 msgr2=0x7f3f1c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:31.497 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.806+0000 7f3f3d200700 1 --2- 172.21.15.27:0/173679941 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f3f1c039600 0x7f3f1c03bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3f20006fd0 tx=0x7f3f20006e40 comp rx=0 tx=0).stop 2024-04-26T22:46:31.497 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.806+0000 7f3f3d200700 1 -- 172.21.15.27:0/173679941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b1a0 msgr2=0x7f3f3819ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:31.497 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.806+0000 7f3f3d200700 1 --2- 172.21.15.27:0/173679941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b1a0 0x7f3f3819ed20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3f28011ac0 tx=0x7f3f28011ba0 comp rx=0 tx=0).stop 2024-04-26T22:46:31.498 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.806+0000 7f3f3d200700 1 -- 172.21.15.27:0/173679941 shutdown_connections 2024-04-26T22:46:31.498 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.806+0000 7f3f3d200700 1 --2- 172.21.15.27:0/173679941 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f3f1c039600 0x7f3f1c03bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:31.498 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.806+0000 7f3f3d200700 1 --2- 172.21.15.27:0/173679941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b1a0 0x7f3f3819ed20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:31.498 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.806+0000 7f3f3d200700 1 -- 172.21.15.27:0/173679941 >> 172.21.15.27:0/173679941 conn(0x7f3f3807e1b0 msgr2=0x7f3f38108cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:31.499 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.806+0000 7f3f3d200700 1 -- 172.21.15.27:0/173679941 shutdown_connections 2024-04-26T22:46:31.499 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:30.806+0000 7f3f3d200700 1 -- 172.21.15.27:0/173679941 wait complete. 2024-04-26T22:46:33.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:33 smithi027 ceph-mon[28034]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:46:33.164 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T22:46:33.165 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.141+0000 7f713e400700 1 Processor -- start 2024-04-26T22:46:33.165 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.142+0000 7f713e400700 1 -- start start 2024-04-26T22:46:33.165 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.142+0000 7f713e400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7138109900 0x7f713810bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:33.165 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.142+0000 7f713e400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f713807ce10 con 0x7f7138109900 2024-04-26T22:46:33.165 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.142+0000 7f7137400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7138109900 0x7f713810bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:33.165 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.142+0000 7f7137400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7138109900 0x7f713810bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36614/0 (socket says 172.21.15.27:36614) 2024-04-26T22:46:33.165 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.142+0000 7f7137400700 1 -- 172.21.15.27:0/4095175861 learned_addr learned my addr 172.21.15.27:0/4095175861 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:33.165 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.143+0000 7f7137400700 1 -- 172.21.15.27:0/4095175861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713810c230 con 0x7f7138109900 2024-04-26T22:46:33.166 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.143+0000 7f7137400700 1 --2- 172.21.15.27:0/4095175861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7138109900 0x7f713810bcf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f71280098a0 tx=0x7f7128009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe539da5ce0fab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:33.170 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.143+0000 7f7136000700 1 -- 172.21.15.27:0/4095175861 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7128004160 con 0x7f7138109900 2024-04-26T22:46:33.170 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.144+0000 7f7136000700 1 -- 172.21.15.27:0/4095175861 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7128004300 con 0x7f7138109900 2024-04-26T22:46:33.170 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.144+0000 7f7136000700 1 -- 172.21.15.27:0/4095175861 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7128011af0 con 0x7f7138109900 2024-04-26T22:46:33.170 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.144+0000 7f713e400700 1 -- 172.21.15.27:0/4095175861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7138109900 msgr2=0x7f713810bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:33.170 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.144+0000 7f713e400700 1 --2- 172.21.15.27:0/4095175861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7138109900 0x7f713810bcf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f71280098a0 tx=0x7f7128009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:33.170 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.144+0000 7f713e400700 1 -- 172.21.15.27:0/4095175861 shutdown_connections 2024-04-26T22:46:33.170 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.144+0000 7f713e400700 1 --2- 172.21.15.27:0/4095175861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7138109900 0x7f713810bcf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:33.170 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.144+0000 7f713e400700 1 -- 172.21.15.27:0/4095175861 >> 172.21.15.27:0/4095175861 conn(0x7f7138103290 msgr2=0x7f7138105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:33.170 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.145+0000 7f713e400700 1 -- 172.21.15.27:0/4095175861 shutdown_connections 2024-04-26T22:46:33.170 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.145+0000 7f713e400700 1 -- 172.21.15.27:0/4095175861 wait complete. 2024-04-26T22:46:33.171 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.146+0000 7f713e400700 1 Processor -- start 2024-04-26T22:46:33.171 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.146+0000 7f713e400700 1 -- start start 2024-04-26T22:46:33.171 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.146+0000 7f713e400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7138109900 0x7f7138109180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:33.171 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.146+0000 7f713e400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f713810d1a0 con 0x7f7138109900 2024-04-26T22:46:33.171 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.146+0000 7f7137400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7138109900 0x7f7138109180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:33.171 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.147+0000 7f7137400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7138109900 0x7f7138109180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36624/0 (socket says 172.21.15.27:36624) 2024-04-26T22:46:33.171 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.147+0000 7f7137400700 1 -- 172.21.15.27:0/1351651218 learned_addr learned my addr 172.21.15.27:0/1351651218 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:33.171 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.147+0000 7f7137400700 1 -- 172.21.15.27:0/1351651218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71381096c0 con 0x7f7138109900 2024-04-26T22:46:33.171 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.147+0000 7f7137400700 1 --2- 172.21.15.27:0/1351651218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7138109900 0x7f7138109180 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7128011ac0 tx=0x7f7128011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:33.171 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.148+0000 7f712fe00700 1 -- 172.21.15.27:0/1351651218 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7128003970 con 0x7f7138109900 2024-04-26T22:46:33.171 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.148+0000 7f712fe00700 1 -- 172.21.15.27:0/1351651218 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7128003b10 con 0x7f7138109900 2024-04-26T22:46:33.171 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.148+0000 7f712fe00700 1 -- 172.21.15.27:0/1351651218 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7128024610 con 0x7f7138109900 2024-04-26T22:46:33.171 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.148+0000 7f713e400700 1 -- 172.21.15.27:0/1351651218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71381078b0 con 0x7f7138109900 2024-04-26T22:46:33.172 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.148+0000 7f713e400700 1 -- 172.21.15.27:0/1351651218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7138107df0 con 0x7f7138109900 2024-04-26T22:46:33.172 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.149+0000 7f713e400700 1 -- 172.21.15.27:0/1351651218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7138064a80 con 0x7f7138109900 2024-04-26T22:46:33.172 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.150+0000 7f712fe00700 1 -- 172.21.15.27:0/1351651218 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f712801a9e0 con 0x7f7138109900 2024-04-26T22:46:33.172 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.150+0000 7f712fe00700 1 --2- 172.21.15.27:0/1351651218 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f711c039600 0x7f711c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:33.172 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.150+0000 7f712fe00700 1 -- 172.21.15.27:0/1351651218 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f712804da40 con 0x7f7138109900 2024-04-26T22:46:33.172 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.150+0000 7f7136a00700 1 --2- 172.21.15.27:0/1351651218 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f711c039600 0x7f711c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:33.172 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.151+0000 7f7136a00700 1 --2- 172.21.15.27:0/1351651218 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f711c039600 0x7f711c03bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7124006fd0 tx=0x7f7124006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:33.172 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.156+0000 7f712fe00700 1 -- 172.21.15.27:0/1351651218 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7128029060 con 0x7f7138109900 2024-04-26T22:46:33.172 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.393+0000 7f713e400700 1 -- 172.21.15.27:0/1351651218 --> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f7138108600 con 0x7f711c039600 2024-04-26T22:46:33.172 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.394+0000 7f712fe00700 1 -- 172.21.15.27:0/1351651218 <== mgr.14120 v2:172.21.15.27:6800/1661629783 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f7138108600 con 0x7f711c039600 2024-04-26T22:46:33.172 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.397+0000 7f713e400700 1 -- 172.21.15.27:0/1351651218 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f711c039600 msgr2=0x7f711c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:33.172 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.397+0000 7f713e400700 1 --2- 172.21.15.27:0/1351651218 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f711c039600 0x7f711c03bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7124006fd0 tx=0x7f7124006e40 comp rx=0 tx=0).stop 2024-04-26T22:46:33.172 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.397+0000 7f713e400700 1 -- 172.21.15.27:0/1351651218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7138109900 msgr2=0x7f7138109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:33.173 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.397+0000 7f713e400700 1 --2- 172.21.15.27:0/1351651218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7138109900 0x7f7138109180 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7128011ac0 tx=0x7f7128011ba0 comp rx=0 tx=0).stop 2024-04-26T22:46:33.174 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.397+0000 7f713e400700 1 -- 172.21.15.27:0/1351651218 shutdown_connections 2024-04-26T22:46:33.174 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.397+0000 7f713e400700 1 --2- 172.21.15.27:0/1351651218 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f711c039600 0x7f711c03bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:33.174 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.397+0000 7f713e400700 1 --2- 172.21.15.27:0/1351651218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7138109900 0x7f7138109180 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:33.174 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.397+0000 7f713e400700 1 -- 172.21.15.27:0/1351651218 >> 172.21.15.27:0/1351651218 conn(0x7f7138103290 msgr2=0x7f7138103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:33.174 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.398+0000 7f713e400700 1 -- 172.21.15.27:0/1351651218 shutdown_connections 2024-04-26T22:46:33.174 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:32.398+0000 7f713e400700 1 -- 172.21.15.27:0/1351651218 wait complete. 2024-04-26T22:46:33.174 INFO:teuthology.orchestra.run.smithi027.stdout:Generating ssh key... 2024-04-26T22:46:34.616 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: Generating public/private rsa key pair. 2024-04-26T22:46:34.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: Your identification has been saved in /tmp/tmprz9d5dtb/key. 2024-04-26T22:46:34.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: Your public key has been saved in /tmp/tmprz9d5dtb/key.pub. 2024-04-26T22:46:34.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: The key fingerprint is: 2024-04-26T22:46:34.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: SHA256:Ildt7P9x7TKhoQHI70/d3GopA6H6uKVn64b8s6EReo0 ceph-95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:46:34.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: The key's randomart image is: 2024-04-26T22:46:34.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: +---[RSA 3072]----+ 2024-04-26T22:46:34.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: | | 2024-04-26T22:46:34.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: | o | 2024-04-26T22:46:34.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: | . .. + | 2024-04-26T22:46:34.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: | o..o. | 2024-04-26T22:46:34.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: | . +.So.. | 2024-04-26T22:46:34.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: | + =o oo.o...| 2024-04-26T22:46:34.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: | ..E++ .+oo+o+| 2024-04-26T22:46:34.618 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: | .+**o. +.== | 2024-04-26T22:46:34.618 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: | =O*=. +oo.| 2024-04-26T22:46:34.618 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: +----[SHA256]-----+ 2024-04-26T22:46:35.087 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.791+0000 7fbd8e600700 1 Processor -- start 2024-04-26T22:46:35.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.791+0000 7fbd8e600700 1 -- start start 2024-04-26T22:46:35.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.791+0000 7fbd8e600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd8807f730 0x7fbd8807db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:35.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.791+0000 7fbd8e600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8807e160 con 0x7fbd8807f730 2024-04-26T22:46:35.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.792+0000 7fbd87400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd8807f730 0x7fbd8807db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:35.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.792+0000 7fbd87400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd8807f730 0x7fbd8807db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36628/0 (socket says 172.21.15.27:36628) 2024-04-26T22:46:35.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.792+0000 7fbd87400700 1 -- 172.21.15.27:0/2191514489 learned_addr learned my addr 172.21.15.27:0/2191514489 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:35.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.793+0000 7fbd87400700 1 -- 172.21.15.27:0/2191514489 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8807e2e0 con 0x7fbd8807f730 2024-04-26T22:46:35.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.793+0000 7fbd87400700 1 --2- 172.21.15.27:0/2191514489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd8807f730 0x7fbd8807db90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbd780098a0 tx=0x7fbd78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65cf7b529a3f128 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:35.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.793+0000 7fbd86a00700 1 -- 172.21.15.27:0/2191514489 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd78004160 con 0x7fbd8807f730 2024-04-26T22:46:35.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.793+0000 7fbd86a00700 1 -- 172.21.15.27:0/2191514489 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd78004300 con 0x7fbd8807f730 2024-04-26T22:46:35.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.793+0000 7fbd86a00700 1 -- 172.21.15.27:0/2191514489 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd78011af0 con 0x7fbd8807f730 2024-04-26T22:46:35.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.794+0000 7fbd8e600700 1 -- 172.21.15.27:0/2191514489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd8807f730 msgr2=0x7fbd8807db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:35.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.794+0000 7fbd8e600700 1 --2- 172.21.15.27:0/2191514489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd8807f730 0x7fbd8807db90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbd780098a0 tx=0x7fbd78009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:35.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.794+0000 7fbd8e600700 1 -- 172.21.15.27:0/2191514489 shutdown_connections 2024-04-26T22:46:35.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.794+0000 7fbd8e600700 1 --2- 172.21.15.27:0/2191514489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd8807f730 0x7fbd8807db90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:35.094 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.794+0000 7fbd8e600700 1 -- 172.21.15.27:0/2191514489 >> 172.21.15.27:0/2191514489 conn(0x7fbd88104380 msgr2=0x7fbd881067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:35.094 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.794+0000 7fbd8e600700 1 -- 172.21.15.27:0/2191514489 shutdown_connections 2024-04-26T22:46:35.094 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.794+0000 7fbd8e600700 1 -- 172.21.15.27:0/2191514489 wait complete. 2024-04-26T22:46:35.094 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.795+0000 7fbd8e600700 1 Processor -- start 2024-04-26T22:46:35.094 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.796+0000 7fbd8e600700 1 -- start start 2024-04-26T22:46:35.094 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.796+0000 7fbd8e600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd8807f730 0x7fbd881a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:35.094 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.796+0000 7fbd8e600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8810afb0 con 0x7fbd8807f730 2024-04-26T22:46:35.094 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.796+0000 7fbd87400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd8807f730 0x7fbd881a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:35.094 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.796+0000 7fbd87400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd8807f730 0x7fbd881a33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36644/0 (socket says 172.21.15.27:36644) 2024-04-26T22:46:35.094 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.796+0000 7fbd87400700 1 -- 172.21.15.27:0/4268633222 learned_addr learned my addr 172.21.15.27:0/4268633222 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:35.094 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.797+0000 7fbd87400700 1 -- 172.21.15.27:0/4268633222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd881a3900 con 0x7fbd8807f730 2024-04-26T22:46:35.094 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.797+0000 7fbd87400700 1 --2- 172.21.15.27:0/4268633222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd8807f730 0x7fbd881a33c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbd780117b0 tx=0x7fbd78011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:35.094 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.797+0000 7fbd84c00700 1 -- 172.21.15.27:0/4268633222 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd78003af0 con 0x7fbd8807f730 2024-04-26T22:46:35.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.797+0000 7fbd84c00700 1 -- 172.21.15.27:0/4268633222 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd78020070 con 0x7fbd8807f730 2024-04-26T22:46:35.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.797+0000 7fbd8e600700 1 -- 172.21.15.27:0/4268633222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd881a3b90 con 0x7fbd8807f730 2024-04-26T22:46:35.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.797+0000 7fbd84c00700 1 -- 172.21.15.27:0/4268633222 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd78019630 con 0x7fbd8807f730 2024-04-26T22:46:35.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.797+0000 7fbd8e600700 1 -- 172.21.15.27:0/4268633222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd881a6880 con 0x7fbd8807f730 2024-04-26T22:46:35.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.799+0000 7fbd84c00700 1 -- 172.21.15.27:0/4268633222 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fbd7801a430 con 0x7fbd8807f730 2024-04-26T22:46:35.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.799+0000 7fbd8e600700 1 -- 172.21.15.27:0/4268633222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd88064a80 con 0x7fbd8807f730 2024-04-26T22:46:35.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.799+0000 7fbd84c00700 1 --2- 172.21.15.27:0/4268633222 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fbd6c039600 0x7fbd6c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:35.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.799+0000 7fbd84c00700 1 -- 172.21.15.27:0/4268633222 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbd7804db90 con 0x7fbd8807f730 2024-04-26T22:46:35.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.799+0000 7fbd7ea00700 1 --2- 172.21.15.27:0/4268633222 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fbd6c039600 0x7fbd6c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:35.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.800+0000 7fbd7ea00700 1 --2- 172.21.15.27:0/4268633222 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fbd6c039600 0x7fbd6c03bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbd70006fd0 tx=0x7fbd70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:35.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:33.805+0000 7fbd84c00700 1 -- 172.21.15.27:0/4268633222 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbd7800e260 con 0x7fbd8807f730 2024-04-26T22:46:35.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:34.035+0000 7fbd8e600700 1 -- 172.21.15.27:0/4268633222 --> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fbd88063860 con 0x7fbd6c039600 2024-04-26T22:46:35.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:34.330+0000 7fbd84c00700 1 -- 172.21.15.27:0/4268633222 <== mgr.14120 v2:172.21.15.27:6800/1661629783 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbd88063860 con 0x7fbd6c039600 2024-04-26T22:46:35.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:34.333+0000 7fbd8e600700 1 -- 172.21.15.27:0/4268633222 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fbd6c039600 msgr2=0x7fbd6c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:35.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:34.333+0000 7fbd8e600700 1 --2- 172.21.15.27:0/4268633222 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fbd6c039600 0x7fbd6c03bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbd70006fd0 tx=0x7fbd70006e40 comp rx=0 tx=0).stop 2024-04-26T22:46:35.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:34.333+0000 7fbd8e600700 1 -- 172.21.15.27:0/4268633222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd8807f730 msgr2=0x7fbd881a33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:35.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:34.333+0000 7fbd8e600700 1 --2- 172.21.15.27:0/4268633222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd8807f730 0x7fbd881a33c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbd780117b0 tx=0x7fbd78011b70 comp rx=0 tx=0).stop 2024-04-26T22:46:35.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:34.333+0000 7fbd8e600700 1 -- 172.21.15.27:0/4268633222 shutdown_connections 2024-04-26T22:46:35.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:34.333+0000 7fbd8e600700 1 --2- 172.21.15.27:0/4268633222 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fbd6c039600 0x7fbd6c03bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:35.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:34.333+0000 7fbd8e600700 1 --2- 172.21.15.27:0/4268633222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd8807f730 0x7fbd881a33c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:35.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:34.333+0000 7fbd8e600700 1 -- 172.21.15.27:0/4268633222 >> 172.21.15.27:0/4268633222 conn(0x7fbd88104380 msgr2=0x7fbd881067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:35.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:34.333+0000 7fbd8e600700 1 -- 172.21.15.27:0/4268633222 shutdown_connections 2024-04-26T22:46:35.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:34.333+0000 7fbd8e600700 1 -- 172.21.15.27:0/4268633222 wait complete. 2024-04-26T22:46:35.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:35 smithi027 ceph-mon[28034]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:46:35.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:35 smithi027 ceph-mon[28034]: Generating ssh key... 2024-04-26T22:46:35.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:35 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' 2024-04-26T22:46:35.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:35 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' 2024-04-26T22:46:36.648 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbQiDkHDWminIjhAUMuY15Ev6aElnRBihvMMt7y17jYAqoTPcdwrjM4pCo3pej4H/tiNqvStwnqY6h1Zno7w9rDWKC1xlOHc1zKdfOLUM3jK8DAcsE2B9pPrdu3V8V6ghb8NJvzcxj3F0LRTpTp6qsPXPpEPEOr/Ub2C22pegckKCvuHkDEOMX1M3qP3WNqgaJP4iptXtpMc3Kk41mkDhifiYDOmgi/Hfa1SIi+DM2lrsuJ0eGUEG4XJrKd8FOXH1mFxOnLSBt1vtuKpE62WbgOToOtR07HVPAl0NAkZGlYxzzt/6GeYNmkJXT17kt+AFPZzXO3QgStWhxQLFtOcWQ3DfSdoVjg/gqP+AuDaFkytbvJnLx9TBghf6c6X6VLg7MfjVl+FFUEbllKPYM5EWiWx9/r1uYPbFWZhHa+7QQcaXLsfU3tiOHS0fhrCADbdKUg+biqsYj3Hoeg49tLwbX6gu3Gl2HmQJCzHDlnijiv4vHphkQdXVcn6tsTQAvL4U= ceph-95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:46:36.648 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.787+0000 7fb145200700 1 Processor -- start 2024-04-26T22:46:36.648 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.788+0000 7fb145200700 1 -- start start 2024-04-26T22:46:36.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.788+0000 7fb145200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb140107710 0x7fb140107b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:36.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.788+0000 7fb145200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb14007cdf0 con 0x7fb140107710 2024-04-26T22:46:36.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.789+0000 7fb13de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb140107710 0x7fb140107b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:36.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.789+0000 7fb13de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb140107710 0x7fb140107b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36648/0 (socket says 172.21.15.27:36648) 2024-04-26T22:46:36.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.789+0000 7fb13de00700 1 -- 172.21.15.27:0/1168741947 learned_addr learned my addr 172.21.15.27:0/1168741947 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:36.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.789+0000 7fb13de00700 1 -- 172.21.15.27:0/1168741947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb140108070 con 0x7fb140107710 2024-04-26T22:46:36.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.789+0000 7fb13de00700 1 --2- 172.21.15.27:0/1168741947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb140107710 0x7fb140107b30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb1300098a0 tx=0x7fb130009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaaeb3f0bc96d149 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:36.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.790+0000 7fb13ca00700 1 -- 172.21.15.27:0/1168741947 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb130004160 con 0x7fb140107710 2024-04-26T22:46:36.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.790+0000 7fb13ca00700 1 -- 172.21.15.27:0/1168741947 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb130004300 con 0x7fb140107710 2024-04-26T22:46:36.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.790+0000 7fb13ca00700 1 -- 172.21.15.27:0/1168741947 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb130011af0 con 0x7fb140107710 2024-04-26T22:46:36.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.790+0000 7fb145200700 1 -- 172.21.15.27:0/1168741947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb140107710 msgr2=0x7fb140107b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:36.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.790+0000 7fb145200700 1 --2- 172.21.15.27:0/1168741947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb140107710 0x7fb140107b30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb1300098a0 tx=0x7fb130009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:36.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.791+0000 7fb145200700 1 -- 172.21.15.27:0/1168741947 shutdown_connections 2024-04-26T22:46:36.652 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.791+0000 7fb145200700 1 --2- 172.21.15.27:0/1168741947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb140107710 0x7fb140107b30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:36.652 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.791+0000 7fb145200700 1 -- 172.21.15.27:0/1168741947 >> 172.21.15.27:0/1168741947 conn(0x7fb140103290 msgr2=0x7fb140105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:36.652 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.791+0000 7fb145200700 1 -- 172.21.15.27:0/1168741947 shutdown_connections 2024-04-26T22:46:36.652 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.791+0000 7fb145200700 1 -- 172.21.15.27:0/1168741947 wait complete. 2024-04-26T22:46:36.652 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.792+0000 7fb145200700 1 Processor -- start 2024-04-26T22:46:36.653 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.792+0000 7fb145200700 1 -- start start 2024-04-26T22:46:36.653 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.792+0000 7fb145200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb140107710 0x7fb14019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:36.653 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.792+0000 7fb145200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb1401090a0 con 0x7fb140107710 2024-04-26T22:46:36.653 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.793+0000 7fb13de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb140107710 0x7fb14019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:36.654 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.793+0000 7fb13de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb140107710 0x7fb14019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36660/0 (socket says 172.21.15.27:36660) 2024-04-26T22:46:36.654 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.793+0000 7fb13de00700 1 -- 172.21.15.27:0/1819586387 learned_addr learned my addr 172.21.15.27:0/1819586387 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:36.654 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.793+0000 7fb13de00700 1 -- 172.21.15.27:0/1819586387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14019f4d0 con 0x7fb140107710 2024-04-26T22:46:36.655 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.793+0000 7fb13de00700 1 --2- 172.21.15.27:0/1819586387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb140107710 0x7fb14019ef90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb130011ac0 tx=0x7fb130011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:36.655 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.794+0000 7fb136a00700 1 -- 172.21.15.27:0/1819586387 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb130003970 con 0x7fb140107710 2024-04-26T22:46:36.655 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.794+0000 7fb136a00700 1 -- 172.21.15.27:0/1819586387 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb130003b10 con 0x7fb140107710 2024-04-26T22:46:36.656 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.794+0000 7fb136a00700 1 -- 172.21.15.27:0/1819586387 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb130024610 con 0x7fb140107710 2024-04-26T22:46:36.656 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.794+0000 7fb145200700 1 -- 172.21.15.27:0/1819586387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14019f760 con 0x7fb140107710 2024-04-26T22:46:36.656 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.794+0000 7fb145200700 1 -- 172.21.15.27:0/1819586387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1401a2450 con 0x7fb140107710 2024-04-26T22:46:36.657 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.795+0000 7fb136a00700 1 -- 172.21.15.27:0/1819586387 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb13001a9e0 con 0x7fb140107710 2024-04-26T22:46:36.657 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.795+0000 7fb145200700 1 -- 172.21.15.27:0/1819586387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb140064a80 con 0x7fb140107710 2024-04-26T22:46:36.657 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.796+0000 7fb136a00700 1 --2- 172.21.15.27:0/1819586387 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fb124039600 0x7fb12403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:36.657 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.796+0000 7fb136a00700 1 -- 172.21.15.27:0/1819586387 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb13004d980 con 0x7fb140107710 2024-04-26T22:46:36.657 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.796+0000 7fb13d400700 1 --2- 172.21.15.27:0/1819586387 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fb124039600 0x7fb12403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:36.658 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.797+0000 7fb13d400700 1 --2- 172.21.15.27:0/1819586387 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fb124039600 0x7fb12403bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb12c006fd0 tx=0x7fb12c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:36.658 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:35.802+0000 7fb136a00700 1 -- 172.21.15.27:0/1819586387 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb13000b3e0 con 0x7fb140107710 2024-04-26T22:46:36.658 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:36.031+0000 7fb145200700 1 -- 172.21.15.27:0/1819586387 --> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fb1400008d0 con 0x7fb124039600 2024-04-26T22:46:36.658 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:36.033+0000 7fb136a00700 1 -- 172.21.15.27:0/1819586387 <== mgr.14120 v2:172.21.15.27:6800/1661629783 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fb1400008d0 con 0x7fb124039600 2024-04-26T22:46:36.658 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:36.036+0000 7fb145200700 1 -- 172.21.15.27:0/1819586387 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fb124039600 msgr2=0x7fb12403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:36.658 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:36.036+0000 7fb145200700 1 --2- 172.21.15.27:0/1819586387 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fb124039600 0x7fb12403bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb12c006fd0 tx=0x7fb12c006e40 comp rx=0 tx=0).stop 2024-04-26T22:46:36.659 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:36.036+0000 7fb145200700 1 -- 172.21.15.27:0/1819586387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb140107710 msgr2=0x7fb14019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:36.659 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:36.036+0000 7fb145200700 1 --2- 172.21.15.27:0/1819586387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb140107710 0x7fb14019ef90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb130011ac0 tx=0x7fb130011ba0 comp rx=0 tx=0).stop 2024-04-26T22:46:36.659 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:36.036+0000 7fb145200700 1 -- 172.21.15.27:0/1819586387 shutdown_connections 2024-04-26T22:46:36.659 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:36.036+0000 7fb145200700 1 --2- 172.21.15.27:0/1819586387 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fb124039600 0x7fb12403bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:36.659 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:36.036+0000 7fb145200700 1 --2- 172.21.15.27:0/1819586387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb140107710 0x7fb14019ef90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:36.660 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:36.036+0000 7fb145200700 1 -- 172.21.15.27:0/1819586387 >> 172.21.15.27:0/1819586387 conn(0x7fb140103290 msgr2=0x7fb140195b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:36.660 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:36.037+0000 7fb145200700 1 -- 172.21.15.27:0/1819586387 shutdown_connections 2024-04-26T22:46:36.660 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:36.037+0000 7fb145200700 1 -- 172.21.15.27:0/1819586387 wait complete. 2024-04-26T22:46:36.661 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T22:46:36.661 INFO:teuthology.orchestra.run.smithi027.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T22:46:36.679 INFO:teuthology.orchestra.run.smithi027.stdout:Adding host smithi027... 2024-04-26T22:46:37.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:37 smithi027 ceph-mon[28034]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:46:38.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:38 smithi027 ceph-mon[28034]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi027", "addr": "172.21.15.27", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:46:40.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:40 smithi027 ceph-mon[28034]: Deploying cephadm binary to smithi027 2024-04-26T22:46:44.720 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Added host 'smithi027' with addr '172.21.15.27' 2024-04-26T22:46:44.721 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.314+0000 7fab0b400700 1 Processor -- start 2024-04-26T22:46:44.721 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.315+0000 7fab0b400700 1 -- start start 2024-04-26T22:46:44.721 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.315+0000 7fab0b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab04108e00 0x7fab04109220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:44.721 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.315+0000 7fab0b400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab04109760 con 0x7fab04108e00 2024-04-26T22:46:44.721 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.316+0000 7fab08a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab04108e00 0x7fab04109220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:44.721 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.316+0000 7fab08a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab04108e00 0x7fab04109220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36664/0 (socket says 172.21.15.27:36664) 2024-04-26T22:46:44.721 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.316+0000 7fab08a00700 1 -- 172.21.15.27:0/4100563167 learned_addr learned my addr 172.21.15.27:0/4100563167 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:44.721 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.316+0000 7fab08a00700 1 -- 172.21.15.27:0/4100563167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab04109f60 con 0x7fab04108e00 2024-04-26T22:46:44.721 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.316+0000 7fab08a00700 1 --2- 172.21.15.27:0/4100563167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab04108e00 0x7fab04109220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009e80 tx=0x7faaf400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9828d18f5f0b699a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:44.721 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.317+0000 7fab03e00700 1 -- 172.21.15.27:0/4100563167 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faaf4017070 con 0x7fab04108e00 2024-04-26T22:46:44.721 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.317+0000 7fab03e00700 1 -- 172.21.15.27:0/4100563167 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faaf400b8c0 con 0x7fab04108e00 2024-04-26T22:46:44.722 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.317+0000 7fab03e00700 1 -- 172.21.15.27:0/4100563167 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faaf4004490 con 0x7fab04108e00 2024-04-26T22:46:44.722 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.317+0000 7fab0b400700 1 -- 172.21.15.27:0/4100563167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab04108e00 msgr2=0x7fab04109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:44.722 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.317+0000 7fab0b400700 1 --2- 172.21.15.27:0/4100563167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab04108e00 0x7fab04109220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009e80 tx=0x7faaf400b330 comp rx=0 tx=0).stop 2024-04-26T22:46:44.722 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.318+0000 7fab0b400700 1 -- 172.21.15.27:0/4100563167 shutdown_connections 2024-04-26T22:46:44.722 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.318+0000 7fab0b400700 1 --2- 172.21.15.27:0/4100563167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab04108e00 0x7fab04109220 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:44.722 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.318+0000 7fab0b400700 1 -- 172.21.15.27:0/4100563167 >> 172.21.15.27:0/4100563167 conn(0x7fab04104460 msgr2=0x7fab04106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:44.722 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.318+0000 7fab0b400700 1 -- 172.21.15.27:0/4100563167 shutdown_connections 2024-04-26T22:46:44.722 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.318+0000 7fab0b400700 1 -- 172.21.15.27:0/4100563167 wait complete. 2024-04-26T22:46:44.722 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.319+0000 7fab0b400700 1 Processor -- start 2024-04-26T22:46:44.722 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.319+0000 7fab0b400700 1 -- start start 2024-04-26T22:46:44.722 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.319+0000 7fab0b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab04108e00 0x7fab0419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:44.722 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.319+0000 7fab0b400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab0410d3a0 con 0x7fab04108e00 2024-04-26T22:46:44.722 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.319+0000 7fab08a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab04108e00 0x7fab0419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:44.723 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.320+0000 7fab08a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab04108e00 0x7fab0419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36668/0 (socket says 172.21.15.27:36668) 2024-04-26T22:46:44.723 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.320+0000 7fab08a00700 1 -- 172.21.15.27:0/1964640405 learned_addr learned my addr 172.21.15.27:0/1964640405 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:44.723 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.320+0000 7fab08a00700 1 -- 172.21.15.27:0/1964640405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0419d1b0 con 0x7fab04108e00 2024-04-26T22:46:44.723 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.320+0000 7fab08a00700 1 --2- 172.21.15.27:0/1964640405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab04108e00 0x7fab0419cc70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faaf400b7b0 tx=0x7faaf400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:44.723 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.321+0000 7fab02000700 1 -- 172.21.15.27:0/1964640405 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faaf4020020 con 0x7fab04108e00 2024-04-26T22:46:44.723 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.321+0000 7fab02000700 1 -- 172.21.15.27:0/1964640405 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faaf4003d10 con 0x7fab04108e00 2024-04-26T22:46:44.723 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.321+0000 7fab02000700 1 -- 172.21.15.27:0/1964640405 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faaf4017040 con 0x7fab04108e00 2024-04-26T22:46:44.723 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.321+0000 7fab0b400700 1 -- 172.21.15.27:0/1964640405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0419d440 con 0x7fab04108e00 2024-04-26T22:46:44.723 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.321+0000 7fab0b400700 1 -- 172.21.15.27:0/1964640405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab041a0130 con 0x7fab04108e00 2024-04-26T22:46:44.723 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.322+0000 7fab0b400700 1 -- 172.21.15.27:0/1964640405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab04064a80 con 0x7fab04108e00 2024-04-26T22:46:44.723 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.322+0000 7fab02000700 1 -- 172.21.15.27:0/1964640405 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7faaf4023020 con 0x7fab04108e00 2024-04-26T22:46:44.723 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.323+0000 7fab02000700 1 --2- 172.21.15.27:0/1964640405 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7faae8039600 0x7faae803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:44.723 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.323+0000 7fab02000700 1 -- 172.21.15.27:0/1964640405 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faaf400f050 con 0x7fab04108e00 2024-04-26T22:46:44.724 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.323+0000 7faafbe00700 1 --2- 172.21.15.27:0/1964640405 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7faae8039600 0x7faae803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:44.724 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.324+0000 7faafbe00700 1 --2- 172.21.15.27:0/1964640405 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7faae8039600 0x7faae803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faaf0006fd0 tx=0x7faaf0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:44.724 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.329+0000 7fab02000700 1 -- 172.21.15.27:0/1964640405 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faaf402a020 con 0x7fab04108e00 2024-04-26T22:46:44.724 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:37.566+0000 7fab0b400700 1 -- 172.21.15.27:0/1964640405 --> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi027", "addr": "172.21.15.27", "target": ["mon-mgr", ""]}) -- 0x7fab04105570 con 0x7faae8039600 2024-04-26T22:46:44.724 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:44.023+0000 7fab02000700 1 -- 172.21.15.27:0/1964640405 <== mgr.14120 v2:172.21.15.27:6800/1661629783 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fab04105570 con 0x7faae8039600 2024-04-26T22:46:44.724 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:44.027+0000 7fab0b400700 1 -- 172.21.15.27:0/1964640405 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7faae8039600 msgr2=0x7faae803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:44.724 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:44.027+0000 7fab0b400700 1 --2- 172.21.15.27:0/1964640405 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7faae8039600 0x7faae803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faaf0006fd0 tx=0x7faaf0006e40 comp rx=0 tx=0).stop 2024-04-26T22:46:44.724 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:44.027+0000 7fab0b400700 1 -- 172.21.15.27:0/1964640405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab04108e00 msgr2=0x7fab0419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:44.724 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:44.027+0000 7fab0b400700 1 --2- 172.21.15.27:0/1964640405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab04108e00 0x7fab0419cc70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faaf400b7b0 tx=0x7faaf400be90 comp rx=0 tx=0).stop 2024-04-26T22:46:44.724 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:44.027+0000 7fab0b400700 1 -- 172.21.15.27:0/1964640405 shutdown_connections 2024-04-26T22:46:44.725 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:44.027+0000 7fab0b400700 1 --2- 172.21.15.27:0/1964640405 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7faae8039600 0x7faae803bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:44.725 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:44.027+0000 7fab0b400700 1 --2- 172.21.15.27:0/1964640405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab04108e00 0x7fab0419cc70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:44.725 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:44.027+0000 7fab0b400700 1 -- 172.21.15.27:0/1964640405 >> 172.21.15.27:0/1964640405 conn(0x7fab04104460 msgr2=0x7fab04104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:44.725 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:44.028+0000 7fab0b400700 1 -- 172.21.15.27:0/1964640405 shutdown_connections 2024-04-26T22:46:44.725 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:44.028+0000 7fab0b400700 1 -- 172.21.15.27:0/1964640405 wait complete. 2024-04-26T22:46:44.725 INFO:teuthology.orchestra.run.smithi027.stdout:Deploying unmanaged mon service... 2024-04-26T22:46:45.279 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:45 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' 2024-04-26T22:46:45.279 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:45 smithi027 ceph-mon[28034]: Added host smithi027 2024-04-26T22:46:45.279 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:45 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:46:46.976 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:46 smithi027 ceph-mon[28034]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:46:46.976 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:46 smithi027 ceph-mon[28034]: Saving service mon spec with placement count:5 2024-04-26T22:46:46.976 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:46 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' 2024-04-26T22:46:46.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T22:46:46.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.439+0000 7f6baca00700 1 Processor -- start 2024-04-26T22:46:46.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.439+0000 7f6baca00700 1 -- start start 2024-04-26T22:46:46.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.439+0000 7f6baca00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba8108e20 0x7f6ba8109240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:46.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.440+0000 7f6baca00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8109780 con 0x7f6ba8108e20 2024-04-26T22:46:46.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.440+0000 7f6ba5e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba8108e20 0x7f6ba8109240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:46.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.440+0000 7f6ba5e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba8108e20 0x7f6ba8109240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37970/0 (socket says 172.21.15.27:37970) 2024-04-26T22:46:46.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.440+0000 7f6ba5e00700 1 -- 172.21.15.27:0/285722817 learned_addr learned my addr 172.21.15.27:0/285722817 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:46.990 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.441+0000 7f6ba5e00700 1 -- 172.21.15.27:0/285722817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba8109f80 con 0x7f6ba8108e20 2024-04-26T22:46:46.994 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.441+0000 7f6ba5e00700 1 --2- 172.21.15.27:0/285722817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba8108e20 0x7f6ba8109240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6b980098a0 tx=0x7f6b98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee13a1aadd6b02c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:46.994 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.441+0000 7f6ba4a00700 1 -- 172.21.15.27:0/285722817 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b98004160 con 0x7f6ba8108e20 2024-04-26T22:46:46.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.441+0000 7f6ba4a00700 1 -- 172.21.15.27:0/285722817 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6b98004300 con 0x7f6ba8108e20 2024-04-26T22:46:46.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.441+0000 7f6ba4a00700 1 -- 172.21.15.27:0/285722817 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b98011af0 con 0x7f6ba8108e20 2024-04-26T22:46:46.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.442+0000 7f6baca00700 1 -- 172.21.15.27:0/285722817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba8108e20 msgr2=0x7f6ba8109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:46.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.442+0000 7f6baca00700 1 --2- 172.21.15.27:0/285722817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba8108e20 0x7f6ba8109240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6b980098a0 tx=0x7f6b98009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:46.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.442+0000 7f6baca00700 1 -- 172.21.15.27:0/285722817 shutdown_connections 2024-04-26T22:46:46.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.442+0000 7f6baca00700 1 --2- 172.21.15.27:0/285722817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba8108e20 0x7f6ba8109240 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:46.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.442+0000 7f6baca00700 1 -- 172.21.15.27:0/285722817 >> 172.21.15.27:0/285722817 conn(0x7f6ba81043c0 msgr2=0x7f6ba8106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:46.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.442+0000 7f6baca00700 1 -- 172.21.15.27:0/285722817 shutdown_connections 2024-04-26T22:46:46.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.443+0000 7f6baca00700 1 -- 172.21.15.27:0/285722817 wait complete. 2024-04-26T22:46:46.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.443+0000 7f6baca00700 1 Processor -- start 2024-04-26T22:46:46.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.444+0000 7f6baca00700 1 -- start start 2024-04-26T22:46:46.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.444+0000 7f6baca00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba8108e20 0x7f6ba819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:46.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.444+0000 7f6baca00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba810d3c0 con 0x7f6ba8108e20 2024-04-26T22:46:46.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.444+0000 7f6ba5e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba8108e20 0x7f6ba819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:46.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.444+0000 7f6ba5e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba8108e20 0x7f6ba819ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37982/0 (socket says 172.21.15.27:37982) 2024-04-26T22:46:46.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.444+0000 7f6ba5e00700 1 -- 172.21.15.27:0/3096998258 learned_addr learned my addr 172.21.15.27:0/3096998258 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:46.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.445+0000 7f6ba5e00700 1 -- 172.21.15.27:0/3096998258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba819f470 con 0x7f6ba8108e20 2024-04-26T22:46:46.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.445+0000 7f6ba5e00700 1 --2- 172.21.15.27:0/3096998258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba8108e20 0x7f6ba819ef30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6b98011ac0 tx=0x7f6b98011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:46.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.445+0000 7f6b9ea00700 1 -- 172.21.15.27:0/3096998258 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b98003970 con 0x7f6ba8108e20 2024-04-26T22:46:46.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.445+0000 7f6b9ea00700 1 -- 172.21.15.27:0/3096998258 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6b98003b10 con 0x7f6ba8108e20 2024-04-26T22:46:46.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.445+0000 7f6b9ea00700 1 -- 172.21.15.27:0/3096998258 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b98024610 con 0x7f6ba8108e20 2024-04-26T22:46:46.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.445+0000 7f6baca00700 1 -- 172.21.15.27:0/3096998258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba819f700 con 0x7f6ba8108e20 2024-04-26T22:46:46.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.445+0000 7f6baca00700 1 -- 172.21.15.27:0/3096998258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba81a23f0 con 0x7f6ba8108e20 2024-04-26T22:46:46.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.447+0000 7f6baca00700 1 -- 172.21.15.27:0/3096998258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba8064a80 con 0x7f6ba8108e20 2024-04-26T22:46:46.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.447+0000 7f6b9ea00700 1 -- 172.21.15.27:0/3096998258 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6b9801a9e0 con 0x7f6ba8108e20 2024-04-26T22:46:46.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.447+0000 7f6b9ea00700 1 --2- 172.21.15.27:0/3096998258 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6b8c039600 0x7f6b8c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:46.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.447+0000 7f6b9ea00700 1 -- 172.21.15.27:0/3096998258 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6b9804d980 con 0x7f6ba8108e20 2024-04-26T22:46:46.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.448+0000 7f6ba5400700 1 --2- 172.21.15.27:0/3096998258 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6b8c039600 0x7f6b8c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:46.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.448+0000 7f6ba5400700 1 --2- 172.21.15.27:0/3096998258 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6b8c039600 0x7f6b8c03bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6b94006fd0 tx=0x7f6b94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:46.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.453+0000 7f6b9ea00700 1 -- 172.21.15.27:0/3096998258 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6b9800b3e0 con 0x7f6ba8108e20 2024-04-26T22:46:46.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.709+0000 7f6baca00700 1 -- 172.21.15.27:0/3096998258 --> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6ba8105590 con 0x7f6b8c039600 2024-04-26T22:46:46.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.720+0000 7f6b9ea00700 1 -- 172.21.15.27:0/3096998258 <== mgr.14120 v2:172.21.15.27:6800/1661629783 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6ba8105590 con 0x7f6b8c039600 2024-04-26T22:46:46.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.723+0000 7f6baca00700 1 -- 172.21.15.27:0/3096998258 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6b8c039600 msgr2=0x7f6b8c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:46.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.723+0000 7f6baca00700 1 --2- 172.21.15.27:0/3096998258 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6b8c039600 0x7f6b8c03bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6b94006fd0 tx=0x7f6b94006e40 comp rx=0 tx=0).stop 2024-04-26T22:46:46.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.723+0000 7f6baca00700 1 -- 172.21.15.27:0/3096998258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba8108e20 msgr2=0x7f6ba819ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:46.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.723+0000 7f6baca00700 1 --2- 172.21.15.27:0/3096998258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba8108e20 0x7f6ba819ef30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6b98011ac0 tx=0x7f6b98011ba0 comp rx=0 tx=0).stop 2024-04-26T22:46:46.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.723+0000 7f6baca00700 1 -- 172.21.15.27:0/3096998258 shutdown_connections 2024-04-26T22:46:46.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.723+0000 7f6baca00700 1 --2- 172.21.15.27:0/3096998258 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6b8c039600 0x7f6b8c03bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:46.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.723+0000 7f6baca00700 1 --2- 172.21.15.27:0/3096998258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba8108e20 0x7f6ba819ef30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:46.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.723+0000 7f6baca00700 1 -- 172.21.15.27:0/3096998258 >> 172.21.15.27:0/3096998258 conn(0x7f6ba81043c0 msgr2=0x7f6ba8104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:46.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.723+0000 7f6baca00700 1 -- 172.21.15.27:0/3096998258 shutdown_connections 2024-04-26T22:46:46.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:45.723+0000 7f6baca00700 1 -- 172.21.15.27:0/3096998258 wait complete. 2024-04-26T22:46:46.998 INFO:teuthology.orchestra.run.smithi027.stdout:Deploying unmanaged mgr service... 2024-04-26T22:46:48.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:48 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' 2024-04-26T22:46:48.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:48 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' 2024-04-26T22:46:48.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:48 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' 2024-04-26T22:46:49.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T22:46:49.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.095+0000 7f03e9800700 1 Processor -- start 2024-04-26T22:46:49.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.095+0000 7f03e9800700 1 -- start start 2024-04-26T22:46:49.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.096+0000 7f03e9800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03e407f730 0x7f03e407db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:49.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.096+0000 7f03e9800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f03e407e160 con 0x7f03e407f730 2024-04-26T22:46:49.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.096+0000 7f03e2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03e407f730 0x7f03e407db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:49.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.096+0000 7f03e2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03e407f730 0x7f03e407db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37792/0 (socket says 172.21.15.27:37792) 2024-04-26T22:46:49.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.096+0000 7f03e2a00700 1 -- 172.21.15.27:0/3594711496 learned_addr learned my addr 172.21.15.27:0/3594711496 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:49.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.097+0000 7f03e2a00700 1 -- 172.21.15.27:0/3594711496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e407e2e0 con 0x7f03e407f730 2024-04-26T22:46:49.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.097+0000 7f03e2a00700 1 --2- 172.21.15.27:0/3594711496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03e407f730 0x7f03e407db90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f03d40098a0 tx=0x7f03d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486b8dcdf8a8b86e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:49.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.097+0000 7f03e1600700 1 -- 172.21.15.27:0/3594711496 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03d4004160 con 0x7f03e407f730 2024-04-26T22:46:49.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.097+0000 7f03e1600700 1 -- 172.21.15.27:0/3594711496 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d4004300 con 0x7f03e407f730 2024-04-26T22:46:49.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.097+0000 7f03e1600700 1 -- 172.21.15.27:0/3594711496 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03d4011a20 con 0x7f03e407f730 2024-04-26T22:46:49.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.098+0000 7f03e9800700 1 -- 172.21.15.27:0/3594711496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03e407f730 msgr2=0x7f03e407db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:49.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.098+0000 7f03e9800700 1 --2- 172.21.15.27:0/3594711496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03e407f730 0x7f03e407db90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f03d40098a0 tx=0x7f03d4009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:49.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.098+0000 7f03e9800700 1 -- 172.21.15.27:0/3594711496 shutdown_connections 2024-04-26T22:46:49.108 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.098+0000 7f03e9800700 1 --2- 172.21.15.27:0/3594711496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03e407f730 0x7f03e407db90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:49.108 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.098+0000 7f03e9800700 1 -- 172.21.15.27:0/3594711496 >> 172.21.15.27:0/3594711496 conn(0x7f03e4104460 msgr2=0x7f03e4106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:49.108 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.098+0000 7f03e9800700 1 -- 172.21.15.27:0/3594711496 shutdown_connections 2024-04-26T22:46:49.108 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.098+0000 7f03e9800700 1 -- 172.21.15.27:0/3594711496 wait complete. 2024-04-26T22:46:49.108 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.099+0000 7f03e9800700 1 Processor -- start 2024-04-26T22:46:49.108 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.100+0000 7f03e9800700 1 -- start start 2024-04-26T22:46:49.109 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.100+0000 7f03e9800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03e407f730 0x7f03e419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:49.109 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.100+0000 7f03e9800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f03e410f410 con 0x7f03e407f730 2024-04-26T22:46:49.109 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.100+0000 7f03e2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03e407f730 0x7f03e419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:49.109 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.100+0000 7f03e2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03e407f730 0x7f03e419eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37796/0 (socket says 172.21.15.27:37796) 2024-04-26T22:46:49.109 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.100+0000 7f03e2a00700 1 -- 172.21.15.27:0/1812546307 learned_addr learned my addr 172.21.15.27:0/1812546307 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:49.109 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.101+0000 7f03e2a00700 1 -- 172.21.15.27:0/1812546307 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e419f420 con 0x7f03e407f730 2024-04-26T22:46:49.109 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.101+0000 7f03e2a00700 1 --2- 172.21.15.27:0/1812546307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03e407f730 0x7f03e419eee0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009520 tx=0x7f03d4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:49.109 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.101+0000 7f03db400700 1 -- 172.21.15.27:0/1812546307 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03d4003af0 con 0x7f03e407f730 2024-04-26T22:46:49.109 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.101+0000 7f03db400700 1 -- 172.21.15.27:0/1812546307 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d401a8e0 con 0x7f03e407f730 2024-04-26T22:46:49.109 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.101+0000 7f03e9800700 1 -- 172.21.15.27:0/1812546307 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e419f6b0 con 0x7f03e407f730 2024-04-26T22:46:49.109 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.101+0000 7f03db400700 1 -- 172.21.15.27:0/1812546307 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03d40247a0 con 0x7f03e407f730 2024-04-26T22:46:49.109 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.101+0000 7f03e9800700 1 -- 172.21.15.27:0/1812546307 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e41a23a0 con 0x7f03e407f730 2024-04-26T22:46:49.109 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.103+0000 7f03db400700 1 -- 172.21.15.27:0/1812546307 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f03d4022020 con 0x7f03e407f730 2024-04-26T22:46:49.109 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.103+0000 7f03e9800700 1 -- 172.21.15.27:0/1812546307 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e4064a80 con 0x7f03e407f730 2024-04-26T22:46:49.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.103+0000 7f03db400700 1 --2- 172.21.15.27:0/1812546307 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f03c8039600 0x7f03c803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:49.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.103+0000 7f03db400700 1 -- 172.21.15.27:0/1812546307 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f03d404d850 con 0x7f03e407f730 2024-04-26T22:46:49.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.103+0000 7f03e2000700 1 --2- 172.21.15.27:0/1812546307 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f03c8039600 0x7f03c803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:49.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.104+0000 7f03e2000700 1 --2- 172.21.15.27:0/1812546307 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f03c8039600 0x7f03c803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f03d0006fd0 tx=0x7f03d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:49.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.109+0000 7f03db400700 1 -- 172.21.15.27:0/1812546307 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f03d40172b0 con 0x7f03e407f730 2024-04-26T22:46:49.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.361+0000 7f03e9800700 1 -- 172.21.15.27:0/1812546307 --> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f03e4105610 con 0x7f03c8039600 2024-04-26T22:46:49.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.368+0000 7f03db400700 1 -- 172.21.15.27:0/1812546307 <== mgr.14120 v2:172.21.15.27:6800/1661629783 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f03e4105610 con 0x7f03c8039600 2024-04-26T22:46:49.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.370+0000 7f03e9800700 1 -- 172.21.15.27:0/1812546307 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f03c8039600 msgr2=0x7f03c803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:49.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.370+0000 7f03e9800700 1 --2- 172.21.15.27:0/1812546307 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f03c8039600 0x7f03c803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f03d0006fd0 tx=0x7f03d0006e40 comp rx=0 tx=0).stop 2024-04-26T22:46:49.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.370+0000 7f03e9800700 1 -- 172.21.15.27:0/1812546307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03e407f730 msgr2=0x7f03e419eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:49.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.370+0000 7f03e9800700 1 --2- 172.21.15.27:0/1812546307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03e407f730 0x7f03e419eee0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009520 tx=0x7f03d4011aa0 comp rx=0 tx=0).stop 2024-04-26T22:46:49.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.371+0000 7f03e9800700 1 -- 172.21.15.27:0/1812546307 shutdown_connections 2024-04-26T22:46:49.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.371+0000 7f03e9800700 1 --2- 172.21.15.27:0/1812546307 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f03c8039600 0x7f03c803bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:49.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.371+0000 7f03e9800700 1 --2- 172.21.15.27:0/1812546307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03e407f730 0x7f03e419eee0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:49.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.371+0000 7f03e9800700 1 -- 172.21.15.27:0/1812546307 >> 172.21.15.27:0/1812546307 conn(0x7f03e4104460 msgr2=0x7f03e4104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:49.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.371+0000 7f03e9800700 1 -- 172.21.15.27:0/1812546307 shutdown_connections 2024-04-26T22:46:49.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:48.371+0000 7f03e9800700 1 -- 172.21.15.27:0/1812546307 wait complete. 2024-04-26T22:46:49.657 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:49 smithi027 ceph-mon[28034]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:46:49.657 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:49 smithi027 ceph-mon[28034]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:46:49.658 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:49 smithi027 ceph-mon[28034]: Saving service mgr spec with placement count:2 2024-04-26T22:46:51.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:50 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3071556937' entity='client.admin' 2024-04-26T22:46:51.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:50 smithi027 ceph-mon[28034]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:46:51.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.704+0000 7f6af8400700 1 Processor -- start 2024-04-26T22:46:51.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.704+0000 7f6af8400700 1 -- start start 2024-04-26T22:46:51.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.705+0000 7f6af8400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af010b110 0x7f6af010b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:51.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.705+0000 7f6af8400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6af010bb00 con 0x7f6af010b110 2024-04-26T22:46:51.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.705+0000 7f6af5a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af010b110 0x7f6af010b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:51.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.705+0000 7f6af5a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af010b110 0x7f6af010b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37802/0 (socket says 172.21.15.27:37802) 2024-04-26T22:46:51.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.705+0000 7f6af5a00700 1 -- 172.21.15.27:0/2609142036 learned_addr learned my addr 172.21.15.27:0/2609142036 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:51.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.705+0000 7f6af5a00700 1 -- 172.21.15.27:0/2609142036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af010c200 con 0x7f6af010b110 2024-04-26T22:46:51.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.706+0000 7f6af5a00700 1 --2- 172.21.15.27:0/2609142036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af010b110 0x7f6af010b530 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40098a0 tx=0x7f6ae4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffe7d47b9afe789 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:51.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.706+0000 7f6aefe00700 1 -- 172.21.15.27:0/2609142036 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ae4004160 con 0x7f6af010b110 2024-04-26T22:46:51.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.706+0000 7f6aefe00700 1 -- 172.21.15.27:0/2609142036 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae4004300 con 0x7f6af010b110 2024-04-26T22:46:51.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.706+0000 7f6aefe00700 1 -- 172.21.15.27:0/2609142036 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ae4011a20 con 0x7f6af010b110 2024-04-26T22:46:51.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.707+0000 7f6af8400700 1 -- 172.21.15.27:0/2609142036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af010b110 msgr2=0x7f6af010b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:51.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.707+0000 7f6af8400700 1 --2- 172.21.15.27:0/2609142036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af010b110 0x7f6af010b530 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40098a0 tx=0x7f6ae4009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:51.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.707+0000 7f6af8400700 1 -- 172.21.15.27:0/2609142036 shutdown_connections 2024-04-26T22:46:51.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.707+0000 7f6af8400700 1 --2- 172.21.15.27:0/2609142036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af010b110 0x7f6af010b530 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:51.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.707+0000 7f6af8400700 1 -- 172.21.15.27:0/2609142036 >> 172.21.15.27:0/2609142036 conn(0x7f6af007e470 msgr2=0x7f6af007e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:51.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.707+0000 7f6af8400700 1 -- 172.21.15.27:0/2609142036 shutdown_connections 2024-04-26T22:46:51.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.707+0000 7f6af8400700 1 -- 172.21.15.27:0/2609142036 wait complete. 2024-04-26T22:46:51.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.708+0000 7f6af8400700 1 Processor -- start 2024-04-26T22:46:51.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.709+0000 7f6af8400700 1 -- start start 2024-04-26T22:46:51.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.709+0000 7f6af8400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af010b110 0x7f6af019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:51.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.709+0000 7f6af8400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6af010f5b0 con 0x7f6af010b110 2024-04-26T22:46:51.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.709+0000 7f6af5a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af010b110 0x7f6af019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:51.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.709+0000 7f6af5a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af010b110 0x7f6af019eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37804/0 (socket says 172.21.15.27:37804) 2024-04-26T22:46:51.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.709+0000 7f6af5a00700 1 -- 172.21.15.27:0/3071556937 learned_addr learned my addr 172.21.15.27:0/3071556937 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:51.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.710+0000 7f6af5a00700 1 -- 172.21.15.27:0/3071556937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af019f410 con 0x7f6af010b110 2024-04-26T22:46:51.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.710+0000 7f6af5a00700 1 --2- 172.21.15.27:0/3071556937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af010b110 0x7f6af019eed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40116e0 tx=0x7f6ae4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:51.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.710+0000 7f6aee000700 1 -- 172.21.15.27:0/3071556937 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ae40039e0 con 0x7f6af010b110 2024-04-26T22:46:51.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.711+0000 7f6aee000700 1 -- 172.21.15.27:0/3071556937 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae401b8f0 con 0x7f6af010b110 2024-04-26T22:46:51.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.711+0000 7f6af8400700 1 -- 172.21.15.27:0/3071556937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af019f6a0 con 0x7f6af010b110 2024-04-26T22:46:51.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.711+0000 7f6aee000700 1 -- 172.21.15.27:0/3071556937 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ae4023760 con 0x7f6af010b110 2024-04-26T22:46:51.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.711+0000 7f6af8400700 1 -- 172.21.15.27:0/3071556937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af01a2390 con 0x7f6af010b110 2024-04-26T22:46:51.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.712+0000 7f6af8400700 1 -- 172.21.15.27:0/3071556937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af0064a80 con 0x7f6af010b110 2024-04-26T22:46:51.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.712+0000 7f6aee000700 1 -- 172.21.15.27:0/3071556937 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6ae401b440 con 0x7f6af010b110 2024-04-26T22:46:51.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.713+0000 7f6aee000700 1 --2- 172.21.15.27:0/3071556937 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6ad8039560 0x7f6ad803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:51.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.713+0000 7f6aee000700 1 -- 172.21.15.27:0/3071556937 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6ae404df80 con 0x7f6af010b110 2024-04-26T22:46:51.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.713+0000 7f6af5000700 1 --2- 172.21.15.27:0/3071556937 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6ad8039560 0x7f6ad803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:51.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.714+0000 7f6af5000700 1 --2- 172.21.15.27:0/3071556937 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6ad8039560 0x7f6ad803ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6adc006fd0 tx=0x7f6adc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:51.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.719+0000 7f6aee000700 1 -- 172.21.15.27:0/3071556937 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6ae401f030 con 0x7f6af010b110 2024-04-26T22:46:51.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.965+0000 7f6af8400700 1 -- 172.21.15.27:0/3071556937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f6af01a2940 con 0x7f6af010b110 2024-04-26T22:46:51.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.971+0000 7f6aee000700 1 -- 172.21.15.27:0/3071556937 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f6ae401a3d0 con 0x7f6af010b110 2024-04-26T22:46:51.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.973+0000 7f6af8400700 1 -- 172.21.15.27:0/3071556937 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6ad8039560 msgr2=0x7f6ad803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:51.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.973+0000 7f6af8400700 1 --2- 172.21.15.27:0/3071556937 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6ad8039560 0x7f6ad803ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6adc006fd0 tx=0x7f6adc006e40 comp rx=0 tx=0).stop 2024-04-26T22:46:51.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.973+0000 7f6af8400700 1 -- 172.21.15.27:0/3071556937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af010b110 msgr2=0x7f6af019eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:51.543 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.973+0000 7f6af8400700 1 --2- 172.21.15.27:0/3071556937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af010b110 0x7f6af019eed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40116e0 tx=0x7f6ae4011aa0 comp rx=0 tx=0).stop 2024-04-26T22:46:51.544 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.974+0000 7f6af8400700 1 -- 172.21.15.27:0/3071556937 shutdown_connections 2024-04-26T22:46:51.544 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.974+0000 7f6af8400700 1 --2- 172.21.15.27:0/3071556937 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6ad8039560 0x7f6ad803ba20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:51.544 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.974+0000 7f6af8400700 1 --2- 172.21.15.27:0/3071556937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af010b110 0x7f6af019eed0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:51.544 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.974+0000 7f6af8400700 1 -- 172.21.15.27:0/3071556937 >> 172.21.15.27:0/3071556937 conn(0x7f6af007e470 msgr2=0x7f6af0108cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:51.544 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.974+0000 7f6af8400700 1 -- 172.21.15.27:0/3071556937 shutdown_connections 2024-04-26T22:46:51.544 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:49.974+0000 7f6af8400700 1 -- 172.21.15.27:0/3071556937 wait complete. 2024-04-26T22:46:52.870 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:52 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' 2024-04-26T22:46:52.870 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:52 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2432991308' entity='client.admin' 2024-04-26T22:46:53.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.212+0000 7f6a58000700 1 Processor -- start 2024-04-26T22:46:53.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.212+0000 7f6a58000700 1 -- start start 2024-04-26T22:46:53.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.212+0000 7f6a58000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5007eea0 0x7f6a5007d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:53.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.212+0000 7f6a58000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5007d8d0 con 0x7f6a5007eea0 2024-04-26T22:46:53.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.213+0000 7f6a55600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5007eea0 0x7f6a5007d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:53.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.213+0000 7f6a55600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5007eea0 0x7f6a5007d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37814/0 (socket says 172.21.15.27:37814) 2024-04-26T22:46:53.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.213+0000 7f6a55600700 1 -- 172.21.15.27:0/1492174509 learned_addr learned my addr 172.21.15.27:0/1492174509 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:53.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.213+0000 7f6a55600700 1 -- 172.21.15.27:0/1492174509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5007da50 con 0x7f6a5007eea0 2024-04-26T22:46:53.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.214+0000 7f6a55600700 1 --2- 172.21.15.27:0/1492174509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5007eea0 0x7f6a5007d300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6a440098a0 tx=0x7f6a44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a70a33c484acb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:53.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.214+0000 7f6a4fe00700 1 -- 172.21.15.27:0/1492174509 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a44004160 con 0x7f6a5007eea0 2024-04-26T22:46:53.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.214+0000 7f6a4fe00700 1 -- 172.21.15.27:0/1492174509 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a44004300 con 0x7f6a5007eea0 2024-04-26T22:46:53.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.214+0000 7f6a4fe00700 1 -- 172.21.15.27:0/1492174509 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a44011a20 con 0x7f6a5007eea0 2024-04-26T22:46:53.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.215+0000 7f6a58000700 1 -- 172.21.15.27:0/1492174509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5007eea0 msgr2=0x7f6a5007d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:53.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.215+0000 7f6a58000700 1 --2- 172.21.15.27:0/1492174509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5007eea0 0x7f6a5007d300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6a440098a0 tx=0x7f6a44009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:53.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.215+0000 7f6a58000700 1 -- 172.21.15.27:0/1492174509 shutdown_connections 2024-04-26T22:46:53.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.215+0000 7f6a58000700 1 --2- 172.21.15.27:0/1492174509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5007eea0 0x7f6a5007d300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:53.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.215+0000 7f6a58000700 1 -- 172.21.15.27:0/1492174509 >> 172.21.15.27:0/1492174509 conn(0x7f6a501043e0 msgr2=0x7f6a50106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:53.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.215+0000 7f6a58000700 1 -- 172.21.15.27:0/1492174509 shutdown_connections 2024-04-26T22:46:53.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.215+0000 7f6a58000700 1 -- 172.21.15.27:0/1492174509 wait complete. 2024-04-26T22:46:53.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.216+0000 7f6a58000700 1 Processor -- start 2024-04-26T22:46:53.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.216+0000 7f6a58000700 1 -- start start 2024-04-26T22:46:53.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.216+0000 7f6a58000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5007eea0 0x7f6a501a3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:53.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.216+0000 7f6a58000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5010b000 con 0x7f6a5007eea0 2024-04-26T22:46:53.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.217+0000 7f6a55600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5007eea0 0x7f6a501a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:53.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.217+0000 7f6a55600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5007eea0 0x7f6a501a3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37820/0 (socket says 172.21.15.27:37820) 2024-04-26T22:46:53.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.217+0000 7f6a55600700 1 -- 172.21.15.27:0/2432991308 learned_addr learned my addr 172.21.15.27:0/2432991308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:53.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.217+0000 7f6a55600700 1 -- 172.21.15.27:0/2432991308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a501a3880 con 0x7f6a5007eea0 2024-04-26T22:46:53.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.217+0000 7f6a55600700 1 --2- 172.21.15.27:0/2432991308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5007eea0 0x7f6a501a3340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6a440119f0 tx=0x7f6a44011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:53.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.218+0000 7f6a4e000700 1 -- 172.21.15.27:0/2432991308 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a440039c0 con 0x7f6a5007eea0 2024-04-26T22:46:53.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.218+0000 7f6a4e000700 1 -- 172.21.15.27:0/2432991308 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a44003b60 con 0x7f6a5007eea0 2024-04-26T22:46:53.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.218+0000 7f6a58000700 1 -- 172.21.15.27:0/2432991308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a501a3b10 con 0x7f6a5007eea0 2024-04-26T22:46:53.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.218+0000 7f6a4e000700 1 -- 172.21.15.27:0/2432991308 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a440197a0 con 0x7f6a5007eea0 2024-04-26T22:46:53.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.218+0000 7f6a58000700 1 -- 172.21.15.27:0/2432991308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a501a6800 con 0x7f6a5007eea0 2024-04-26T22:46:53.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.219+0000 7f6a4e000700 1 -- 172.21.15.27:0/2432991308 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6a4401ab90 con 0x7f6a5007eea0 2024-04-26T22:46:53.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.219+0000 7f6a58000700 1 -- 172.21.15.27:0/2432991308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a50064a80 con 0x7f6a5007eea0 2024-04-26T22:46:53.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.219+0000 7f6a4e000700 1 --2- 172.21.15.27:0/2432991308 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6a38039560 0x7f6a3803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:53.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.220+0000 7f6a4e000700 1 -- 172.21.15.27:0/2432991308 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6a4404dc10 con 0x7f6a5007eea0 2024-04-26T22:46:53.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.220+0000 7f6a54c00700 1 --2- 172.21.15.27:0/2432991308 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6a38039560 0x7f6a3803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:53.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.220+0000 7f6a54c00700 1 --2- 172.21.15.27:0/2432991308 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6a38039560 0x7f6a3803ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c006fd0 tx=0x7f6a3c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:53.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.230+0000 7f6a4e000700 1 -- 172.21.15.27:0/2432991308 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6a4400b3e0 con 0x7f6a5007eea0 2024-04-26T22:46:53.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.469+0000 7f6a58000700 1 -- 172.21.15.27:0/2432991308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f6a5019c8a0 con 0x7f6a5007eea0 2024-04-26T22:46:53.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.473+0000 7f6a4e000700 1 -- 172.21.15.27:0/2432991308 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f6a5019c8a0 con 0x7f6a5007eea0 2024-04-26T22:46:53.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.477+0000 7f6a58000700 1 -- 172.21.15.27:0/2432991308 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6a38039560 msgr2=0x7f6a3803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:53.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.477+0000 7f6a58000700 1 --2- 172.21.15.27:0/2432991308 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6a38039560 0x7f6a3803ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c006fd0 tx=0x7f6a3c006e40 comp rx=0 tx=0).stop 2024-04-26T22:46:53.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.477+0000 7f6a58000700 1 -- 172.21.15.27:0/2432991308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5007eea0 msgr2=0x7f6a501a3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:53.233 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.477+0000 7f6a58000700 1 --2- 172.21.15.27:0/2432991308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5007eea0 0x7f6a501a3340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6a440119f0 tx=0x7f6a44011ad0 comp rx=0 tx=0).stop 2024-04-26T22:46:53.233 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.477+0000 7f6a58000700 1 -- 172.21.15.27:0/2432991308 shutdown_connections 2024-04-26T22:46:53.233 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.477+0000 7f6a58000700 1 --2- 172.21.15.27:0/2432991308 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f6a38039560 0x7f6a3803ba20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:53.233 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.477+0000 7f6a58000700 1 --2- 172.21.15.27:0/2432991308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5007eea0 0x7f6a501a3340 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:53.233 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.477+0000 7f6a58000700 1 -- 172.21.15.27:0/2432991308 >> 172.21.15.27:0/2432991308 conn(0x7f6a501043e0 msgr2=0x7f6a501067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:53.233 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.478+0000 7f6a58000700 1 -- 172.21.15.27:0/2432991308 shutdown_connections 2024-04-26T22:46:53.233 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:52.478+0000 7f6a58000700 1 -- 172.21.15.27:0/2432991308 wait complete. 2024-04-26T22:46:53.233 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling the dashboard module... 2024-04-26T22:46:53.820 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:53 smithi027 ceph-mon[28034]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:46:55.282 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:55 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: ignoring --setuser ceph since I am not root 2024-04-26T22:46:55.282 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:55 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: ignoring --setgroup ceph since I am not root 2024-04-26T22:46:55.282 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:55 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:55.280+0000 7f5bcd916200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T22:46:55.282 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:54 smithi027 ceph-mon[28034]: from='mgr.14120 172.21.15.27:0/2397169387' entity='mgr.x' 2024-04-26T22:46:55.283 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:54 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2566662400' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T22:46:55.796 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:55 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:55.536+0000 7f5bcd916200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:46:55.796 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:55 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:55.679+0000 7f5bcd916200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T22:46:56.108 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:55 smithi027 ceph-mon[28034]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:46:56.108 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:55 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2566662400' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T22:46:56.108 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:55 smithi027 ceph-mon[28034]: mgrmap e9: x(active, since 26s) 2024-04-26T22:46:56.253 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.994+0000 7fadcec00700 1 Processor -- start 2024-04-26T22:46:56.260 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.994+0000 7fadcec00700 1 -- start start 2024-04-26T22:46:56.260 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.994+0000 7fadcec00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc807eea0 0x7fadc807d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:56.260 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.994+0000 7fadcec00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fadc807d8d0 con 0x7fadc807eea0 2024-04-26T22:46:56.260 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.995+0000 7fadcd800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc807eea0 0x7fadc807d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:56.260 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.995+0000 7fadcd800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc807eea0 0x7fadc807d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37824/0 (socket says 172.21.15.27:37824) 2024-04-26T22:46:56.260 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.995+0000 7fadcd800700 1 -- 172.21.15.27:0/2381769012 learned_addr learned my addr 172.21.15.27:0/2381769012 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:56.261 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.996+0000 7fadcd800700 1 -- 172.21.15.27:0/2381769012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc807da50 con 0x7fadc807eea0 2024-04-26T22:46:56.261 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.996+0000 7fadcd800700 1 --2- 172.21.15.27:0/2381769012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc807eea0 0x7fadc807d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0098a0 tx=0x7fadbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54fe053a8e1bd030 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:56.261 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.996+0000 7fadc7e00700 1 -- 172.21.15.27:0/2381769012 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadbc004160 con 0x7fadc807eea0 2024-04-26T22:46:56.261 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.997+0000 7fadc7e00700 1 -- 172.21.15.27:0/2381769012 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadbc004300 con 0x7fadc807eea0 2024-04-26T22:46:56.261 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.997+0000 7fadc7e00700 1 -- 172.21.15.27:0/2381769012 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadbc011a20 con 0x7fadc807eea0 2024-04-26T22:46:56.261 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.997+0000 7fadcec00700 1 -- 172.21.15.27:0/2381769012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc807eea0 msgr2=0x7fadc807d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:56.261 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.997+0000 7fadcec00700 1 --2- 172.21.15.27:0/2381769012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc807eea0 0x7fadc807d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0098a0 tx=0x7fadbc009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:56.261 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.997+0000 7fadcec00700 1 -- 172.21.15.27:0/2381769012 shutdown_connections 2024-04-26T22:46:56.261 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.997+0000 7fadcec00700 1 --2- 172.21.15.27:0/2381769012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc807eea0 0x7fadc807d300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:56.261 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.997+0000 7fadcec00700 1 -- 172.21.15.27:0/2381769012 >> 172.21.15.27:0/2381769012 conn(0x7fadc8104250 msgr2=0x7fadc8106670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:56.261 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.997+0000 7fadcec00700 1 -- 172.21.15.27:0/2381769012 shutdown_connections 2024-04-26T22:46:56.261 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.998+0000 7fadcec00700 1 -- 172.21.15.27:0/2381769012 wait complete. 2024-04-26T22:46:56.261 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.998+0000 7fadcec00700 1 Processor -- start 2024-04-26T22:46:56.261 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.999+0000 7fadcec00700 1 -- start start 2024-04-26T22:46:56.262 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.999+0000 7fadcec00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc807eea0 0x7fadc81a3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:56.262 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.999+0000 7fadcec00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fadc810ae30 con 0x7fadc807eea0 2024-04-26T22:46:56.262 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.999+0000 7fadcd800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc807eea0 0x7fadc81a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:56.262 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.999+0000 7fadcd800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc807eea0 0x7fadc81a3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37830/0 (socket says 172.21.15.27:37830) 2024-04-26T22:46:56.262 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:53.999+0000 7fadcd800700 1 -- 172.21.15.27:0/2566662400 learned_addr learned my addr 172.21.15.27:0/2566662400 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:56.262 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.000+0000 7fadcd800700 1 -- 172.21.15.27:0/2566662400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc81a3640 con 0x7fadc807eea0 2024-04-26T22:46:56.262 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.000+0000 7fadcd800700 1 --2- 172.21.15.27:0/2566662400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc807eea0 0x7fadc81a3100 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009520 tx=0x7fadbc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:56.262 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.000+0000 7fadc6000700 1 -- 172.21.15.27:0/2566662400 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadbc003af0 con 0x7fadc807eea0 2024-04-26T22:46:56.262 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.000+0000 7fadc6000700 1 -- 172.21.15.27:0/2566662400 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadbc01a8e0 con 0x7fadc807eea0 2024-04-26T22:46:56.262 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.000+0000 7fadcec00700 1 -- 172.21.15.27:0/2566662400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc81a38d0 con 0x7fadc807eea0 2024-04-26T22:46:56.262 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.001+0000 7fadc6000700 1 -- 172.21.15.27:0/2566662400 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadbc0247a0 con 0x7fadc807eea0 2024-04-26T22:46:56.262 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.001+0000 7fadcec00700 1 -- 172.21.15.27:0/2566662400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc81a65c0 con 0x7fadc807eea0 2024-04-26T22:46:56.262 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.002+0000 7fadc6000700 1 -- 172.21.15.27:0/2566662400 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fadbc022020 con 0x7fadc807eea0 2024-04-26T22:46:56.263 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.002+0000 7fadcec00700 1 -- 172.21.15.27:0/2566662400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc8064a80 con 0x7fadc807eea0 2024-04-26T22:46:56.263 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.003+0000 7fadc6000700 1 --2- 172.21.15.27:0/2566662400 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fadb00395b0 0x7fadb003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:56.263 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.003+0000 7fadc6000700 1 -- 172.21.15.27:0/2566662400 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fadbc053060 con 0x7fadc807eea0 2024-04-26T22:46:56.263 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.003+0000 7fadcce00700 1 --2- 172.21.15.27:0/2566662400 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fadb00395b0 0x7fadb003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:56.263 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.004+0000 7fadcce00700 1 --2- 172.21.15.27:0/2566662400 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fadb00395b0 0x7fadb003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fadb4006fd0 tx=0x7fadb4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:56.263 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.009+0000 7fadc6000700 1 -- 172.21.15.27:0/2566662400 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fadbc01ad40 con 0x7fadc807eea0 2024-04-26T22:46:56.263 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.313+0000 7fadcec00700 1 -- 172.21.15.27:0/2566662400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fadc802e790 con 0x7fadc807eea0 2024-04-26T22:46:56.263 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.953+0000 7fadc6000700 1 -- 172.21.15.27:0/2566662400 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fadbc019930 con 0x7fadc807eea0 2024-04-26T22:46:56.263 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.956+0000 7fadc6000700 1 -- 172.21.15.27:0/2566662400 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fadbc055070 con 0x7fadc807eea0 2024-04-26T22:46:56.263 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.957+0000 7fadcec00700 1 -- 172.21.15.27:0/2566662400 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fadb00395b0 msgr2=0x7fadb003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:56.263 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.957+0000 7fadcec00700 1 --2- 172.21.15.27:0/2566662400 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fadb00395b0 0x7fadb003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fadb4006fd0 tx=0x7fadb4006e40 comp rx=0 tx=0).stop 2024-04-26T22:46:56.263 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.957+0000 7fadcec00700 1 -- 172.21.15.27:0/2566662400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc807eea0 msgr2=0x7fadc81a3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:56.264 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.957+0000 7fadcec00700 1 --2- 172.21.15.27:0/2566662400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc807eea0 0x7fadc81a3100 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009520 tx=0x7fadbc011aa0 comp rx=0 tx=0).stop 2024-04-26T22:46:56.264 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.957+0000 7fadcec00700 1 -- 172.21.15.27:0/2566662400 shutdown_connections 2024-04-26T22:46:56.264 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.957+0000 7fadcec00700 1 --2- 172.21.15.27:0/2566662400 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7fadb00395b0 0x7fadb003ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:56.264 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.957+0000 7fadcec00700 1 --2- 172.21.15.27:0/2566662400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc807eea0 0x7fadc81a3100 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:56.264 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.957+0000 7fadcec00700 1 -- 172.21.15.27:0/2566662400 >> 172.21.15.27:0/2566662400 conn(0x7fadc8104250 msgr2=0x7fadc8106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:56.264 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.957+0000 7fadcec00700 1 -- 172.21.15.27:0/2566662400 shutdown_connections 2024-04-26T22:46:56.264 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:54.957+0000 7fadcec00700 1 -- 172.21.15.27:0/2566662400 wait complete. 2024-04-26T22:46:56.364 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:56 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:56.107+0000 7f5bcd916200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T22:46:56.364 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:56 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:56.227+0000 7f5bcd916200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T22:46:57.852 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:46:57 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/613767466' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T22:46:58.114 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:57 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:57.852+0000 7f5bcd916200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T22:46:58.115 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:58 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:58.093+0000 7f5bcd916200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T22:46:58.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-04-26T22:46:58.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T22:46:58.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T22:46:58.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T22:46:58.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T22:46:58.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-04-26T22:46:58.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.376+0000 7f7dcb600700 1 Processor -- start 2024-04-26T22:46:58.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.376+0000 7f7dcb600700 1 -- start start 2024-04-26T22:46:58.366 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.376+0000 7f7dcb600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dc407eea0 0x7f7dc407d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:58.366 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.376+0000 7f7dcb600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc407d8d0 con 0x7f7dc407eea0 2024-04-26T22:46:58.366 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.376+0000 7f7dc8c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dc407eea0 0x7f7dc407d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:58.366 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.376+0000 7f7dc8c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dc407eea0 0x7f7dc407d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37864/0 (socket says 172.21.15.27:37864) 2024-04-26T22:46:58.366 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.376+0000 7f7dc8c00700 1 -- 172.21.15.27:0/234079561 learned_addr learned my addr 172.21.15.27:0/234079561 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:58.366 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.377+0000 7f7dc8c00700 1 -- 172.21.15.27:0/234079561 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc407da50 con 0x7f7dc407eea0 2024-04-26T22:46:58.367 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.377+0000 7f7dc8c00700 1 --2- 172.21.15.27:0/234079561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dc407eea0 0x7f7dc407d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7db80098a0 tx=0x7f7db8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b627f9251c0ebbbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:58.367 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.378+0000 7f7dc3400700 1 -- 172.21.15.27:0/234079561 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7db8004160 con 0x7f7dc407eea0 2024-04-26T22:46:58.367 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.378+0000 7f7dc3400700 1 -- 172.21.15.27:0/234079561 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7db8004300 con 0x7f7dc407eea0 2024-04-26T22:46:58.367 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.378+0000 7f7dc3400700 1 -- 172.21.15.27:0/234079561 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7db8011a20 con 0x7f7dc407eea0 2024-04-26T22:46:58.367 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.378+0000 7f7dcb600700 1 -- 172.21.15.27:0/234079561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dc407eea0 msgr2=0x7f7dc407d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:58.367 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.378+0000 7f7dcb600700 1 --2- 172.21.15.27:0/234079561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dc407eea0 0x7f7dc407d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7db80098a0 tx=0x7f7db8009bb0 comp rx=0 tx=0).stop 2024-04-26T22:46:58.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.379+0000 7f7dcb600700 1 -- 172.21.15.27:0/234079561 shutdown_connections 2024-04-26T22:46:58.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.379+0000 7f7dcb600700 1 --2- 172.21.15.27:0/234079561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dc407eea0 0x7f7dc407d300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:58.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.379+0000 7f7dcb600700 1 -- 172.21.15.27:0/234079561 >> 172.21.15.27:0/234079561 conn(0x7f7dc41043e0 msgr2=0x7f7dc4106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:58.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.379+0000 7f7dcb600700 1 -- 172.21.15.27:0/234079561 shutdown_connections 2024-04-26T22:46:58.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.379+0000 7f7dcb600700 1 -- 172.21.15.27:0/234079561 wait complete. 2024-04-26T22:46:58.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.380+0000 7f7dcb600700 1 Processor -- start 2024-04-26T22:46:58.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.380+0000 7f7dcb600700 1 -- start start 2024-04-26T22:46:58.369 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.381+0000 7f7dcb600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dc407eea0 0x7f7dc410c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:58.369 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.381+0000 7f7dcb600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc410f410 con 0x7f7dc407eea0 2024-04-26T22:46:58.369 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.381+0000 7f7dc8c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dc407eea0 0x7f7dc410c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:46:58.369 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.381+0000 7f7dc8c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dc407eea0 0x7f7dc410c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37880/0 (socket says 172.21.15.27:37880) 2024-04-26T22:46:58.369 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.381+0000 7f7dc8c00700 1 -- 172.21.15.27:0/613767466 learned_addr learned my addr 172.21.15.27:0/613767466 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:46:58.369 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.382+0000 7f7dc8c00700 1 -- 172.21.15.27:0/613767466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc410c590 con 0x7f7dc407eea0 2024-04-26T22:46:58.369 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.382+0000 7f7dc8c00700 1 --2- 172.21.15.27:0/613767466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dc407eea0 0x7f7dc410c050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7db8009520 tx=0x7f7db8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:46:58.369 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.382+0000 7f7dc1600700 1 -- 172.21.15.27:0/613767466 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7db80038b0 con 0x7f7dc407eea0 2024-04-26T22:46:58.369 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.382+0000 7f7dc1600700 1 -- 172.21.15.27:0/613767466 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7db8003a50 con 0x7f7dc407eea0 2024-04-26T22:46:58.370 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.382+0000 7f7dc1600700 1 -- 172.21.15.27:0/613767466 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7db8019690 con 0x7f7dc407eea0 2024-04-26T22:46:58.370 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.382+0000 7f7dcb600700 1 -- 172.21.15.27:0/613767466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc410c820 con 0x7f7dc407eea0 2024-04-26T22:46:58.370 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.383+0000 7f7dcb600700 1 -- 172.21.15.27:0/613767466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc4108de0 con 0x7f7dc407eea0 2024-04-26T22:46:58.370 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.384+0000 7f7dcb600700 1 -- 172.21.15.27:0/613767466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc410c9f0 con 0x7f7dc407eea0 2024-04-26T22:46:58.370 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.384+0000 7f7dc1600700 1 -- 172.21.15.27:0/613767466 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f7db801aa90 con 0x7f7dc407eea0 2024-04-26T22:46:58.370 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.385+0000 7f7dc1600700 1 --2- 172.21.15.27:0/613767466 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f7dac039650 0x7f7dac03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:46:58.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.385+0000 7f7dc1600700 1 -- 172.21.15.27:0/613767466 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7db804dd60 con 0x7f7dc407eea0 2024-04-26T22:46:58.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.385+0000 7f7dc3e00700 1 -- 172.21.15.27:0/613767466 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f7dac039650 msgr2=0x7f7dac03bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1661629783 2024-04-26T22:46:58.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.385+0000 7f7dc3e00700 1 --2- 172.21.15.27:0/613767466 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f7dac039650 0x7f7dac03bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T22:46:58.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.391+0000 7f7dc1600700 1 -- 172.21.15.27:0/613767466 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7dc410c9f0 con 0x7f7dc407eea0 2024-04-26T22:46:58.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.586+0000 7f7dc3e00700 1 -- 172.21.15.27:0/613767466 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f7dac039650 msgr2=0x7f7dac03bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1661629783 2024-04-26T22:46:58.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.586+0000 7f7dc3e00700 1 --2- 172.21.15.27:0/613767466 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f7dac039650 0x7f7dac03bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T22:46:58.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.682+0000 7f7dcb600700 1 -- 172.21.15.27:0/613767466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f7dc4109430 con 0x7f7dc407eea0 2024-04-26T22:46:58.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.682+0000 7f7dc1600700 1 -- 172.21.15.27:0/613767466 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f7db800b3e0 con 0x7f7dc407eea0 2024-04-26T22:46:58.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.685+0000 7f7dcb600700 1 -- 172.21.15.27:0/613767466 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f7dac039650 msgr2=0x7f7dac03bb10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:46:58.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.685+0000 7f7dcb600700 1 --2- 172.21.15.27:0/613767466 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f7dac039650 0x7f7dac03bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:58.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.685+0000 7f7dcb600700 1 -- 172.21.15.27:0/613767466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dc407eea0 msgr2=0x7f7dc410c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:46:58.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.685+0000 7f7dcb600700 1 --2- 172.21.15.27:0/613767466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dc407eea0 0x7f7dc410c050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7db8009520 tx=0x7f7db8011aa0 comp rx=0 tx=0).stop 2024-04-26T22:46:58.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.685+0000 7f7dcb600700 1 -- 172.21.15.27:0/613767466 shutdown_connections 2024-04-26T22:46:58.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.685+0000 7f7dcb600700 1 --2- 172.21.15.27:0/613767466 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f7dac039650 0x7f7dac03bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:58.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.685+0000 7f7dcb600700 1 --2- 172.21.15.27:0/613767466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dc407eea0 0x7f7dc410c050 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:46:58.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.685+0000 7f7dcb600700 1 -- 172.21.15.27:0/613767466 >> 172.21.15.27:0/613767466 conn(0x7f7dc41043e0 msgr2=0x7f7dc4104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:46:58.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.686+0000 7f7dcb600700 1 -- 172.21.15.27:0/613767466 shutdown_connections 2024-04-26T22:46:58.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:57.686+0000 7f7dcb600700 1 -- 172.21.15.27:0/613767466 wait complete. 2024-04-26T22:46:58.373 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for the mgr to restart... 2024-04-26T22:46:58.373 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr epoch 9... 2024-04-26T22:46:58.614 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:58 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:58.317+0000 7f5bcd916200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T22:46:58.914 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:58 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:58.731+0000 7f5bcd916200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T22:46:58.914 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:58 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:58.861+0000 7f5bcd916200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T22:46:59.198 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:59 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:59.130+0000 7f5bcd916200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T22:46:59.599 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:46:59 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:46:59.396+0000 7f5bcd916200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:47:00.614 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:47:00 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:47:00.220+0000 7f5bcd916200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T22:47:01.114 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:47:00 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:47:00.793+0000 7f5bcd916200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T22:47:01.115 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:47:00 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:47:00.927+0000 7f5bcd916200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T22:47:01.614 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:47:01 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:47:01.203+0000 7f5bcd916200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T22:47:02.342 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:47:01 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:47:01.879+0000 7f5bcd916200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T22:47:02.343 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:47:02 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:47:02.008+0000 7f5bcd916200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T22:47:02.343 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:47:02 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:47:02.341+0000 7f5bcd916200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T22:47:02.614 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:47:02 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:47:02.526+0000 7f5bcd916200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T22:47:03.114 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:47:02 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:47:02.659+0000 7f5bcd916200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T22:47:03.115 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:47:02 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:47:02.815+0000 7f5bcd916200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T22:47:03.614 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:47:03 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:47:03.200+0000 7f5bcd916200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T22:47:04.288 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:47:03 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:47:03.990+0000 7f5bcd916200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T22:47:04.614 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:47:04 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:47:04.287+0000 7f5bcd916200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T22:47:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:04 smithi027 ceph-mon[28034]: Active manager daemon x restarted 2024-04-26T22:47:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:04 smithi027 ceph-mon[28034]: Activating manager daemon x 2024-04-26T22:47:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:04 smithi027 ceph-mon[28034]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T22:47:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:04 smithi027 ceph-mon[28034]: mgrmap e10: x(active, starting, since 0.0117477s) 2024-04-26T22:47:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:47:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T22:47:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T22:47:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T22:47:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T22:47:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:04 smithi027 ceph-mon[28034]: Manager daemon x is now available 2024-04-26T22:47:05.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:47:05.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T22:47:05.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T22:47:05.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:05 smithi027 ceph-mon[28034]: mgrmap e11: x(active, since 1.01306s) 2024-04-26T22:47:06.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-04-26T22:47:06.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T22:47:06.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T22:47:06.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-04-26T22:47:06.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.987+0000 7f5761000700 1 Processor -- start 2024-04-26T22:47:06.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.988+0000 7f5761000700 1 -- start start 2024-04-26T22:47:06.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.988+0000 7f5761000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f575c108df0 0x7f575c109210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:06.190 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.988+0000 7f5761000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f575c1097e0 con 0x7f575c108df0 2024-04-26T22:47:06.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.989+0000 7f5759e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f575c108df0 0x7f575c109210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:06.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.989+0000 7f5759e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f575c108df0 0x7f575c109210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38128/0 (socket says 172.21.15.27:38128) 2024-04-26T22:47:06.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.989+0000 7f5759e00700 1 -- 172.21.15.27:0/1413375363 learned_addr learned my addr 172.21.15.27:0/1413375363 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:06.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.989+0000 7f5759e00700 1 -- 172.21.15.27:0/1413375363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575c109ff0 con 0x7f575c108df0 2024-04-26T22:47:06.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.990+0000 7f5759e00700 1 --2- 172.21.15.27:0/1413375363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f575c108df0 0x7f575c109210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f574c0098a0 tx=0x7f574c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edb42fcdea997e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:06.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.990+0000 7f5758a00700 1 -- 172.21.15.27:0/1413375363 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f574c004160 con 0x7f575c108df0 2024-04-26T22:47:06.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.990+0000 7f5758a00700 1 -- 172.21.15.27:0/1413375363 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f574c004300 con 0x7f575c108df0 2024-04-26T22:47:06.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.990+0000 7f5758a00700 1 -- 172.21.15.27:0/1413375363 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f574c011a20 con 0x7f575c108df0 2024-04-26T22:47:06.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.991+0000 7f5761000700 1 -- 172.21.15.27:0/1413375363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f575c108df0 msgr2=0x7f575c109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:06.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.991+0000 7f5761000700 1 --2- 172.21.15.27:0/1413375363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f575c108df0 0x7f575c109210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f574c0098a0 tx=0x7f574c009bb0 comp rx=0 tx=0).stop 2024-04-26T22:47:06.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.991+0000 7f5761000700 1 -- 172.21.15.27:0/1413375363 shutdown_connections 2024-04-26T22:47:06.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.991+0000 7f5761000700 1 --2- 172.21.15.27:0/1413375363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f575c108df0 0x7f575c109210 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:06.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.991+0000 7f5761000700 1 -- 172.21.15.27:0/1413375363 >> 172.21.15.27:0/1413375363 conn(0x7f575c104390 msgr2=0x7f575c1067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:06.191 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.991+0000 7f5761000700 1 -- 172.21.15.27:0/1413375363 shutdown_connections 2024-04-26T22:47:06.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.991+0000 7f5761000700 1 -- 172.21.15.27:0/1413375363 wait complete. 2024-04-26T22:47:06.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.993+0000 7f5761000700 1 Processor -- start 2024-04-26T22:47:06.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.993+0000 7f5761000700 1 -- start start 2024-04-26T22:47:06.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.993+0000 7f5761000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f575c108df0 0x7f575c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:06.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.993+0000 7f5761000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f575c10d3a0 con 0x7f575c108df0 2024-04-26T22:47:06.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.993+0000 7f5759e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f575c108df0 0x7f575c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:06.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.993+0000 7f5759e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f575c108df0 0x7f575c19eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38142/0 (socket says 172.21.15.27:38142) 2024-04-26T22:47:06.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.993+0000 7f5759e00700 1 -- 172.21.15.27:0/2213897591 learned_addr learned my addr 172.21.15.27:0/2213897591 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:06.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.994+0000 7f5759e00700 1 -- 172.21.15.27:0/2213897591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575c19f430 con 0x7f575c108df0 2024-04-26T22:47:06.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.994+0000 7f5759e00700 1 --2- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f575c108df0 0x7f575c19eef0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f574c009520 tx=0x7f574c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:06.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.994+0000 7f5752a00700 1 -- 172.21.15.27:0/2213897591 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f574c0039e0 con 0x7f575c108df0 2024-04-26T22:47:06.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.994+0000 7f5752a00700 1 -- 172.21.15.27:0/2213897591 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f574c01b8f0 con 0x7f575c108df0 2024-04-26T22:47:06.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.995+0000 7f5761000700 1 -- 172.21.15.27:0/2213897591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575c19f6c0 con 0x7f575c108df0 2024-04-26T22:47:06.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.995+0000 7f5752a00700 1 -- 172.21.15.27:0/2213897591 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f574c019770 con 0x7f575c108df0 2024-04-26T22:47:06.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.995+0000 7f5761000700 1 -- 172.21.15.27:0/2213897591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f575c1a23b0 con 0x7f575c108df0 2024-04-26T22:47:06.192 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.996+0000 7f5752a00700 1 -- 172.21.15.27:0/2213897591 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f574c019a70 con 0x7f575c108df0 2024-04-26T22:47:06.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.996+0000 7f5752a00700 1 --2- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5740039600 0x7f574003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:06.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.997+0000 7f5752a00700 1 -- 172.21.15.27:0/2213897591 --> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f574003c1d0 con 0x7f5740039600 2024-04-26T22:47:06.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.997+0000 7f5752a00700 1 -- 172.21.15.27:0/2213897591 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f574c04e030 con 0x7f575c108df0 2024-04-26T22:47:06.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.997+0000 7f5759400700 1 -- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5740039600 msgr2=0x7f574003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1661629783 2024-04-26T22:47:06.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:58.997+0000 7f5759400700 1 --2- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5740039600 0x7f574003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T22:47:06.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:59.198+0000 7f5759400700 1 -- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5740039600 msgr2=0x7f574003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1661629783 2024-04-26T22:47:06.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:59.198+0000 7f5759400700 1 --2- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5740039600 0x7f574003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T22:47:06.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:59.599+0000 7f5759400700 1 -- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5740039600 msgr2=0x7f574003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1661629783 2024-04-26T22:47:06.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:46:59.599+0000 7f5759400700 1 --2- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5740039600 0x7f574003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T22:47:06.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:00.400+0000 7f5759400700 1 -- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5740039600 msgr2=0x7f574003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1661629783 2024-04-26T22:47:06.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:00.400+0000 7f5759400700 1 --2- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5740039600 0x7f574003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T22:47:06.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:02.002+0000 7f5759400700 1 -- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5740039600 msgr2=0x7f574003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1661629783 2024-04-26T22:47:06.193 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:02.002+0000 7f5759400700 1 --2- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5740039600 0x7f574003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T22:47:06.194 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:04.299+0000 7f5752a00700 1 -- 172.21.15.27:0/2213897591 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f574c01a8f0 con 0x7f575c108df0 2024-04-26T22:47:06.194 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:04.299+0000 7f5752a00700 1 -- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5740039600 msgr2=0x7f574003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:47:06.194 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:04.299+0000 7f5752a00700 1 --2- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5740039600 0x7f574003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:06.194 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.301+0000 7f5752a00700 1 -- 172.21.15.27:0/2213897591 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f574c0500f0 con 0x7f575c108df0 2024-04-26T22:47:06.194 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.301+0000 7f5752a00700 1 --2- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f574003cfe0 0x7f574003f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:06.194 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.301+0000 7f5752a00700 1 -- 172.21.15.27:0/2213897591 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f574003c1d0 con 0x7f574003cfe0 2024-04-26T22:47:06.194 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.304+0000 7f5759400700 1 --2- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f574003cfe0 0x7f574003f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:06.194 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.306+0000 7f5759400700 1 --2- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f574003cfe0 0x7f574003f3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5748003a10 tx=0x7f57480092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:06.194 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.308+0000 7f5752a00700 1 -- 172.21.15.27:0/2213897591 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f574003c1d0 con 0x7f574003cfe0 2024-04-26T22:47:06.194 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.316+0000 7f5761000700 1 -- 172.21.15.27:0/2213897591 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f575c10d1f0 con 0x7f574003cfe0 2024-04-26T22:47:06.194 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.316+0000 7f5752a00700 1 -- 172.21.15.27:0/2213897591 <== mgr.14152 v2:172.21.15.27:6800/3636526714 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f575c10d1f0 con 0x7f574003cfe0 2024-04-26T22:47:06.194 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.317+0000 7f5761000700 1 -- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f574003cfe0 msgr2=0x7f574003f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:06.194 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.317+0000 7f5761000700 1 --2- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f574003cfe0 0x7f574003f3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5748003a10 tx=0x7f57480092b0 comp rx=0 tx=0).stop 2024-04-26T22:47:06.194 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.317+0000 7f5761000700 1 -- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f575c108df0 msgr2=0x7f575c19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:06.195 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.317+0000 7f5761000700 1 --2- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f575c108df0 0x7f575c19eef0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f574c009520 tx=0x7f574c011aa0 comp rx=0 tx=0).stop 2024-04-26T22:47:06.195 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.317+0000 7f5761000700 1 -- 172.21.15.27:0/2213897591 shutdown_connections 2024-04-26T22:47:06.195 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.317+0000 7f5761000700 1 --2- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f574003cfe0 0x7f574003f3d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:06.195 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.317+0000 7f5761000700 1 --2- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f575c108df0 0x7f575c19eef0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:06.195 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.317+0000 7f5761000700 1 --2- 172.21.15.27:0/2213897591 >> [v2:172.21.15.27:6800/1661629783,v1:172.21.15.27:6801/1661629783] conn(0x7f5740039600 0x7f574003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:06.195 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.317+0000 7f5761000700 1 -- 172.21.15.27:0/2213897591 >> 172.21.15.27:0/2213897591 conn(0x7f575c104390 msgr2=0x7f575c104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:06.195 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.318+0000 7f5761000700 1 -- 172.21.15.27:0/2213897591 shutdown_connections 2024-04-26T22:47:06.195 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:05.318+0000 7f5761000700 1 -- 172.21.15.27:0/2213897591 wait complete. 2024-04-26T22:47:06.195 INFO:teuthology.orchestra.run.smithi027.stdout:mgr epoch 9 is available 2024-04-26T22:47:06.195 INFO:teuthology.orchestra.run.smithi027.stdout:Generating a dashboard self-signed certificate... 2024-04-26T22:47:06.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:06 smithi027 ceph-mon[28034]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T22:47:06.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:06 smithi027 ceph-mon[28034]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T22:47:07.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:07 smithi027 ceph-mon[28034]: mgrmap e12: x(active, since 2s) 2024-04-26T22:47:07.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:07 smithi027 ceph-mon[28034]: [26/Apr/2024:22:47:06] ENGINE Bus STARTING 2024-04-26T22:47:07.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:07 smithi027 ceph-mon[28034]: [26/Apr/2024:22:47:06] ENGINE Serving on http://172.21.15.27:8765 2024-04-26T22:47:07.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:07 smithi027 ceph-mon[28034]: [26/Apr/2024:22:47:06] ENGINE Serving on https://172.21.15.27:7150 2024-04-26T22:47:07.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:07 smithi027 ceph-mon[28034]: [26/Apr/2024:22:47:06] ENGINE Bus STARTED 2024-04-26T22:47:07.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:08.511 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T22:47:08.511 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.486+0000 7f06a9000700 1 Processor -- start 2024-04-26T22:47:08.511 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.487+0000 7f06a9000700 1 -- start start 2024-04-26T22:47:08.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.487+0000 7f06a9000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a4108e00 0x7f06a4109220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:08.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.487+0000 7f06a9000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f06a4109760 con 0x7f06a4108e00 2024-04-26T22:47:08.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.488+0000 7f06a1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a4108e00 0x7f06a4109220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:08.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.488+0000 7f06a1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a4108e00 0x7f06a4109220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38222/0 (socket says 172.21.15.27:38222) 2024-04-26T22:47:08.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.488+0000 7f06a1e00700 1 -- 172.21.15.27:0/3036932455 learned_addr learned my addr 172.21.15.27:0/3036932455 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:08.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.488+0000 7f06a1e00700 1 -- 172.21.15.27:0/3036932455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a4109f60 con 0x7f06a4108e00 2024-04-26T22:47:08.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.489+0000 7f06a1e00700 1 --2- 172.21.15.27:0/3036932455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a4108e00 0x7f06a4109220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f06940098a0 tx=0x7f0694009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec6013537db1b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:08.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.489+0000 7f06a0a00700 1 -- 172.21.15.27:0/3036932455 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0694004160 con 0x7f06a4108e00 2024-04-26T22:47:08.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.489+0000 7f06a0a00700 1 -- 172.21.15.27:0/3036932455 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0694004300 con 0x7f06a4108e00 2024-04-26T22:47:08.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.489+0000 7f06a0a00700 1 -- 172.21.15.27:0/3036932455 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0694011a20 con 0x7f06a4108e00 2024-04-26T22:47:08.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.490+0000 7f06a9000700 1 -- 172.21.15.27:0/3036932455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a4108e00 msgr2=0x7f06a4109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:08.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.490+0000 7f06a9000700 1 --2- 172.21.15.27:0/3036932455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a4108e00 0x7f06a4109220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f06940098a0 tx=0x7f0694009bb0 comp rx=0 tx=0).stop 2024-04-26T22:47:08.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.490+0000 7f06a9000700 1 -- 172.21.15.27:0/3036932455 shutdown_connections 2024-04-26T22:47:08.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.490+0000 7f06a9000700 1 --2- 172.21.15.27:0/3036932455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a4108e00 0x7f06a4109220 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:08.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.490+0000 7f06a9000700 1 -- 172.21.15.27:0/3036932455 >> 172.21.15.27:0/3036932455 conn(0x7f06a41043e0 msgr2=0x7f06a4106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:08.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.490+0000 7f06a9000700 1 -- 172.21.15.27:0/3036932455 shutdown_connections 2024-04-26T22:47:08.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.490+0000 7f06a9000700 1 -- 172.21.15.27:0/3036932455 wait complete. 2024-04-26T22:47:08.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.491+0000 7f06a9000700 1 Processor -- start 2024-04-26T22:47:08.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.492+0000 7f06a9000700 1 -- start start 2024-04-26T22:47:08.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.492+0000 7f06a9000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a4108e00 0x7f06a419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:08.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.492+0000 7f06a9000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f06a410d3a0 con 0x7f06a4108e00 2024-04-26T22:47:08.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.492+0000 7f06a1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a4108e00 0x7f06a419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:08.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.492+0000 7f06a1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a4108e00 0x7f06a419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38236/0 (socket says 172.21.15.27:38236) 2024-04-26T22:47:08.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.492+0000 7f06a1e00700 1 -- 172.21.15.27:0/3888552624 learned_addr learned my addr 172.21.15.27:0/3888552624 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:08.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.493+0000 7f06a1e00700 1 -- 172.21.15.27:0/3888552624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a419f3f0 con 0x7f06a4108e00 2024-04-26T22:47:08.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.493+0000 7f06a1e00700 1 --2- 172.21.15.27:0/3888552624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a4108e00 0x7f06a419eeb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0694004660 tx=0x7f0694011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:08.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.493+0000 7f069aa00700 1 -- 172.21.15.27:0/3888552624 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0694003aa0 con 0x7f06a4108e00 2024-04-26T22:47:08.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.493+0000 7f069aa00700 1 -- 172.21.15.27:0/3888552624 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f069401b8f0 con 0x7f06a4108e00 2024-04-26T22:47:08.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.493+0000 7f06a9000700 1 -- 172.21.15.27:0/3888552624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a419f680 con 0x7f06a4108e00 2024-04-26T22:47:08.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.493+0000 7f069aa00700 1 -- 172.21.15.27:0/3888552624 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0694023900 con 0x7f06a4108e00 2024-04-26T22:47:08.514 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.494+0000 7f06a9000700 1 -- 172.21.15.27:0/3888552624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a41a2370 con 0x7f06a4108e00 2024-04-26T22:47:08.514 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.495+0000 7f06a9000700 1 -- 172.21.15.27:0/3888552624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a410d520 con 0x7f06a4108e00 2024-04-26T22:47:08.514 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.495+0000 7f069aa00700 1 -- 172.21.15.27:0/3888552624 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f0694023b70 con 0x7f06a4108e00 2024-04-26T22:47:08.514 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.496+0000 7f069aa00700 1 --2- 172.21.15.27:0/3888552624 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0688039660 0x7f068803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:08.514 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.496+0000 7f069aa00700 1 -- 172.21.15.27:0/3888552624 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f069404e010 con 0x7f06a4108e00 2024-04-26T22:47:08.514 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.496+0000 7f06a1400700 1 --2- 172.21.15.27:0/3888552624 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0688039660 0x7f068803bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:08.514 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.497+0000 7f06a1400700 1 --2- 172.21.15.27:0/3888552624 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0688039660 0x7f068803bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0690006fd0 tx=0x7f0690006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:08.514 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.502+0000 7f069aa00700 1 -- 172.21.15.27:0/3888552624 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f069401f030 con 0x7f06a4108e00 2024-04-26T22:47:08.514 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.743+0000 7f06a9000700 1 -- 172.21.15.27:0/3888552624 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f06a4105520 con 0x7f0688039660 2024-04-26T22:47:08.514 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.825+0000 7f069aa00700 1 -- 172.21.15.27:0/3888552624 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f06a4105520 con 0x7f0688039660 2024-04-26T22:47:08.514 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.828+0000 7f06a9000700 1 -- 172.21.15.27:0/3888552624 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0688039660 msgr2=0x7f068803bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:08.514 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.828+0000 7f06a9000700 1 --2- 172.21.15.27:0/3888552624 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0688039660 0x7f068803bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0690006fd0 tx=0x7f0690006e40 comp rx=0 tx=0).stop 2024-04-26T22:47:08.514 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.828+0000 7f06a9000700 1 -- 172.21.15.27:0/3888552624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a4108e00 msgr2=0x7f06a419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:08.514 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.828+0000 7f06a9000700 1 --2- 172.21.15.27:0/3888552624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a4108e00 0x7f06a419eeb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0694004660 tx=0x7f0694011aa0 comp rx=0 tx=0).stop 2024-04-26T22:47:08.515 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.828+0000 7f06a9000700 1 -- 172.21.15.27:0/3888552624 shutdown_connections 2024-04-26T22:47:08.515 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.828+0000 7f06a9000700 1 --2- 172.21.15.27:0/3888552624 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0688039660 0x7f068803bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:08.515 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.828+0000 7f06a9000700 1 --2- 172.21.15.27:0/3888552624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a4108e00 0x7f06a419eeb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:08.515 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.828+0000 7f06a9000700 1 -- 172.21.15.27:0/3888552624 >> 172.21.15.27:0/3888552624 conn(0x7f06a41043e0 msgr2=0x7f06a4104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:08.515 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.828+0000 7f06a9000700 1 -- 172.21.15.27:0/3888552624 shutdown_connections 2024-04-26T22:47:08.515 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:07.828+0000 7f06a9000700 1 -- 172.21.15.27:0/3888552624 wait complete. 2024-04-26T22:47:08.515 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial admin user... 2024-04-26T22:47:09.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:08 smithi027 ceph-mon[28034]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:47:09.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:08 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:09.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:08 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:10.643 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$9vAw8lhBDl6EcgFkXe/uCei4t6PTgdjS.a9Ks1ac3sV88Od2aLgS.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714171630, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T22:47:10.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.442+0000 7f8592e00700 1 Processor -- start 2024-04-26T22:47:10.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.442+0000 7f8592e00700 1 -- start start 2024-04-26T22:47:10.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.443+0000 7f8592e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f858c108e00 0x7f858c109220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:10.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.443+0000 7f8592e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f858c109760 con 0x7f858c108e00 2024-04-26T22:47:10.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.443+0000 7f858be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f858c108e00 0x7f858c109220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:10.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.443+0000 7f858be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f858c108e00 0x7f858c109220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57822/0 (socket says 172.21.15.27:57822) 2024-04-26T22:47:10.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.443+0000 7f858be00700 1 -- 172.21.15.27:0/2044925788 learned_addr learned my addr 172.21.15.27:0/2044925788 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:10.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.444+0000 7f858be00700 1 -- 172.21.15.27:0/2044925788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858c109f60 con 0x7f858c108e00 2024-04-26T22:47:10.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.444+0000 7f858be00700 1 --2- 172.21.15.27:0/2044925788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f858c108e00 0x7f858c109220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f857c0098a0 tx=0x7f857c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c868e7e39dc68e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:10.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.445+0000 7f858aa00700 1 -- 172.21.15.27:0/2044925788 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f857c004160 con 0x7f858c108e00 2024-04-26T22:47:10.648 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.445+0000 7f858aa00700 1 -- 172.21.15.27:0/2044925788 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f857c004300 con 0x7f858c108e00 2024-04-26T22:47:10.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.445+0000 7f858aa00700 1 -- 172.21.15.27:0/2044925788 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f857c011a20 con 0x7f858c108e00 2024-04-26T22:47:10.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.445+0000 7f8592e00700 1 -- 172.21.15.27:0/2044925788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f858c108e00 msgr2=0x7f858c109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:10.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.445+0000 7f8592e00700 1 --2- 172.21.15.27:0/2044925788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f858c108e00 0x7f858c109220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f857c0098a0 tx=0x7f857c009bb0 comp rx=0 tx=0).stop 2024-04-26T22:47:10.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.446+0000 7f8592e00700 1 -- 172.21.15.27:0/2044925788 shutdown_connections 2024-04-26T22:47:10.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.446+0000 7f8592e00700 1 --2- 172.21.15.27:0/2044925788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f858c108e00 0x7f858c109220 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:10.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.446+0000 7f8592e00700 1 -- 172.21.15.27:0/2044925788 >> 172.21.15.27:0/2044925788 conn(0x7f858c104460 msgr2=0x7f858c106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:10.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.446+0000 7f8592e00700 1 -- 172.21.15.27:0/2044925788 shutdown_connections 2024-04-26T22:47:10.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.446+0000 7f8592e00700 1 -- 172.21.15.27:0/2044925788 wait complete. 2024-04-26T22:47:10.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.447+0000 7f8592e00700 1 Processor -- start 2024-04-26T22:47:10.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.447+0000 7f8592e00700 1 -- start start 2024-04-26T22:47:10.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.448+0000 7f8592e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f858c108e00 0x7f858c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:10.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.448+0000 7f8592e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f858c10d3a0 con 0x7f858c108e00 2024-04-26T22:47:10.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.448+0000 7f858be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f858c108e00 0x7f858c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:10.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.448+0000 7f858be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f858c108e00 0x7f858c19eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57826/0 (socket says 172.21.15.27:57826) 2024-04-26T22:47:10.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.448+0000 7f858be00700 1 -- 172.21.15.27:0/1374080836 learned_addr learned my addr 172.21.15.27:0/1374080836 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:10.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.448+0000 7f858be00700 1 -- 172.21.15.27:0/1374080836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858c19f3f0 con 0x7f858c108e00 2024-04-26T22:47:10.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.449+0000 7f858be00700 1 --2- 172.21.15.27:0/1374080836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f858c108e00 0x7f858c19eeb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f857c0116e0 tx=0x7f857c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:10.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.449+0000 7f8588c00700 1 -- 172.21.15.27:0/1374080836 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f857c003a60 con 0x7f858c108e00 2024-04-26T22:47:10.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.449+0000 7f8588c00700 1 -- 172.21.15.27:0/1374080836 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f857c01b8f0 con 0x7f858c108e00 2024-04-26T22:47:10.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.449+0000 7f8588c00700 1 -- 172.21.15.27:0/1374080836 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f857c019860 con 0x7f858c108e00 2024-04-26T22:47:10.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.449+0000 7f8592e00700 1 -- 172.21.15.27:0/1374080836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f858c19f680 con 0x7f858c108e00 2024-04-26T22:47:10.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.449+0000 7f8592e00700 1 -- 172.21.15.27:0/1374080836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f858c1a2370 con 0x7f858c108e00 2024-04-26T22:47:10.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.451+0000 7f8592e00700 1 -- 172.21.15.27:0/1374080836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f858c10d520 con 0x7f858c108e00 2024-04-26T22:47:10.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.451+0000 7f8588c00700 1 -- 172.21.15.27:0/1374080836 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f857c01b440 con 0x7f858c108e00 2024-04-26T22:47:10.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.451+0000 7f8588c00700 1 --2- 172.21.15.27:0/1374080836 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8570039610 0x7f857003bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:10.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.451+0000 7f8588c00700 1 -- 172.21.15.27:0/1374080836 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f857c04de20 con 0x7f858c108e00 2024-04-26T22:47:10.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.452+0000 7f858b400700 1 --2- 172.21.15.27:0/1374080836 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8570039610 0x7f857003bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:10.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.453+0000 7f858b400700 1 --2- 172.21.15.27:0/1374080836 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8570039610 0x7f857003bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8578006fd0 tx=0x7f8578006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:10.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.457+0000 7f8588c00700 1 -- 172.21.15.27:0/1374080836 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f857c01f030 con 0x7f858c108e00 2024-04-26T22:47:10.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:09.736+0000 7f8592e00700 1 -- 172.21.15.27:0/1374080836 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f858c1a2600 con 0x7f8570039610 2024-04-26T22:47:10.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:10.011+0000 7f8588c00700 1 -- 172.21.15.27:0/1374080836 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f858c1a2600 con 0x7f8570039610 2024-04-26T22:47:10.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:10.014+0000 7f8592e00700 1 -- 172.21.15.27:0/1374080836 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8570039610 msgr2=0x7f857003bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:10.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:10.014+0000 7f8592e00700 1 --2- 172.21.15.27:0/1374080836 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8570039610 0x7f857003bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8578006fd0 tx=0x7f8578006e40 comp rx=0 tx=0).stop 2024-04-26T22:47:10.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:10.015+0000 7f8592e00700 1 -- 172.21.15.27:0/1374080836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f858c108e00 msgr2=0x7f858c19eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:10.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:10.015+0000 7f8592e00700 1 --2- 172.21.15.27:0/1374080836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f858c108e00 0x7f858c19eeb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f857c0116e0 tx=0x7f857c011aa0 comp rx=0 tx=0).stop 2024-04-26T22:47:10.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:10.015+0000 7f8592e00700 1 -- 172.21.15.27:0/1374080836 shutdown_connections 2024-04-26T22:47:10.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:10.015+0000 7f8592e00700 1 --2- 172.21.15.27:0/1374080836 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8570039610 0x7f857003bad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:10.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:10.015+0000 7f8592e00700 1 --2- 172.21.15.27:0/1374080836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f858c108e00 0x7f858c19eeb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:10.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:10.015+0000 7f8592e00700 1 -- 172.21.15.27:0/1374080836 >> 172.21.15.27:0/1374080836 conn(0x7f858c104460 msgr2=0x7f858c104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:10.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:10.015+0000 7f8592e00700 1 -- 172.21.15.27:0/1374080836 shutdown_connections 2024-04-26T22:47:10.652 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:10.015+0000 7f8592e00700 1 -- 172.21.15.27:0/1374080836 wait complete. 2024-04-26T22:47:10.652 INFO:teuthology.orchestra.run.smithi027.stdout:Fetching dashboard port number... 2024-04-26T22:47:11.008 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:10 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:11.008 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:10 smithi027 ceph-mon[28034]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:47:11.008 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:10 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:12.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:12 smithi027 ceph-mon[28034]: mgrmap e13: x(active, since 6s) 2024-04-26T22:47:12.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:12 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3608372017' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T22:47:13.177 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T22:47:13.177 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.335+0000 7fdce2800700 1 Processor -- start 2024-04-26T22:47:13.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.335+0000 7fdce2800700 1 -- start start 2024-04-26T22:47:13.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.336+0000 7fdce2800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcdc07f820 0x7fdcdc07fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:13.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.336+0000 7fdce2800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc080210 con 0x7fdcdc07f820 2024-04-26T22:47:13.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.336+0000 7fdcdb400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcdc07f820 0x7fdcdc07fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:13.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.336+0000 7fdcdb400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcdc07f820 0x7fdcdc07fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57832/0 (socket says 172.21.15.27:57832) 2024-04-26T22:47:13.182 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.336+0000 7fdcdb400700 1 -- 172.21.15.27:0/174312498 learned_addr learned my addr 172.21.15.27:0/174312498 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:13.183 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.337+0000 7fdcdb400700 1 -- 172.21.15.27:0/174312498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc080a70 con 0x7fdcdc07f820 2024-04-26T22:47:13.183 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.337+0000 7fdcdb400700 1 --2- 172.21.15.27:0/174312498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcdc07f820 0x7fdcdc07fc40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0098a0 tx=0x7fdccc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4686884f3e3dc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:13.183 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.337+0000 7fdcda000700 1 -- 172.21.15.27:0/174312498 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdccc004160 con 0x7fdcdc07f820 2024-04-26T22:47:13.183 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.337+0000 7fdcda000700 1 -- 172.21.15.27:0/174312498 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdccc004300 con 0x7fdcdc07f820 2024-04-26T22:47:13.183 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.338+0000 7fdcda000700 1 -- 172.21.15.27:0/174312498 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdccc011a20 con 0x7fdcdc07f820 2024-04-26T22:47:13.183 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.338+0000 7fdce2800700 1 -- 172.21.15.27:0/174312498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcdc07f820 msgr2=0x7fdcdc07fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:13.183 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.338+0000 7fdce2800700 1 --2- 172.21.15.27:0/174312498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcdc07f820 0x7fdcdc07fc40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0098a0 tx=0x7fdccc009bb0 comp rx=0 tx=0).stop 2024-04-26T22:47:13.183 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.338+0000 7fdce2800700 1 -- 172.21.15.27:0/174312498 shutdown_connections 2024-04-26T22:47:13.183 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.338+0000 7fdce2800700 1 --2- 172.21.15.27:0/174312498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcdc07f820 0x7fdcdc07fc40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:13.183 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.338+0000 7fdce2800700 1 -- 172.21.15.27:0/174312498 >> 172.21.15.27:0/174312498 conn(0x7fdcdc07e430 msgr2=0x7fdcdc07e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:13.183 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.339+0000 7fdce2800700 1 -- 172.21.15.27:0/174312498 shutdown_connections 2024-04-26T22:47:13.183 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.339+0000 7fdce2800700 1 -- 172.21.15.27:0/174312498 wait complete. 2024-04-26T22:47:13.183 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.340+0000 7fdce2800700 1 Processor -- start 2024-04-26T22:47:13.184 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.340+0000 7fdce2800700 1 -- start start 2024-04-26T22:47:13.184 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.340+0000 7fdce2800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcdc07f820 0x7fdcdc1a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:13.184 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.340+0000 7fdce2800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc081e30 con 0x7fdcdc07f820 2024-04-26T22:47:13.184 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.341+0000 7fdcdb400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcdc07f820 0x7fdcdc1a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:13.184 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.341+0000 7fdcdb400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcdc07f820 0x7fdcdc1a32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57836/0 (socket says 172.21.15.27:57836) 2024-04-26T22:47:13.184 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.341+0000 7fdcdb400700 1 -- 172.21.15.27:0/3608372017 learned_addr learned my addr 172.21.15.27:0/3608372017 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:13.184 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.341+0000 7fdcdb400700 1 -- 172.21.15.27:0/3608372017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc1a3830 con 0x7fdcdc07f820 2024-04-26T22:47:13.184 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.341+0000 7fdcdb400700 1 --2- 172.21.15.27:0/3608372017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcdc07f820 0x7fdcdc1a32f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdccc004660 tx=0x7fdccc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:13.184 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.342+0000 7fdcd3e00700 1 -- 172.21.15.27:0/3608372017 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdccc003aa0 con 0x7fdcdc07f820 2024-04-26T22:47:13.184 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.342+0000 7fdcd3e00700 1 -- 172.21.15.27:0/3608372017 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdccc01b8f0 con 0x7fdcdc07f820 2024-04-26T22:47:13.184 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.342+0000 7fdcd3e00700 1 -- 172.21.15.27:0/3608372017 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdccc023900 con 0x7fdcdc07f820 2024-04-26T22:47:13.184 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.342+0000 7fdce2800700 1 -- 172.21.15.27:0/3608372017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcdc1a3ac0 con 0x7fdcdc07f820 2024-04-26T22:47:13.184 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.342+0000 7fdce2800700 1 -- 172.21.15.27:0/3608372017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcdc1a67b0 con 0x7fdcdc07f820 2024-04-26T22:47:13.184 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.343+0000 7fdcd3e00700 1 -- 172.21.15.27:0/3608372017 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdccc01baa0 con 0x7fdcdc07f820 2024-04-26T22:47:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.343+0000 7fdce2800700 1 -- 172.21.15.27:0/3608372017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcdc064a80 con 0x7fdcdc07f820 2024-04-26T22:47:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.344+0000 7fdcd3e00700 1 --2- 172.21.15.27:0/3608372017 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdcc0039700 0x7fdcc003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.344+0000 7fdcd3e00700 1 -- 172.21.15.27:0/3608372017 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdccc04e170 con 0x7fdcdc07f820 2024-04-26T22:47:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.344+0000 7fdcdaa00700 1 --2- 172.21.15.27:0/3608372017 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdcc0039700 0x7fdcc003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.345+0000 7fdcdaa00700 1 --2- 172.21.15.27:0/3608372017 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdcc0039700 0x7fdcc003bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8006fd0 tx=0x7fdcc8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.350+0000 7fdcd3e00700 1 -- 172.21.15.27:0/3608372017 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdccc01f030 con 0x7fdcdc07f820 2024-04-26T22:47:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.588+0000 7fdce2800700 1 -- 172.21.15.27:0/3608372017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fdcdc02e760 con 0x7fdcdc07f820 2024-04-26T22:47:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.589+0000 7fdcd3e00700 1 -- 172.21.15.27:0/3608372017 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fdccc0202b0 con 0x7fdcdc07f820 2024-04-26T22:47:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.592+0000 7fdce2800700 1 -- 172.21.15.27:0/3608372017 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdcc0039700 msgr2=0x7fdcc003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.593+0000 7fdce2800700 1 --2- 172.21.15.27:0/3608372017 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdcc0039700 0x7fdcc003bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8006fd0 tx=0x7fdcc8006e40 comp rx=0 tx=0).stop 2024-04-26T22:47:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.593+0000 7fdce2800700 1 -- 172.21.15.27:0/3608372017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcdc07f820 msgr2=0x7fdcdc1a32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.593+0000 7fdce2800700 1 --2- 172.21.15.27:0/3608372017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcdc07f820 0x7fdcdc1a32f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdccc004660 tx=0x7fdccc011aa0 comp rx=0 tx=0).stop 2024-04-26T22:47:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.593+0000 7fdce2800700 1 -- 172.21.15.27:0/3608372017 shutdown_connections 2024-04-26T22:47:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.593+0000 7fdce2800700 1 --2- 172.21.15.27:0/3608372017 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdcc0039700 0x7fdcc003bbc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.593+0000 7fdce2800700 1 --2- 172.21.15.27:0/3608372017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcdc07f820 0x7fdcdc1a32f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.593+0000 7fdce2800700 1 -- 172.21.15.27:0/3608372017 >> 172.21.15.27:0/3608372017 conn(0x7fdcdc07e430 msgr2=0x7fdcdc199f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:13.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.594+0000 7fdce2800700 1 -- 172.21.15.27:0/3608372017 shutdown_connections 2024-04-26T22:47:13.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:11.594+0000 7fdce2800700 1 -- 172.21.15.27:0/3608372017 wait complete. 2024-04-26T22:47:13.209 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T22:47:13.209 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-04-26T22:47:13.231 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T22:47:13.231 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-04-26T22:47:13.231 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-04-26T22:47:13.231 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T22:47:13.238 INFO:teuthology.orchestra.run.smithi027.stdout:Ceph Dashboard is now available at: 2024-04-26T22:47:13.238 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:47:13.238 INFO:teuthology.orchestra.run.smithi027.stdout: URL: https://smithi027.front.sepia.ceph.com:8443/ 2024-04-26T22:47:13.239 INFO:teuthology.orchestra.run.smithi027.stdout: User: admin 2024-04-26T22:47:13.239 INFO:teuthology.orchestra.run.smithi027.stdout: Password: z98stm9uun 2024-04-26T22:47:13.239 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:47:13.240 INFO:teuthology.orchestra.run.smithi027.stdout:Saving cluster configuration to /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config directory 2024-04-26T22:47:13.242 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling autotune for osd_memory_target 2024-04-26T22:47:14.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.907+0000 7f4173c00700 1 Processor -- start 2024-04-26T22:47:14.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.908+0000 7f4173c00700 1 -- start start 2024-04-26T22:47:14.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.908+0000 7f4173c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c07ee40 0x7f416c07d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:14.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.908+0000 7f4173c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f416c07d870 con 0x7f416c07ee40 2024-04-26T22:47:14.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.908+0000 7f4171200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c07ee40 0x7f416c07d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:14.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.909+0000 7f4171200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c07ee40 0x7f416c07d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57846/0 (socket says 172.21.15.27:57846) 2024-04-26T22:47:14.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.909+0000 7f4171200700 1 -- 172.21.15.27:0/44561598 learned_addr learned my addr 172.21.15.27:0/44561598 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:14.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.909+0000 7f4171200700 1 -- 172.21.15.27:0/44561598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c07d9f0 con 0x7f416c07ee40 2024-04-26T22:47:14.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.909+0000 7f4171200700 1 --2- 172.21.15.27:0/44561598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c07ee40 0x7f416c07d2a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f41600098a0 tx=0x7f4160009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309a3160f3d79e31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:14.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.910+0000 7f416b400700 1 -- 172.21.15.27:0/44561598 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4160004160 con 0x7f416c07ee40 2024-04-26T22:47:14.875 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.910+0000 7f416b400700 1 -- 172.21.15.27:0/44561598 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4160004300 con 0x7f416c07ee40 2024-04-26T22:47:14.875 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.910+0000 7f416b400700 1 -- 172.21.15.27:0/44561598 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4160011a20 con 0x7f416c07ee40 2024-04-26T22:47:14.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.911+0000 7f4173c00700 1 -- 172.21.15.27:0/44561598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c07ee40 msgr2=0x7f416c07d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:14.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.911+0000 7f4173c00700 1 --2- 172.21.15.27:0/44561598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c07ee40 0x7f416c07d2a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f41600098a0 tx=0x7f4160009bb0 comp rx=0 tx=0).stop 2024-04-26T22:47:14.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.911+0000 7f4173c00700 1 -- 172.21.15.27:0/44561598 shutdown_connections 2024-04-26T22:47:14.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.911+0000 7f4173c00700 1 --2- 172.21.15.27:0/44561598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c07ee40 0x7f416c07d2a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:14.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.911+0000 7f4173c00700 1 -- 172.21.15.27:0/44561598 >> 172.21.15.27:0/44561598 conn(0x7f416c104390 msgr2=0x7f416c1067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:14.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.911+0000 7f4173c00700 1 -- 172.21.15.27:0/44561598 shutdown_connections 2024-04-26T22:47:14.880 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.911+0000 7f4173c00700 1 -- 172.21.15.27:0/44561598 wait complete. 2024-04-26T22:47:14.880 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.912+0000 7f4173c00700 1 Processor -- start 2024-04-26T22:47:14.880 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.912+0000 7f4173c00700 1 -- start start 2024-04-26T22:47:14.881 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.912+0000 7f4173c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c07ee40 0x7f416c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:14.881 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.913+0000 7f4173c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f416c10f410 con 0x7f416c07ee40 2024-04-26T22:47:14.881 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.913+0000 7f4171200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c07ee40 0x7f416c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:14.881 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.913+0000 7f4171200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c07ee40 0x7f416c19ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57856/0 (socket says 172.21.15.27:57856) 2024-04-26T22:47:14.881 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.913+0000 7f4171200700 1 -- 172.21.15.27:0/2320750394 learned_addr learned my addr 172.21.15.27:0/2320750394 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:14.881 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.913+0000 7f4171200700 1 -- 172.21.15.27:0/2320750394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c19f3a0 con 0x7f416c07ee40 2024-04-26T22:47:14.881 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.913+0000 7f4171200700 1 --2- 172.21.15.27:0/2320750394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c07ee40 0x7f416c19ee60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4160000c00 tx=0x7f4160011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:14.881 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.914+0000 7f4169600700 1 -- 172.21.15.27:0/2320750394 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4160003aa0 con 0x7f416c07ee40 2024-04-26T22:47:14.881 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.914+0000 7f4169600700 1 -- 172.21.15.27:0/2320750394 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f416001b8f0 con 0x7f416c07ee40 2024-04-26T22:47:14.881 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.914+0000 7f4169600700 1 -- 172.21.15.27:0/2320750394 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4160023870 con 0x7f416c07ee40 2024-04-26T22:47:14.881 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.914+0000 7f4173c00700 1 -- 172.21.15.27:0/2320750394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c19f630 con 0x7f416c07ee40 2024-04-26T22:47:14.881 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.914+0000 7f4173c00700 1 -- 172.21.15.27:0/2320750394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c1a2320 con 0x7f416c07ee40 2024-04-26T22:47:14.881 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.915+0000 7f4169600700 1 -- 172.21.15.27:0/2320750394 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4160023b70 con 0x7f416c07ee40 2024-04-26T22:47:14.881 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.915+0000 7f4173c00700 1 -- 172.21.15.27:0/2320750394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c064a80 con 0x7f416c07ee40 2024-04-26T22:47:14.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.916+0000 7f4169600700 1 --2- 172.21.15.27:0/2320750394 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f41540396a0 0x7f415403bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:14.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.916+0000 7f4169600700 1 -- 172.21.15.27:0/2320750394 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f416004e150 con 0x7f416c07ee40 2024-04-26T22:47:14.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.916+0000 7f416be00700 1 --2- 172.21.15.27:0/2320750394 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f41540396a0 0x7f415403bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:14.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.917+0000 7f416be00700 1 --2- 172.21.15.27:0/2320750394 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f41540396a0 0x7f415403bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4158006fd0 tx=0x7f4158006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:14.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:13.921+0000 7f4169600700 1 -- 172.21.15.27:0/2320750394 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f416001f030 con 0x7f416c07ee40 2024-04-26T22:47:14.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:14.152+0000 7f4173c00700 1 -- 172.21.15.27:0/2320750394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f416c1a24f0 con 0x7f416c07ee40 2024-04-26T22:47:14.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:14.153+0000 7f4169600700 1 -- 172.21.15.27:0/2320750394 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f4160021dc0 con 0x7f416c07ee40 2024-04-26T22:47:14.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:14.156+0000 7f4173c00700 1 -- 172.21.15.27:0/2320750394 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f41540396a0 msgr2=0x7f415403bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:14.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:14.156+0000 7f4173c00700 1 --2- 172.21.15.27:0/2320750394 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f41540396a0 0x7f415403bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4158006fd0 tx=0x7f4158006e40 comp rx=0 tx=0).stop 2024-04-26T22:47:14.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:14.156+0000 7f4173c00700 1 -- 172.21.15.27:0/2320750394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c07ee40 msgr2=0x7f416c19ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:14.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:14.156+0000 7f4173c00700 1 --2- 172.21.15.27:0/2320750394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c07ee40 0x7f416c19ee60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4160000c00 tx=0x7f4160011aa0 comp rx=0 tx=0).stop 2024-04-26T22:47:14.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:14.156+0000 7f4173c00700 1 -- 172.21.15.27:0/2320750394 shutdown_connections 2024-04-26T22:47:14.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:14.156+0000 7f4173c00700 1 --2- 172.21.15.27:0/2320750394 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f41540396a0 0x7f415403bb60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:14.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:14.156+0000 7f4173c00700 1 --2- 172.21.15.27:0/2320750394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c07ee40 0x7f416c19ee60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:14.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:14.156+0000 7f4173c00700 1 -- 172.21.15.27:0/2320750394 >> 172.21.15.27:0/2320750394 conn(0x7f416c104390 msgr2=0x7f416c104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:14.883 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:14.157+0000 7f4173c00700 1 -- 172.21.15.27:0/2320750394 shutdown_connections 2024-04-26T22:47:14.883 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:14.157+0000 7f4173c00700 1 -- 172.21.15.27:0/2320750394 wait complete. 2024-04-26T22:47:16.502 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.556+0000 7f388cc00700 1 Processor -- start 2024-04-26T22:47:16.503 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.557+0000 7f388cc00700 1 -- start start 2024-04-26T22:47:16.503 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.557+0000 7f388cc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f388810b110 0x7f388810b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:16.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.557+0000 7f388cc00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f388810bb00 con 0x7f388810b110 2024-04-26T22:47:16.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.557+0000 7f3885e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f388810b110 0x7f388810b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:16.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.558+0000 7f3885e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f388810b110 0x7f388810b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57870/0 (socket says 172.21.15.27:57870) 2024-04-26T22:47:16.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.558+0000 7f3885e00700 1 -- 172.21.15.27:0/2618873879 learned_addr learned my addr 172.21.15.27:0/2618873879 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:16.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.558+0000 7f3885e00700 1 -- 172.21.15.27:0/2618873879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388810c220 con 0x7f388810b110 2024-04-26T22:47:16.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.558+0000 7f3885e00700 1 --2- 172.21.15.27:0/2618873879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f388810b110 0x7f388810b530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f38780098a0 tx=0x7f3878009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dfbc428603d061 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:16.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.559+0000 7f3884a00700 1 -- 172.21.15.27:0/2618873879 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3878004160 con 0x7f388810b110 2024-04-26T22:47:16.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.559+0000 7f3884a00700 1 -- 172.21.15.27:0/2618873879 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3878004300 con 0x7f388810b110 2024-04-26T22:47:16.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.559+0000 7f3884a00700 1 -- 172.21.15.27:0/2618873879 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3878011a20 con 0x7f388810b110 2024-04-26T22:47:16.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.559+0000 7f388cc00700 1 -- 172.21.15.27:0/2618873879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f388810b110 msgr2=0x7f388810b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:16.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.559+0000 7f388cc00700 1 --2- 172.21.15.27:0/2618873879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f388810b110 0x7f388810b530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f38780098a0 tx=0x7f3878009bb0 comp rx=0 tx=0).stop 2024-04-26T22:47:16.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.559+0000 7f388cc00700 1 -- 172.21.15.27:0/2618873879 shutdown_connections 2024-04-26T22:47:16.506 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.559+0000 7f388cc00700 1 --2- 172.21.15.27:0/2618873879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f388810b110 0x7f388810b530 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:16.506 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.559+0000 7f388cc00700 1 -- 172.21.15.27:0/2618873879 >> 172.21.15.27:0/2618873879 conn(0x7f388807e430 msgr2=0x7f388807e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:16.506 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.560+0000 7f388cc00700 1 -- 172.21.15.27:0/2618873879 shutdown_connections 2024-04-26T22:47:16.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.560+0000 7f388cc00700 1 -- 172.21.15.27:0/2618873879 wait complete. 2024-04-26T22:47:16.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.561+0000 7f388cc00700 1 Processor -- start 2024-04-26T22:47:16.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.561+0000 7f388cc00700 1 -- start start 2024-04-26T22:47:16.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.561+0000 7f388cc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f388810b110 0x7f388819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:16.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.561+0000 7f388cc00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f388810f5d0 con 0x7f388810b110 2024-04-26T22:47:16.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.562+0000 7f3885e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f388810b110 0x7f388819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:16.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.562+0000 7f3885e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f388810b110 0x7f388819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57884/0 (socket says 172.21.15.27:57884) 2024-04-26T22:47:16.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.562+0000 7f3885e00700 1 -- 172.21.15.27:0/3952903662 learned_addr learned my addr 172.21.15.27:0/3952903662 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:16.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.562+0000 7f3885e00700 1 -- 172.21.15.27:0/3952903662 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388819f450 con 0x7f388810b110 2024-04-26T22:47:16.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.562+0000 7f3885e00700 1 --2- 172.21.15.27:0/3952903662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f388810b110 0x7f388819ef10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3878000c00 tx=0x7f3878011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:16.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.563+0000 7f387ea00700 1 -- 172.21.15.27:0/3952903662 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3878003aa0 con 0x7f388810b110 2024-04-26T22:47:16.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.563+0000 7f387ea00700 1 -- 172.21.15.27:0/3952903662 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f387801b8f0 con 0x7f388810b110 2024-04-26T22:47:16.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.563+0000 7f388cc00700 1 -- 172.21.15.27:0/3952903662 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388819f6e0 con 0x7f388810b110 2024-04-26T22:47:16.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.563+0000 7f387ea00700 1 -- 172.21.15.27:0/3952903662 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3878023890 con 0x7f388810b110 2024-04-26T22:47:16.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.563+0000 7f388cc00700 1 -- 172.21.15.27:0/3952903662 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38881a23d0 con 0x7f388810b110 2024-04-26T22:47:16.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.564+0000 7f387ea00700 1 -- 172.21.15.27:0/3952903662 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f387801b440 con 0x7f388810b110 2024-04-26T22:47:16.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.564+0000 7f388cc00700 1 -- 172.21.15.27:0/3952903662 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3888064a80 con 0x7f388810b110 2024-04-26T22:47:16.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.565+0000 7f387ea00700 1 --2- 172.21.15.27:0/3952903662 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f386c039700 0x7f386c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:16.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.565+0000 7f387ea00700 1 -- 172.21.15.27:0/3952903662 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f387804dff0 con 0x7f388810b110 2024-04-26T22:47:16.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.565+0000 7f3885400700 1 --2- 172.21.15.27:0/3952903662 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f386c039700 0x7f386c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:16.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.566+0000 7f3885400700 1 --2- 172.21.15.27:0/3952903662 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f386c039700 0x7f386c03bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3874006fd0 tx=0x7f3874006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:16.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.570+0000 7f387ea00700 1 -- 172.21.15.27:0/3952903662 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f387801f030 con 0x7f388810b110 2024-04-26T22:47:16.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.922+0000 7f388cc00700 1 -- 172.21.15.27:0/3952903662 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f38881a25a0 con 0x7f388810b110 2024-04-26T22:47:16.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.928+0000 7f387ea00700 1 -- 172.21.15.27:0/3952903662 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f3878016260 con 0x7f388810b110 2024-04-26T22:47:16.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.930+0000 7f388cc00700 1 -- 172.21.15.27:0/3952903662 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f386c039700 msgr2=0x7f386c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:16.511 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.930+0000 7f388cc00700 1 --2- 172.21.15.27:0/3952903662 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f386c039700 0x7f386c03bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3874006fd0 tx=0x7f3874006e40 comp rx=0 tx=0).stop 2024-04-26T22:47:16.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.930+0000 7f388cc00700 1 -- 172.21.15.27:0/3952903662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f388810b110 msgr2=0x7f388819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:16.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.930+0000 7f388cc00700 1 --2- 172.21.15.27:0/3952903662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f388810b110 0x7f388819ef10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3878000c00 tx=0x7f3878011aa0 comp rx=0 tx=0).stop 2024-04-26T22:47:16.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.931+0000 7f388cc00700 1 -- 172.21.15.27:0/3952903662 shutdown_connections 2024-04-26T22:47:16.512 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.931+0000 7f388cc00700 1 --2- 172.21.15.27:0/3952903662 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f386c039700 0x7f386c03bbc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:16.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.931+0000 7f388cc00700 1 --2- 172.21.15.27:0/3952903662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f388810b110 0x7f388819ef10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:16.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.931+0000 7f388cc00700 1 -- 172.21.15.27:0/3952903662 >> 172.21.15.27:0/3952903662 conn(0x7f388807e430 msgr2=0x7f3888108cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:16.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.931+0000 7f388cc00700 1 -- 172.21.15.27:0/3952903662 shutdown_connections 2024-04-26T22:47:16.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T22:47:15.931+0000 7f388cc00700 1 -- 172.21.15.27:0/3952903662 wait complete. 2024-04-26T22:47:16.513 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T22:47:16.513 INFO:teuthology.orchestra.run.smithi027.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T22:47:16.513 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:47:16.513 INFO:teuthology.orchestra.run.smithi027.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:47:16.513 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:47:16.513 INFO:teuthology.orchestra.run.smithi027.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T22:47:16.513 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:47:16.513 INFO:teuthology.orchestra.run.smithi027.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T22:47:16.514 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:47:16.514 INFO:teuthology.orchestra.run.smithi027.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T22:47:16.514 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:47:16.514 INFO:teuthology.orchestra.run.smithi027.stdout: ceph telemetry on 2024-04-26T22:47:16.514 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:47:16.514 INFO:teuthology.orchestra.run.smithi027.stdout:For more information see: 2024-04-26T22:47:16.514 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:47:16.514 INFO:teuthology.orchestra.run.smithi027.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-26T22:47:16.514 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:47:16.514 INFO:teuthology.orchestra.run.smithi027.stdout:Bootstrap complete. 2024-04-26T22:47:16.625 INFO:tasks.cephadm:Fetching config... 2024-04-26T22:47:16.625 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:47:16.625 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T22:47:16.676 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T22:47:16.676 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:47:16.677 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T22:47:16.767 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T22:47:16.767 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:47:16.767 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-26T22:47:16.890 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T22:47:16.890 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:47:16.890 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T22:47:16.943 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T22:47:16.943 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbQiDkHDWminIjhAUMuY15Ev6aElnRBihvMMt7y17jYAqoTPcdwrjM4pCo3pej4H/tiNqvStwnqY6h1Zno7w9rDWKC1xlOHc1zKdfOLUM3jK8DAcsE2B9pPrdu3V8V6ghb8NJvzcxj3F0LRTpTp6qsPXPpEPEOr/Ub2C22pegckKCvuHkDEOMX1M3qP3WNqgaJP4iptXtpMc3Kk41mkDhifiYDOmgi/Hfa1SIi+DM2lrsuJ0eGUEG4XJrKd8FOXH1mFxOnLSBt1vtuKpE62WbgOToOtR07HVPAl0NAkZGlYxzzt/6GeYNmkJXT17kt+AFPZzXO3QgStWhxQLFtOcWQ3DfSdoVjg/gqP+AuDaFkytbvJnLx9TBghf6c6X6VLg7MfjVl+FFUEbllKPYM5EWiWx9/r1uYPbFWZhHa+7QQcaXLsfU3tiOHS0fhrCADbdKUg+biqsYj3Hoeg49tLwbX6gu3Gl2HmQJCzHDlnijiv4vHphkQdXVcn6tsTQAvL4U= ceph-95ebdda6-041e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T22:47:17.050 INFO:teuthology.orchestra.run.smithi027.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbQiDkHDWminIjhAUMuY15Ev6aElnRBihvMMt7y17jYAqoTPcdwrjM4pCo3pej4H/tiNqvStwnqY6h1Zno7w9rDWKC1xlOHc1zKdfOLUM3jK8DAcsE2B9pPrdu3V8V6ghb8NJvzcxj3F0LRTpTp6qsPXPpEPEOr/Ub2C22pegckKCvuHkDEOMX1M3qP3WNqgaJP4iptXtpMc3Kk41mkDhifiYDOmgi/Hfa1SIi+DM2lrsuJ0eGUEG4XJrKd8FOXH1mFxOnLSBt1vtuKpE62WbgOToOtR07HVPAl0NAkZGlYxzzt/6GeYNmkJXT17kt+AFPZzXO3QgStWhxQLFtOcWQ3DfSdoVjg/gqP+AuDaFkytbvJnLx9TBghf6c6X6VLg7MfjVl+FFUEbllKPYM5EWiWx9/r1uYPbFWZhHa+7QQcaXLsfU3tiOHS0fhrCADbdKUg+biqsYj3Hoeg49tLwbX6gu3Gl2HmQJCzHDlnijiv4vHphkQdXVcn6tsTQAvL4U= ceph-95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:47:17.103 DEBUG:teuthology.orchestra.run.smithi080:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbQiDkHDWminIjhAUMuY15Ev6aElnRBihvMMt7y17jYAqoTPcdwrjM4pCo3pej4H/tiNqvStwnqY6h1Zno7w9rDWKC1xlOHc1zKdfOLUM3jK8DAcsE2B9pPrdu3V8V6ghb8NJvzcxj3F0LRTpTp6qsPXPpEPEOr/Ub2C22pegckKCvuHkDEOMX1M3qP3WNqgaJP4iptXtpMc3Kk41mkDhifiYDOmgi/Hfa1SIi+DM2lrsuJ0eGUEG4XJrKd8FOXH1mFxOnLSBt1vtuKpE62WbgOToOtR07HVPAl0NAkZGlYxzzt/6GeYNmkJXT17kt+AFPZzXO3QgStWhxQLFtOcWQ3DfSdoVjg/gqP+AuDaFkytbvJnLx9TBghf6c6X6VLg7MfjVl+FFUEbllKPYM5EWiWx9/r1uYPbFWZhHa+7QQcaXLsfU3tiOHS0fhrCADbdKUg+biqsYj3Hoeg49tLwbX6gu3Gl2HmQJCzHDlnijiv4vHphkQdXVcn6tsTQAvL4U= ceph-95ebdda6-041e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T22:47:17.124 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:16 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3952903662' entity='client.admin' 2024-04-26T22:47:17.142 INFO:teuthology.orchestra.run.smithi080.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbQiDkHDWminIjhAUMuY15Ev6aElnRBihvMMt7y17jYAqoTPcdwrjM4pCo3pej4H/tiNqvStwnqY6h1Zno7w9rDWKC1xlOHc1zKdfOLUM3jK8DAcsE2B9pPrdu3V8V6ghb8NJvzcxj3F0LRTpTp6qsPXPpEPEOr/Ub2C22pegckKCvuHkDEOMX1M3qP3WNqgaJP4iptXtpMc3Kk41mkDhifiYDOmgi/Hfa1SIi+DM2lrsuJ0eGUEG4XJrKd8FOXH1mFxOnLSBt1vtuKpE62WbgOToOtR07HVPAl0NAkZGlYxzzt/6GeYNmkJXT17kt+AFPZzXO3QgStWhxQLFtOcWQ3DfSdoVjg/gqP+AuDaFkytbvJnLx9TBghf6c6X6VLg7MfjVl+FFUEbllKPYM5EWiWx9/r1uYPbFWZhHa+7QQcaXLsfU3tiOHS0fhrCADbdKUg+biqsYj3Hoeg49tLwbX6gu3Gl2HmQJCzHDlnijiv4vHphkQdXVcn6tsTQAvL4U= ceph-95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:47:17.153 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbQiDkHDWminIjhAUMuY15Ev6aElnRBihvMMt7y17jYAqoTPcdwrjM4pCo3pej4H/tiNqvStwnqY6h1Zno7w9rDWKC1xlOHc1zKdfOLUM3jK8DAcsE2B9pPrdu3V8V6ghb8NJvzcxj3F0LRTpTp6qsPXPpEPEOr/Ub2C22pegckKCvuHkDEOMX1M3qP3WNqgaJP4iptXtpMc3Kk41mkDhifiYDOmgi/Hfa1SIi+DM2lrsuJ0eGUEG4XJrKd8FOXH1mFxOnLSBt1vtuKpE62WbgOToOtR07HVPAl0NAkZGlYxzzt/6GeYNmkJXT17kt+AFPZzXO3QgStWhxQLFtOcWQ3DfSdoVjg/gqP+AuDaFkytbvJnLx9TBghf6c6X6VLg7MfjVl+FFUEbllKPYM5EWiWx9/r1uYPbFWZhHa+7QQcaXLsfU3tiOHS0fhrCADbdKUg+biqsYj3Hoeg49tLwbX6gu3Gl2HmQJCzHDlnijiv4vHphkQdXVcn6tsTQAvL4U= ceph-95ebdda6-041e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T22:47:17.191 INFO:teuthology.orchestra.run.smithi094.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbQiDkHDWminIjhAUMuY15Ev6aElnRBihvMMt7y17jYAqoTPcdwrjM4pCo3pej4H/tiNqvStwnqY6h1Zno7w9rDWKC1xlOHc1zKdfOLUM3jK8DAcsE2B9pPrdu3V8V6ghb8NJvzcxj3F0LRTpTp6qsPXPpEPEOr/Ub2C22pegckKCvuHkDEOMX1M3qP3WNqgaJP4iptXtpMc3Kk41mkDhifiYDOmgi/Hfa1SIi+DM2lrsuJ0eGUEG4XJrKd8FOXH1mFxOnLSBt1vtuKpE62WbgOToOtR07HVPAl0NAkZGlYxzzt/6GeYNmkJXT17kt+AFPZzXO3QgStWhxQLFtOcWQ3DfSdoVjg/gqP+AuDaFkytbvJnLx9TBghf6c6X6VLg7MfjVl+FFUEbllKPYM5EWiWx9/r1uYPbFWZhHa+7QQcaXLsfU3tiOHS0fhrCADbdKUg+biqsYj3Hoeg49tLwbX6gu3Gl2HmQJCzHDlnijiv4vHphkQdXVcn6tsTQAvL4U= ceph-95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:47:17.203 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T22:47:17.657 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:47:20.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.519+0000 7f6cc2e00700 1 -- 172.21.15.27:0/2282578969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cbc1050b0 msgr2=0x7f6cbc1054d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:20.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.519+0000 7f6cc2e00700 1 --2- 172.21.15.27:0/2282578969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cbc1050b0 0x7f6cbc1054d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009910 tx=0x7f6cac009c20 comp rx=0 tx=0).stop 2024-04-26T22:47:20.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.519+0000 7f6cc2e00700 1 -- 172.21.15.27:0/2282578969 shutdown_connections 2024-04-26T22:47:20.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.519+0000 7f6cc2e00700 1 --2- 172.21.15.27:0/2282578969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cbc1050b0 0x7f6cbc1054d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:20.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.519+0000 7f6cc2e00700 1 -- 172.21.15.27:0/2282578969 >> 172.21.15.27:0/2282578969 conn(0x7f6cbc100670 msgr2=0x7f6cbc102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:20.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.519+0000 7f6cc2e00700 1 -- 172.21.15.27:0/2282578969 shutdown_connections 2024-04-26T22:47:20.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.519+0000 7f6cc2e00700 1 -- 172.21.15.27:0/2282578969 wait complete. 2024-04-26T22:47:20.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.521+0000 7f6cc2e00700 1 Processor -- start 2024-04-26T22:47:20.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.521+0000 7f6cc2e00700 1 -- start start 2024-04-26T22:47:20.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.522+0000 7f6cc2e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cbc1050b0 0x7f6cbc19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:20.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.522+0000 7f6cc2e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc10aff0 con 0x7f6cbc1050b0 2024-04-26T22:47:20.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.522+0000 7f6cbbe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cbc1050b0 0x7f6cbc19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:20.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.522+0000 7f6cbbe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cbc1050b0 0x7f6cbc19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44464/0 (socket says 172.21.15.27:44464) 2024-04-26T22:47:20.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.522+0000 7f6cbbe00700 1 -- 172.21.15.27:0/3497838203 learned_addr learned my addr 172.21.15.27:0/3497838203 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:20.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.523+0000 7f6cbbe00700 1 -- 172.21.15.27:0/3497838203 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc19ac10 con 0x7f6cbc1050b0 2024-04-26T22:47:20.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.523+0000 7f6cbbe00700 1 --2- 172.21.15.27:0/3497838203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cbc1050b0 0x7f6cbc19a6d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6cac005950 tx=0x7f6cac004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:20.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.523+0000 7f6cb8c00700 1 -- 172.21.15.27:0/3497838203 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cac026070 con 0x7f6cbc1050b0 2024-04-26T22:47:20.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.523+0000 7f6cb8c00700 1 -- 172.21.15.27:0/3497838203 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cac00bc00 con 0x7f6cbc1050b0 2024-04-26T22:47:20.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.523+0000 7f6cb8c00700 1 -- 172.21.15.27:0/3497838203 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cac003a10 con 0x7f6cbc1050b0 2024-04-26T22:47:20.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.523+0000 7f6cc2e00700 1 -- 172.21.15.27:0/3497838203 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cbc19aea0 con 0x7f6cbc1050b0 2024-04-26T22:47:20.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.523+0000 7f6cc2e00700 1 -- 172.21.15.27:0/3497838203 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cbc19db90 con 0x7f6cbc1050b0 2024-04-26T22:47:20.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.525+0000 7f6cb8c00700 1 -- 172.21.15.27:0/3497838203 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6cac029070 con 0x7f6cbc1050b0 2024-04-26T22:47:20.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.525+0000 7f6cc2e00700 1 -- 172.21.15.27:0/3497838203 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cbc10b770 con 0x7f6cbc1050b0 2024-04-26T22:47:20.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.525+0000 7f6cb8c00700 1 --2- 172.21.15.27:0/3497838203 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6ca00396b0 0x7f6ca003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:20.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.525+0000 7f6cb8c00700 1 -- 172.21.15.27:0/3497838203 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6cac01e070 con 0x7f6cbc1050b0 2024-04-26T22:47:20.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.525+0000 7f6cbb400700 1 --2- 172.21.15.27:0/3497838203 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6ca00396b0 0x7f6ca003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:20.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.526+0000 7f6cbb400700 1 --2- 172.21.15.27:0/3497838203 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6ca00396b0 0x7f6ca003bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8006fd0 tx=0x7f6ca8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:20.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.531+0000 7f6cb8c00700 1 -- 172.21.15.27:0/3497838203 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6cac013070 con 0x7f6cbc1050b0 2024-04-26T22:47:20.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.774+0000 7f6cc2e00700 1 -- 172.21.15.27:0/3497838203 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f6cbc02e5a0 con 0x7f6cbc1050b0 2024-04-26T22:47:20.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.779+0000 7f6cb8c00700 1 -- 172.21.15.27:0/3497838203 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f6cac0300c0 con 0x7f6cbc1050b0 2024-04-26T22:47:20.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.782+0000 7f6cc2e00700 1 -- 172.21.15.27:0/3497838203 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6ca00396b0 msgr2=0x7f6ca003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:20.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.782+0000 7f6cc2e00700 1 --2- 172.21.15.27:0/3497838203 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6ca00396b0 0x7f6ca003bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8006fd0 tx=0x7f6ca8006e40 comp rx=0 tx=0).stop 2024-04-26T22:47:20.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.782+0000 7f6cc2e00700 1 -- 172.21.15.27:0/3497838203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cbc1050b0 msgr2=0x7f6cbc19a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:20.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.782+0000 7f6cc2e00700 1 --2- 172.21.15.27:0/3497838203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cbc1050b0 0x7f6cbc19a6d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6cac005950 tx=0x7f6cac004fa0 comp rx=0 tx=0).stop 2024-04-26T22:47:20.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.782+0000 7f6cc2e00700 1 -- 172.21.15.27:0/3497838203 shutdown_connections 2024-04-26T22:47:20.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.782+0000 7f6cc2e00700 1 --2- 172.21.15.27:0/3497838203 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6ca00396b0 0x7f6ca003bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:20.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.782+0000 7f6cc2e00700 1 --2- 172.21.15.27:0/3497838203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cbc1050b0 0x7f6cbc19a6d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:20.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.782+0000 7f6cc2e00700 1 -- 172.21.15.27:0/3497838203 >> 172.21.15.27:0/3497838203 conn(0x7f6cbc100670 msgr2=0x7f6cbc101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:20.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.783+0000 7f6cc2e00700 1 -- 172.21.15.27:0/3497838203 shutdown_connections 2024-04-26T22:47:20.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:20.783+0000 7f6cc2e00700 1 -- 172.21.15.27:0/3497838203 wait complete. 2024-04-26T22:47:21.759 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T22:47:21.759 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T22:47:22.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:21 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3497838203' entity='client.admin' 2024-04-26T22:47:22.220 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:47:24.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:24 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:24.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:24 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:24.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:24 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:47:24.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:24 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:24.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:24 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:47:24.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:24 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:25.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.048+0000 7fef86200700 1 -- 172.21.15.27:0/4190208812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef80102e80 msgr2=0x7fef801032a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:25.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.048+0000 7fef86200700 1 --2- 172.21.15.27:0/4190208812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef80102e80 0x7fef801032a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fef70009910 tx=0x7fef70009c20 comp rx=0 tx=0).stop 2024-04-26T22:47:25.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.049+0000 7fef86200700 1 -- 172.21.15.27:0/4190208812 shutdown_connections 2024-04-26T22:47:25.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.049+0000 7fef86200700 1 --2- 172.21.15.27:0/4190208812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef80102e80 0x7fef801032a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:25.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.049+0000 7fef86200700 1 -- 172.21.15.27:0/4190208812 >> 172.21.15.27:0/4190208812 conn(0x7fef800fe420 msgr2=0x7fef80100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:25.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.049+0000 7fef86200700 1 -- 172.21.15.27:0/4190208812 shutdown_connections 2024-04-26T22:47:25.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.049+0000 7fef86200700 1 -- 172.21.15.27:0/4190208812 wait complete. 2024-04-26T22:47:25.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.050+0000 7fef86200700 1 Processor -- start 2024-04-26T22:47:25.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.050+0000 7fef86200700 1 -- start start 2024-04-26T22:47:25.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.051+0000 7fef86200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef80102e80 0x7fef8019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:25.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.051+0000 7fef86200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fef80108d40 con 0x7fef80102e80 2024-04-26T22:47:25.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.051+0000 7fef7f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef80102e80 0x7fef8019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:25.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.051+0000 7fef7f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef80102e80 0x7fef8019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44468/0 (socket says 172.21.15.27:44468) 2024-04-26T22:47:25.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.051+0000 7fef7f400700 1 -- 172.21.15.27:0/882368463 learned_addr learned my addr 172.21.15.27:0/882368463 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:25.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.052+0000 7fef7f400700 1 -- 172.21.15.27:0/882368463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8019ac10 con 0x7fef80102e80 2024-04-26T22:47:25.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.052+0000 7fef7f400700 1 --2- 172.21.15.27:0/882368463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef80102e80 0x7fef8019a6d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fef70005190 tx=0x7fef70004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:25.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.052+0000 7fef7cc00700 1 -- 172.21.15.27:0/882368463 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef70027070 con 0x7fef80102e80 2024-04-26T22:47:25.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.052+0000 7fef7cc00700 1 -- 172.21.15.27:0/882368463 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef70022470 con 0x7fef80102e80 2024-04-26T22:47:25.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.052+0000 7fef86200700 1 -- 172.21.15.27:0/882368463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8019aea0 con 0x7fef80102e80 2024-04-26T22:47:25.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.052+0000 7fef7cc00700 1 -- 172.21.15.27:0/882368463 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef7000f5f0 con 0x7fef80102e80 2024-04-26T22:47:25.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.052+0000 7fef86200700 1 -- 172.21.15.27:0/882368463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef8019db90 con 0x7fef80102e80 2024-04-26T22:47:25.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.053+0000 7fef7cc00700 1 -- 172.21.15.27:0/882368463 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fef7002a070 con 0x7fef80102e80 2024-04-26T22:47:25.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.053+0000 7fef86200700 1 -- 172.21.15.27:0/882368463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef801094c0 con 0x7fef80102e80 2024-04-26T22:47:25.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.054+0000 7fef7cc00700 1 --2- 172.21.15.27:0/882368463 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fef640396b0 0x7fef6403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:25.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.054+0000 7fef7cc00700 1 -- 172.21.15.27:0/882368463 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fef7001e030 con 0x7fef80102e80 2024-04-26T22:47:25.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.054+0000 7fef76a00700 1 --2- 172.21.15.27:0/882368463 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fef640396b0 0x7fef6403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:25.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.055+0000 7fef76a00700 1 --2- 172.21.15.27:0/882368463 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fef640396b0 0x7fef6403bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fef68006fd0 tx=0x7fef68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:25.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.060+0000 7fef7cc00700 1 -- 172.21.15.27:0/882368463 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fef70013070 con 0x7fef80102e80 2024-04-26T22:47:25.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.297+0000 7fef86200700 1 -- 172.21.15.27:0/882368463 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fef8002e5a0 con 0x7fef640396b0 2024-04-26T22:47:25.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.309+0000 7fef7cc00700 1 -- 172.21.15.27:0/882368463 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fef8002e5a0 con 0x7fef640396b0 2024-04-26T22:47:25.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.312+0000 7fef86200700 1 -- 172.21.15.27:0/882368463 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fef640396b0 msgr2=0x7fef6403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:25.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.312+0000 7fef86200700 1 --2- 172.21.15.27:0/882368463 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fef640396b0 0x7fef6403bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fef68006fd0 tx=0x7fef68006e40 comp rx=0 tx=0).stop 2024-04-26T22:47:25.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.312+0000 7fef86200700 1 -- 172.21.15.27:0/882368463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef80102e80 msgr2=0x7fef8019a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:25.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.312+0000 7fef86200700 1 --2- 172.21.15.27:0/882368463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef80102e80 0x7fef8019a6d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fef70005190 tx=0x7fef70004f80 comp rx=0 tx=0).stop 2024-04-26T22:47:25.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.312+0000 7fef86200700 1 -- 172.21.15.27:0/882368463 shutdown_connections 2024-04-26T22:47:25.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.312+0000 7fef86200700 1 --2- 172.21.15.27:0/882368463 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fef640396b0 0x7fef6403bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:25.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.312+0000 7fef86200700 1 --2- 172.21.15.27:0/882368463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef80102e80 0x7fef8019a6d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:25.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.312+0000 7fef86200700 1 -- 172.21.15.27:0/882368463 >> 172.21.15.27:0/882368463 conn(0x7fef800fe420 msgr2=0x7fef80100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:25.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.312+0000 7fef86200700 1 -- 172.21.15.27:0/882368463 shutdown_connections 2024-04-26T22:47:25.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:25.312+0000 7fef86200700 1 -- 172.21.15.27:0/882368463 wait complete. 2024-04-26T22:47:25.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:25 smithi027 ceph-mon[28034]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:26.135 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi080 2024-04-26T22:47:26.135 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-26T22:47:26.135 DEBUG:teuthology.orchestra.run.smithi080:> dd of=/etc/ceph/ceph.conf 2024-04-26T22:47:26.153 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-26T22:47:26.153 DEBUG:teuthology.orchestra.run.smithi080:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:47:26.210 INFO:tasks.cephadm:Adding host smithi080 to orchestrator... 2024-04-26T22:47:26.211 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch host add smithi080 2024-04-26T22:47:26.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:26 smithi027 ceph-mon[28034]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:47:26.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:26.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:47:26.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:47:26.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:47:26.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:26 smithi027 ceph-mon[28034]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-26T22:47:26.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:26 smithi027 ceph-mon[28034]: Updating smithi027:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:47:26.648 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:47:27.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:27 smithi027 ceph-mon[28034]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:27.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:27 smithi027 ceph-mon[28034]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:47:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.739+0000 7f0aa6a00700 1 -- 172.21.15.27:0/2982615574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0aa0102ef0 msgr2=0x7f0aa01032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.739+0000 7f0aa6a00700 1 --2- 172.21.15.27:0/2982615574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0aa0102ef0 0x7f0aa01032d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0a94009910 tx=0x7f0a94009c20 comp rx=0 tx=0).stop 2024-04-26T22:47:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.739+0000 7f0aa6a00700 1 -- 172.21.15.27:0/2982615574 shutdown_connections 2024-04-26T22:47:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.739+0000 7f0aa6a00700 1 --2- 172.21.15.27:0/2982615574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0aa0102ef0 0x7f0aa01032d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.739+0000 7f0aa6a00700 1 -- 172.21.15.27:0/2982615574 >> 172.21.15.27:0/2982615574 conn(0x7f0aa00fe7c0 msgr2=0x7f0aa0100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.739+0000 7f0aa6a00700 1 -- 172.21.15.27:0/2982615574 shutdown_connections 2024-04-26T22:47:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.739+0000 7f0aa6a00700 1 -- 172.21.15.27:0/2982615574 wait complete. 2024-04-26T22:47:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.740+0000 7f0aa6a00700 1 Processor -- start 2024-04-26T22:47:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.741+0000 7f0aa6a00700 1 -- start start 2024-04-26T22:47:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.741+0000 7f0aa6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0aa0102ef0 0x7f0aa01986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:28.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.741+0000 7f0aa6a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa010d170 con 0x7f0aa0102ef0 2024-04-26T22:47:28.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.742+0000 7f0aa5600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0aa0102ef0 0x7f0aa01986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:28.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.742+0000 7f0aa5600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0aa0102ef0 0x7f0aa01986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35264/0 (socket says 172.21.15.27:35264) 2024-04-26T22:47:28.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.742+0000 7f0aa5600700 1 -- 172.21.15.27:0/1093880936 learned_addr learned my addr 172.21.15.27:0/1093880936 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:28.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.742+0000 7f0aa5600700 1 -- 172.21.15.27:0/1093880936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa0198bf0 con 0x7f0aa0102ef0 2024-04-26T22:47:28.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.743+0000 7f0aa5600700 1 --2- 172.21.15.27:0/1093880936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0aa0102ef0 0x7f0aa01986b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0a94006010 tx=0x7f0a94005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:28.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.743+0000 7f0a9e000700 1 -- 172.21.15.27:0/1093880936 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a94026070 con 0x7f0aa0102ef0 2024-04-26T22:47:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.743+0000 7f0a9e000700 1 -- 172.21.15.27:0/1093880936 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a9400bba0 con 0x7f0aa0102ef0 2024-04-26T22:47:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.743+0000 7f0a9e000700 1 -- 172.21.15.27:0/1093880936 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a94003910 con 0x7f0aa0102ef0 2024-04-26T22:47:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.743+0000 7f0aa6a00700 1 -- 172.21.15.27:0/1093880936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa0198ee0 con 0x7f0aa0102ef0 2024-04-26T22:47:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.743+0000 7f0aa6a00700 1 -- 172.21.15.27:0/1093880936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa019cf80 con 0x7f0aa0102ef0 2024-04-26T22:47:28.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.744+0000 7f0aa6a00700 1 -- 172.21.15.27:0/1093880936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa01043d0 con 0x7f0aa0102ef0 2024-04-26T22:47:28.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.744+0000 7f0a9e000700 1 -- 172.21.15.27:0/1093880936 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0a94029070 con 0x7f0aa0102ef0 2024-04-26T22:47:28.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.745+0000 7f0a9e000700 1 --2- 172.21.15.27:0/1093880936 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0a880396b0 0x7f0a8803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:28.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.745+0000 7f0a9e000700 1 -- 172.21.15.27:0/1093880936 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0a9401e070 con 0x7f0aa0102ef0 2024-04-26T22:47:28.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.745+0000 7f0aa4c00700 1 --2- 172.21.15.27:0/1093880936 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0a880396b0 0x7f0a8803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:28.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.746+0000 7f0aa4c00700 1 --2- 172.21.15.27:0/1093880936 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0a880396b0 0x7f0a8803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c006fd0 tx=0x7f0a8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:28.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.753+0000 7f0a9e000700 1 -- 172.21.15.27:0/1093880936 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0a94013070 con 0x7f0aa0102ef0 2024-04-26T22:47:28.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:28.989+0000 7f0aa6a00700 1 -- 172.21.15.27:0/1093880936 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi080", "target": ["mon-mgr", ""]}) -- 0x7f0aa0199a10 con 0x7f0a880396b0 2024-04-26T22:47:29.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:29 smithi027 ceph-mon[28034]: Updating smithi027:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:47:29.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:29.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:29.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:30.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:30 smithi027 ceph-mon[28034]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:30.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:30 smithi027 ceph-mon[28034]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi080", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:47:31.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:31 smithi027 ceph-mon[28034]: Deploying cephadm binary to smithi080 2024-04-26T22:47:32.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:32 smithi027 ceph-mon[28034]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:33.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:33.954+0000 7f0a9e000700 1 -- 172.21.15.27:0/1093880936 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f0aa0199a10 con 0x7f0a880396b0 2024-04-26T22:47:33.955 INFO:teuthology.orchestra.run.smithi027.stdout:Added host 'smithi080' with addr '172.21.15.80' 2024-04-26T22:47:33.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:33.957+0000 7f0aa6a00700 1 -- 172.21.15.27:0/1093880936 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0a880396b0 msgr2=0x7f0a8803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:33.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:33.957+0000 7f0aa6a00700 1 --2- 172.21.15.27:0/1093880936 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0a880396b0 0x7f0a8803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c006fd0 tx=0x7f0a8c006e40 comp rx=0 tx=0).stop 2024-04-26T22:47:33.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:33.957+0000 7f0aa6a00700 1 -- 172.21.15.27:0/1093880936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0aa0102ef0 msgr2=0x7f0aa01986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:33.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:33.957+0000 7f0aa6a00700 1 --2- 172.21.15.27:0/1093880936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0aa0102ef0 0x7f0aa01986b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0a94006010 tx=0x7f0a94005780 comp rx=0 tx=0).stop 2024-04-26T22:47:33.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:33.957+0000 7f0aa6a00700 1 -- 172.21.15.27:0/1093880936 shutdown_connections 2024-04-26T22:47:33.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:33.957+0000 7f0aa6a00700 1 --2- 172.21.15.27:0/1093880936 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0a880396b0 0x7f0a8803bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:33.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:33.957+0000 7f0aa6a00700 1 --2- 172.21.15.27:0/1093880936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0aa0102ef0 0x7f0aa01986b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:33.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:33.957+0000 7f0aa6a00700 1 -- 172.21.15.27:0/1093880936 >> 172.21.15.27:0/1093880936 conn(0x7f0aa00fe7c0 msgr2=0x7f0aa01048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:33.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:33.958+0000 7f0aa6a00700 1 -- 172.21.15.27:0/1093880936 shutdown_connections 2024-04-26T22:47:33.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:33.958+0000 7f0aa6a00700 1 -- 172.21.15.27:0/1093880936 wait complete. 2024-04-26T22:47:34.354 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:34 smithi027 ceph-mon[28034]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:34.354 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:34 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:34.354 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:34 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:47:34.777 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T22:47:35.245 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:47:35.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:35 smithi027 ceph-mon[28034]: Added host smithi080 2024-04-26T22:47:35.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:35.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:36.290 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:36 smithi027 ceph-mon[28034]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.243+0000 7ff55be00700 1 -- 172.21.15.27:0/896800578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff55c075740 msgr2=0x7ff55c075b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.243+0000 7ff55be00700 1 --2- 172.21.15.27:0/896800578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff55c075740 0x7ff55c075b20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009910 tx=0x7ff54c009c20 comp rx=0 tx=0).stop 2024-04-26T22:47:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.243+0000 7ff55be00700 1 -- 172.21.15.27:0/896800578 shutdown_connections 2024-04-26T22:47:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.243+0000 7ff55be00700 1 --2- 172.21.15.27:0/896800578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff55c075740 0x7ff55c075b20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.243+0000 7ff55be00700 1 -- 172.21.15.27:0/896800578 >> 172.21.15.27:0/896800578 conn(0x7ff55c0fe780 msgr2=0x7ff55c100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.243+0000 7ff55be00700 1 -- 172.21.15.27:0/896800578 shutdown_connections 2024-04-26T22:47:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.244+0000 7ff55be00700 1 -- 172.21.15.27:0/896800578 wait complete. 2024-04-26T22:47:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.245+0000 7ff55be00700 1 Processor -- start 2024-04-26T22:47:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.245+0000 7ff55be00700 1 -- start start 2024-04-26T22:47:37.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.245+0000 7ff55be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff55c075740 0x7ff55c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.245+0000 7ff55be00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c10f3b0 con 0x7ff55c075740 2024-04-26T22:47:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.245+0000 7ff55aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff55c075740 0x7ff55c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.245+0000 7ff55aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff55c075740 0x7ff55c1a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35282/0 (socket says 172.21.15.27:35282) 2024-04-26T22:47:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.245+0000 7ff55aa00700 1 -- 172.21.15.27:0/2423067667 learned_addr learned my addr 172.21.15.27:0/2423067667 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.246+0000 7ff55aa00700 1 -- 172.21.15.27:0/2423067667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c1a0b40 con 0x7ff55c075740 2024-04-26T22:47:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.246+0000 7ff55aa00700 1 --2- 172.21.15.27:0/2423067667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff55c075740 0x7ff55c1a0600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff54c006010 tx=0x7ff54c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.246+0000 7ff553400700 1 -- 172.21.15.27:0/2423067667 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff54c026070 con 0x7ff55c075740 2024-04-26T22:47:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.246+0000 7ff553400700 1 -- 172.21.15.27:0/2423067667 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff54c00bc00 con 0x7ff55c075740 2024-04-26T22:47:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.246+0000 7ff55be00700 1 -- 172.21.15.27:0/2423067667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55c1a0dd0 con 0x7ff55c075740 2024-04-26T22:47:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.247+0000 7ff553400700 1 -- 172.21.15.27:0/2423067667 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff54c003910 con 0x7ff55c075740 2024-04-26T22:47:37.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.247+0000 7ff55be00700 1 -- 172.21.15.27:0/2423067667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff55c19a780 con 0x7ff55c075740 2024-04-26T22:47:37.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.248+0000 7ff553400700 1 -- 172.21.15.27:0/2423067667 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff54c029070 con 0x7ff55c075740 2024-04-26T22:47:37.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.248+0000 7ff55be00700 1 -- 172.21.15.27:0/2423067667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff55c076e80 con 0x7ff55c075740 2024-04-26T22:47:37.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.248+0000 7ff553400700 1 --2- 172.21.15.27:0/2423067667 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff5400396b0 0x7ff54003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:37.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.248+0000 7ff553400700 1 -- 172.21.15.27:0/2423067667 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff54c01e070 con 0x7ff55c075740 2024-04-26T22:47:37.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.249+0000 7ff55a000700 1 --2- 172.21.15.27:0/2423067667 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff5400396b0 0x7ff54003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:37.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.249+0000 7ff55a000700 1 --2- 172.21.15.27:0/2423067667 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff5400396b0 0x7ff54003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff548006fd0 tx=0x7ff548006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:37.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.254+0000 7ff553400700 1 -- 172.21.15.27:0/2423067667 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff54c013070 con 0x7ff55c075740 2024-04-26T22:47:37.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.498+0000 7ff55be00700 1 -- 172.21.15.27:0/2423067667 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff55c19bcb0 con 0x7ff5400396b0 2024-04-26T22:47:37.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.500+0000 7ff553400700 1 -- 172.21.15.27:0/2423067667 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7ff55c19bcb0 con 0x7ff5400396b0 2024-04-26T22:47:37.500 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:47:37.500 INFO:teuthology.orchestra.run.smithi027.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.80", "hostname": "smithi080", "labels": [], "status": ""}] 2024-04-26T22:47:37.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.503+0000 7ff55be00700 1 -- 172.21.15.27:0/2423067667 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff5400396b0 msgr2=0x7ff54003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:37.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.503+0000 7ff55be00700 1 --2- 172.21.15.27:0/2423067667 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff5400396b0 0x7ff54003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff548006fd0 tx=0x7ff548006e40 comp rx=0 tx=0).stop 2024-04-26T22:47:37.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.503+0000 7ff55be00700 1 -- 172.21.15.27:0/2423067667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff55c075740 msgr2=0x7ff55c1a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:37.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.503+0000 7ff55be00700 1 --2- 172.21.15.27:0/2423067667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff55c075740 0x7ff55c1a0600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff54c006010 tx=0x7ff54c005780 comp rx=0 tx=0).stop 2024-04-26T22:47:37.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.503+0000 7ff55be00700 1 -- 172.21.15.27:0/2423067667 shutdown_connections 2024-04-26T22:47:37.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.503+0000 7ff55be00700 1 --2- 172.21.15.27:0/2423067667 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff5400396b0 0x7ff54003bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:37.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.503+0000 7ff55be00700 1 --2- 172.21.15.27:0/2423067667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff55c075740 0x7ff55c1a0600 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:37.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.503+0000 7ff55be00700 1 -- 172.21.15.27:0/2423067667 >> 172.21.15.27:0/2423067667 conn(0x7ff55c0fe780 msgr2=0x7ff55c108fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:37.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.503+0000 7ff55be00700 1 -- 172.21.15.27:0/2423067667 shutdown_connections 2024-04-26T22:47:37.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:37.504+0000 7ff55be00700 1 -- 172.21.15.27:0/2423067667 wait complete. 2024-04-26T22:47:37.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:37 smithi027 ceph-mon[28034]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:37.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:37 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:38.302 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi094 2024-04-26T22:47:38.302 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T22:47:38.302 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/etc/ceph/ceph.conf 2024-04-26T22:47:38.322 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T22:47:38.322 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:47:38.379 INFO:tasks.cephadm:Adding host smithi094 to orchestrator... 2024-04-26T22:47:38.379 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch host add smithi094 2024-04-26T22:47:38.836 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:47:38.982 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:38 smithi027 ceph-mon[28034]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:47:40.000 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:39 smithi027 ceph-mon[28034]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:41.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.005+0000 7f24bda00700 1 -- 172.21.15.27:0/2393497435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24b8103090 msgr2=0x7f24b8103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:41.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.005+0000 7f24bda00700 1 --2- 172.21.15.27:0/2393497435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24b8103090 0x7f24b8103470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009910 tx=0x7f24a8009c20 comp rx=0 tx=0).stop 2024-04-26T22:47:41.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.005+0000 7f24bda00700 1 -- 172.21.15.27:0/2393497435 shutdown_connections 2024-04-26T22:47:41.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.005+0000 7f24bda00700 1 --2- 172.21.15.27:0/2393497435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24b8103090 0x7f24b8103470 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:41.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.006+0000 7f24bda00700 1 -- 172.21.15.27:0/2393497435 >> 172.21.15.27:0/2393497435 conn(0x7f24b80fe960 msgr2=0x7f24b8100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:41.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.006+0000 7f24bda00700 1 -- 172.21.15.27:0/2393497435 shutdown_connections 2024-04-26T22:47:41.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.006+0000 7f24bda00700 1 -- 172.21.15.27:0/2393497435 wait complete. 2024-04-26T22:47:41.006 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:40 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:41.007 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:40 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:41.007 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:40 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:41.007 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:40 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:41.007 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:40 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi080", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:47:41.007 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:40 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:47:41.007 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:40 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:47:41.007 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:40 smithi027 ceph-mon[28034]: Updating smithi080:/etc/ceph/ceph.conf 2024-04-26T22:47:41.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.007+0000 7f24bda00700 1 Processor -- start 2024-04-26T22:47:41.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.007+0000 7f24bda00700 1 -- start start 2024-04-26T22:47:41.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.007+0000 7f24bda00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24b8103090 0x7f24b8075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:41.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.007+0000 7f24bda00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24b810d310 con 0x7f24b8103090 2024-04-26T22:47:41.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.008+0000 7f24b6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24b8103090 0x7f24b8075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:41.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.008+0000 7f24b6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24b8103090 0x7f24b8075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41152/0 (socket says 172.21.15.27:41152) 2024-04-26T22:47:41.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.008+0000 7f24b6a00700 1 -- 172.21.15.27:0/455439979 learned_addr learned my addr 172.21.15.27:0/455439979 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:41.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.008+0000 7f24b6a00700 1 -- 172.21.15.27:0/455439979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b8077610 con 0x7f24b8103090 2024-04-26T22:47:41.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.008+0000 7f24b6a00700 1 --2- 172.21.15.27:0/455439979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24b8103090 0x7f24b8075720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f24a8006010 tx=0x7f24a8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.009+0000 7f24af400700 1 -- 172.21.15.27:0/455439979 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24a8026070 con 0x7f24b8103090 2024-04-26T22:47:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.009+0000 7f24af400700 1 -- 172.21.15.27:0/455439979 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24a800bc00 con 0x7f24b8103090 2024-04-26T22:47:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.009+0000 7f24af400700 1 -- 172.21.15.27:0/455439979 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24a8003910 con 0x7f24b8103090 2024-04-26T22:47:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.009+0000 7f24bda00700 1 -- 172.21.15.27:0/455439979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b8075cc0 con 0x7f24b8103090 2024-04-26T22:47:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.009+0000 7f24bda00700 1 -- 172.21.15.27:0/455439979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b8076180 con 0x7f24b8103090 2024-04-26T22:47:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.010+0000 7f24af400700 1 -- 172.21.15.27:0/455439979 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f24a8029070 con 0x7f24b8103090 2024-04-26T22:47:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.010+0000 7f24bda00700 1 -- 172.21.15.27:0/455439979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b8104570 con 0x7f24b8103090 2024-04-26T22:47:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.011+0000 7f24af400700 1 --2- 172.21.15.27:0/455439979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f249c0396b0 0x7f249c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.011+0000 7f24af400700 1 -- 172.21.15.27:0/455439979 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f24a801e070 con 0x7f24b8103090 2024-04-26T22:47:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.011+0000 7f24b6000700 1 --2- 172.21.15.27:0/455439979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f249c0396b0 0x7f249c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.012+0000 7f24b6000700 1 --2- 172.21.15.27:0/455439979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f249c0396b0 0x7f249c03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f24a4006fd0 tx=0x7f24a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:41.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.017+0000 7f24af400700 1 -- 172.21.15.27:0/455439979 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f24a8013070 con 0x7f24b8103090 2024-04-26T22:47:41.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:41.255+0000 7f24bda00700 1 -- 172.21.15.27:0/455439979 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi094", "target": ["mon-mgr", ""]}) -- 0x7f24b8076410 con 0x7f249c0396b0 2024-04-26T22:47:42.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:42 smithi027 ceph-mon[28034]: Updating smithi080:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:47:42.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:42 smithi027 ceph-mon[28034]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:42.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:42 smithi027 ceph-mon[28034]: Updating smithi080:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:47:42.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:42 smithi027 ceph-mon[28034]: Updating smithi080:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:47:42.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:42 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:42.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:42 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:42.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:42 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:43.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:43 smithi027 ceph-mon[28034]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi094", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:47:43.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:43 smithi027 ceph-mon[28034]: Deploying cephadm binary to smithi094 2024-04-26T22:47:44.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:44 smithi027 ceph-mon[28034]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:46.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:46.146+0000 7f24af400700 1 -- 172.21.15.27:0/455439979 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f24b8076410 con 0x7f249c0396b0 2024-04-26T22:47:46.147 INFO:teuthology.orchestra.run.smithi027.stdout:Added host 'smithi094' with addr '172.21.15.94' 2024-04-26T22:47:46.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:46.150+0000 7f24bda00700 1 -- 172.21.15.27:0/455439979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f249c0396b0 msgr2=0x7f249c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:46.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:46.150+0000 7f24bda00700 1 --2- 172.21.15.27:0/455439979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f249c0396b0 0x7f249c03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f24a4006fd0 tx=0x7f24a4006e40 comp rx=0 tx=0).stop 2024-04-26T22:47:46.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:46.150+0000 7f24bda00700 1 -- 172.21.15.27:0/455439979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24b8103090 msgr2=0x7f24b8075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:46.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:46.150+0000 7f24bda00700 1 --2- 172.21.15.27:0/455439979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24b8103090 0x7f24b8075720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f24a8006010 tx=0x7f24a8005780 comp rx=0 tx=0).stop 2024-04-26T22:47:46.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:46.150+0000 7f24bda00700 1 -- 172.21.15.27:0/455439979 shutdown_connections 2024-04-26T22:47:46.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:46.150+0000 7f24bda00700 1 --2- 172.21.15.27:0/455439979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f249c0396b0 0x7f249c03bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:46.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:46.150+0000 7f24bda00700 1 --2- 172.21.15.27:0/455439979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24b8103090 0x7f24b8075720 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:46.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:46.150+0000 7f24bda00700 1 -- 172.21.15.27:0/455439979 >> 172.21.15.27:0/455439979 conn(0x7f24b80fe960 msgr2=0x7f24b8104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:46.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:46.151+0000 7f24bda00700 1 -- 172.21.15.27:0/455439979 shutdown_connections 2024-04-26T22:47:46.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:46.151+0000 7f24bda00700 1 -- 172.21.15.27:0/455439979 wait complete. 2024-04-26T22:47:46.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:46 smithi027 ceph-mon[28034]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:46.942 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T22:47:47.419 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:47:47.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:47 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:47.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:47 smithi027 ceph-mon[28034]: Added host smithi094 2024-04-26T22:47:47.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:47 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:47:47.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:47 smithi027 ceph-mon[28034]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:47.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:47 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:48.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:49.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.605+0000 7f0338800700 1 -- 172.21.15.27:0/3677287438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0330075740 msgr2=0x7f0330075b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:49.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.605+0000 7f0338800700 1 --2- 172.21.15.27:0/3677287438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0330075740 0x7f0330075b20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0324009910 tx=0x7f0324009c20 comp rx=0 tx=0).stop 2024-04-26T22:47:49.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.605+0000 7f0338800700 1 -- 172.21.15.27:0/3677287438 shutdown_connections 2024-04-26T22:47:49.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.605+0000 7f0338800700 1 --2- 172.21.15.27:0/3677287438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0330075740 0x7f0330075b20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:49.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.605+0000 7f0338800700 1 -- 172.21.15.27:0/3677287438 >> 172.21.15.27:0/3677287438 conn(0x7f03300fe960 msgr2=0x7f0330100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:49.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.605+0000 7f0338800700 1 -- 172.21.15.27:0/3677287438 shutdown_connections 2024-04-26T22:47:49.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.605+0000 7f0338800700 1 -- 172.21.15.27:0/3677287438 wait complete. 2024-04-26T22:47:49.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.607+0000 7f0338800700 1 Processor -- start 2024-04-26T22:47:49.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.607+0000 7f0338800700 1 -- start start 2024-04-26T22:47:49.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.608+0000 7f0338800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f033019ad50 0x7f033019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:49.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.608+0000 7f0338800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f033010f570 con 0x7f033019ad50 2024-04-26T22:47:49.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.608+0000 7f0335e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f033019ad50 0x7f033019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:49.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.608+0000 7f0335e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f033019ad50 0x7f033019b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49690/0 (socket says 172.21.15.27:49690) 2024-04-26T22:47:49.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.608+0000 7f0335e00700 1 -- 172.21.15.27:0/1750765611 learned_addr learned my addr 172.21.15.27:0/1750765611 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:49.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.609+0000 7f0335e00700 1 -- 172.21.15.27:0/1750765611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033019b670 con 0x7f033019ad50 2024-04-26T22:47:49.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.609+0000 7f0335e00700 1 --2- 172.21.15.27:0/1750765611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f033019ad50 0x7f033019b130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0324005950 tx=0x7f0324005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:49.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.609+0000 7f032aa00700 1 -- 172.21.15.27:0/1750765611 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0324026070 con 0x7f033019ad50 2024-04-26T22:47:49.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.609+0000 7f032aa00700 1 -- 172.21.15.27:0/1750765611 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f032400b750 con 0x7f033019ad50 2024-04-26T22:47:49.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.610+0000 7f0338800700 1 -- 172.21.15.27:0/1750765611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033019b900 con 0x7f033019ad50 2024-04-26T22:47:49.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.610+0000 7f032aa00700 1 -- 172.21.15.27:0/1750765611 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0324003810 con 0x7f033019ad50 2024-04-26T22:47:49.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.610+0000 7f0338800700 1 -- 172.21.15.27:0/1750765611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f033019f7f0 con 0x7f033019ad50 2024-04-26T22:47:49.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.611+0000 7f032aa00700 1 -- 172.21.15.27:0/1750765611 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0324029070 con 0x7f033019ad50 2024-04-26T22:47:49.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.611+0000 7f0338800700 1 -- 172.21.15.27:0/1750765611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0330076e80 con 0x7f033019ad50 2024-04-26T22:47:49.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.612+0000 7f032aa00700 1 --2- 172.21.15.27:0/1750765611 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f03180396b0 0x7f031803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:49.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.612+0000 7f032aa00700 1 -- 172.21.15.27:0/1750765611 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f032401e070 con 0x7f033019ad50 2024-04-26T22:47:49.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.612+0000 7f0335400700 1 --2- 172.21.15.27:0/1750765611 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f03180396b0 0x7f031803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:49.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.613+0000 7f0335400700 1 --2- 172.21.15.27:0/1750765611 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f03180396b0 0x7f031803bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0320006fd0 tx=0x7f0320006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:49.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.618+0000 7f032aa00700 1 -- 172.21.15.27:0/1750765611 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0324013070 con 0x7f033019ad50 2024-04-26T22:47:49.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.854+0000 7f0338800700 1 -- 172.21.15.27:0/1750765611 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f033019c120 con 0x7f03180396b0 2024-04-26T22:47:49.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.857+0000 7f032aa00700 1 -- 172.21.15.27:0/1750765611 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f033019c120 con 0x7f03180396b0 2024-04-26T22:47:49.858 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:47:49.858 INFO:teuthology.orchestra.run.smithi027.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.80", "hostname": "smithi080", "labels": [], "status": ""}, {"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}] 2024-04-26T22:47:49.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.860+0000 7f0338800700 1 -- 172.21.15.27:0/1750765611 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f03180396b0 msgr2=0x7f031803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:49.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.860+0000 7f0338800700 1 --2- 172.21.15.27:0/1750765611 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f03180396b0 0x7f031803bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0320006fd0 tx=0x7f0320006e40 comp rx=0 tx=0).stop 2024-04-26T22:47:49.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.860+0000 7f0338800700 1 -- 172.21.15.27:0/1750765611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f033019ad50 msgr2=0x7f033019b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:49.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.860+0000 7f0338800700 1 --2- 172.21.15.27:0/1750765611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f033019ad50 0x7f033019b130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0324005950 tx=0x7f0324005780 comp rx=0 tx=0).stop 2024-04-26T22:47:49.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.860+0000 7f0338800700 1 -- 172.21.15.27:0/1750765611 shutdown_connections 2024-04-26T22:47:49.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.860+0000 7f0338800700 1 --2- 172.21.15.27:0/1750765611 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f03180396b0 0x7f031803bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:49.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.860+0000 7f0338800700 1 --2- 172.21.15.27:0/1750765611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f033019ad50 0x7f033019b130 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:49.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.860+0000 7f0338800700 1 -- 172.21.15.27:0/1750765611 >> 172.21.15.27:0/1750765611 conn(0x7f03300fe960 msgr2=0x7f033010bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:49.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.861+0000 7f0338800700 1 -- 172.21.15.27:0/1750765611 shutdown_connections 2024-04-26T22:47:49.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:49.861+0000 7f0338800700 1 -- 172.21.15.27:0/1750765611 wait complete. 2024-04-26T22:47:50.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:49 smithi027 ceph-mon[28034]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:50.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:49 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:50.699 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T22:47:50.700 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T22:47:51.159 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:47:51.305 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:50 smithi027 ceph-mon[28034]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:47:52.156 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:51 smithi027 ceph-mon[28034]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:53.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:53.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:53.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:53.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:53.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:47:53.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:47:53.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:47:53.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.395+0000 7f276e000700 1 -- 172.21.15.27:0/13603226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768107d60 msgr2=0x7f276806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:53.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.395+0000 7f276e000700 1 --2- 172.21.15.27:0/13603226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768107d60 0x7f276806a840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2758009910 tx=0x7f2758009c20 comp rx=0 tx=0).stop 2024-04-26T22:47:53.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.395+0000 7f276e000700 1 -- 172.21.15.27:0/13603226 shutdown_connections 2024-04-26T22:47:53.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.395+0000 7f276e000700 1 --2- 172.21.15.27:0/13603226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768107d60 0x7f276806a840 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:53.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.395+0000 7f276e000700 1 -- 172.21.15.27:0/13603226 >> 172.21.15.27:0/13603226 conn(0x7f2768077880 msgr2=0x7f2768077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:53.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.395+0000 7f276e000700 1 -- 172.21.15.27:0/13603226 shutdown_connections 2024-04-26T22:47:53.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.395+0000 7f276e000700 1 -- 172.21.15.27:0/13603226 wait complete. 2024-04-26T22:47:53.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.396+0000 7f276e000700 1 Processor -- start 2024-04-26T22:47:53.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.397+0000 7f276e000700 1 -- start start 2024-04-26T22:47:53.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.397+0000 7f276e000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768107d60 0x7f276819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:53.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.397+0000 7f276e000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2768113990 con 0x7f2768107d60 2024-04-26T22:47:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.397+0000 7f2767400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768107d60 0x7f276819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.397+0000 7f2767400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768107d60 0x7f276819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49702/0 (socket says 172.21.15.27:49702) 2024-04-26T22:47:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.398+0000 7f2767400700 1 -- 172.21.15.27:0/2596343239 learned_addr learned my addr 172.21.15.27:0/2596343239 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:47:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.398+0000 7f2767400700 1 -- 172.21.15.27:0/2596343239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276819f4b0 con 0x7f2768107d60 2024-04-26T22:47:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.398+0000 7f2767400700 1 --2- 172.21.15.27:0/2596343239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768107d60 0x7f276819ef70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f27580056d0 tx=0x7f27580057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.399+0000 7f275fe00700 1 -- 172.21.15.27:0/2596343239 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2758026070 con 0x7f2768107d60 2024-04-26T22:47:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.399+0000 7f275fe00700 1 -- 172.21.15.27:0/2596343239 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f275800bc00 con 0x7f2768107d60 2024-04-26T22:47:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.399+0000 7f276e000700 1 -- 172.21.15.27:0/2596343239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276819f740 con 0x7f2768107d60 2024-04-26T22:47:53.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.399+0000 7f275fe00700 1 -- 172.21.15.27:0/2596343239 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2758003920 con 0x7f2768107d60 2024-04-26T22:47:53.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.399+0000 7f276e000700 1 -- 172.21.15.27:0/2596343239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f276819fba0 con 0x7f2768107d60 2024-04-26T22:47:53.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.400+0000 7f275fe00700 1 -- 172.21.15.27:0/2596343239 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2758029070 con 0x7f2768107d60 2024-04-26T22:47:53.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.400+0000 7f276e000700 1 -- 172.21.15.27:0/2596343239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2768102350 con 0x7f2768107d60 2024-04-26T22:47:53.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.401+0000 7f275fe00700 1 --2- 172.21.15.27:0/2596343239 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f274c0396b0 0x7f274c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:53.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.401+0000 7f275fe00700 1 -- 172.21.15.27:0/2596343239 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f275801e070 con 0x7f2768107d60 2024-04-26T22:47:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.401+0000 7f2766a00700 1 --2- 172.21.15.27:0/2596343239 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f274c0396b0 0x7f274c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.402+0000 7f2766a00700 1 --2- 172.21.15.27:0/2596343239 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f274c0396b0 0x7f274c03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2754006fd0 tx=0x7f2754006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:53.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.407+0000 7f275fe00700 1 -- 172.21.15.27:0/2596343239 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2758013070 con 0x7f2768107d60 2024-04-26T22:47:53.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:53.646+0000 7f276e000700 1 -- 172.21.15.27:0/2596343239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f27681a02d0 con 0x7f2768107d60 2024-04-26T22:47:54.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:54 smithi027 ceph-mon[28034]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T22:47:54.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:54 smithi027 ceph-mon[28034]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:54.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:54 smithi027 ceph-mon[28034]: Updating smithi094:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:47:54.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:54 smithi027 ceph-mon[28034]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:47:54.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:54 smithi027 ceph-mon[28034]: Updating smithi094:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:47:54.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:54.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:54.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:54.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:54 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2596343239' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T22:47:54.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:54.398+0000 7f275fe00700 1 -- 172.21.15.27:0/2596343239 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f275800e5a0 con 0x7f2768107d60 2024-04-26T22:47:54.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:54.402+0000 7f276e000700 1 -- 172.21.15.27:0/2596343239 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f274c0396b0 msgr2=0x7f274c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:54.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:54.402+0000 7f276e000700 1 --2- 172.21.15.27:0/2596343239 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f274c0396b0 0x7f274c03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2754006fd0 tx=0x7f2754006e40 comp rx=0 tx=0).stop 2024-04-26T22:47:54.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:54.402+0000 7f276e000700 1 -- 172.21.15.27:0/2596343239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768107d60 msgr2=0x7f276819ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:54.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:54.402+0000 7f276e000700 1 --2- 172.21.15.27:0/2596343239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768107d60 0x7f276819ef70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f27580056d0 tx=0x7f27580057b0 comp rx=0 tx=0).stop 2024-04-26T22:47:54.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:54.402+0000 7f276e000700 1 -- 172.21.15.27:0/2596343239 shutdown_connections 2024-04-26T22:47:54.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:54.402+0000 7f276e000700 1 --2- 172.21.15.27:0/2596343239 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f274c0396b0 0x7f274c03bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:54.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:54.402+0000 7f276e000700 1 --2- 172.21.15.27:0/2596343239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768107d60 0x7f276819ef70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:54.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:54.402+0000 7f276e000700 1 -- 172.21.15.27:0/2596343239 >> 172.21.15.27:0/2596343239 conn(0x7f2768077880 msgr2=0x7f27680fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:54.402+0000 7f276e000700 1 -- 172.21.15.27:0/2596343239 shutdown_connections 2024-04-26T22:47:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:47:54.403+0000 7f276e000700 1 -- 172.21.15.27:0/2596343239 wait complete. 2024-04-26T22:47:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:adjusted tunables profile to default 2024-04-26T22:47:55.188 INFO:tasks.cephadm:Adding mon.a on smithi027 2024-04-26T22:47:55.188 INFO:tasks.cephadm:Adding mon.b on smithi080 2024-04-26T22:47:55.188 INFO:tasks.cephadm:Adding mon.c on smithi094 2024-04-26T22:47:55.188 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi027:172.21.15.27=a;smithi080:172.21.15.80=b;smithi094:172.21.15.94=c' 2024-04-26T22:47:55.426 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:47:55.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:55 smithi027 ceph-mon[28034]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:55.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:55 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2596343239' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T22:47:55.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:55 smithi027 ceph-mon[28034]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T22:47:57.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.033+0000 7fb0bbce9700 1 -- 172.21.15.94:0/3454470913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0b4101f90 msgr2=0x7fb0b4102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:57.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.033+0000 7fb0bbce9700 1 --2- 172.21.15.94:0/3454470913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0b4101f90 0x7fb0b4102370 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009910 tx=0x7fb0b0009c20 comp rx=0 tx=0).stop 2024-04-26T22:47:57.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.033+0000 7fb0bbce9700 1 -- 172.21.15.94:0/3454470913 shutdown_connections 2024-04-26T22:47:57.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.033+0000 7fb0bbce9700 1 --2- 172.21.15.94:0/3454470913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0b4101f90 0x7fb0b4102370 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:57.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.033+0000 7fb0bbce9700 1 -- 172.21.15.94:0/3454470913 >> 172.21.15.94:0/3454470913 conn(0x7fb0b40fd840 msgr2=0x7fb0b40ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:57.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.034+0000 7fb0bbce9700 1 -- 172.21.15.94:0/3454470913 shutdown_connections 2024-04-26T22:47:57.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.034+0000 7fb0bbce9700 1 -- 172.21.15.94:0/3454470913 wait complete. 2024-04-26T22:47:57.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.034+0000 7fb0bbce9700 1 Processor -- start 2024-04-26T22:47:57.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.034+0000 7fb0bbce9700 1 -- start start 2024-04-26T22:47:57.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.035+0000 7fb0bbce9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0b4101f90 0x7fb0b4199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:57.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.035+0000 7fb0bbce9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b410c210 con 0x7fb0b4101f90 2024-04-26T22:47:57.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.035+0000 7fb0b9a85700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0b4101f90 0x7fb0b4199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:57.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.035+0000 7fb0b9a85700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0b4101f90 0x7fb0b4199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.94:44480/0 (socket says 172.21.15.94:44480) 2024-04-26T22:47:57.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.035+0000 7fb0b9a85700 1 -- 172.21.15.94:0/947904124 learned_addr learned my addr 172.21.15.94:0/947904124 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T22:47:57.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.036+0000 7fb0b9a85700 1 -- 172.21.15.94:0/947904124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b4199bb0 con 0x7fb0b4101f90 2024-04-26T22:47:57.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.036+0000 7fb0b9a85700 1 --2- 172.21.15.94:0/947904124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0b4101f90 0x7fb0b4199670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0006010 tx=0x7fb0b0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:57.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.037+0000 7fb0aaffd700 1 -- 172.21.15.94:0/947904124 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0b0026070 con 0x7fb0b4101f90 2024-04-26T22:47:57.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.037+0000 7fb0bbce9700 1 -- 172.21.15.94:0/947904124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b4199ea0 con 0x7fb0b4101f90 2024-04-26T22:47:57.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.037+0000 7fb0aaffd700 1 -- 172.21.15.94:0/947904124 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b000bba0 con 0x7fb0b4101f90 2024-04-26T22:47:57.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.037+0000 7fb0aaffd700 1 -- 172.21.15.94:0/947904124 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0b0003910 con 0x7fb0b4101f90 2024-04-26T22:47:57.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.037+0000 7fb0bbce9700 1 -- 172.21.15.94:0/947904124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b419a360 con 0x7fb0b4101f90 2024-04-26T22:47:57.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.038+0000 7fb0bbce9700 1 -- 172.21.15.94:0/947904124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b4103470 con 0x7fb0b4101f90 2024-04-26T22:47:57.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.038+0000 7fb0aaffd700 1 -- 172.21.15.94:0/947904124 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb0b0029070 con 0x7fb0b4101f90 2024-04-26T22:47:57.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.038+0000 7fb0aaffd700 1 --2- 172.21.15.94:0/947904124 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb0a0039650 0x7fb0a003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:47:57.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.038+0000 7fb0aaffd700 1 -- 172.21.15.94:0/947904124 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb0b001e070 con 0x7fb0b4101f90 2024-04-26T22:47:57.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.039+0000 7fb0b9284700 1 --2- 172.21.15.94:0/947904124 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb0a0039650 0x7fb0a003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:47:57.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.040+0000 7fb0b9284700 1 --2- 172.21.15.94:0/947904124 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb0a0039650 0x7fb0a003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4006fd0 tx=0x7fb0a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:47:57.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.055+0000 7fb0aaffd700 1 -- 172.21.15.94:0/947904124 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb0b0030020 con 0x7fb0b4101f90 2024-04-26T22:47:57.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.265+0000 7fb0bbce9700 1 -- 172.21.15.94:0/947904124 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi080:172.21.15.80=b;smithi094:172.21.15.94=c", "target": ["mon-mgr", ""]}) -- 0x7fb0b406b900 con 0x7fb0a0039650 2024-04-26T22:47:57.280 INFO:teuthology.orchestra.run.smithi094.stdout:Scheduled mon update... 2024-04-26T22:47:57.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.280+0000 7fb0aaffd700 1 -- 172.21.15.94:0/947904124 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb0b406b900 con 0x7fb0a0039650 2024-04-26T22:47:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.282+0000 7fb0bbce9700 1 -- 172.21.15.94:0/947904124 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb0a0039650 msgr2=0x7fb0a003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.282+0000 7fb0bbce9700 1 --2- 172.21.15.94:0/947904124 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb0a0039650 0x7fb0a003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4006fd0 tx=0x7fb0a4006e40 comp rx=0 tx=0).stop 2024-04-26T22:47:57.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.282+0000 7fb0bbce9700 1 -- 172.21.15.94:0/947904124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0b4101f90 msgr2=0x7fb0b4199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:47:57.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.282+0000 7fb0bbce9700 1 --2- 172.21.15.94:0/947904124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0b4101f90 0x7fb0b4199670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0006010 tx=0x7fb0b0005780 comp rx=0 tx=0).stop 2024-04-26T22:47:57.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.282+0000 7fb0bbce9700 1 -- 172.21.15.94:0/947904124 shutdown_connections 2024-04-26T22:47:57.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.282+0000 7fb0bbce9700 1 --2- 172.21.15.94:0/947904124 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb0a0039650 0x7fb0a003bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:57.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.282+0000 7fb0bbce9700 1 --2- 172.21.15.94:0/947904124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0b4101f90 0x7fb0b4199670 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:47:57.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.282+0000 7fb0bbce9700 1 -- 172.21.15.94:0/947904124 >> 172.21.15.94:0/947904124 conn(0x7fb0b40fd840 msgr2=0x7fb0b4103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:47:57.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.282+0000 7fb0bbce9700 1 -- 172.21.15.94:0/947904124 shutdown_connections 2024-04-26T22:47:57.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:47:57.282+0000 7fb0bbce9700 1 -- 172.21.15.94:0/947904124 wait complete. 2024-04-26T22:47:57.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:57 smithi027 ceph-mon[28034]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:47:57.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:57.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:47:57.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:47:57.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:47:57.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:47:57.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T22:47:57.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:47:57.888 DEBUG:teuthology.orchestra.run.smithi080:mon.b> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mon.b.service 2024-04-26T22:47:57.891 DEBUG:teuthology.orchestra.run.smithi094:mon.c> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mon.c.service 2024-04-26T22:47:57.893 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T22:47:57.893 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T22:47:58.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:58 smithi027 ceph-mon[28034]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi080:172.21.15.80=b;smithi094:172.21.15.94=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:47:58.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:58 smithi027 ceph-mon[28034]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi080:172.21.15.80=b;smithi094:172.21.15.94=c;count:3 2024-04-26T22:47:58.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:58 smithi027 ceph-mon[28034]: Deploying daemon mon.c on smithi094 2024-04-26T22:47:59.067 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:47:59.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:47:59 smithi027 ceph-mon[28034]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.530+0000 7f2e03bde700 1 -- 172.21.15.94:0/2357890200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dfc108fb0 msgr2=0x7f2dfc109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.530+0000 7f2e03bde700 1 --2- 172.21.15.94:0/2357890200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dfc108fb0 0x7f2dfc109390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2df400b0a0 tx=0x7f2df400b3b0 comp rx=0 tx=0).stop 2024-04-26T22:48:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.530+0000 7f2e03bde700 1 -- 172.21.15.94:0/2357890200 shutdown_connections 2024-04-26T22:48:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.530+0000 7f2e03bde700 1 --2- 172.21.15.94:0/2357890200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dfc108fb0 0x7f2dfc109390 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.530+0000 7f2e03bde700 1 -- 172.21.15.94:0/2357890200 >> 172.21.15.94:0/2357890200 conn(0x7f2dfc0784a0 msgr2=0x7f2dfc0788b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:48:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.530+0000 7f2e03bde700 1 -- 172.21.15.94:0/2357890200 shutdown_connections 2024-04-26T22:48:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.530+0000 7f2e03bde700 1 -- 172.21.15.94:0/2357890200 wait complete. 2024-04-26T22:48:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.531+0000 7f2e03bde700 1 Processor -- start 2024-04-26T22:48:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.531+0000 7f2e03bde700 1 -- start start 2024-04-26T22:48:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.531+0000 7f2e03bde700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dfc108fb0 0x7f2dfc102590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.531+0000 7f2e03bde700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc10d460 con 0x7f2dfc108fb0 2024-04-26T22:48:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.532+0000 7f2e0197a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dfc108fb0 0x7f2dfc102590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.532+0000 7f2e0197a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dfc108fb0 0x7f2dfc102590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.94:39558/0 (socket says 172.21.15.94:39558) 2024-04-26T22:48:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.532+0000 7f2e0197a700 1 -- 172.21.15.94:0/10083236 learned_addr learned my addr 172.21.15.94:0/10083236 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T22:48:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.532+0000 7f2e0197a700 1 -- 172.21.15.94:0/10083236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dfc104480 con 0x7f2dfc108fb0 2024-04-26T22:48:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.533+0000 7f2e0197a700 1 --2- 172.21.15.94:0/10083236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dfc108fb0 0x7f2dfc102590 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2df400d010 tx=0x7f2df4009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:48:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.533+0000 7f2dfaffd700 1 -- 172.21.15.94:0/10083236 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2df4017050 con 0x7f2dfc108fb0 2024-04-26T22:48:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.533+0000 7f2e03bde700 1 -- 172.21.15.94:0/10083236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dfc102ad0 con 0x7f2dfc108fb0 2024-04-26T22:48:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.533+0000 7f2e03bde700 1 -- 172.21.15.94:0/10083236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dfc102f90 con 0x7f2dfc108fb0 2024-04-26T22:48:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.533+0000 7f2dfaffd700 1 -- 172.21.15.94:0/10083236 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2df4023070 con 0x7f2dfc108fb0 2024-04-26T22:48:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.533+0000 7f2dfaffd700 1 -- 172.21.15.94:0/10083236 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2df401e740 con 0x7f2dfc108fb0 2024-04-26T22:48:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.534+0000 7f2e03bde700 1 -- 172.21.15.94:0/10083236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dfc064a80 con 0x7f2dfc108fb0 2024-04-26T22:48:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.535+0000 7f2dfaffd700 1 -- 172.21.15.94:0/10083236 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2df4007d10 con 0x7f2dfc108fb0 2024-04-26T22:48:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.535+0000 7f2dfaffd700 1 --2- 172.21.15.94:0/10083236 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2de4039650 0x7f2de403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.535+0000 7f2dfaffd700 1 -- 172.21.15.94:0/10083236 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2df400f050 con 0x7f2dfc108fb0 2024-04-26T22:48:01.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.535+0000 7f2e01179700 1 --2- 172.21.15.94:0/10083236 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2de4039650 0x7f2de403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:01.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.536+0000 7f2e01179700 1 --2- 172.21.15.94:0/10083236 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2de4039650 0x7f2de403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2df0006fd0 tx=0x7f2df0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:48:01.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.551+0000 7f2dfaffd700 1 -- 172.21.15.94:0/10083236 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2df4028460 con 0x7f2dfc108fb0 2024-04-26T22:48:01.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.850+0000 7f2e03bde700 1 -- 172.21.15.94:0/10083236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2dfc1046b0 con 0x7f2dfc108fb0 2024-04-26T22:48:01.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.852+0000 7f2dfaffd700 1 -- 172.21.15.94:0/10083236 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f2df404f030 con 0x7f2dfc108fb0 2024-04-26T22:48:01.852 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:48:01.852 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":1,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","modified":"2024-04-26T22:45:47.264621Z","created":"2024-04-26T22:45:47.264621Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T22:48:01.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.854+0000 7f2e03bde700 1 -- 172.21.15.94:0/10083236 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2de4039650 msgr2=0x7f2de403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:01.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.854+0000 7f2e03bde700 1 --2- 172.21.15.94:0/10083236 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2de4039650 0x7f2de403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2df0006fd0 tx=0x7f2df0006e40 comp rx=0 tx=0).stop 2024-04-26T22:48:01.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.854+0000 7f2e03bde700 1 -- 172.21.15.94:0/10083236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dfc108fb0 msgr2=0x7f2dfc102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:01.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.854+0000 7f2e03bde700 1 --2- 172.21.15.94:0/10083236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dfc108fb0 0x7f2dfc102590 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2df400d010 tx=0x7f2df4009790 comp rx=0 tx=0).stop 2024-04-26T22:48:01.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.854+0000 7f2e03bde700 1 -- 172.21.15.94:0/10083236 shutdown_connections 2024-04-26T22:48:01.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.854+0000 7f2e03bde700 1 --2- 172.21.15.94:0/10083236 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2de4039650 0x7f2de403bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:01.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.854+0000 7f2e03bde700 1 --2- 172.21.15.94:0/10083236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dfc108fb0 0x7f2dfc102590 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:01.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.854+0000 7f2e03bde700 1 -- 172.21.15.94:0/10083236 >> 172.21.15.94:0/10083236 conn(0x7f2dfc0784a0 msgr2=0x7f2dfc104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:48:01.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.854+0000 7f2e03bde700 1 -- 172.21.15.94:0/10083236 shutdown_connections 2024-04-26T22:48:01.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:01.854+0000 7f2e03bde700 1 -- 172.21.15.94:0/10083236 wait complete. 2024-04-26T22:48:01.855 INFO:teuthology.orchestra.run.smithi094.stderr:dumped monmap epoch 1 2024-04-26T22:48:01.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:01 smithi027 ceph-mon[28034]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:01.968 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: mon.c@-1(???) e0 preinit fsid 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:48:01.969 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: mon.c@-1(synchronizing).mds e1 new map 2024-04-26T22:48:01.969 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-26T22:48:01.969 INFO:journalctl@ceph.mon.c.smithi094.stdout: e1 2024-04-26T22:48:01.969 INFO:journalctl@ceph.mon.c.smithi094.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T22:48:01.969 INFO:journalctl@ceph.mon.c.smithi094.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T22:48:01.969 INFO:journalctl@ceph.mon.c.smithi094.stdout: legacy client fscid: -1 2024-04-26T22:48:01.969 INFO:journalctl@ceph.mon.c.smithi094.stdout: 2024-04-26T22:48:01.969 INFO:journalctl@ceph.mon.c.smithi094.stdout: No filesystems configured 2024-04-26T22:48:01.969 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T22:48:01.969 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T22:48:01.969 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T22:48:01.969 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T22:48:01.969 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T22:48:02.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T22:48:02.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T22:48:02.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T22:48:02.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T22:48:02.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T22:48:02.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: Deploying cephadm binary to smithi080 2024-04-26T22:48:02.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:02.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:02.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:02.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: Added host smithi080 2024-04-26T22:48:02.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:02.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:02.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:48:02.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:02.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi080", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:02.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:02.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:02.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: Updating smithi080:/etc/ceph/ceph.conf 2024-04-26T22:48:02.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: Updating smithi080:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:48:02.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:02.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: Updating smithi080:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:48:02.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: Updating smithi080:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:48:02.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.288 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.288 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi094", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:48:02.288 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: Deploying cephadm binary to smithi094 2024-04-26T22:48:02.288 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:02.288 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:02.288 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.288 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: Added host smithi094 2024-04-26T22:48:02.288 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:02.288 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:02.288 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.288 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.288 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:02.288 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.288 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:48:02.289 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:02.289 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.289 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.289 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.289 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.289 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:02.289 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:02.289 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:02.289 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T22:48:02.289 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:02.289 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: Updating smithi094:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:48:02.289 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:48:02.289 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: Updating smithi094:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:48:02.289 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.289 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.289 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.290 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2596343239' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T22:48:02.290 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:02.290 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2596343239' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T22:48:02.290 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T22:48:02.290 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:02.290 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.290 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:02.290 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:02.290 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:02.290 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:02.290 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T22:48:02.290 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:02.290 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi080:172.21.15.80=b;smithi094:172.21.15.94=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:48:02.290 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi080:172.21.15.80=b;smithi094:172.21.15.94=c;count:3 2024-04-26T22:48:02.291 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: Deploying daemon mon.c on smithi094 2024-04-26T22:48:02.291 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:02.291 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:02.291 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:01 smithi094 ceph-mon[32889]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-26T22:48:02.291 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:02 smithi094 bash[32855]: d9d25be2ae96ac3d9cfbd7c36002f8af5f8dfc687da0eaf188830ccb5897bdfb 2024-04-26T22:48:02.291 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:02 smithi094 systemd[1]: Started Ceph mon.c for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:48:03.612 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T22:48:03.613 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T22:48:03.873 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:48:07.320 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:48:07.320 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: mon.a calling monitor election 2024-04-26T22:48:07.320 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:07.320 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:07.320 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:07.320 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:07.320 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: mon.c calling monitor election 2024-04-26T22:48:07.321 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:07.321 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:07.321 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:07.321 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:07.321 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T22:48:07.321 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: monmap epoch 2 2024-04-26T22:48:07.321 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: fsid 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:48:07.321 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: last_changed 2024-04-26T22:48:01.975202+0000 2024-04-26T22:48:07.321 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: created 2024-04-26T22:45:47.264621+0000 2024-04-26T22:48:07.321 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: min_mon_release 19 (squid) 2024-04-26T22:48:07.321 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: election_strategy: 1 2024-04-26T22:48:07.321 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T22:48:07.321 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: 1: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.c 2024-04-26T22:48:07.321 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: fsmap 2024-04-26T22:48:07.321 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T22:48:07.321 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: mgrmap e13: x(active, since 62s) 2024-04-26T22:48:07.322 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: overall HEALTH_OK 2024-04-26T22:48:07.322 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:07.322 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:07.322 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:07.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:48:07.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: mon.a calling monitor election 2024-04-26T22:48:07.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:07.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:07.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:07.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:07.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: mon.c calling monitor election 2024-04-26T22:48:07.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:07.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:07.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:07.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:07.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T22:48:07.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: monmap epoch 2 2024-04-26T22:48:07.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: fsid 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:48:07.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: last_changed 2024-04-26T22:48:01.975202+0000 2024-04-26T22:48:07.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: created 2024-04-26T22:45:47.264621+0000 2024-04-26T22:48:07.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: min_mon_release 19 (squid) 2024-04-26T22:48:07.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: election_strategy: 1 2024-04-26T22:48:07.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T22:48:07.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: 1: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.c 2024-04-26T22:48:07.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: fsmap 2024-04-26T22:48:07.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T22:48:07.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: mgrmap e13: x(active, since 62s) 2024-04-26T22:48:07.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: overall HEALTH_OK 2024-04-26T22:48:07.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:07.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:07.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:08.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.283+0000 7f23c0a12700 1 -- 172.21.15.94:0/3978918623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23a80035c0 msgr2=0x7f23a8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:08.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.283+0000 7f23c0a12700 1 --2- 172.21.15.94:0/3978918623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23a80035c0 0x7f23a8005a50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f23bc101d40 tx=0x7f23b4009e80 comp rx=0 tx=0).stop 2024-04-26T22:48:08.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.283+0000 7f23c0a12700 1 -- 172.21.15.94:0/3978918623 shutdown_connections 2024-04-26T22:48:08.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.283+0000 7f23c0a12700 1 --2- 172.21.15.94:0/3978918623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23a80035c0 0x7f23a8005a50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:08.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.283+0000 7f23c0a12700 1 -- 172.21.15.94:0/3978918623 >> 172.21.15.94:0/3978918623 conn(0x7f23bc0fd900 msgr2=0x7f23bc0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:48:08.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.283+0000 7f23c0a12700 1 -- 172.21.15.94:0/3978918623 shutdown_connections 2024-04-26T22:48:08.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.283+0000 7f23c0a12700 1 -- 172.21.15.94:0/3978918623 wait complete. 2024-04-26T22:48:08.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.285+0000 7f23c0a12700 1 Processor -- start 2024-04-26T22:48:08.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.285+0000 7f23c0a12700 1 -- start start 2024-04-26T22:48:08.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.285+0000 7f23c0a12700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a80035c0 0x7f23bc19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:08.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.285+0000 7f23c0a12700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23bc19ec60 0x7f23bc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.285+0000 7f23c0a12700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc109d60 con 0x7f23bc19ec60 2024-04-26T22:48:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.285+0000 7f23c0a12700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc109be0 con 0x7f23a80035c0 2024-04-26T22:48:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.285+0000 7f23ba59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a80035c0 0x7f23bc19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.285+0000 7f23ba59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a80035c0 0x7f23bc19e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55894/0 (socket says 172.21.15.94:55894) 2024-04-26T22:48:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.285+0000 7f23ba59c700 1 -- 172.21.15.94:0/2260160139 learned_addr learned my addr 172.21.15.94:0/2260160139 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T22:48:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.285+0000 7f23ba59c700 1 -- 172.21.15.94:0/2260160139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a80035c0 msgr2=0x7f23bc19e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T22:48:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.285+0000 7f23ba59c700 1 -- 172.21.15.94:0/2260160139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a80035c0 msgr2=0x7f23bc19e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T22:48:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.285+0000 7f23ba59c700 1 --2- 172.21.15.94:0/2260160139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a80035c0 0x7f23bc19e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T22:48:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.286+0000 7f23ba59c700 1 --2- 172.21.15.94:0/2260160139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a80035c0 0x7f23bc19e720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T22:48:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.286+0000 7f23b9d9b700 1 --2- 172.21.15.94:0/2260160139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23bc19ec60 0x7f23bc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.286+0000 7f23b9d9b700 1 -- 172.21.15.94:0/2260160139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a80035c0 msgr2=0x7f23bc19e720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:48:08.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.286+0000 7f23b9d9b700 1 --2- 172.21.15.94:0/2260160139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a80035c0 0x7f23bc19e720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:08.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.286+0000 7f23b9d9b700 1 -- 172.21.15.94:0/2260160139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc19f2a0 con 0x7f23bc19ec60 2024-04-26T22:48:08.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.287+0000 7f23b9d9b700 1 --2- 172.21.15.94:0/2260160139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23bc19ec60 0x7f23bc1988e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f23a400b4d0 tx=0x7f23a400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:48:08.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.287+0000 7f23b37fe700 1 -- 172.21.15.94:0/2260160139 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f23a4011630 con 0x7f23bc19ec60 2024-04-26T22:48:08.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.287+0000 7f23c0a12700 1 -- 172.21.15.94:0/2260160139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23bc198e20 con 0x7f23bc19ec60 2024-04-26T22:48:08.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.287+0000 7f23c0a12700 1 -- 172.21.15.94:0/2260160139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23bc1993b0 con 0x7f23bc19ec60 2024-04-26T22:48:08.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.287+0000 7f23b37fe700 1 -- 172.21.15.94:0/2260160139 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23a4011c80 con 0x7f23bc19ec60 2024-04-26T22:48:08.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.287+0000 7f23b37fe700 1 -- 172.21.15.94:0/2260160139 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f23a401adc0 con 0x7f23bc19ec60 2024-04-26T22:48:08.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.288+0000 7f23c0a12700 1 -- 172.21.15.94:0/2260160139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23bc103550 con 0x7f23bc19ec60 2024-04-26T22:48:08.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.289+0000 7f23b37fe700 1 -- 172.21.15.94:0/2260160139 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f23a4018020 con 0x7f23bc19ec60 2024-04-26T22:48:08.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.289+0000 7f23b37fe700 1 --2- 172.21.15.94:0/2260160139 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f239c039410 0x7f239c03b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:08.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.289+0000 7f23b37fe700 1 -- 172.21.15.94:0/2260160139 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f23a404c280 con 0x7f23bc19ec60 2024-04-26T22:48:08.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.289+0000 7f23ba59c700 1 --2- 172.21.15.94:0/2260160139 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f239c039410 0x7f239c03b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:08.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.290+0000 7f23ba59c700 1 --2- 172.21.15.94:0/2260160139 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f239c039410 0x7f239c03b8d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f23b4007590 tx=0x7f23b40074e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:48:08.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.304+0000 7f23b37fe700 1 -- 172.21.15.94:0/2260160139 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f23a401f020 con 0x7f23bc19ec60 2024-04-26T22:48:08.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:08 smithi027 ceph-mon[28034]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:08.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:08 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T22:48:08.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:08 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:08.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:08 smithi027 ceph-mon[28034]: Deploying daemon mon.b on smithi080 2024-04-26T22:48:08.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:08 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:08.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:08 smithi094 ceph-mon[32889]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:08.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:08 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T22:48:08.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:08 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:08.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:08 smithi094 ceph-mon[32889]: Deploying daemon mon.b on smithi080 2024-04-26T22:48:08.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:08 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:08.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.606+0000 7f23c0a12700 1 -- 172.21.15.94:0/2260160139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f23bc199e10 con 0x7f23bc19ec60 2024-04-26T22:48:08.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.608+0000 7f23b37fe700 1 -- 172.21.15.94:0/2260160139 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f23a4017dd0 con 0x7f23bc19ec60 2024-04-26T22:48:08.608 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:48:08.608 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":2,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","modified":"2024-04-26T22:48:01.975202Z","created":"2024-04-26T22:45:47.264621Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T22:48:08.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.610+0000 7f23c0a12700 1 -- 172.21.15.94:0/2260160139 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f239c039410 msgr2=0x7f239c03b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:08.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.610+0000 7f23c0a12700 1 --2- 172.21.15.94:0/2260160139 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f239c039410 0x7f239c03b8d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f23b4007590 tx=0x7f23b40074e0 comp rx=0 tx=0).stop 2024-04-26T22:48:08.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.610+0000 7f23c0a12700 1 -- 172.21.15.94:0/2260160139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23bc19ec60 msgr2=0x7f23bc1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:08.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.610+0000 7f23c0a12700 1 --2- 172.21.15.94:0/2260160139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23bc19ec60 0x7f23bc1988e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f23a400b4d0 tx=0x7f23a400b890 comp rx=0 tx=0).stop 2024-04-26T22:48:08.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.610+0000 7f23c0a12700 1 -- 172.21.15.94:0/2260160139 shutdown_connections 2024-04-26T22:48:08.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.610+0000 7f23c0a12700 1 --2- 172.21.15.94:0/2260160139 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f239c039410 0x7f239c03b8d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:08.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.610+0000 7f23c0a12700 1 --2- 172.21.15.94:0/2260160139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a80035c0 0x7f23bc19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:08.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.610+0000 7f23c0a12700 1 --2- 172.21.15.94:0/2260160139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23bc19ec60 0x7f23bc1988e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:08.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.610+0000 7f23c0a12700 1 -- 172.21.15.94:0/2260160139 >> 172.21.15.94:0/2260160139 conn(0x7f23bc0fd900 msgr2=0x7f23bc0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:48:08.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.610+0000 7f23c0a12700 1 -- 172.21.15.94:0/2260160139 shutdown_connections 2024-04-26T22:48:08.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:08.610+0000 7f23c0a12700 1 -- 172.21.15.94:0/2260160139 wait complete. 2024-04-26T22:48:08.612 INFO:teuthology.orchestra.run.smithi094.stderr:dumped monmap epoch 2 2024-04-26T22:48:09.201 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:09 smithi094 ceph-mon[32889]: from='client.? 172.21.15.94:0/2260160139' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T22:48:09.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:09 smithi027 ceph-mon[28034]: from='client.? 172.21.15.94:0/2260160139' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T22:48:10.233 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T22:48:10.234 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T22:48:10.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:10 smithi027 ceph-mon[28034]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:10.474 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:48:10.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:10 smithi094 ceph-mon[32889]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.108 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T22:48:11.108 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T22:48:11.108 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T22:48:11.109 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T22:48:11.109 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T22:48:11.109 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T22:48:11.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T22:48:11.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T22:48:11.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T22:48:11.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T22:48:11.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: Deploying cephadm binary to smithi080 2024-04-26T22:48:11.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:11.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: Added host smithi080 2024-04-26T22:48:11.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:48:11.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi080", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:11.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:11.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:11.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: Updating smithi080:/etc/ceph/ceph.conf 2024-04-26T22:48:11.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: Updating smithi080:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:48:11.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: Updating smithi080:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:48:11.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: Updating smithi080:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:48:11.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi094", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:48:11.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: Deploying cephadm binary to smithi094 2024-04-26T22:48:11.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: Added host smithi094 2024-04-26T22:48:11.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:11.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:48:11.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:11.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:11.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:11.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T22:48:11.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: Updating smithi094:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:48:11.526 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:48:11.526 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: Updating smithi094:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:48:11.526 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.526 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.526 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.526 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2596343239' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T22:48:11.526 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.526 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2596343239' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T22:48:11.526 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T22:48:11.526 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.526 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.526 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:11.526 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:11.527 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:11.527 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.527 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T22:48:11.527 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:11.527 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi080:172.21.15.80=b;smithi094:172.21.15.94=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:48:11.527 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi080:172.21.15.80=b;smithi094:172.21.15.94=c;count:3 2024-04-26T22:48:11.527 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: Deploying daemon mon.c on smithi094 2024-04-26T22:48:11.527 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:48:11.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: mon.a calling monitor election 2024-04-26T22:48:11.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:11.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:11.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:11.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: mon.c calling monitor election 2024-04-26T22:48:11.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:11.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:11.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:11.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T22:48:11.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: monmap epoch 2 2024-04-26T22:48:11.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: fsid 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:48:11.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: last_changed 2024-04-26T22:48:01.975202+0000 2024-04-26T22:48:11.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: created 2024-04-26T22:45:47.264621+0000 2024-04-26T22:48:11.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: min_mon_release 19 (squid) 2024-04-26T22:48:11.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: election_strategy: 1 2024-04-26T22:48:11.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T22:48:11.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: 1: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.c 2024-04-26T22:48:11.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: fsmap 2024-04-26T22:48:11.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T22:48:11.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: mgrmap e13: x(active, since 62s) 2024-04-26T22:48:11.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: overall HEALTH_OK 2024-04-26T22:48:11.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:11.530 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.530 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T22:48:11.530 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:11.530 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: Deploying daemon mon.b on smithi080 2024-04-26T22:48:11.530 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:11.530 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: from='client.? 172.21.15.94:0/2260160139' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T22:48:11.530 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:11.530 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 ceph-mon[32324]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-26T22:48:11.530 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 bash[32305]: fe1b7809b5c131b406023f199bb443aa97a5337ebd1a926c67d61d7aeb9879fa 2024-04-26T22:48:11.530 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:11 smithi080 systemd[1]: Started Ceph mon.b for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:48:16.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.407+0000 7fd0ed0cc700 1 -- 172.21.15.94:0/649707569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0d00057d0 msgr2=0x7fd0d0007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:16.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.407+0000 7fd0ed0cc700 1 --2- 172.21.15.94:0/649707569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0d00057d0 0x7fd0d0007c60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8101d40 tx=0x7fd0d800fe60 comp rx=0 tx=0).stop 2024-04-26T22:48:16.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.407+0000 7fd0ed0cc700 1 -- 172.21.15.94:0/649707569 shutdown_connections 2024-04-26T22:48:16.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.407+0000 7fd0ed0cc700 1 --2- 172.21.15.94:0/649707569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0d00057d0 0x7fd0d0007c60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:16.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.407+0000 7fd0ed0cc700 1 -- 172.21.15.94:0/649707569 >> 172.21.15.94:0/649707569 conn(0x7fd0e80fd900 msgr2=0x7fd0e80ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:48:16.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.408+0000 7fd0ed0cc700 1 -- 172.21.15.94:0/649707569 shutdown_connections 2024-04-26T22:48:16.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.408+0000 7fd0ed0cc700 1 -- 172.21.15.94:0/649707569 wait complete. 2024-04-26T22:48:16.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.409+0000 7fd0ed0cc700 1 Processor -- start 2024-04-26T22:48:16.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.409+0000 7fd0ed0cc700 1 -- start start 2024-04-26T22:48:16.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.409+0000 7fd0ed0cc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0d00057d0 0x7fd0e819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:16.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.409+0000 7fd0ed0cc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e819cbd0 0x7fd0e8196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:16.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.409+0000 7fd0ed0cc700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0e8196ca0 0x7fd0e8197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:16.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.409+0000 7fd0ed0cc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e8108cf0 con 0x7fd0d00057d0 2024-04-26T22:48:16.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.409+0000 7fd0ed0cc700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e806bc30 con 0x7fd0e8196ca0 2024-04-26T22:48:16.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.409+0000 7fd0ed0cc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e8108e70 con 0x7fd0e819cbd0 2024-04-26T22:48:16.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.410+0000 7fd0e659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e819cbd0 0x7fd0e8196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:16.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.410+0000 7fd0e659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e819cbd0 0x7fd0e8196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41854/0 (socket says 172.21.15.94:41854) 2024-04-26T22:48:16.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.410+0000 7fd0e659c700 1 -- 172.21.15.94:0/2638763850 learned_addr learned my addr 172.21.15.94:0/2638763850 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T22:48:16.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.410+0000 7fd0e759e700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0e8196ca0 0x7fd0e8197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:16.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.410+0000 7fd0e659c700 1 -- 172.21.15.94:0/2638763850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e819cbd0 msgr2=0x7fd0e8196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T22:48:16.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.410+0000 7fd0e659c700 1 -- 172.21.15.94:0/2638763850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e819cbd0 msgr2=0x7fd0e8196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T22:48:16.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.410+0000 7fd0e659c700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e819cbd0 0x7fd0e8196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T22:48:16.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.410+0000 7fd0e659c700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e819cbd0 0x7fd0e8196760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T22:48:16.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.410+0000 7fd0e759e700 1 -- 172.21.15.94:0/2638763850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0e8196ca0 msgr2=0x7fd0e8197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-26T22:48:16.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.410+0000 7fd0e759e700 1 -- 172.21.15.94:0/2638763850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0e8196ca0 msgr2=0x7fd0e8197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T22:48:16.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.410+0000 7fd0e759e700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0e8196ca0 0x7fd0e8197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T22:48:16.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.410+0000 7fd0e759e700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0e8196ca0 0x7fd0e8197150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T22:48:16.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.410+0000 7fd0e6d9d700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0d00057d0 0x7fd0e819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:16.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.411+0000 7fd0e6d9d700 1 -- 172.21.15.94:0/2638763850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e819cbd0 msgr2=0x7fd0e8196760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:48:16.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.411+0000 7fd0e6d9d700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e819cbd0 0x7fd0e8196760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:16.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.411+0000 7fd0e6d9d700 1 -- 172.21.15.94:0/2638763850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0e8196ca0 msgr2=0x7fd0e8197150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:48:16.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.411+0000 7fd0e6d9d700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0e8196ca0 0x7fd0e8197150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:16.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.411+0000 7fd0e6d9d700 1 -- 172.21.15.94:0/2638763850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e81a3440 con 0x7fd0d00057d0 2024-04-26T22:48:16.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.411+0000 7fd0e6d9d700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0d00057d0 0x7fd0e819c690 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8101d40 tx=0x7fd0d8003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:48:16.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.412+0000 7fd0cffff700 1 -- 172.21.15.94:0/2638763850 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d802b020 con 0x7fd0d00057d0 2024-04-26T22:48:16.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.412+0000 7fd0ed0cc700 1 -- 172.21.15.94:0/2638763850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e81a3670 con 0x7fd0d00057d0 2024-04-26T22:48:16.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.412+0000 7fd0cffff700 1 -- 172.21.15.94:0/2638763850 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0d80056c0 con 0x7fd0d00057d0 2024-04-26T22:48:16.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.412+0000 7fd0cffff700 1 -- 172.21.15.94:0/2638763850 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8022040 con 0x7fd0d00057d0 2024-04-26T22:48:16.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.412+0000 7fd0ed0cc700 1 -- 172.21.15.94:0/2638763850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e81a3c00 con 0x7fd0d00057d0 2024-04-26T22:48:16.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.413+0000 7fd0ed0cc700 1 -- 172.21.15.94:0/2638763850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e8103550 con 0x7fd0d00057d0 2024-04-26T22:48:16.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.414+0000 7fd0cffff700 1 -- 172.21.15.94:0/2638763850 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd0d802e020 con 0x7fd0d00057d0 2024-04-26T22:48:16.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.414+0000 7fd0cffff700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fd0d4039ae0 0x7fd0d403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:16.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.414+0000 7fd0cffff700 1 -- 172.21.15.94:0/2638763850 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd0d804f290 con 0x7fd0d00057d0 2024-04-26T22:48:16.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.414+0000 7fd0e659c700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fd0d4039ae0 0x7fd0d403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:16.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.415+0000 7fd0e659c700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fd0d4039ae0 0x7fd0d403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009180 tx=0x7fd0dc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:48:16.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.432+0000 7fd0cffff700 1 -- 172.21.15.94:0/2638763850 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd0d8035020 con 0x7fd0d00057d0 2024-04-26T22:48:16.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:16.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: mon.c calling monitor election 2024-04-26T22:48:16.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:48:16.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: mon.a calling monitor election 2024-04-26T22:48:16.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:16.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:16.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:16.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:16.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:16.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:16.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:16.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:16.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:16.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T22:48:16.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: monmap epoch 3 2024-04-26T22:48:16.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: fsid 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:48:16.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: last_changed 2024-04-26T22:48:11.114675+0000 2024-04-26T22:48:16.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: created 2024-04-26T22:45:47.264621+0000 2024-04-26T22:48:16.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: min_mon_release 19 (squid) 2024-04-26T22:48:16.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: election_strategy: 1 2024-04-26T22:48:16.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T22:48:16.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: 1: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.c 2024-04-26T22:48:16.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: 2: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.b 2024-04-26T22:48:16.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: fsmap 2024-04-26T22:48:16.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T22:48:16.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: mgrmap e13: x(active, since 71s) 2024-04-26T22:48:16.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: overall HEALTH_OK 2024-04-26T22:48:16.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:16.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:16.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:16.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:16.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: mon.c calling monitor election 2024-04-26T22:48:16.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:48:16.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: mon.a calling monitor election 2024-04-26T22:48:16.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:16.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:16.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:16.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:16.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:16.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:16.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:16.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:16.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:16.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T22:48:16.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: monmap epoch 3 2024-04-26T22:48:16.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: fsid 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:48:16.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: last_changed 2024-04-26T22:48:11.114675+0000 2024-04-26T22:48:16.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: created 2024-04-26T22:45:47.264621+0000 2024-04-26T22:48:16.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: min_mon_release 19 (squid) 2024-04-26T22:48:16.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: election_strategy: 1 2024-04-26T22:48:16.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T22:48:16.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: 1: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.c 2024-04-26T22:48:16.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: 2: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.b 2024-04-26T22:48:16.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: fsmap 2024-04-26T22:48:16.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T22:48:16.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: mgrmap e13: x(active, since 71s) 2024-04-26T22:48:16.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: overall HEALTH_OK 2024-04-26T22:48:16.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:16.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:16.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:16.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.731+0000 7fd0ed0cc700 1 -- 172.21.15.94:0/2638763850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd0e802e490 con 0x7fd0d00057d0 2024-04-26T22:48:16.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.732+0000 7fd0cffff700 1 -- 172.21.15.94:0/2638763850 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7fd0d801d440 con 0x7fd0d00057d0 2024-04-26T22:48:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T22:48:16.733 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":3,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","modified":"2024-04-26T22:48:11.114675Z","created":"2024-04-26T22:45:47.264621Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:3300","nonce":0},{"type":"v1","addr":"172.21.15.80:6789","nonce":0}]},"addr":"172.21.15.80:6789/0","public_addr":"172.21.15.80:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T22:48:16.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.734+0000 7fd0ed0cc700 1 -- 172.21.15.94:0/2638763850 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fd0d4039ae0 msgr2=0x7fd0d403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:16.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.734+0000 7fd0ed0cc700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fd0d4039ae0 0x7fd0d403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009180 tx=0x7fd0dc008040 comp rx=0 tx=0).stop 2024-04-26T22:48:16.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.734+0000 7fd0ed0cc700 1 -- 172.21.15.94:0/2638763850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0d00057d0 msgr2=0x7fd0e819c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:16.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.734+0000 7fd0ed0cc700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0d00057d0 0x7fd0e819c690 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8101d40 tx=0x7fd0d8003d50 comp rx=0 tx=0).stop 2024-04-26T22:48:16.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.734+0000 7fd0ed0cc700 1 -- 172.21.15.94:0/2638763850 shutdown_connections 2024-04-26T22:48:16.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.734+0000 7fd0ed0cc700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fd0d4039ae0 0x7fd0d403bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:16.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.734+0000 7fd0ed0cc700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0e8196ca0 0x7fd0e8197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:16.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.734+0000 7fd0ed0cc700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0d00057d0 0x7fd0e819c690 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:16.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.734+0000 7fd0ed0cc700 1 --2- 172.21.15.94:0/2638763850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e819cbd0 0x7fd0e8196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:16.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.734+0000 7fd0ed0cc700 1 -- 172.21.15.94:0/2638763850 >> 172.21.15.94:0/2638763850 conn(0x7fd0e80fd900 msgr2=0x7fd0e80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:48:16.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.734+0000 7fd0ed0cc700 1 -- 172.21.15.94:0/2638763850 shutdown_connections 2024-04-26T22:48:16.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:16.734+0000 7fd0ed0cc700 1 -- 172.21.15.94:0/2638763850 wait complete. 2024-04-26T22:48:16.736 INFO:teuthology.orchestra.run.smithi094.stderr:dumped monmap epoch 3 2024-04-26T22:48:17.314 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:17 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:17.314 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:17 smithi094 ceph-mon[32889]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:17.314 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:17 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:17.314 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:17 smithi094 ceph-mon[32889]: from='client.? 172.21.15.94:0/2638763850' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T22:48:17.314 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:17 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:17.337 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T22:48:17.338 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T22:48:17.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:17 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:17.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:17 smithi027 ceph-mon[28034]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:17.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:17 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:17.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:17 smithi027 ceph-mon[28034]: from='client.? 172.21.15.94:0/2638763850' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T22:48:17.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:17 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:17.779 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:48:18.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:18.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: mon.c calling monitor election 2024-04-26T22:48:18.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:48:18.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: mon.a calling monitor election 2024-04-26T22:48:18.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:18.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:48:18.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:18.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:18.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:18.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:18.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:18.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:18.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:18.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T22:48:18.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: monmap epoch 3 2024-04-26T22:48:18.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: fsid 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:48:18.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: last_changed 2024-04-26T22:48:11.114675+0000 2024-04-26T22:48:18.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: created 2024-04-26T22:45:47.264621+0000 2024-04-26T22:48:18.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: min_mon_release 19 (squid) 2024-04-26T22:48:18.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: election_strategy: 1 2024-04-26T22:48:18.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T22:48:18.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: 1: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.c 2024-04-26T22:48:18.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: 2: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.b 2024-04-26T22:48:18.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: fsmap 2024-04-26T22:48:18.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T22:48:18.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: mgrmap e13: x(active, since 71s) 2024-04-26T22:48:18.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: overall HEALTH_OK 2024-04-26T22:48:18.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:18.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:18.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:18.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:18.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:18.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:18.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: from='client.? 172.21.15.94:0/2638763850' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T22:48:18.524 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:19.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: mon.b calling monitor election 2024-04-26T22:48:19.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-26T22:48:19.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: Updating smithi080:/etc/ceph/ceph.conf 2024-04-26T22:48:19.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T22:48:19.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: mon.b calling monitor election 2024-04-26T22:48:19.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: mon.c calling monitor election 2024-04-26T22:48:19.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: mon.a calling monitor election 2024-04-26T22:48:19.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T22:48:19.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: monmap epoch 3 2024-04-26T22:48:19.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: fsid 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:48:19.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: last_changed 2024-04-26T22:48:11.114675+0000 2024-04-26T22:48:19.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: created 2024-04-26T22:45:47.264621+0000 2024-04-26T22:48:19.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: min_mon_release 19 (squid) 2024-04-26T22:48:19.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: election_strategy: 1 2024-04-26T22:48:19.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T22:48:19.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: 1: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.c 2024-04-26T22:48:19.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: 2: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.b 2024-04-26T22:48:19.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: fsmap 2024-04-26T22:48:19.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T22:48:19.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: mgrmap e13: x(active, since 73s) 2024-04-26T22:48:19.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: overall HEALTH_OK 2024-04-26T22:48:19.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:19.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:19.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:19.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:19 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:19.392 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: mon.b calling monitor election 2024-04-26T22:48:19.392 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-26T22:48:19.392 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: Updating smithi080:/etc/ceph/ceph.conf 2024-04-26T22:48:19.392 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T22:48:19.392 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: mon.b calling monitor election 2024-04-26T22:48:19.392 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: mon.c calling monitor election 2024-04-26T22:48:19.392 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: mon.a calling monitor election 2024-04-26T22:48:19.393 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T22:48:19.393 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: monmap epoch 3 2024-04-26T22:48:19.393 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: fsid 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:48:19.393 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: last_changed 2024-04-26T22:48:11.114675+0000 2024-04-26T22:48:19.393 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: created 2024-04-26T22:45:47.264621+0000 2024-04-26T22:48:19.393 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: min_mon_release 19 (squid) 2024-04-26T22:48:19.393 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: election_strategy: 1 2024-04-26T22:48:19.393 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T22:48:19.393 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: 1: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.c 2024-04-26T22:48:19.393 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: 2: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.b 2024-04-26T22:48:19.393 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: fsmap 2024-04-26T22:48:19.393 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T22:48:19.393 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: mgrmap e13: x(active, since 73s) 2024-04-26T22:48:19.393 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: overall HEALTH_OK 2024-04-26T22:48:19.393 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:19.394 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:19.394 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:19.394 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:19 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:19.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: mon.b calling monitor election 2024-04-26T22:48:19.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-26T22:48:19.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: Updating smithi080:/etc/ceph/ceph.conf 2024-04-26T22:48:19.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T22:48:19.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: mon.b calling monitor election 2024-04-26T22:48:19.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: mon.c calling monitor election 2024-04-26T22:48:19.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: mon.a calling monitor election 2024-04-26T22:48:19.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T22:48:19.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: monmap epoch 3 2024-04-26T22:48:19.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: fsid 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:48:19.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: last_changed 2024-04-26T22:48:11.114675+0000 2024-04-26T22:48:19.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: created 2024-04-26T22:45:47.264621+0000 2024-04-26T22:48:19.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: min_mon_release 19 (squid) 2024-04-26T22:48:19.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: election_strategy: 1 2024-04-26T22:48:19.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T22:48:19.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: 1: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.c 2024-04-26T22:48:19.537 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: 2: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.b 2024-04-26T22:48:19.537 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: fsmap 2024-04-26T22:48:19.537 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T22:48:19.537 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: mgrmap e13: x(active, since 73s) 2024-04-26T22:48:19.537 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: overall HEALTH_OK 2024-04-26T22:48:19.537 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:19.537 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:19.537 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:19.537 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:19 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.908+0000 7f68bd000700 1 -- 172.21.15.27:0/2791217238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b8103090 msgr2=0x7f68b8103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.908+0000 7f68bd000700 1 --2- 172.21.15.27:0/2791217238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b8103090 0x7f68b8103470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009910 tx=0x7f68a8009c20 comp rx=0 tx=0).stop 2024-04-26T22:48:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.908+0000 7f68bd000700 1 -- 172.21.15.27:0/2791217238 shutdown_connections 2024-04-26T22:48:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.908+0000 7f68bd000700 1 --2- 172.21.15.27:0/2791217238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b8103090 0x7f68b8103470 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.908+0000 7f68bd000700 1 -- 172.21.15.27:0/2791217238 >> 172.21.15.27:0/2791217238 conn(0x7f68b80fe960 msgr2=0x7f68b8100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:48:19.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.908+0000 7f68bd000700 1 -- 172.21.15.27:0/2791217238 shutdown_connections 2024-04-26T22:48:19.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.908+0000 7f68bd000700 1 -- 172.21.15.27:0/2791217238 wait complete. 2024-04-26T22:48:19.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.909+0000 7f68bd000700 1 Processor -- start 2024-04-26T22:48:19.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.910+0000 7f68bd000700 1 -- start start 2024-04-26T22:48:19.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.910+0000 7f68bd000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b8103090 0x7f68b81a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:19.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.910+0000 7f68bd000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68b81a0e50 0x7f68b819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:19.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.910+0000 7f68bd000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b819b010 0x7f68b819b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:19.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.910+0000 7f68bd000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f68b810d310 con 0x7f68b819b010 2024-04-26T22:48:19.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.910+0000 7f68bd000700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f68b810d190 con 0x7f68b81a0e50 2024-04-26T22:48:19.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.910+0000 7f68bd000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68b810d490 con 0x7f68b8103090 2024-04-26T22:48:19.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.910+0000 7f68b5e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b8103090 0x7f68b81a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:19.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.910+0000 7f68b6800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b819b010 0x7f68b819b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:19.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.910+0000 7f68b5e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b8103090 0x7f68b81a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:39472/0 (socket says 172.21.15.27:39472) 2024-04-26T22:48:19.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.910+0000 7f68b5e00700 1 -- 172.21.15.27:0/313711908 learned_addr learned my addr 172.21.15.27:0/313711908 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:48:19.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.911+0000 7f68b5400700 1 --2- 172.21.15.27:0/313711908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68b81a0e50 0x7f68b819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:19.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.911+0000 7f68b5e00700 1 -- 172.21.15.27:0/313711908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68b81a0e50 msgr2=0x7f68b819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:19.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.911+0000 7f68b5e00700 1 --2- 172.21.15.27:0/313711908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68b81a0e50 0x7f68b819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:19.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.911+0000 7f68b5e00700 1 -- 172.21.15.27:0/313711908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b819b010 msgr2=0x7f68b819b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:19.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.911+0000 7f68b5e00700 1 --2- 172.21.15.27:0/313711908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b819b010 0x7f68b819b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:19.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.911+0000 7f68b5e00700 1 -- 172.21.15.27:0/313711908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b81a77e0 con 0x7f68b8103090 2024-04-26T22:48:19.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.911+0000 7f68b5e00700 1 --2- 172.21.15.27:0/313711908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b8103090 0x7f68b81a0910 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f68a8005b40 tx=0x7f68a8004f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:48:19.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.911+0000 7f68aea00700 1 -- 172.21.15.27:0/313711908 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8026070 con 0x7f68b8103090 2024-04-26T22:48:19.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.912+0000 7f68aea00700 1 -- 172.21.15.27:0/313711908 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68a800b6f0 con 0x7f68b8103090 2024-04-26T22:48:19.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.912+0000 7f68bd000700 1 -- 172.21.15.27:0/313711908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b81a7a10 con 0x7f68b8103090 2024-04-26T22:48:19.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.912+0000 7f68aea00700 1 -- 172.21.15.27:0/313711908 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a80038e0 con 0x7f68b8103090 2024-04-26T22:48:19.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.912+0000 7f68bd000700 1 -- 172.21.15.27:0/313711908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b81a7fa0 con 0x7f68b8103090 2024-04-26T22:48:19.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.913+0000 7f68aea00700 1 -- 172.21.15.27:0/313711908 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f68a8029070 con 0x7f68b8103090 2024-04-26T22:48:19.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.913+0000 7f68bd000700 1 -- 172.21.15.27:0/313711908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b8064a80 con 0x7f68b8103090 2024-04-26T22:48:19.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.913+0000 7f68aea00700 1 --2- 172.21.15.27:0/313711908 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f689c039ae0 0x7f689c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:19.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.913+0000 7f68aea00700 1 -- 172.21.15.27:0/313711908 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f68a804d4a0 con 0x7f68b8103090 2024-04-26T22:48:19.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.914+0000 7f68b5400700 1 --2- 172.21.15.27:0/313711908 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f689c039ae0 0x7f689c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:19.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.914+0000 7f68b5400700 1 --2- 172.21.15.27:0/313711908 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f689c039ae0 0x7f689c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009850 tx=0x7f68a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:48:19.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:19.919+0000 7f68aea00700 1 -- 172.21.15.27:0/313711908 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f68a8013070 con 0x7f68b8103090 2024-04-26T22:48:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:20.156+0000 7f68bd000700 1 -- 172.21.15.27:0/313711908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f68b81a81d0 con 0x7f68b8103090 2024-04-26T22:48:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:20.157+0000 7f68aea00700 1 -- 172.21.15.27:0/313711908 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7f68a8030020 con 0x7f68b8103090 2024-04-26T22:48:20.158 INFO:teuthology.orchestra.run.smithi027.stdout:# minimal ceph.conf for 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:48:20.158 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2024-04-26T22:48:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:48:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: mon_host = [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] 2024-04-26T22:48:20.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:20.160+0000 7f68bd000700 1 -- 172.21.15.27:0/313711908 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f689c039ae0 msgr2=0x7f689c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:20.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:20.160+0000 7f68bd000700 1 --2- 172.21.15.27:0/313711908 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f689c039ae0 0x7f689c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009850 tx=0x7f68a4008040 comp rx=0 tx=0).stop 2024-04-26T22:48:20.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:20.160+0000 7f68bd000700 1 -- 172.21.15.27:0/313711908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b8103090 msgr2=0x7f68b81a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:20.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:20.160+0000 7f68bd000700 1 --2- 172.21.15.27:0/313711908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b8103090 0x7f68b81a0910 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f68a8005b40 tx=0x7f68a8004f80 comp rx=0 tx=0).stop 2024-04-26T22:48:20.161 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:48:20.115+0000 7f5b92800700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T22:48:20.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:20.160+0000 7f68bd000700 1 -- 172.21.15.27:0/313711908 shutdown_connections 2024-04-26T22:48:20.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:20.160+0000 7f68bd000700 1 --2- 172.21.15.27:0/313711908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b819b010 0x7f68b819b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:20.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:20.160+0000 7f68bd000700 1 --2- 172.21.15.27:0/313711908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b8103090 0x7f68b81a0910 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:20.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:20.160+0000 7f68bd000700 1 --2- 172.21.15.27:0/313711908 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f689c039ae0 0x7f689c03bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:20.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:20.160+0000 7f68bd000700 1 --2- 172.21.15.27:0/313711908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68b81a0e50 0x7f68b819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:20.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:20.160+0000 7f68bd000700 1 -- 172.21.15.27:0/313711908 >> 172.21.15.27:0/313711908 conn(0x7f68b80fe960 msgr2=0x7f68b8075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:48:20.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:20.160+0000 7f68bd000700 1 -- 172.21.15.27:0/313711908 shutdown_connections 2024-04-26T22:48:20.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-mon[28034]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:20.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-mon[28034]: Updating smithi080:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:48:20.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-mon[28034]: Updating smithi094:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:48:20.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-mon[28034]: Updating smithi027:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:48:20.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:20.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T22:48:20.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T22:48:20.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:20 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:20.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:20.161+0000 7f68bd000700 1 -- 172.21.15.27:0/313711908 wait complete. 2024-04-26T22:48:20.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:20 smithi080 ceph-mon[32324]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:20.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:20 smithi080 ceph-mon[32324]: Updating smithi080:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:48:20.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:20 smithi080 ceph-mon[32324]: Updating smithi094:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:48:20.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:20 smithi080 ceph-mon[32324]: Updating smithi027:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:48:20.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:20 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:20.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:20 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:20 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:20 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:20 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:20 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:20 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:20 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:20 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:20 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T22:48:20.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:20 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T22:48:20.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:20 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:20.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:20 smithi094 ceph-mon[32889]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:20.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:20 smithi094 ceph-mon[32889]: Updating smithi080:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:48:20.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:20 smithi094 ceph-mon[32889]: Updating smithi094:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:48:20.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:20 smithi094 ceph-mon[32889]: Updating smithi027:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:48:20.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:20 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:48:20.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:20 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:20 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:20 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:20 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:20 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:20 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:20 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:20 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:20.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:20 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T22:48:20.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:20 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T22:48:20.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:20 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:20.960 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T22:48:20.961 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:48:20.961 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T22:48:21.032 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:48:21.032 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:48:21.149 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-26T22:48:21.149 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T22:48:21.180 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-26T22:48:21.180 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:48:21.245 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T22:48:21.246 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T22:48:21.277 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T22:48:21.277 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:48:21.280 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:21 smithi027 ceph-mon[28034]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T22:48:21.280 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:21 smithi027 ceph-mon[28034]: Reconfiguring daemon mon.a on smithi027 2024-04-26T22:48:21.280 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:21 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/313711908' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:21.339 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:21 smithi094 ceph-mon[32889]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T22:48:21.340 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:21 smithi094 ceph-mon[32889]: Reconfiguring daemon mon.a on smithi027 2024-04-26T22:48:21.340 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:21 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/313711908' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:21.343 INFO:tasks.cephadm:Adding mgr.x on smithi027 2024-04-26T22:48:21.343 INFO:tasks.cephadm:Adding mgr.y on smithi080 2024-04-26T22:48:21.343 INFO:tasks.cephadm:Adding mgr.z on smithi094 2024-04-26T22:48:21.344 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi027=x;smithi080=y;smithi094=z' 2024-04-26T22:48:21.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:21 smithi080 ceph-mon[32324]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T22:48:21.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:21 smithi080 ceph-mon[32324]: Reconfiguring daemon mon.a on smithi027 2024-04-26T22:48:21.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:21 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/313711908' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:48:22.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:22 smithi027 ceph-mon[28034]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:22.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:22 smithi080 ceph-mon[32324]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:22.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:22 smithi094 ceph-mon[32889]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:23.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.093+0000 7ff2ca8e7700 1 -- 172.21.15.94:0/2518453011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2c4101840 msgr2=0x7ff2c4103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:23.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.093+0000 7ff2ca8e7700 1 --2- 172.21.15.94:0/2518453011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2c4101840 0x7ff2c4103c70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009910 tx=0x7ff2b8009c20 comp rx=0 tx=0).stop 2024-04-26T22:48:23.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.093+0000 7ff2ca8e7700 1 -- 172.21.15.94:0/2518453011 shutdown_connections 2024-04-26T22:48:23.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.093+0000 7ff2ca8e7700 1 --2- 172.21.15.94:0/2518453011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2c4101840 0x7ff2c4103c70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:23.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.093+0000 7ff2ca8e7700 1 -- 172.21.15.94:0/2518453011 >> 172.21.15.94:0/2518453011 conn(0x7ff2c40fd460 msgr2=0x7ff2c40ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:48:23.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.093+0000 7ff2ca8e7700 1 -- 172.21.15.94:0/2518453011 shutdown_connections 2024-04-26T22:48:23.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.093+0000 7ff2ca8e7700 1 -- 172.21.15.94:0/2518453011 wait complete. 2024-04-26T22:48:23.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.094+0000 7ff2ca8e7700 1 Processor -- start 2024-04-26T22:48:23.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.094+0000 7ff2ca8e7700 1 -- start start 2024-04-26T22:48:23.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.094+0000 7ff2ca8e7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2c4101840 0x7ff2c419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:23.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.094+0000 7ff2ca8e7700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2c419f280 0x7ff2c41a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:23.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.094+0000 7ff2ca8e7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2c419f700 0x7ff2c41a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:23.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.094+0000 7ff2ca8e7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c4106db0 con 0x7ff2c4101840 2024-04-26T22:48:23.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.094+0000 7ff2ca8e7700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c4106c30 con 0x7ff2c419f280 2024-04-26T22:48:23.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.094+0000 7ff2ca8e7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c4106f30 con 0x7ff2c419f700 2024-04-26T22:48:23.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.094+0000 7ff2c8e84700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2c419f700 0x7ff2c41a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:23.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.094+0000 7ff2c8e84700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2c419f700 0x7ff2c41a6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42590/0 (socket says 172.21.15.94:42590) 2024-04-26T22:48:23.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.094+0000 7ff2c8e84700 1 -- 172.21.15.94:0/496567058 learned_addr learned my addr 172.21.15.94:0/496567058 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T22:48:23.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.095+0000 7ff2c37fe700 1 --2- 172.21.15.94:0/496567058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2c419f280 0x7ff2c41a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:23.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.095+0000 7ff2c8e84700 1 -- 172.21.15.94:0/496567058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2c419f280 msgr2=0x7ff2c41a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:23.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.095+0000 7ff2c8e84700 1 --2- 172.21.15.94:0/496567058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2c419f280 0x7ff2c41a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:23.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.095+0000 7ff2c8e84700 1 -- 172.21.15.94:0/496567058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2c4101840 msgr2=0x7ff2c419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:23.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.095+0000 7ff2c8e84700 1 --2- 172.21.15.94:0/496567058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2c4101840 0x7ff2c419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:23.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.095+0000 7ff2c8e84700 1 -- 172.21.15.94:0/496567058 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c41a7160 con 0x7ff2c419f700 2024-04-26T22:48:23.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.095+0000 7ff2c8e84700 1 --2- 172.21.15.94:0/496567058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2c419f700 0x7ff2c41a6860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00ba50 tx=0x7ff2bc00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:48:23.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.095+0000 7ff2c17fa700 1 -- 172.21.15.94:0/496567058 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc019070 con 0x7ff2c419f700 2024-04-26T22:48:23.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.095+0000 7ff2c17fa700 1 -- 172.21.15.94:0/496567058 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2bc014440 con 0x7ff2c419f700 2024-04-26T22:48:23.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.095+0000 7ff2ca8e7700 1 -- 172.21.15.94:0/496567058 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c41a7450 con 0x7ff2c419f700 2024-04-26T22:48:23.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.095+0000 7ff2c17fa700 1 -- 172.21.15.94:0/496567058 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc007590 con 0x7ff2c419f700 2024-04-26T22:48:23.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.095+0000 7ff2ca8e7700 1 -- 172.21.15.94:0/496567058 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c41a7a10 con 0x7ff2c419f700 2024-04-26T22:48:23.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.096+0000 7ff2c17fa700 1 -- 172.21.15.94:0/496567058 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff2bc01c070 con 0x7ff2c419f700 2024-04-26T22:48:23.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.096+0000 7ff2c17fa700 1 --2- 172.21.15.94:0/496567058 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff2ac039ae0 0x7ff2ac03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:23.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.096+0000 7ff2c17fa700 1 -- 172.21.15.94:0/496567058 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff2bc04bf50 con 0x7ff2c419f700 2024-04-26T22:48:23.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.096+0000 7ff2ca8e7700 1 -- 172.21.15.94:0/496567058 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c4064a80 con 0x7ff2c419f700 2024-04-26T22:48:23.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.097+0000 7ff2c3fff700 1 --2- 172.21.15.94:0/496567058 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff2ac039ae0 0x7ff2ac03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:23.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.098+0000 7ff2c3fff700 1 --2- 172.21.15.94:0/496567058 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff2ac039ae0 0x7ff2ac03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8005e00 tx=0x7ff2b8005260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:48:23.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.102+0000 7ff2c17fa700 1 -- 172.21.15.94:0/496567058 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff2bc00e0a0 con 0x7ff2c419f700 2024-04-26T22:48:23.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.312+0000 7ff2ca8e7700 1 -- 172.21.15.94:0/496567058 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi080=y;smithi094=z", "target": ["mon-mgr", ""]}) -- 0x7ff2c4063920 con 0x7ff2ac039ae0 2024-04-26T22:48:23.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.327+0000 7ff2c17fa700 1 -- 172.21.15.94:0/496567058 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff2c4063920 con 0x7ff2ac039ae0 2024-04-26T22:48:23.327 INFO:teuthology.orchestra.run.smithi094.stdout:Scheduled mgr update... 2024-04-26T22:48:23.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.329+0000 7ff2ca8e7700 1 -- 172.21.15.94:0/496567058 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff2ac039ae0 msgr2=0x7ff2ac03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:23.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.329+0000 7ff2ca8e7700 1 --2- 172.21.15.94:0/496567058 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff2ac039ae0 0x7ff2ac03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8005e00 tx=0x7ff2b8005260 comp rx=0 tx=0).stop 2024-04-26T22:48:23.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.329+0000 7ff2ca8e7700 1 -- 172.21.15.94:0/496567058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2c419f700 msgr2=0x7ff2c41a6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:23.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.329+0000 7ff2ca8e7700 1 --2- 172.21.15.94:0/496567058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2c419f700 0x7ff2c41a6860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00ba50 tx=0x7ff2bc00be10 comp rx=0 tx=0).stop 2024-04-26T22:48:23.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.329+0000 7ff2ca8e7700 1 -- 172.21.15.94:0/496567058 shutdown_connections 2024-04-26T22:48:23.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.329+0000 7ff2ca8e7700 1 --2- 172.21.15.94:0/496567058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2c419f700 0x7ff2c41a6860 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:23.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.329+0000 7ff2ca8e7700 1 --2- 172.21.15.94:0/496567058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2c4101840 0x7ff2c419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:23.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.329+0000 7ff2ca8e7700 1 --2- 172.21.15.94:0/496567058 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff2ac039ae0 0x7ff2ac03bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:23.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.329+0000 7ff2ca8e7700 1 --2- 172.21.15.94:0/496567058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2c419f280 0x7ff2c41a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:23.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.329+0000 7ff2ca8e7700 1 -- 172.21.15.94:0/496567058 >> 172.21.15.94:0/496567058 conn(0x7ff2c40fd460 msgr2=0x7ff2c40fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:48:23.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.329+0000 7ff2ca8e7700 1 -- 172.21.15.94:0/496567058 shutdown_connections 2024-04-26T22:48:23.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:48:23.329+0000 7ff2ca8e7700 1 -- 172.21.15.94:0/496567058 wait complete. 2024-04-26T22:48:23.694 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:23 smithi080 ceph-mon[32324]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:23.695 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:23.695 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:23.695 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:23 smithi080 ceph-mon[32324]: Reconfiguring mon.b (monmap changed)... 2024-04-26T22:48:23.695 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T22:48:23.695 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T22:48:23.695 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:23.695 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:23 smithi080 ceph-mon[32324]: Reconfiguring daemon mon.b on smithi080 2024-04-26T22:48:23.695 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:23.729 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:23 smithi094 ceph-mon[32889]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:23.730 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:23.730 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:23.730 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:23 smithi094 ceph-mon[32889]: Reconfiguring mon.b (monmap changed)... 2024-04-26T22:48:23.730 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T22:48:23.730 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T22:48:23.730 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:23.730 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:23 smithi094 ceph-mon[32889]: Reconfiguring daemon mon.b on smithi080 2024-04-26T22:48:23.730 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:23.816 DEBUG:teuthology.orchestra.run.smithi080:mgr.y> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mgr.y.service 2024-04-26T22:48:23.819 DEBUG:teuthology.orchestra.run.smithi094:mgr.z> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mgr.z.service 2024-04-26T22:48:23.821 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T22:48:23.821 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:48:23.821 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T22:48:23.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:23 smithi027 ceph-mon[28034]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:23.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:23.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:23.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:23 smithi027 ceph-mon[28034]: Reconfiguring mon.b (monmap changed)... 2024-04-26T22:48:23.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T22:48:23.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T22:48:23.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:23.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:23 smithi027 ceph-mon[28034]: Reconfiguring daemon mon.b on smithi080 2024-04-26T22:48:23.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:23.884 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T22:48:23.884 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2024-04-26T22:48:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T22:48:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:48:23.986 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-26T22:48:23.986 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:48:23.986 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:48:23.986 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-26 22:47:21.961492125 +0000 2024-04-26T22:48:23.986 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-26 22:42:04.548545406 +0000 2024-04-26T22:48:23.986 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-26 22:42:04.548545406 +0000 2024-04-26T22:48:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-26 22:42:04.548545406 +0000 2024-04-26T22:48:23.986 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T22:48:24.078 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-04-26T22:48:24.078 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-04-26T22:48:24.079 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000404911 s, 1.3 MB/s 2024-04-26T22:48:24.083 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T22:48:24.144 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2024-04-26T22:48:24.195 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T22:48:24.195 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:48:24.195 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-26T22:48:24.195 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:48:24.195 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:48:24.195 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-26 22:47:22.030490352 +0000 2024-04-26T22:48:24.195 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-26 22:42:05.201528789 +0000 2024-04-26T22:48:24.195 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-26 22:42:05.201528789 +0000 2024-04-26T22:48:24.195 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-26 22:42:05.201528789 +0000 2024-04-26T22:48:24.196 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T22:48:24.273 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-04-26T22:48:24.273 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-04-26T22:48:24.273 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.00037094 s, 1.4 MB/s 2024-04-26T22:48:24.276 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T22:48:24.334 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2024-04-26T22:48:24.390 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T22:48:24.390 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:48:24.390 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-26T22:48:24.390 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:48:24.391 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:48:24.391 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-26 22:47:22.088488862 +0000 2024-04-26T22:48:24.391 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-26 22:42:05.808513343 +0000 2024-04-26T22:48:24.391 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-26 22:42:05.808513343 +0000 2024-04-26T22:48:24.391 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-26 22:42:05.808513343 +0000 2024-04-26T22:48:24.391 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T22:48:24.463 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-04-26T22:48:24.463 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-04-26T22:48:24.464 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000339537 s, 1.5 MB/s 2024-04-26T22:48:24.467 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T22:48:24.523 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2024-04-26T22:48:24.574 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T22:48:24.574 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:48:24.574 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-26T22:48:24.574 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:48:24.574 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:48:24.574 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-26 22:47:22.146487372 +0000 2024-04-26T22:48:24.574 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-26 22:42:06.439497286 +0000 2024-04-26T22:48:24.575 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-26 22:42:06.439497286 +0000 2024-04-26T22:48:24.575 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-26 22:42:06.439497286 +0000 2024-04-26T22:48:24.575 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T22:48:24.659 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-04-26T22:48:24.660 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-04-26T22:48:24.660 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000376375 s, 1.4 MB/s 2024-04-26T22:48:24.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:24 smithi027 ceph-mon[28034]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi080=y;smithi094=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:48:24.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:24 smithi027 ceph-mon[28034]: Saving service mgr spec with placement smithi027=x;smithi080=y;smithi094=z;count:3 2024-04-26T22:48:24.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:24 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:24.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:24 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:24.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:24 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T22:48:24.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:24 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T22:48:24.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:24 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:24.665 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T22:48:24.728 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-26T22:48:24.728 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T22:48:24.746 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T22:48:24.746 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_1 2024-04-26T22:48:24.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:24 smithi080 ceph-mon[32324]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi080=y;smithi094=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:48:24.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:24 smithi080 ceph-mon[32324]: Saving service mgr spec with placement smithi027=x;smithi080=y;smithi094=z;count:3 2024-04-26T22:48:24.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:24 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:24.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:24 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:24.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:24 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T22:48:24.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:24 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T22:48:24.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:24 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:24.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:24 smithi094 ceph-mon[32889]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi080=y;smithi094=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:48:24.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:24 smithi094 ceph-mon[32889]: Saving service mgr spec with placement smithi027=x;smithi080=y;smithi094=z;count:3 2024-04-26T22:48:24.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:24 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:24.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:24 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:24.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:24 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T22:48:24.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:24 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T22:48:24.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:24 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:24.787 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T22:48:24.787 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:48:24.787 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-26T22:48:24.787 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:48:24.787 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:48:24.787 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-26 22:48:10.557821163 +0000 2024-04-26T22:48:24.788 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-26 22:40:53.822810866 +0000 2024-04-26T22:48:24.788 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-26 22:40:53.822810866 +0000 2024-04-26T22:48:24.788 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-04-26 22:40:53.822810866 +0000 2024-04-26T22:48:24.788 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T22:48:24.851 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-26T22:48:24.851 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-26T22:48:24.852 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000220588 s, 2.3 MB/s 2024-04-26T22:48:24.853 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T22:48:24.909 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_2 2024-04-26T22:48:24.965 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T22:48:24.965 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:48:24.965 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-26T22:48:24.966 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:48:24.966 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:48:24.966 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-26 22:48:10.557821163 +0000 2024-04-26T22:48:24.966 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-26 22:40:54.154803057 +0000 2024-04-26T22:48:24.966 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-26 22:40:54.154803057 +0000 2024-04-26T22:48:24.966 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-04-26 22:40:54.154803057 +0000 2024-04-26T22:48:24.966 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T22:48:25.028 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-26T22:48:25.028 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-26T22:48:25.028 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000207625 s, 2.5 MB/s 2024-04-26T22:48:25.029 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T22:48:25.086 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_3 2024-04-26T22:48:25.140 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T22:48:25.141 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:48:25.141 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-26T22:48:25.141 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:48:25.141 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:48:25.141 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-26 22:48:10.558821140 +0000 2024-04-26T22:48:25.141 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-26 22:40:54.483795318 +0000 2024-04-26T22:48:25.141 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-26 22:40:54.483795318 +0000 2024-04-26T22:48:25.141 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-04-26 22:40:54.483795318 +0000 2024-04-26T22:48:25.141 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T22:48:25.203 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-26T22:48:25.203 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-26T22:48:25.203 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000194408 s, 2.6 MB/s 2024-04-26T22:48:25.204 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T22:48:25.260 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_4 2024-04-26T22:48:25.315 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T22:48:25.315 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:48:25.316 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-26T22:48:25.316 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:48:25.316 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:48:25.316 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-26 22:48:10.558821140 +0000 2024-04-26T22:48:25.316 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-26 22:40:54.851786661 +0000 2024-04-26T22:48:25.316 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-26 22:40:54.851786661 +0000 2024-04-26T22:48:25.316 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-04-26 22:40:54.851786661 +0000 2024-04-26T22:48:25.316 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T22:48:25.377 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-26T22:48:25.378 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-26T22:48:25.378 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000212446 s, 2.4 MB/s 2024-04-26T22:48:25.379 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T22:48:25.434 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T22:48:25.434 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T22:48:25.450 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T22:48:25.450 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-04-26T22:48:25.506 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T22:48:25.506 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:48:25.506 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-26T22:48:25.506 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:48:25.506 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:48:25.506 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-26 22:48:01.268660559 +0000 2024-04-26T22:48:25.506 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-26 22:40:43.471206347 +0000 2024-04-26T22:48:25.506 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-26 22:40:43.471206347 +0000 2024-04-26T22:48:25.506 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-26 22:40:43.471206347 +0000 2024-04-26T22:48:25.507 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T22:48:25.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:25 smithi080 ceph-mon[32324]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:25.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:25 smithi080 ceph-mon[32324]: Reconfiguring mon.c (monmap changed)... 2024-04-26T22:48:25.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:25 smithi080 ceph-mon[32324]: Reconfiguring daemon mon.c on smithi094 2024-04-26T22:48:25.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:25 smithi094 ceph-mon[32889]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:25 smithi094 ceph-mon[32889]: Reconfiguring mon.c (monmap changed)... 2024-04-26T22:48:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:25 smithi094 ceph-mon[32889]: Reconfiguring daemon mon.c on smithi094 2024-04-26T22:48:25.556 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-26T22:48:25.556 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-26T22:48:25.557 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000106158 s, 4.8 MB/s 2024-04-26T22:48:25.558 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T22:48:25.613 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-04-26T22:48:25.668 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T22:48:25.668 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:48:25.668 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-26T22:48:25.669 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:48:25.669 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:48:25.669 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-26 22:48:01.268660559 +0000 2024-04-26T22:48:25.669 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-26 22:40:43.799201269 +0000 2024-04-26T22:48:25.669 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-26 22:40:43.799201269 +0000 2024-04-26T22:48:25.669 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-26 22:40:43.799201269 +0000 2024-04-26T22:48:25.669 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T22:48:25.730 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-26T22:48:25.730 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-26T22:48:25.730 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000105476 s, 4.9 MB/s 2024-04-26T22:48:25.731 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T22:48:25.787 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-04-26T22:48:25.840 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T22:48:25.841 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:48:25.841 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-26T22:48:25.841 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:48:25.841 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:48:25.841 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-26 22:48:01.269660544 +0000 2024-04-26T22:48:25.841 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-26 22:40:44.117196346 +0000 2024-04-26T22:48:25.841 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-26 22:40:44.117196346 +0000 2024-04-26T22:48:25.841 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-26 22:40:44.117196346 +0000 2024-04-26T22:48:25.841 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T22:48:25.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:25 smithi027 ceph-mon[28034]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:25.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:25 smithi027 ceph-mon[28034]: Reconfiguring mon.c (monmap changed)... 2024-04-26T22:48:25.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:25 smithi027 ceph-mon[28034]: Reconfiguring daemon mon.c on smithi094 2024-04-26T22:48:25.902 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-26T22:48:25.902 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-26T22:48:25.902 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000105706 s, 4.8 MB/s 2024-04-26T22:48:25.903 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T22:48:25.959 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-04-26T22:48:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T22:48:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:48:26.015 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-26T22:48:26.015 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:48:26.015 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:48:26.015 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-26 22:48:01.269660544 +0000 2024-04-26T22:48:26.015 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-26 22:40:44.452191160 +0000 2024-04-26T22:48:26.015 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-26 22:40:44.452191160 +0000 2024-04-26T22:48:26.015 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-26 22:40:44.452191160 +0000 2024-04-26T22:48:26.015 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T22:48:26.076 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-26T22:48:26.076 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-26T22:48:26.076 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 8.9459e-05 s, 5.7 MB/s 2024-04-26T22:48:26.077 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T22:48:26.132 INFO:tasks.cephadm:Deploying osd.0 on smithi027 with /dev/vg_nvme/lv_4... 2024-04-26T22:48:26.132 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T22:48:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:48:26.999 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:26.999 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:26.999 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:26.999 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:26.999 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:26.999 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:27.000 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T22:48:27.000 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T22:48:27.000 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T22:48:27.000 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:27.000 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:26 smithi027 ceph-mon[28034]: Deploying daemon mgr.z on smithi094 2024-04-26T22:48:27.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:27.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:27.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:27.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:27.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:27.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:27.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T22:48:27.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T22:48:27.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T22:48:27.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:27.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:26 smithi080 ceph-mon[32324]: Deploying daemon mgr.z on smithi094 2024-04-26T22:48:27.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:27.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:27.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:27.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:27.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:27.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:27.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T22:48:27.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T22:48:27.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T22:48:27.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:27.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:26 smithi094 ceph-mon[32889]: Deploying daemon mgr.z on smithi094 2024-04-26T22:48:28.363 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:28 smithi027 ceph-mon[28034]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:28.364 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:28 smithi094 ceph-mon[32889]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:28.443 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:28 smithi080 ceph-mon[32324]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:28.621 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:28 smithi094 systemd[1]: Started Ceph mgr.z for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:48:28.911 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:28 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:28.620+0000 7fbfc50ec200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T22:48:28.911 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:28 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:28.809+0000 7fbfc50ec200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:48:28.911 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:28 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:28.911+0000 7fbfc50ec200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T22:48:29.534 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:29 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:29.219+0000 7fbfc50ec200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T22:48:29.534 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:29 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:29.308+0000 7fbfc50ec200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T22:48:29.680 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T22:48:29.681 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T22:48:29.681 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-04-26T22:48:29.681 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-04-26T22:48:29.681 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0527274 s, 199 MB/s 2024-04-26T22:48:29.681 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-04-26T22:48:30.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:29 smithi080 ceph-mon[32324]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:30.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:30.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:30.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:30.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T22:48:30.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T22:48:30.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T22:48:30.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:30.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:29 smithi080 ceph-mon[32324]: Deploying daemon mgr.y on smithi080 2024-04-26T22:48:30.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:29 smithi094 ceph-mon[32889]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:30.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:30.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:30.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:30.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T22:48:30.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T22:48:30.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T22:48:30.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:30.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:29 smithi094 ceph-mon[32889]: Deploying daemon mgr.y on smithi080 2024-04-26T22:48:30.103 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:29 smithi027 ceph-mon[28034]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:30.103 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:30.103 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:30.103 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:30.103 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T22:48:30.103 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T22:48:30.103 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T22:48:30.103 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:30.103 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:29 smithi027 ceph-mon[28034]: Deploying daemon mgr.y on smithi080 2024-04-26T22:48:30.518 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_4 2024-04-26T22:48:30.574 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:30 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:30.511+0000 7fbfc50ec200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T22:48:30.726 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:30 smithi080 systemd[1]: Starting Ceph mgr.y for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T22:48:30.871 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:30 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:30.694+0000 7fbfc50ec200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T22:48:30.985 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:48:30.987 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:30 smithi080 podman[33695]: 2024-04-26 22:48:30.629614042 +0000 UTC m=+0.018911342 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:48:30.987 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:30 smithi080 podman[33695]: 2024-04-26 22:48:30.733747108 +0000 UTC m=+0.123044446 container create 6fd3b77644f81d3b2a8800e968c119f5c004d9484831e13f822847fcc97712f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-26T22:48:30.987 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:30 smithi080 podman[33695]: 2024-04-26 22:48:30.960953348 +0000 UTC m=+0.350250643 container init 6fd3b77644f81d3b2a8800e968c119f5c004d9484831e13f822847fcc97712f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T22:48:30.987 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:30 smithi080 podman[33695]: 2024-04-26 22:48:30.963414954 +0000 UTC m=+0.352712253 container start 6fd3b77644f81d3b2a8800e968c119f5c004d9484831e13f822847fcc97712f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:48:31.170 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:30 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:30.871+0000 7fbfc50ec200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T22:48:31.170 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:31.169+0000 7fbfc50ec200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T22:48:31.271 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:30 smithi080 ceph-mgr[33714]: -- 172.21.15.80:0/2550320343 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560b8243e5a0 con 0x560b82412800 2024-04-26T22:48:31.271 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:31 smithi080 bash[33695]: 6fd3b77644f81d3b2a8800e968c119f5c004d9484831e13f822847fcc97712f8 2024-04-26T22:48:31.271 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:31 smithi080 systemd[1]: Started Ceph mgr.y for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:48:31.271 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:31.114+0000 7f802c2cd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T22:48:31.474 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:31.263+0000 7fbfc50ec200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T22:48:31.474 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:31.474+0000 7fbfc50ec200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T22:48:31.722 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:31.298+0000 7f802c2cd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:48:31.722 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:31.401+0000 7f802c2cd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T22:48:31.722 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:31.721+0000 7f802c2cd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T22:48:31.784 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:31.679+0000 7fbfc50ec200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:48:32.021 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:31.809+0000 7f802c2cd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T22:48:32.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:32 smithi027 ceph-mon[28034]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:32.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:32 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:32.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:32 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:32.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:32 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:32.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:32 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:32.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:32 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:32.410 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:32 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:32.328+0000 7fbfc50ec200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T22:48:32.411 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:32 smithi094 ceph-mon[32889]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:32.411 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:32 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:32.411 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:32 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:32.411 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:32 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:32.411 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:32 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:32.411 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:32 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:32.451 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:32 smithi080 ceph-mon[32324]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:32.452 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:32 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:32.452 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:32 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:32.452 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:32 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:32.452 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:32 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:32.452 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:32 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:33.034 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:32 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:32.768+0000 7fbfc50ec200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T22:48:33.035 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:32 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:32.868+0000 7fbfc50ec200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T22:48:33.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.390+0000 7fc003000700 1 -- 172.21.15.27:0/3524644113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbffc1033b0 msgr2=0x7fbffc1037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:33.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.390+0000 7fc003000700 1 --2- 172.21.15.27:0/3524644113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbffc1033b0 0x7fbffc1037d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbff00099c0 tx=0x7fbff0009cd0 comp rx=0 tx=0).stop 2024-04-26T22:48:33.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.390+0000 7fc003000700 1 -- 172.21.15.27:0/3524644113 shutdown_connections 2024-04-26T22:48:33.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.390+0000 7fc003000700 1 --2- 172.21.15.27:0/3524644113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbffc104f50 0x7fbffc10b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:33.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.390+0000 7fc003000700 1 --2- 172.21.15.27:0/3524644113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbffc1033b0 0x7fbffc1037d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:33.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.390+0000 7fc003000700 1 --2- 172.21.15.27:0/3524644113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc1045b0 0x7fbffc104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:33.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.390+0000 7fc003000700 1 -- 172.21.15.27:0/3524644113 >> 172.21.15.27:0/3524644113 conn(0x7fbffc0fe950 msgr2=0x7fbffc100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:48:33.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.390+0000 7fc003000700 1 -- 172.21.15.27:0/3524644113 shutdown_connections 2024-04-26T22:48:33.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.391+0000 7fc003000700 1 -- 172.21.15.27:0/3524644113 wait complete. 2024-04-26T22:48:33.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.392+0000 7fc003000700 1 Processor -- start 2024-04-26T22:48:33.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.392+0000 7fc003000700 1 -- start start 2024-04-26T22:48:33.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.392+0000 7fc003000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbffc1033b0 0x7fbffc0771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:33.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.392+0000 7fc003000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc1045b0 0x7fbffc075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:33.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.392+0000 7fc003000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbffc104f50 0x7fbffc075d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:33.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.392+0000 7fc003000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc10ee80 con 0x7fbffc104f50 2024-04-26T22:48:33.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.392+0000 7fc003000700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc10ed00 con 0x7fbffc1033b0 2024-04-26T22:48:33.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.392+0000 7fc003000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc10f000 con 0x7fbffc1045b0 2024-04-26T22:48:33.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.392+0000 7fc001c00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbffc1033b0 0x7fbffc0771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:33.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.392+0000 7fc001200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc1045b0 0x7fbffc075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:33.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.392+0000 7fc001c00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbffc1033b0 0x7fbffc0771c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33740/0 (socket says 172.21.15.27:33740) 2024-04-26T22:48:33.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.392+0000 7fc001c00700 1 -- 172.21.15.27:0/2249956397 learned_addr learned my addr 172.21.15.27:0/2249956397 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:48:33.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.393+0000 7fc002600700 1 --2- 172.21.15.27:0/2249956397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbffc104f50 0x7fbffc075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:33.394 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:33.060+0000 7f802c2cd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T22:48:33.394 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:33.225+0000 7f802c2cd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T22:48:33.394 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:33.392+0000 7f802c2cd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T22:48:33.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.393+0000 7fc001200700 1 -- 172.21.15.27:0/2249956397 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbffc1033b0 msgr2=0x7fbffc0771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:33.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.393+0000 7fc001200700 1 --2- 172.21.15.27:0/2249956397 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbffc1033b0 0x7fbffc0771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:33.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.393+0000 7fc001200700 1 -- 172.21.15.27:0/2249956397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbffc104f50 msgr2=0x7fbffc075d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:48:33.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.393+0000 7fc001200700 1 --2- 172.21.15.27:0/2249956397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbffc104f50 0x7fbffc075d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:48:33.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.393+0000 7fc001200700 1 -- 172.21.15.27:0/2249956397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc076600 con 0x7fbffc1045b0 2024-04-26T22:48:33.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.393+0000 7fc001c00700 1 --2- 172.21.15.27:0/2249956397 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbffc1033b0 0x7fbffc0771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:48:33.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.393+0000 7fc002600700 1 --2- 172.21.15.27:0/2249956397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbffc104f50 0x7fbffc075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:48:33.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.393+0000 7fc001200700 1 --2- 172.21.15.27:0/2249956397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc1045b0 0x7fbffc075810 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800e9f0 tx=0x7fbfe800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:48:33.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.394+0000 7fbffa000700 1 -- 172.21.15.27:0/2249956397 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe800cbd0 con 0x7fbffc1045b0 2024-04-26T22:48:33.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.394+0000 7fbffa000700 1 -- 172.21.15.27:0/2249956397 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfe800cd70 con 0x7fbffc1045b0 2024-04-26T22:48:33.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.394+0000 7fc003000700 1 -- 172.21.15.27:0/2249956397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbffc1a3670 con 0x7fbffc1045b0 2024-04-26T22:48:33.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.394+0000 7fbffa000700 1 -- 172.21.15.27:0/2249956397 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8018680 con 0x7fbffc1045b0 2024-04-26T22:48:33.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.394+0000 7fc003000700 1 -- 172.21.15.27:0/2249956397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbffc1a3bb0 con 0x7fbffc1045b0 2024-04-26T22:48:33.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.395+0000 7fc003000700 1 -- 172.21.15.27:0/2249956397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbffc06a110 con 0x7fbffc1045b0 2024-04-26T22:48:33.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.395+0000 7fbffa000700 1 -- 172.21.15.27:0/2249956397 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbfe8018820 con 0x7fbffc1045b0 2024-04-26T22:48:33.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.396+0000 7fbffa000700 1 --2- 172.21.15.27:0/2249956397 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbfe4039890 0x7fbfe403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:48:33.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.396+0000 7fbffa000700 1 -- 172.21.15.27:0/2249956397 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbfe804cbd0 con 0x7fbffc1045b0 2024-04-26T22:48:33.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.396+0000 7fc001c00700 1 --2- 172.21.15.27:0/2249956397 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbfe4039890 0x7fbfe403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:48:33.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.397+0000 7fc001c00700 1 --2- 172.21.15.27:0/2249956397 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbfe4039890 0x7fbfe403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbff0003820 tx=0x7fbff0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:48:33.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.401+0000 7fbffa000700 1 -- 172.21.15.27:0/2249956397 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbfe80071c0 con 0x7fbffc1045b0 2024-04-26T22:48:33.534 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:33.073+0000 7fbfc50ec200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T22:48:33.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:33.643+0000 7fc003000700 1 -- 172.21.15.27:0/2249956397 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbffc063860 con 0x7fbfe4039890 2024-04-26T22:48:33.925 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:33.591+0000 7fbfc50ec200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T22:48:33.925 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:33.683+0000 7fbfc50ec200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T22:48:33.925 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-mon[32889]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:33.925 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:33.926 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:33.926 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:33.926 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:33.926 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:33.926 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:33.926 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:33.926 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-mon[32889]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T22:48:33.926 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T22:48:33.926 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T22:48:33.926 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:33.926 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-mon[32889]: Reconfiguring daemon mgr.x on smithi027 2024-04-26T22:48:33.926 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:48:33.926 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:48:33.927 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:33.953 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:33 smithi027 ceph-mon[28034]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:33.953 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:33.953 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:33.953 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:33.953 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:33.953 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:33.954 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:33.954 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:33.954 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:33 smithi027 ceph-mon[28034]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T22:48:33.954 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T22:48:33.954 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T22:48:33.954 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:33.954 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:33 smithi027 ceph-mon[28034]: Reconfiguring daemon mgr.x on smithi027 2024-04-26T22:48:33.954 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:48:33.954 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:48:33.954 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:33.961 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-mon[32324]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:33.961 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:33.961 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:33.961 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:33.961 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:33.961 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:33.961 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:33.961 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:33.962 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-mon[32324]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T22:48:33.962 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T22:48:33.962 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T22:48:33.962 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:33.962 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-mon[32324]: Reconfiguring daemon mgr.x on smithi027 2024-04-26T22:48:33.962 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:48:33.962 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:48:33.962 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:33.962 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:33.672+0000 7f802c2cd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T22:48:33.962 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:33.763+0000 7f802c2cd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T22:48:33.962 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:33 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:33.960+0000 7f802c2cd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T22:48:34.264 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:33 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:33.924+0000 7fbfc50ec200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T22:48:34.264 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:34 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:34.057+0000 7fbfc50ec200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T22:48:34.264 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:34 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:34.150+0000 7fbfc50ec200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T22:48:34.264 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:34 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:34.264+0000 7fbfc50ec200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T22:48:34.271 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:34 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:34.148+0000 7f802c2cd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:48:35.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:34 smithi080 ceph-mon[32324]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:48:35.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:34 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:35.021 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:34 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:34.737+0000 7f802c2cd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T22:48:35.034 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:34 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:34.566+0000 7fbfc50ec200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T22:48:35.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:34 smithi094 ceph-mon[32889]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:48:35.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:34 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:35.089 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:34 smithi027 ceph-mon[28034]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:48:35.089 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:34 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:35.441 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:35 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:35.148+0000 7f802c2cd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T22:48:35.441 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:35 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:35.243+0000 7f802c2cd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T22:48:35.441 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:35 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:35.440+0000 7f802c2cd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T22:48:35.534 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:35 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:35.162+0000 7fbfc50ec200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T22:48:35.535 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:48:35 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:48:35.377+0000 7fbfc50ec200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T22:48:36.021 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:35 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:35.953+0000 7f802c2cd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T22:48:36.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:35 smithi080 ceph-mon[32324]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:35 smithi080 ceph-mon[32324]: Standby manager daemon z started 2024-04-26T22:48:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:35 smithi080 ceph-mon[32324]: from='mgr.? 172.21.15.94:0/4261823825' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T22:48:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:35 smithi080 ceph-mon[32324]: from='mgr.? 172.21.15.94:0/4261823825' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:48:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:35 smithi080 ceph-mon[32324]: from='mgr.? 172.21.15.94:0/4261823825' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T22:48:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:35 smithi080 ceph-mon[32324]: from='mgr.? 172.21.15.94:0/4261823825' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:48:36.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:35 smithi094 ceph-mon[32889]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:36.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:36.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:36.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:36.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:35 smithi094 ceph-mon[32889]: Standby manager daemon z started 2024-04-26T22:48:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:35 smithi094 ceph-mon[32889]: from='mgr.? 172.21.15.94:0/4261823825' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T22:48:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:35 smithi094 ceph-mon[32889]: from='mgr.? 172.21.15.94:0/4261823825' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:48:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:35 smithi094 ceph-mon[32889]: from='mgr.? 172.21.15.94:0/4261823825' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T22:48:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:35 smithi094 ceph-mon[32889]: from='mgr.? 172.21.15.94:0/4261823825' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:48:36.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:35 smithi027 ceph-mon[28034]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:36.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:36.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:36.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:36.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:36.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:36.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:36.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:35 smithi027 ceph-mon[28034]: Standby manager daemon z started 2024-04-26T22:48:36.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:35 smithi027 ceph-mon[28034]: from='mgr.? 172.21.15.94:0/4261823825' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T22:48:36.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:35 smithi027 ceph-mon[28034]: from='mgr.? 172.21.15.94:0/4261823825' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:48:36.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:35 smithi027 ceph-mon[28034]: from='mgr.? 172.21.15.94:0/4261823825' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T22:48:36.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:35 smithi027 ceph-mon[28034]: from='mgr.? 172.21.15.94:0/4261823825' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:48:36.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:36.317+0000 7fbffa000700 1 -- 172.21.15.27:0/2249956397 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7fbfe80178f0 con 0x7fbffc1045b0 2024-04-26T22:48:36.407 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:36 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:36.043+0000 7f802c2cd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T22:48:36.407 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:36 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:36.277+0000 7f802c2cd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T22:48:36.407 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:36 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:36.406+0000 7f802c2cd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T22:48:36.771 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:36 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:36.498+0000 7f802c2cd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T22:48:36.771 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:36 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:36.609+0000 7f802c2cd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T22:48:37.271 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:36 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:36.902+0000 7f802c2cd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T22:48:37.475 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:37 smithi027 ceph-mon[28034]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:37.475 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:37 smithi027 ceph-mon[28034]: mgrmap e14: x(active, since 92s), standbys: z 2024-04-26T22:48:37.475 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:37 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T22:48:37.699 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:37 smithi080 ceph-mon[32324]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:37.699 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:37 smithi080 ceph-mon[32324]: mgrmap e14: x(active, since 92s), standbys: z 2024-04-26T22:48:37.699 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:37 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T22:48:37.699 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:37 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:37.489+0000 7f802c2cd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T22:48:37.699 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:48:37 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:48:37.698+0000 7f802c2cd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T22:48:37.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:37 smithi094 ceph-mon[32889]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:37.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:37 smithi094 ceph-mon[32889]: mgrmap e14: x(active, since 92s), standbys: z 2024-04-26T22:48:37.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:37 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T22:48:38.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:48:38.338+0000 7fbffa000700 1 -- 172.21.15.27:0/2249956397 <== mon.1 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fbfe8018c10 con 0x7fbffc1045b0 2024-04-26T22:48:38.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:38 smithi027 ceph-mon[28034]: from='mgr.? 172.21.15.80:0/1427340649' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T22:48:38.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:38 smithi027 ceph-mon[28034]: Standby manager daemon y started 2024-04-26T22:48:38.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:38 smithi027 ceph-mon[28034]: from='mgr.? 172.21.15.80:0/1427340649' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:48:38.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:38 smithi027 ceph-mon[28034]: from='mgr.? 172.21.15.80:0/1427340649' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T22:48:38.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:38 smithi027 ceph-mon[28034]: from='mgr.? 172.21.15.80:0/1427340649' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:48:38.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:38 smithi080 ceph-mon[32324]: from='mgr.? 172.21.15.80:0/1427340649' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T22:48:38.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:38 smithi080 ceph-mon[32324]: Standby manager daemon y started 2024-04-26T22:48:38.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:38 smithi080 ceph-mon[32324]: from='mgr.? 172.21.15.80:0/1427340649' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:48:38.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:38 smithi080 ceph-mon[32324]: from='mgr.? 172.21.15.80:0/1427340649' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T22:48:38.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:38 smithi080 ceph-mon[32324]: from='mgr.? 172.21.15.80:0/1427340649' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:48:38.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:38 smithi094 ceph-mon[32889]: from='mgr.? 172.21.15.80:0/1427340649' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T22:48:38.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:38 smithi094 ceph-mon[32889]: Standby manager daemon y started 2024-04-26T22:48:38.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:38 smithi094 ceph-mon[32889]: from='mgr.? 172.21.15.80:0/1427340649' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:48:38.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:38 smithi094 ceph-mon[32889]: from='mgr.? 172.21.15.80:0/1427340649' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T22:48:38.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:38 smithi094 ceph-mon[32889]: from='mgr.? 172.21.15.80:0/1427340649' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:48:39.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:39 smithi027 ceph-mon[28034]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:39.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:39 smithi027 ceph-mon[28034]: mgrmap e15: x(active, since 94s), standbys: z, y 2024-04-26T22:48:39.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:39 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T22:48:39.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:39 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/768909504' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f6f4675-af17-4d78-87cd-9b1ccef44def"}]: dispatch 2024-04-26T22:48:39.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:39 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f6f4675-af17-4d78-87cd-9b1ccef44def"}]: dispatch 2024-04-26T22:48:39.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:39 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f6f4675-af17-4d78-87cd-9b1ccef44def"}]': finished 2024-04-26T22:48:39.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:39 smithi027 ceph-mon[28034]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T22:48:39.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:39 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:48:39.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:39 smithi080 ceph-mon[32324]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:39.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:39 smithi080 ceph-mon[32324]: mgrmap e15: x(active, since 94s), standbys: z, y 2024-04-26T22:48:39.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:39 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T22:48:39.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:39 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/768909504' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f6f4675-af17-4d78-87cd-9b1ccef44def"}]: dispatch 2024-04-26T22:48:39.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:39 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f6f4675-af17-4d78-87cd-9b1ccef44def"}]: dispatch 2024-04-26T22:48:39.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:39 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f6f4675-af17-4d78-87cd-9b1ccef44def"}]': finished 2024-04-26T22:48:39.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:39 smithi080 ceph-mon[32324]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T22:48:39.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:39 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:48:39.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:39 smithi094 ceph-mon[32889]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:39.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:39 smithi094 ceph-mon[32889]: mgrmap e15: x(active, since 94s), standbys: z, y 2024-04-26T22:48:39.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:39 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T22:48:39.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:39 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/768909504' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f6f4675-af17-4d78-87cd-9b1ccef44def"}]: dispatch 2024-04-26T22:48:39.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:39 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f6f4675-af17-4d78-87cd-9b1ccef44def"}]: dispatch 2024-04-26T22:48:39.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:39 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f6f4675-af17-4d78-87cd-9b1ccef44def"}]': finished 2024-04-26T22:48:39.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:39 smithi094 ceph-mon[32889]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T22:48:39.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:39 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:48:40.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:40 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1218329960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:48:40.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:40 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1218329960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:48:40.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:40 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1218329960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:48:41.606 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:41 smithi094 ceph-mon[32889]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:41.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:41 smithi027 ceph-mon[28034]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:41.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:41 smithi080 ceph-mon[32324]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:43.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:43 smithi027 ceph-mon[28034]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:43.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:43 smithi080 ceph-mon[32324]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:43.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:43 smithi094 ceph-mon[32889]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:45.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:45 smithi027 ceph-mon[28034]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:45.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:45 smithi080 ceph-mon[32324]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:45.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:45 smithi094 ceph-mon[32889]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:47.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:47 smithi080 ceph-mon[32324]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:47.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:47 smithi094 ceph-mon[32889]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:47.857 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:47 smithi027 ceph-mon[28034]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:48.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T22:48:48.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:48.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:48 smithi080 ceph-mon[32324]: Deploying daemon osd.0 on smithi027 2024-04-26T22:48:48.773 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T22:48:48.773 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:48.773 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:48 smithi094 ceph-mon[32889]: Deploying daemon osd.0 on smithi027 2024-04-26T22:48:48.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T22:48:48.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:48.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:48 smithi027 ceph-mon[28034]: Deploying daemon osd.0 on smithi027 2024-04-26T22:48:49.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:49 smithi080 ceph-mon[32324]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:49.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:49 smithi094 ceph-mon[32889]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:49.819 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:49 smithi027 ceph-mon[28034]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:51.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:51 smithi027 ceph-mon[28034]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:51.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:51 smithi080 ceph-mon[32324]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:51.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:51 smithi094 ceph-mon[32889]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:53.580 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:53 smithi027 ceph-mon[28034]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:53.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:53 smithi080 ceph-mon[32324]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:53.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:53 smithi094 ceph-mon[32889]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:55.761 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:55 smithi027 ceph-mon[28034]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:55.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:55 smithi080 ceph-mon[32324]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:55.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:55 smithi094 ceph-mon[32889]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:57.900 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:57 smithi027 ceph-mon[28034]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:57.900 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:57.900 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:57.900 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:58.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:57 smithi080 ceph-mon[32324]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:58.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:57 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:58.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:57 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:58.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:57 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:58.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:57 smithi094 ceph-mon[32889]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:48:58.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:57 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:58.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:57 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:48:58.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:57 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:59.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:48:59 smithi027 ceph-mon[28034]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:49:00.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:48:59 smithi080 ceph-mon[32324]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:49:00.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:48:59 smithi094 ceph-mon[32889]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:49:01.514 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:01 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:01.514 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:01 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:01.514 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:01 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:01.514 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:01 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:01.514 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:01 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:01.514 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:01 smithi027 ceph-mon[28034]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:49:01.514 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:01 smithi027 ceph-mon[28034]: from='osd.0 [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T22:49:01.515 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:01 smithi027 ceph-mon[28034]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T22:49:01.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:01 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:01.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:01 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:01.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:01 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:01.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:01 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:01.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:01 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:01.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:01 smithi080 ceph-mon[32324]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:49:01.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:01 smithi080 ceph-mon[32324]: from='osd.0 [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T22:49:01.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:01 smithi080 ceph-mon[32324]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T22:49:01.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:01.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:01.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:01.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:01.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:01.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:01 smithi094 ceph-mon[32889]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:49:01.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:01 smithi094 ceph-mon[32889]: from='osd.0 [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T22:49:01.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:01 smithi094 ceph-mon[32889]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T22:49:01.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:01.598+0000 7fbffa000700 1 -- 172.21.15.27:0/2249956397 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbffc063860 con 0x7fbfe4039890 2024-04-26T22:49:01.599 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 0 on host 'smithi027' 2024-04-26T22:49:01.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:01.602+0000 7fc003000700 1 -- 172.21.15.27:0/2249956397 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbfe4039890 msgr2=0x7fbfe403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:49:01.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:01.602+0000 7fc003000700 1 --2- 172.21.15.27:0/2249956397 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbfe4039890 0x7fbfe403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbff0003820 tx=0x7fbff0005fb0 comp rx=0 tx=0).stop 2024-04-26T22:49:01.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:01.602+0000 7fc003000700 1 -- 172.21.15.27:0/2249956397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc1045b0 msgr2=0x7fbffc075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:49:01.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:01.602+0000 7fc003000700 1 --2- 172.21.15.27:0/2249956397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc1045b0 0x7fbffc075810 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800e9f0 tx=0x7fbfe800ed00 comp rx=0 tx=0).stop 2024-04-26T22:49:01.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:01.602+0000 7fc003000700 1 -- 172.21.15.27:0/2249956397 shutdown_connections 2024-04-26T22:49:01.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:01.602+0000 7fc003000700 1 --2- 172.21.15.27:0/2249956397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbffc104f50 0x7fbffc075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:01.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:01.602+0000 7fc003000700 1 --2- 172.21.15.27:0/2249956397 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbfe4039890 0x7fbfe403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:01.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:01.602+0000 7fc003000700 1 --2- 172.21.15.27:0/2249956397 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbffc1033b0 0x7fbffc0771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:01.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:01.602+0000 7fc003000700 1 --2- 172.21.15.27:0/2249956397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc1045b0 0x7fbffc075810 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:01.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:01.602+0000 7fc003000700 1 -- 172.21.15.27:0/2249956397 >> 172.21.15.27:0/2249956397 conn(0x7fbffc0fe950 msgr2=0x7fbffc107950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:49:01.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:01.602+0000 7fc003000700 1 -- 172.21.15.27:0/2249956397 shutdown_connections 2024-04-26T22:49:01.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:01.602+0000 7fc003000700 1 -- 172.21.15.27:0/2249956397 wait complete. 2024-04-26T22:49:02.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:02 smithi027 ceph-mon[28034]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T22:49:02.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:02 smithi027 ceph-mon[28034]: from='osd.0 [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T22:49:02.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:02 smithi027 ceph-mon[28034]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T22:49:02.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:02 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:49:02.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:02 smithi027 ceph-mon[28034]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T22:49:02.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:02 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:02.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:02 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:02.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:02 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:02.455 DEBUG:teuthology.orchestra.run.smithi027:osd.0> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T22:49:02.459 INFO:tasks.cephadm:Deploying osd.1 on smithi080 with /dev/vg_nvme/lv_4... 2024-04-26T22:49:02.459 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T22:49:02.700 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T22:49:02.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:02 smithi080 ceph-mon[32324]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T22:49:02.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:02 smithi080 ceph-mon[32324]: from='osd.0 [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T22:49:02.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:02 smithi080 ceph-mon[32324]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T22:49:02.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:02 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:49:02.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:02 smithi080 ceph-mon[32324]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T22:49:02.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:02 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:02.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:02 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:02.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:02 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:02.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:02 smithi094 ceph-mon[32889]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T22:49:02.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:02 smithi094 ceph-mon[32889]: from='osd.0 [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T22:49:02.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:02 smithi094 ceph-mon[32889]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T22:49:02.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:02 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:49:02.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:02 smithi094 ceph-mon[32889]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T22:49:02.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:02 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:02.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:02 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:02.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:02 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:03.276 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 22:49:02 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-0[42699]: 2024-04-26T22:49:02.919+0000 7f58d6c00700 -1 osd.0 0 waiting for initial osdmap 2024-04-26T22:49:03.276 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 22:49:02 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-0[42699]: 2024-04-26T22:49:02.935+0000 7f58cca00700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:49:03.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:03 smithi080 ceph-mon[32324]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T22:49:03.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:03 smithi080 ceph-mon[32324]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T22:49:03.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:03 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:49:03.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:03 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:49:03.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:03 smithi080 ceph-mon[32324]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:49:03.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:03 smithi080 ceph-mon[32324]: from='osd.0 ' entity='osd.0' 2024-04-26T22:49:03.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:03 smithi027 ceph-mon[28034]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T22:49:03.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:03 smithi027 ceph-mon[28034]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T22:49:03.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:03 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:49:03.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:03 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:49:03.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:03 smithi027 ceph-mon[28034]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:49:03.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:03 smithi027 ceph-mon[28034]: from='osd.0 ' entity='osd.0' 2024-04-26T22:49:03.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:03 smithi094 ceph-mon[32889]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T22:49:03.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:03 smithi094 ceph-mon[32889]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T22:49:03.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:03 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:49:03.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:03 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:49:03.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:03 smithi094 ceph-mon[32889]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T22:49:03.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:03 smithi094 ceph-mon[32889]: from='osd.0 ' entity='osd.0' 2024-04-26T22:49:04.554 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:04 smithi080 ceph-mon[32324]: purged_snaps scrub starts 2024-04-26T22:49:04.555 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:04 smithi080 ceph-mon[32324]: purged_snaps scrub ok 2024-04-26T22:49:04.555 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:49:04.555 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:04 smithi080 ceph-mon[32324]: osd.0 [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880] boot 2024-04-26T22:49:04.555 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:04 smithi080 ceph-mon[32324]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T22:49:04.555 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:49:04.555 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T22:49:04.555 INFO:teuthology.orchestra.run.smithi080.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T22:49:04.555 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10+0 records in 2024-04-26T22:49:04.555 INFO:teuthology.orchestra.run.smithi080.stderr:10+0 records out 2024-04-26T22:49:04.556 INFO:teuthology.orchestra.run.smithi080.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112401 s, 933 MB/s 2024-04-26T22:49:04.556 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping successful for: 2024-04-26T22:49:04.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:04 smithi027 ceph-mon[28034]: purged_snaps scrub starts 2024-04-26T22:49:04.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:04 smithi027 ceph-mon[28034]: purged_snaps scrub ok 2024-04-26T22:49:04.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:49:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:04 smithi027 ceph-mon[28034]: osd.0 [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880] boot 2024-04-26T22:49:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:04 smithi027 ceph-mon[28034]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T22:49:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:49:04.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:04 smithi094 ceph-mon[32889]: purged_snaps scrub starts 2024-04-26T22:49:04.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:04 smithi094 ceph-mon[32889]: purged_snaps scrub ok 2024-04-26T22:49:04.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:49:04.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:04 smithi094 ceph-mon[32889]: osd.0 [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880] boot 2024-04-26T22:49:04.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:04 smithi094 ceph-mon[32889]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T22:49:04.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:49:05.168 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi080:vg_nvme/lv_4 2024-04-26T22:49:05.402 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T22:49:05.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:05 smithi080 ceph-mon[32324]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:05.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:05 smithi027 ceph-mon[28034]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:05.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:05 smithi094 ceph-mon[32889]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:06.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:06 smithi080 ceph-mon[32324]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T22:49:06.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:06 smithi027 ceph-mon[28034]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T22:49:06.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:06 smithi094 ceph-mon[32889]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T22:49:06.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 -- 172.21.15.80:0/450409572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6c01068c0 msgr2=0x7ff6c0108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:49:06.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 --2- 172.21.15.80:0/450409572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6c01068c0 0x7ff6c0108cb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009990 tx=0x7ff6b0009ca0 comp rx=0 tx=0).stop 2024-04-26T22:49:06.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 -- 172.21.15.80:0/450409572 shutdown_connections 2024-04-26T22:49:06.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 --2- 172.21.15.80:0/450409572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c01091f0 0x7ff6c010b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:06.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 --2- 172.21.15.80:0/450409572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c0103f90 0x7ff6c0106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:06.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 --2- 172.21.15.80:0/450409572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6c01068c0 0x7ff6c0108cb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:06.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 -- 172.21.15.80:0/450409572 >> 172.21.15.80:0/450409572 conn(0x7ff6c00fd940 msgr2=0x7ff6c00ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:49:06.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 -- 172.21.15.80:0/450409572 shutdown_connections 2024-04-26T22:49:06.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 -- 172.21.15.80:0/450409572 wait complete. 2024-04-26T22:49:06.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 Processor -- start 2024-04-26T22:49:06.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 -- start start 2024-04-26T22:49:06.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6c0103f90 0x7ff6c0199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:49:06.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c01068c0 0x7ff6c0199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:49:06.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c01091f0 0x7ff6c01a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:49:06.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c010eff0 con 0x7ff6c01091f0 2024-04-26T22:49:06.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c010ee70 con 0x7ff6c0103f90 2024-04-26T22:49:06.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c7c2c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c010f170 con 0x7ff6c01068c0 2024-04-26T22:49:06.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.785+0000 7ff6c59c8700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6c0103f90 0x7ff6c0199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:49:06.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.786+0000 7ff6c59c8700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6c0103f90 0x7ff6c0199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56504/0 (socket says 172.21.15.80:56504) 2024-04-26T22:49:06.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.786+0000 7ff6c59c8700 1 -- 172.21.15.80:0/2035638049 learned_addr learned my addr 172.21.15.80:0/2035638049 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-26T22:49:06.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.786+0000 7ff6c51c7700 1 --2- 172.21.15.80:0/2035638049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c01068c0 0x7ff6c0199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:49:06.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.786+0000 7ff6c59c8700 1 -- 172.21.15.80:0/2035638049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c01068c0 msgr2=0x7ff6c0199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:49:06.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.786+0000 7ff6c59c8700 1 --2- 172.21.15.80:0/2035638049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c01068c0 0x7ff6c0199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:06.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.786+0000 7ff6c59c8700 1 -- 172.21.15.80:0/2035638049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c01091f0 msgr2=0x7ff6c01a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:49:06.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.786+0000 7ff6c59c8700 1 --2- 172.21.15.80:0/2035638049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c01091f0 0x7ff6c01a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:06.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.786+0000 7ff6c59c8700 1 -- 172.21.15.80:0/2035638049 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c01a12a0 con 0x7ff6c0103f90 2024-04-26T22:49:06.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.786+0000 7ff6c51c7700 1 --2- 172.21.15.80:0/2035638049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c01068c0 0x7ff6c0199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:49:06.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.786+0000 7ff6c59c8700 1 --2- 172.21.15.80:0/2035638049 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6c0103f90 0x7ff6c0199640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00c920 tx=0x7ff6bc00cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:49:06.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.786+0000 7ff6b6ffd700 1 -- 172.21.15.80:0/2035638049 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc007b00 con 0x7ff6c0103f90 2024-04-26T22:49:06.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.786+0000 7ff6b6ffd700 1 -- 172.21.15.80:0/2035638049 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6bc004ce0 con 0x7ff6c0103f90 2024-04-26T22:49:06.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.786+0000 7ff6c7c2c700 1 -- 172.21.15.80:0/2035638049 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c01a1590 con 0x7ff6c0103f90 2024-04-26T22:49:06.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.786+0000 7ff6b6ffd700 1 -- 172.21.15.80:0/2035638049 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc005680 con 0x7ff6c0103f90 2024-04-26T22:49:06.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.786+0000 7ff6c7c2c700 1 -- 172.21.15.80:0/2035638049 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c01a1b50 con 0x7ff6c0103f90 2024-04-26T22:49:06.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.787+0000 7ff6c7c2c700 1 -- 172.21.15.80:0/2035638049 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c006a110 con 0x7ff6c0103f90 2024-04-26T22:49:06.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.788+0000 7ff6b6ffd700 1 -- 172.21.15.80:0/2035638049 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7ff6bc004750 con 0x7ff6c0103f90 2024-04-26T22:49:06.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.789+0000 7ff6b6ffd700 1 --2- 172.21.15.80:0/2035638049 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff6ac0a3bb0 0x7ff6ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:49:06.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.789+0000 7ff6b6ffd700 1 -- 172.21.15.80:0/2035638049 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7ff6bc0ce550 con 0x7ff6c0103f90 2024-04-26T22:49:06.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.789+0000 7ff6c51c7700 1 --2- 172.21.15.80:0/2035638049 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff6ac0a3bb0 0x7ff6ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:49:06.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.790+0000 7ff6c51c7700 1 --2- 172.21.15.80:0/2035638049 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff6ac0a3bb0 0x7ff6ac0a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000b5c0 tx=0x7ff6b0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:49:06.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:06.793+0000 7ff6b6ffd700 1 -- 172.21.15.80:0/2035638049 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff6bc014030 con 0x7ff6c0103f90 2024-04-26T22:49:07.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:07.001+0000 7ff6c7c2c700 1 -- 172.21.15.80:0/2035638049 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff6c0063860 con 0x7ff6ac0a3bb0 2024-04-26T22:49:07.599 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:07 smithi080 ceph-mon[32324]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:07.599 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:07 smithi080 ceph-mon[32324]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:49:07.600 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:07 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:49:07.600 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:07 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:49:07.600 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:07 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:07.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:07 smithi027 ceph-mon[28034]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:07.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:07 smithi027 ceph-mon[28034]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:49:07.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:49:07.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:49:07.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:07.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:07 smithi094 ceph-mon[32889]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:07.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:07 smithi094 ceph-mon[32889]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:49:07.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:49:07.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:49:07.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:09.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:08 smithi080 ceph-mon[32324]: Detected new or changed devices on smithi027 2024-04-26T22:49:09.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:08 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:09.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:08 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:09.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:08 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:49:09.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:08 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi027 to 17089M 2024-04-26T22:49:09.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:08 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:09.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:08 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:09.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:08 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:09.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:08 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:09.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:08 smithi094 ceph-mon[32889]: Detected new or changed devices on smithi027 2024-04-26T22:49:09.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:08 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:09.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:08 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:09.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:08 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:49:09.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:08 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi027 to 17089M 2024-04-26T22:49:09.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:08 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:09.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:08 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:09.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:08 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:09.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:08 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:09.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:08 smithi027 ceph-mon[28034]: Detected new or changed devices on smithi027 2024-04-26T22:49:09.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:08 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:09.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:08 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:09.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:08 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:49:09.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:08 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi027 to 17089M 2024-04-26T22:49:09.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:08 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:09.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:08 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:09.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:08 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:09.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:08 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:10.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:09 smithi027 ceph-mon[28034]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:10.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:09 smithi027 ceph-mon[28034]: from='client.? 172.21.15.80:0/1575718035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e980f676-0a81-4253-8a39-db33f37ed79d"}]: dispatch 2024-04-26T22:49:10.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:09 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e980f676-0a81-4253-8a39-db33f37ed79d"}]: dispatch 2024-04-26T22:49:10.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:09 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e980f676-0a81-4253-8a39-db33f37ed79d"}]': finished 2024-04-26T22:49:10.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:09 smithi027 ceph-mon[28034]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T22:49:10.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:09 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:10.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:09 smithi080 ceph-mon[32324]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:10.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:09 smithi080 ceph-mon[32324]: from='client.? 172.21.15.80:0/1575718035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e980f676-0a81-4253-8a39-db33f37ed79d"}]: dispatch 2024-04-26T22:49:10.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:09 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e980f676-0a81-4253-8a39-db33f37ed79d"}]: dispatch 2024-04-26T22:49:10.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:09 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e980f676-0a81-4253-8a39-db33f37ed79d"}]': finished 2024-04-26T22:49:10.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:09 smithi080 ceph-mon[32324]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T22:49:10.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:09 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:10.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:09 smithi094 ceph-mon[32889]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:10.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:09 smithi094 ceph-mon[32889]: from='client.? 172.21.15.80:0/1575718035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e980f676-0a81-4253-8a39-db33f37ed79d"}]: dispatch 2024-04-26T22:49:10.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:09 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e980f676-0a81-4253-8a39-db33f37ed79d"}]: dispatch 2024-04-26T22:49:10.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:09 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e980f676-0a81-4253-8a39-db33f37ed79d"}]': finished 2024-04-26T22:49:10.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:09 smithi094 ceph-mon[32889]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T22:49:10.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:09 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:11.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:11 smithi027 ceph-mon[28034]: from='client.? 172.21.15.80:0/4257189454' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:49:11.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:11 smithi080 ceph-mon[32324]: from='client.? 172.21.15.80:0/4257189454' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:49:11.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:11 smithi094 ceph-mon[32889]: from='client.? 172.21.15.80:0/4257189454' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:49:12.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:12 smithi080 ceph-mon[32324]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:12.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:12 smithi094 ceph-mon[32889]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:12.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:12 smithi027 ceph-mon[28034]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:14.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:14 smithi080 ceph-mon[32324]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:14.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:14 smithi094 ceph-mon[32889]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:14.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:14 smithi027 ceph-mon[28034]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:16.113 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:16 smithi080 ceph-mon[32324]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:16.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:16 smithi094 ceph-mon[32889]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:16.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:16 smithi027 ceph-mon[28034]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:17.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:17 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T22:49:17.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:17 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:17.364 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:17 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T22:49:17.364 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:17 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:17.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:17 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T22:49:17.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:17 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:18.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:18 smithi080 ceph-mon[32324]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:18.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:18 smithi080 ceph-mon[32324]: Deploying daemon osd.1 on smithi080 2024-04-26T22:49:18.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:18 smithi094 ceph-mon[32889]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:18.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:18 smithi094 ceph-mon[32889]: Deploying daemon osd.1 on smithi080 2024-04-26T22:49:18.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:18 smithi027 ceph-mon[28034]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:18.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:18 smithi027 ceph-mon[28034]: Deploying daemon osd.1 on smithi080 2024-04-26T22:49:20.205 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:20 smithi080 ceph-mon[32324]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:20.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:20 smithi027 ceph-mon[28034]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:20.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:20 smithi094 ceph-mon[32889]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:22.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:22 smithi080 ceph-mon[32324]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:22.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:22.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:22.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:22.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:22 smithi027 ceph-mon[28034]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:22.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:22.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:22.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:22.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:22 smithi094 ceph-mon[32889]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:22.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:22.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:22.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:24.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:24 smithi080 ceph-mon[32324]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:24.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:24 smithi027 ceph-mon[28034]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:24.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:24 smithi094 ceph-mon[32889]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:25.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:25.492+0000 7ff6b6ffd700 1 -- 172.21.15.80:0/2035638049 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff6c0063860 con 0x7ff6ac0a3bb0 2024-04-26T22:49:25.493 INFO:teuthology.orchestra.run.smithi080.stdout:Created osd(s) 1 on host 'smithi080' 2024-04-26T22:49:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:25.494+0000 7ff6c7c2c700 1 -- 172.21.15.80:0/2035638049 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff6ac0a3bb0 msgr2=0x7ff6ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:49:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:25.494+0000 7ff6c7c2c700 1 --2- 172.21.15.80:0/2035638049 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff6ac0a3bb0 0x7ff6ac0a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000b5c0 tx=0x7ff6b0005fb0 comp rx=0 tx=0).stop 2024-04-26T22:49:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:25.494+0000 7ff6c7c2c700 1 -- 172.21.15.80:0/2035638049 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6c0103f90 msgr2=0x7ff6c0199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:49:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:25.494+0000 7ff6c7c2c700 1 --2- 172.21.15.80:0/2035638049 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6c0103f90 0x7ff6c0199640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00c920 tx=0x7ff6bc00cce0 comp rx=0 tx=0).stop 2024-04-26T22:49:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:25.494+0000 7ff6c7c2c700 1 -- 172.21.15.80:0/2035638049 shutdown_connections 2024-04-26T22:49:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:25.494+0000 7ff6c7c2c700 1 --2- 172.21.15.80:0/2035638049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c01091f0 0x7ff6c01a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:25.494+0000 7ff6c7c2c700 1 --2- 172.21.15.80:0/2035638049 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff6ac0a3bb0 0x7ff6ac0a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:25.494+0000 7ff6c7c2c700 1 --2- 172.21.15.80:0/2035638049 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6c0103f90 0x7ff6c0199640 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:25.494+0000 7ff6c7c2c700 1 --2- 172.21.15.80:0/2035638049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c01068c0 0x7ff6c0199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:25.494+0000 7ff6c7c2c700 1 -- 172.21.15.80:0/2035638049 >> 172.21.15.80:0/2035638049 conn(0x7ff6c00fd940 msgr2=0x7ff6c01048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:49:25.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:25.494+0000 7ff6c7c2c700 1 -- 172.21.15.80:0/2035638049 shutdown_connections 2024-04-26T22:49:25.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:49:25.494+0000 7ff6c7c2c700 1 -- 172.21.15.80:0/2035638049 wait complete. 2024-04-26T22:49:26.071 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:26 smithi080 ceph-mon[32324]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:26.071 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:26.071 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:26.072 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:26.072 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:26.072 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:26.072 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:26.072 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:26.072 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:26.072 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:26 smithi080 ceph-mon[32324]: from='osd.1 [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T22:49:26.072 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:26 smithi080 ceph-mon[32324]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T22:49:26.073 DEBUG:teuthology.orchestra.run.smithi080:osd.1> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T22:49:26.075 INFO:tasks.cephadm:Deploying osd.2 on smithi094 with /dev/vg_nvme/lv_4... 2024-04-26T22:49:26.076 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T22:49:26.316 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:49:26.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:26 smithi027 ceph-mon[28034]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:26.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:26.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:26.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:26.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:26.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:26.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:26.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:26.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:26.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:26 smithi027 ceph-mon[28034]: from='osd.1 [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T22:49:26.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:26 smithi027 ceph-mon[28034]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T22:49:26.438 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:26 smithi094 ceph-mon[32889]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:26.438 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:26.438 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:26.438 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:26.438 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:26.438 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:26.438 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:26.438 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:26.438 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:26.438 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:26 smithi094 ceph-mon[32889]: from='osd.1 [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T22:49:26.439 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:26 smithi094 ceph-mon[32889]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T22:49:27.583 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:27 smithi080 ceph-mon[32324]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:27.584 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:27 smithi080 ceph-mon[32324]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T22:49:27.584 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:27 smithi080 ceph-mon[32324]: from='osd.1 [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:49:27.584 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:27 smithi080 ceph-mon[32324]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T22:49:27.584 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:27.584 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:27 smithi080 ceph-mon[32324]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:49:27.650 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:27 smithi094 ceph-mon[32889]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:27.650 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:27 smithi094 ceph-mon[32889]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T22:49:27.650 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:27 smithi094 ceph-mon[32889]: from='osd.1 [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:49:27.650 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:27 smithi094 ceph-mon[32889]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T22:49:27.650 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:27.650 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:27 smithi094 ceph-mon[32889]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:49:27.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:27 smithi027 ceph-mon[28034]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:27.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:27 smithi027 ceph-mon[28034]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T22:49:27.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:27 smithi027 ceph-mon[28034]: from='osd.1 [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:49:27.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:27 smithi027 ceph-mon[28034]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T22:49:27.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:27.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:27 smithi027 ceph-mon[28034]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:49:28.223 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T22:49:28.223 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T22:49:28.223 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-26T22:49:28.223 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-26T22:49:28.223 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00809457 s, 1.3 GB/s 2024-04-26T22:49:28.223 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-26T22:49:28.233 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 22:49:27 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-1[36926]: 2024-04-26T22:49:27.917+0000 7f0815dc4700 -1 osd.1 0 waiting for initial osdmap 2024-04-26T22:49:28.233 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 22:49:27 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-1[36926]: 2024-04-26T22:49:27.920+0000 7f080dbaf700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:49:28.500 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:28 smithi080 ceph-mon[32324]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-26T22:49:28.500 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:28 smithi080 ceph-mon[32324]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T22:49:28.500 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:28 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:28.500 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:28 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:28.500 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:28 smithi080 ceph-mon[32324]: from='osd.1 ' entity='osd.1' 2024-04-26T22:49:28.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:28 smithi094 ceph-mon[32889]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-26T22:49:28.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:28 smithi094 ceph-mon[32889]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T22:49:28.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:28 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:28.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:28 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:28.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:28 smithi094 ceph-mon[32889]: from='osd.1 ' entity='osd.1' 2024-04-26T22:49:28.856 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_4 2024-04-26T22:49:28.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:28 smithi027 ceph-mon[28034]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-26T22:49:28.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:28 smithi027 ceph-mon[28034]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T22:49:28.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:28 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:28.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:28 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:28.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:28 smithi027 ceph-mon[28034]: from='osd.1 ' entity='osd.1' 2024-04-26T22:49:29.093 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:49:29.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:29 smithi094 ceph-mon[32889]: purged_snaps scrub starts 2024-04-26T22:49:29.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:29 smithi094 ceph-mon[32889]: purged_snaps scrub ok 2024-04-26T22:49:29.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:29 smithi094 ceph-mon[32889]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:29.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:29.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:29 smithi094 ceph-mon[32889]: Detected new or changed devices on smithi080 2024-04-26T22:49:29.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:29.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:29.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:49:29.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:29 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi080 to 17088M 2024-04-26T22:49:29.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:29.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:29.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:29.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:29.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:29 smithi094 ceph-mon[32889]: osd.1 [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245] boot 2024-04-26T22:49:29.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:29 smithi094 ceph-mon[32889]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T22:49:29.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:29.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:29 smithi080 ceph-mon[32324]: purged_snaps scrub starts 2024-04-26T22:49:29.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:29 smithi080 ceph-mon[32324]: purged_snaps scrub ok 2024-04-26T22:49:29.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:29 smithi080 ceph-mon[32324]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:29.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:29.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:29 smithi080 ceph-mon[32324]: Detected new or changed devices on smithi080 2024-04-26T22:49:29.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:29.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:29.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:49:29.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:29 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi080 to 17088M 2024-04-26T22:49:29.773 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:29.773 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:29.773 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:29.773 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:29.773 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:29 smithi080 ceph-mon[32324]: osd.1 [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245] boot 2024-04-26T22:49:29.773 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:29 smithi080 ceph-mon[32324]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T22:49:29.773 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:29.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:29 smithi027 ceph-mon[28034]: purged_snaps scrub starts 2024-04-26T22:49:29.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:29 smithi027 ceph-mon[28034]: purged_snaps scrub ok 2024-04-26T22:49:29.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:29 smithi027 ceph-mon[28034]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T22:49:29.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:29.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:29 smithi027 ceph-mon[28034]: Detected new or changed devices on smithi080 2024-04-26T22:49:29.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:29.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:29.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:49:29.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:29 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi080 to 17088M 2024-04-26T22:49:29.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:29.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:29.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:29.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:29.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:29 smithi027 ceph-mon[28034]: osd.1 [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245] boot 2024-04-26T22:49:29.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:29 smithi027 ceph-mon[28034]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T22:49:29.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:49:30.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.741+0000 7f2f00ac7700 1 -- 172.21.15.94:0/754942270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2efc102390 msgr2=0x7f2efc1027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:49:30.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.741+0000 7f2f00ac7700 1 --2- 172.21.15.94:0/754942270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2efc102390 0x7f2efc1027b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009990 tx=0x7f2eec009ca0 comp rx=0 tx=0).stop 2024-04-26T22:49:30.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.742+0000 7f2f00ac7700 1 -- 172.21.15.94:0/754942270 shutdown_connections 2024-04-26T22:49:30.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.742+0000 7f2f00ac7700 1 --2- 172.21.15.94:0/754942270 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2efc103f30 0x7f2efc10a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:30.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.742+0000 7f2f00ac7700 1 --2- 172.21.15.94:0/754942270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2efc102390 0x7f2efc1027b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:30.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.742+0000 7f2f00ac7700 1 --2- 172.21.15.94:0/754942270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2efc103590 0x7f2efc1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:30.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.742+0000 7f2f00ac7700 1 -- 172.21.15.94:0/754942270 >> 172.21.15.94:0/754942270 conn(0x7f2efc0fd950 msgr2=0x7f2efc0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:49:30.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.742+0000 7f2f00ac7700 1 -- 172.21.15.94:0/754942270 shutdown_connections 2024-04-26T22:49:30.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.742+0000 7f2f00ac7700 1 -- 172.21.15.94:0/754942270 wait complete. 2024-04-26T22:49:30.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.742+0000 7f2f00ac7700 1 Processor -- start 2024-04-26T22:49:30.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2f00ac7700 1 -- start start 2024-04-26T22:49:30.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2f00ac7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2efc102390 0x7f2efc06c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:49:30.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2f00ac7700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2efc103590 0x7f2efc19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:49:30.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2f00ac7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2efc103f30 0x7f2efc1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:49:30.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2f00ac7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc10ded0 con 0x7f2efc102390 2024-04-26T22:49:30.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2f00ac7700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc10dd50 con 0x7f2efc103590 2024-04-26T22:49:30.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2f00ac7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc10e050 con 0x7f2efc103f30 2024-04-26T22:49:30.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2efad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2efc103f30 0x7f2efc1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:49:30.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2efad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2efc103f30 0x7f2efc1a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42150/0 (socket says 172.21.15.94:42150) 2024-04-26T22:49:30.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2efad9d700 1 -- 172.21.15.94:0/3400594600 learned_addr learned my addr 172.21.15.94:0/3400594600 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T22:49:30.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2ef9d9b700 1 --2- 172.21.15.94:0/3400594600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2efc103590 0x7f2efc19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:49:30.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2efa59c700 1 --2- 172.21.15.94:0/3400594600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2efc102390 0x7f2efc06c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:49:30.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2efad9d700 1 -- 172.21.15.94:0/3400594600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2efc103590 msgr2=0x7f2efc19e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:49:30.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2efad9d700 1 --2- 172.21.15.94:0/3400594600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2efc103590 0x7f2efc19e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:30.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2efad9d700 1 -- 172.21.15.94:0/3400594600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2efc102390 msgr2=0x7f2efc06c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:49:30.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2efad9d700 1 --2- 172.21.15.94:0/3400594600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2efc102390 0x7f2efc06c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:30.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.743+0000 7f2efad9d700 1 -- 172.21.15.94:0/3400594600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2efc1a12e0 con 0x7f2efc103f30 2024-04-26T22:49:30.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.744+0000 7f2efad9d700 1 --2- 172.21.15.94:0/3400594600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2efc103f30 0x7f2efc1a0a10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400b970 tx=0x7f2ef400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:49:30.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.744+0000 7f2eeb7fe700 1 -- 172.21.15.94:0/3400594600 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4019070 con 0x7f2efc103f30 2024-04-26T22:49:30.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.744+0000 7f2eeb7fe700 1 -- 172.21.15.94:0/3400594600 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef400cde0 con 0x7f2efc103f30 2024-04-26T22:49:30.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.744+0000 7f2eeb7fe700 1 -- 172.21.15.94:0/3400594600 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4007370 con 0x7f2efc103f30 2024-04-26T22:49:30.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.744+0000 7f2f00ac7700 1 -- 172.21.15.94:0/3400594600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2efc1a15d0 con 0x7f2efc103f30 2024-04-26T22:49:30.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.744+0000 7f2f00ac7700 1 -- 172.21.15.94:0/3400594600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2efc1a1ac0 con 0x7f2efc103f30 2024-04-26T22:49:30.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.745+0000 7f2f00ac7700 1 -- 172.21.15.94:0/3400594600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2efc06a110 con 0x7f2efc103f30 2024-04-26T22:49:30.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.746+0000 7f2eeb7fe700 1 -- 172.21.15.94:0/3400594600 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f2ef401c070 con 0x7f2efc103f30 2024-04-26T22:49:30.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.746+0000 7f2eeb7fe700 1 --2- 172.21.15.94:0/3400594600 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2ee40a3c30 0x7f2ee40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:49:30.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.747+0000 7f2eeb7fe700 1 -- 172.21.15.94:0/3400594600 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f2ef40cdfd0 con 0x7f2efc103f30 2024-04-26T22:49:30.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.747+0000 7f2efa59c700 1 --2- 172.21.15.94:0/3400594600 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2ee40a3c30 0x7f2ee40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:49:30.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.748+0000 7f2efa59c700 1 --2- 172.21.15.94:0/3400594600 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2ee40a3c30 0x7f2ee40a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00b5c0 tx=0x7f2eec003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:49:30.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.751+0000 7f2eeb7fe700 1 -- 172.21.15.94:0/3400594600 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2ef409c200 con 0x7f2efc103f30 2024-04-26T22:49:30.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:30.958+0000 7f2f00ac7700 1 -- 172.21.15.94:0/3400594600 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2efc063860 con 0x7f2ee40a3c30 2024-04-26T22:49:31.723 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:31 smithi094 ceph-mon[32889]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:31.723 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:31 smithi094 ceph-mon[32889]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T22:49:31.724 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:31 smithi094 ceph-mon[32889]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:49:31.724 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:31 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:49:31.724 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:31 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:49:31.724 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:31 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:31.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:31 smithi080 ceph-mon[32324]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:31.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:31 smithi080 ceph-mon[32324]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T22:49:31.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:31 smithi080 ceph-mon[32324]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:49:31.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:31 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:49:31.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:31 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:49:31.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:31 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:31.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:31 smithi027 ceph-mon[28034]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:31 smithi027 ceph-mon[28034]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T22:49:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:31 smithi027 ceph-mon[28034]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:49:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:31 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:49:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:31 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:49:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:31 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:33.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:33 smithi080 ceph-mon[32324]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:33.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:33 smithi094 ceph-mon[32889]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:33.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:33 smithi027 ceph-mon[28034]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:34.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:34 smithi080 ceph-mon[32324]: from='client.? 172.21.15.94:0/3501571402' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0"}]: dispatch 2024-04-26T22:49:34.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:34 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0"}]: dispatch 2024-04-26T22:49:34.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:34 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0"}]': finished 2024-04-26T22:49:34.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:34 smithi080 ceph-mon[32324]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T22:49:34.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:34 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:34.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:34 smithi080 ceph-mon[32324]: from='client.? 172.21.15.94:0/3778163772' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:49:34.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:34 smithi094 ceph-mon[32889]: from='client.? 172.21.15.94:0/3501571402' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0"}]: dispatch 2024-04-26T22:49:34.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:34 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0"}]: dispatch 2024-04-26T22:49:34.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:34 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0"}]': finished 2024-04-26T22:49:34.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:34 smithi094 ceph-mon[32889]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T22:49:34.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:34 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:34.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:34 smithi094 ceph-mon[32889]: from='client.? 172.21.15.94:0/3778163772' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:49:34.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:34 smithi027 ceph-mon[28034]: from='client.? 172.21.15.94:0/3501571402' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0"}]: dispatch 2024-04-26T22:49:34.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:34 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0"}]: dispatch 2024-04-26T22:49:34.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:34 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0"}]': finished 2024-04-26T22:49:34.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:34 smithi027 ceph-mon[28034]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T22:49:34.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:34 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:34.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:34 smithi027 ceph-mon[28034]: from='client.? 172.21.15.94:0/3778163772' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:49:36.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:35 smithi080 ceph-mon[32324]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:36.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:35 smithi094 ceph-mon[32889]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:36.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:35 smithi027 ceph-mon[28034]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:38.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:37 smithi080 ceph-mon[32324]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:38.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:37 smithi094 ceph-mon[32889]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:38.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:37 smithi027 ceph-mon[28034]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:39.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:39 smithi080 ceph-mon[32324]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:40.018 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:39 smithi094 ceph-mon[32889]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:40.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:39 smithi027 ceph-mon[28034]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:42.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:41 smithi080 ceph-mon[32324]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:42.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:41 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T22:49:42.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:41 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:42.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:41 smithi080 ceph-mon[32324]: Deploying daemon osd.2 on smithi094 2024-04-26T22:49:42.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:41 smithi094 ceph-mon[32889]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:42.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:41 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T22:49:42.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:41 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:42.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:41 smithi094 ceph-mon[32889]: Deploying daemon osd.2 on smithi094 2024-04-26T22:49:42.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:41 smithi027 ceph-mon[28034]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:42.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:41 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T22:49:42.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:41 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:42.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:41 smithi027 ceph-mon[28034]: Deploying daemon osd.2 on smithi094 2024-04-26T22:49:44.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:43 smithi080 ceph-mon[32324]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:44.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:43 smithi094 ceph-mon[32889]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:44.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:43 smithi027 ceph-mon[28034]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:45.998 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:45 smithi094 ceph-mon[32889]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:46.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:45 smithi080 ceph-mon[32324]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:46.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:45 smithi027 ceph-mon[28034]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:47.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:47 smithi027 ceph-mon[28034]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:47.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:47 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:47.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:47 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:47.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:47 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:48.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:47 smithi080 ceph-mon[32324]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:48.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:47 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:48.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:47 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:48.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:47 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:48.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:47 smithi094 ceph-mon[32889]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:48.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:47 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:48.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:47 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:48.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:47 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:49.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:49 smithi027 ceph-mon[28034]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:49.902 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:49 smithi094 ceph-mon[32889]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:49.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:49.996+0000 7f2eeb7fe700 1 -- 172.21.15.94:0/3400594600 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2efc063860 con 0x7f2ee40a3c30 2024-04-26T22:49:49.997 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 2 on host 'smithi094' 2024-04-26T22:49:49.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:49.998+0000 7f2f00ac7700 1 -- 172.21.15.94:0/3400594600 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2ee40a3c30 msgr2=0x7f2ee40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:49:49.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:49.998+0000 7f2f00ac7700 1 --2- 172.21.15.94:0/3400594600 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2ee40a3c30 0x7f2ee40a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00b5c0 tx=0x7f2eec003860 comp rx=0 tx=0).stop 2024-04-26T22:49:50.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:49.998+0000 7f2f00ac7700 1 -- 172.21.15.94:0/3400594600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2efc103f30 msgr2=0x7f2efc1a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:49:50.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:49.998+0000 7f2f00ac7700 1 --2- 172.21.15.94:0/3400594600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2efc103f30 0x7f2efc1a0a10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400b970 tx=0x7f2ef400bc80 comp rx=0 tx=0).stop 2024-04-26T22:49:50.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:49.998+0000 7f2f00ac7700 1 -- 172.21.15.94:0/3400594600 shutdown_connections 2024-04-26T22:49:50.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:49.998+0000 7f2f00ac7700 1 --2- 172.21.15.94:0/3400594600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2efc103f30 0x7f2efc1a0a10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:50.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:49.998+0000 7f2f00ac7700 1 --2- 172.21.15.94:0/3400594600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2efc102390 0x7f2efc06c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:50.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:49.998+0000 7f2f00ac7700 1 --2- 172.21.15.94:0/3400594600 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2ee40a3c30 0x7f2ee40a60f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:50.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:49.998+0000 7f2f00ac7700 1 --2- 172.21.15.94:0/3400594600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2efc103590 0x7f2efc19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:50.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:49.998+0000 7f2f00ac7700 1 -- 172.21.15.94:0/3400594600 >> 172.21.15.94:0/3400594600 conn(0x7f2efc0fd950 msgr2=0x7f2efc106930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:49:50.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:49.998+0000 7f2f00ac7700 1 -- 172.21.15.94:0/3400594600 shutdown_connections 2024-04-26T22:49:50.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:49:49.998+0000 7f2f00ac7700 1 -- 172.21.15.94:0/3400594600 wait complete. 2024-04-26T22:49:50.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:49 smithi080 ceph-mon[32324]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:50.653 DEBUG:teuthology.orchestra.run.smithi094:osd.2> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T22:49:50.655 INFO:tasks.cephadm:Deploying osd.3 on smithi027 with /dev/vg_nvme/lv_3... 2024-04-26T22:49:50.655 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T22:49:51.109 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:49:51.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:50 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:51.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:50 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:51.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:50 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:51.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:50 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:51.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:50 smithi027 ceph-mon[28034]: from='osd.2 [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T22:49:51.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:50 smithi027 ceph-mon[28034]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T22:49:51.261 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:50 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:51.261 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:50 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:51.261 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:50 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:51.261 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:50 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:51.261 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:50 smithi094 ceph-mon[32889]: from='osd.2 [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T22:49:51.261 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:50 smithi094 ceph-mon[32889]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T22:49:51.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:50 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:51.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:50 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:51.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:50 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:51.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:50 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:51.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:50 smithi080 ceph-mon[32324]: from='osd.2 [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T22:49:51.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:50 smithi080 ceph-mon[32324]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T22:49:52.228 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:51 smithi094 ceph-mon[32889]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:52.228 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:51 smithi094 ceph-mon[32889]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T22:49:52.228 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:51 smithi094 ceph-mon[32889]: from='osd.2 [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:49:52.229 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:51 smithi094 ceph-mon[32889]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T22:49:52.229 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:51 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:52.229 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:51 smithi094 ceph-mon[32889]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:49:52.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:51 smithi027 ceph-mon[28034]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:52.252 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:51 smithi027 ceph-mon[28034]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T22:49:52.252 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:51 smithi027 ceph-mon[28034]: from='osd.2 [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:49:52.252 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:51 smithi027 ceph-mon[28034]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T22:49:52.252 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:51 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:52.252 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:51 smithi027 ceph-mon[28034]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:49:52.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:51 smithi080 ceph-mon[32324]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:52.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:51 smithi080 ceph-mon[32324]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T22:49:52.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:51 smithi080 ceph-mon[32324]: from='osd.2 [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:49:52.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:51 smithi080 ceph-mon[32324]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T22:49:52.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:51 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:52.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:51 smithi080 ceph-mon[32324]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:49:53.035 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 22:49:52 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-2[38042]: 2024-04-26T22:49:52.713+0000 7fa0a3529700 -1 osd.2 0 waiting for initial osdmap 2024-04-26T22:49:53.035 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 22:49:52 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-2[38042]: 2024-04-26T22:49:52.717+0000 7fa09b314700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:49:53.488 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:53 smithi094 ceph-mon[32889]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T22:49:53.488 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:53 smithi094 ceph-mon[32889]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T22:49:53.488 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:53.488 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:53.488 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:53 smithi094 ceph-mon[32889]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:53.488 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:53 smithi094 ceph-mon[32889]: from='osd.2 ' entity='osd.2' 2024-04-26T22:49:53.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:53 smithi027 ceph-mon[28034]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T22:49:53.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:53 smithi027 ceph-mon[28034]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T22:49:53.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:53.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:53.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:53 smithi027 ceph-mon[28034]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:53.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:53 smithi027 ceph-mon[28034]: from='osd.2 ' entity='osd.2' 2024-04-26T22:49:53.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:53 smithi080 ceph-mon[32324]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T22:49:53.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:53 smithi080 ceph-mon[32324]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T22:49:53.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:53.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:53.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:53 smithi080 ceph-mon[32324]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T22:49:53.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:53 smithi080 ceph-mon[32324]: from='osd.2 ' entity='osd.2' 2024-04-26T22:49:54.211 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T22:49:54.211 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T22:49:54.211 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-04-26T22:49:54.211 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-04-26T22:49:54.211 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0336928 s, 311 MB/s 2024-04-26T22:49:54.211 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-04-26T22:49:54.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: purged_snaps scrub starts 2024-04-26T22:49:54.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: purged_snaps scrub ok 2024-04-26T22:49:54.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:54.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: osd.2 [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187] boot 2024-04-26T22:49:54.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T22:49:54.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:54.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: Detected new or changed devices on smithi094 2024-04-26T22:49:54.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:54.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:54.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:49:54.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi094 to 17088M 2024-04-26T22:49:54.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:54.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:54.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:54.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:54.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:54.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:54.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:54.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:54.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: purged_snaps scrub starts 2024-04-26T22:49:54.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: purged_snaps scrub ok 2024-04-26T22:49:54.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:54.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: osd.2 [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187] boot 2024-04-26T22:49:54.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T22:49:54.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:54.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: Detected new or changed devices on smithi094 2024-04-26T22:49:54.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:54.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:54.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:49:54.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi094 to 17088M 2024-04-26T22:49:54.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:54.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:54.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:54.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:54.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:54.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:54.773 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:54.773 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:54 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:54.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: purged_snaps scrub starts 2024-04-26T22:49:54.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: purged_snaps scrub ok 2024-04-26T22:49:54.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:54.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: osd.2 [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187] boot 2024-04-26T22:49:54.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T22:49:54.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:49:54.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: Detected new or changed devices on smithi094 2024-04-26T22:49:54.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:54.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:54.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:49:54.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi094 to 17088M 2024-04-26T22:49:54.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:54.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:54.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:54.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:54.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:54.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:54.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:54.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:54 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:55.079 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_3 2024-04-26T22:49:55.545 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:49:55.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:55 smithi027 ceph-mon[28034]: pgmap v93: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:49:55.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:55 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T22:49:55.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:55 smithi080 ceph-mon[32324]: pgmap v93: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:49:55.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:55 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T22:49:55.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:55 smithi094 ceph-mon[32889]: pgmap v93: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:49:55.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:55 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T22:49:56.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T22:49:56.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:56 smithi027 ceph-mon[28034]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T22:49:56.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T22:49:56.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T22:49:56.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:56 smithi080 ceph-mon[32324]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T22:49:56.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T22:49:56.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T22:49:56.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:56 smithi094 ceph-mon[32889]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T22:49:56.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T22:49:57.234 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 22:49:57 smithi027 sudo[46903]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T22:49:57.234 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 22:49:57 smithi027 sudo[46903]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T22:49:57.234 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 22:49:57 smithi027 sudo[46903]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T22:49:57.234 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 22:49:57 smithi027 sudo[46903]: pam_unix(sudo:session): session closed for user root 2024-04-26T22:49:57.502 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 22:49:57 smithi027 sudo[46918]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T22:49:57.502 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 22:49:57 smithi027 sudo[46918]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T22:49:57.502 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 22:49:57 smithi027 sudo[46918]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T22:49:57.502 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 22:49:57 smithi027 sudo[46918]: pam_unix(sudo:session): session closed for user root 2024-04-26T22:49:57.502 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T22:49:57.502 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:57 smithi027 ceph-mon[28034]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T22:49:57.502 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:57 smithi027 ceph-mon[28034]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:49:57.503 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:57.604 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 22:49:57 smithi094 sudo[41807]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T22:49:57.605 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 22:49:57 smithi094 sudo[41807]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T22:49:57.605 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 22:49:57 smithi094 sudo[41807]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T22:49:57.605 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 22:49:57 smithi094 sudo[41807]: pam_unix(sudo:session): session closed for user root 2024-04-26T22:49:57.605 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 22:49:57 smithi094 sudo[41810]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T22:49:57.605 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 22:49:57 smithi094 sudo[41810]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T22:49:57.605 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 22:49:57 smithi094 sudo[41810]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T22:49:57.605 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 22:49:57 smithi094 sudo[41810]: pam_unix(sudo:session): session closed for user root 2024-04-26T22:49:57.605 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:57 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T22:49:57.605 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:57 smithi094 ceph-mon[32889]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T22:49:57.605 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:57 smithi094 ceph-mon[32889]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:49:57.605 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:57 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:57.605 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:57 smithi094 sudo[41813]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T22:49:57.606 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:57 smithi094 sudo[41813]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T22:49:57.606 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:57 smithi094 sudo[41813]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T22:49:57.606 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:57 smithi094 sudo[41813]: pam_unix(sudo:session): session closed for user root 2024-04-26T22:49:57.630 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:57 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T22:49:57.630 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:57 smithi080 ceph-mon[32324]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T22:49:57.630 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:57 smithi080 ceph-mon[32324]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:49:57.630 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:57 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:57.631 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 22:49:57 smithi080 sudo[41072]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T22:49:57.631 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 22:49:57 smithi080 sudo[41072]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T22:49:57.631 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 22:49:57 smithi080 sudo[41072]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T22:49:57.631 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 22:49:57 smithi080 sudo[41072]: pam_unix(sudo:session): session closed for user root 2024-04-26T22:49:57.631 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 22:49:57 smithi080 sudo[41075]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T22:49:57.631 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 22:49:57 smithi080 sudo[41075]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T22:49:57.631 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 22:49:57 smithi080 sudo[41075]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T22:49:57.631 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 22:49:57 smithi080 sudo[41075]: pam_unix(sudo:session): session closed for user root 2024-04-26T22:49:57.772 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:57 smithi027 sudo[46922]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T22:49:57.772 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:57 smithi027 sudo[46922]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T22:49:57.772 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:57 smithi027 sudo[46922]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T22:49:57.772 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:57 smithi027 sudo[46922]: pam_unix(sudo:session): session closed for user root 2024-04-26T22:49:57.772 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:57 smithi027 sudo[46928]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T22:49:57.772 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:57 smithi027 sudo[46928]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T22:49:57.772 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:57 smithi027 sudo[46928]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T22:49:57.772 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:57 smithi027 sudo[46928]: pam_unix(sudo:session): session closed for user root 2024-04-26T22:49:57.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.834+0000 7f4a4a000700 1 -- 172.21.15.27:0/663991398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44077410 msgr2=0x7f4a44075870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:49:57.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.834+0000 7f4a4a000700 1 --2- 172.21.15.27:0/663991398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44077410 0x7f4a44075870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4a34009990 tx=0x7f4a34009ca0 comp rx=0 tx=0).stop 2024-04-26T22:49:57.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.834+0000 7f4a4a000700 1 -- 172.21.15.27:0/663991398 shutdown_connections 2024-04-26T22:49:57.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.834+0000 7f4a4a000700 1 --2- 172.21.15.27:0/663991398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a4410b390 0x7f4a4410d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:57.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.834+0000 7f4a4a000700 1 --2- 172.21.15.27:0/663991398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44077410 0x7f4a44075870 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:57.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.834+0000 7f4a4a000700 1 --2- 172.21.15.27:0/663991398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a44075e40 0x7f4a440762a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:57.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.834+0000 7f4a4a000700 1 -- 172.21.15.27:0/663991398 >> 172.21.15.27:0/663991398 conn(0x7f4a440fea70 msgr2=0x7f4a44100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:49:57.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.834+0000 7f4a4a000700 1 -- 172.21.15.27:0/663991398 shutdown_connections 2024-04-26T22:49:57.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.834+0000 7f4a4a000700 1 -- 172.21.15.27:0/663991398 wait complete. 2024-04-26T22:49:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.835+0000 7f4a4a000700 1 Processor -- start 2024-04-26T22:49:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.836+0000 7f4a4a000700 1 -- start start 2024-04-26T22:49:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.836+0000 7f4a4a000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44075e40 0x7f4a4419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:49:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.836+0000 7f4a4a000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a44077410 0x7f4a4419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:49:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.836+0000 7f4a4a000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a4410b390 0x7f4a441a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:49:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.836+0000 7f4a4a000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4a44111180 con 0x7f4a44075e40 2024-04-26T22:49:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.836+0000 7f4a4a000700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4a44111000 con 0x7f4a44077410 2024-04-26T22:49:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.836+0000 7f4a4a000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a44111300 con 0x7f4a4410b390 2024-04-26T22:49:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.836+0000 7f4a43400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44075e40 0x7f4a4419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:49:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.836+0000 7f4a43400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44075e40 0x7f4a4419b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32782/0 (socket says 172.21.15.27:32782) 2024-04-26T22:49:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.836+0000 7f4a43400700 1 -- 172.21.15.27:0/998397835 learned_addr learned my addr 172.21.15.27:0/998397835 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:49:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.836+0000 7f4a42a00700 1 --2- 172.21.15.27:0/998397835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a44077410 0x7f4a4419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:49:57.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.836+0000 7f4a43e00700 1 --2- 172.21.15.27:0/998397835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a4410b390 0x7f4a441a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:49:57.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.837+0000 7f4a43400700 1 -- 172.21.15.27:0/998397835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a4410b390 msgr2=0x7f4a441a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:49:57.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.837+0000 7f4a43400700 1 --2- 172.21.15.27:0/998397835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a4410b390 0x7f4a441a2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:57.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.837+0000 7f4a43400700 1 -- 172.21.15.27:0/998397835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a44077410 msgr2=0x7f4a4419bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:49:57.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.837+0000 7f4a43400700 1 --2- 172.21.15.27:0/998397835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a44077410 0x7f4a4419bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:49:57.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.837+0000 7f4a43400700 1 -- 172.21.15.27:0/998397835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a441a3470 con 0x7f4a44075e40 2024-04-26T22:49:57.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.837+0000 7f4a42a00700 1 --2- 172.21.15.27:0/998397835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a44077410 0x7f4a4419bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:49:57.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.837+0000 7f4a43400700 1 --2- 172.21.15.27:0/998397835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44075e40 0x7f4a4419b810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4a340047e0 tx=0x7f4a34009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:49:57.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.837+0000 7f4a3be00700 1 -- 172.21.15.27:0/998397835 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34026070 con 0x7f4a44075e40 2024-04-26T22:49:57.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.837+0000 7f4a3be00700 1 -- 172.21.15.27:0/998397835 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a34021410 con 0x7f4a44075e40 2024-04-26T22:49:57.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.837+0000 7f4a4a000700 1 -- 172.21.15.27:0/998397835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a441a3700 con 0x7f4a44075e40 2024-04-26T22:49:57.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.837+0000 7f4a3be00700 1 -- 172.21.15.27:0/998397835 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34017650 con 0x7f4a44075e40 2024-04-26T22:49:57.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.838+0000 7f4a4a000700 1 -- 172.21.15.27:0/998397835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a441a3bb0 con 0x7f4a44075e40 2024-04-26T22:49:57.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.839+0000 7f4a4a000700 1 -- 172.21.15.27:0/998397835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a4406a110 con 0x7f4a44075e40 2024-04-26T22:49:57.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.842+0000 7f4a3be00700 1 -- 172.21.15.27:0/998397835 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f4a34029070 con 0x7f4a44075e40 2024-04-26T22:49:57.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.843+0000 7f4a3be00700 1 --2- 172.21.15.27:0/998397835 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f4a280a3c30 0x7f4a280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:49:57.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.843+0000 7f4a3be00700 1 -- 172.21.15.27:0/998397835 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f4a340d04a0 con 0x7f4a44075e40 2024-04-26T22:49:57.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.843+0000 7f4a42a00700 1 --2- 172.21.15.27:0/998397835 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f4a280a3c30 0x7f4a280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:49:57.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.844+0000 7f4a42a00700 1 --2- 172.21.15.27:0/998397835 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f4a280a3c30 0x7f4a280a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c011490 tx=0x7f4a2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:49:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:57.847+0000 7f4a3be00700 1 -- 172.21.15.27:0/998397835 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4a34030070 con 0x7f4a44075e40 2024-04-26T22:49:58.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:57 smithi080 sudo[41078]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T22:49:58.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:57 smithi080 sudo[41078]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T22:49:58.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:57 smithi080 sudo[41078]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T22:49:58.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:57 smithi080 sudo[41078]: pam_unix(sudo:session): session closed for user root 2024-04-26T22:49:58.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:57 smithi080 sudo[41081]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T22:49:58.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:57 smithi080 sudo[41081]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T22:49:58.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:57 smithi080 sudo[41081]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T22:49:58.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:57 smithi080 sudo[41081]: pam_unix(sudo:session): session closed for user root 2024-04-26T22:49:58.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:57 smithi094 sudo[41816]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T22:49:58.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:57 smithi094 sudo[41816]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T22:49:58.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:57 smithi094 sudo[41816]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T22:49:58.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:57 smithi094 sudo[41816]: pam_unix(sudo:session): session closed for user root 2024-04-26T22:49:58.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:58.090+0000 7f4a4a000700 1 -- 172.21.15.27:0/998397835 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4a44063860 con 0x7f4a280a3c30 2024-04-26T22:49:58.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T22:49:58.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T22:49:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T22:49:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:49:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:49:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:49:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T22:49:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:49:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:49:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:49:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T22:49:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T22:49:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:49:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:49:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:49:58.617 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T22:49:58.617 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:49:58.617 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:49:58.617 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:49:58.617 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:58.695 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T22:49:58.695 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:58.695 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:58.695 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T22:49:58.695 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T22:49:58.695 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:49:58.695 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:49:58.695 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:49:58.695 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T22:49:58.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:49:58.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:49:58.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:49:58.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T22:49:58.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T22:49:58.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:49:58.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:49:58.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:49:58.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T22:49:58.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:49:58.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:49:58.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:49:58.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:58.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T22:49:58.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:58.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:49:58.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T22:49:58.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T22:49:58.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:49:58.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:49:58.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:49:58.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T22:49:58.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:49:58.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:49:58.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:49:58.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T22:49:58.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T22:49:58.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:49:58.773 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:49:58.773 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:49:58.773 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T22:49:58.773 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:49:58.773 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:49:58.773 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:49:58.773 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:59.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:49:59.383+0000 7f4a3be00700 1 -- 172.21.15.27:0/998397835 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4a34009d10 con 0x7f4a44075e40 2024-04-26T22:49:59.660 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:59 smithi027 ceph-mon[28034]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:49:59.660 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:49:59 smithi027 ceph-mon[28034]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T22:49:59.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:59 smithi080 ceph-mon[32324]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:49:59.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:49:59 smithi080 ceph-mon[32324]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T22:49:59.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:59 smithi094 ceph-mon[32889]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:49:59.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:49:59 smithi094 ceph-mon[32889]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T22:50:00.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:00 smithi080 ceph-mon[32324]: overall HEALTH_OK 2024-04-26T22:50:00.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:00 smithi094 ceph-mon[32889]: overall HEALTH_OK 2024-04-26T22:50:00.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:00 smithi027 ceph-mon[28034]: overall HEALTH_OK 2024-04-26T22:50:01.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:01 smithi080 ceph-mon[32324]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:01.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:01 smithi094 ceph-mon[32889]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:01.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:01 smithi027 ceph-mon[28034]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:03.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:03 smithi080 ceph-mon[32324]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:03.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:03 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3126762236' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95f248ab-e3e8-425c-a0f8-c26cfc6f1be8"}]: dispatch 2024-04-26T22:50:03.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:03 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3126762236' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95f248ab-e3e8-425c-a0f8-c26cfc6f1be8"}]': finished 2024-04-26T22:50:03.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:03 smithi080 ceph-mon[32324]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T22:50:03.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:03 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:50:03.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:03 smithi094 ceph-mon[32889]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:03.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:03 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3126762236' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95f248ab-e3e8-425c-a0f8-c26cfc6f1be8"}]: dispatch 2024-04-26T22:50:03.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:03 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3126762236' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95f248ab-e3e8-425c-a0f8-c26cfc6f1be8"}]': finished 2024-04-26T22:50:03.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:03 smithi094 ceph-mon[32889]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T22:50:03.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:03 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:50:03.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:03 smithi027 ceph-mon[28034]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:03.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:03 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3126762236' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95f248ab-e3e8-425c-a0f8-c26cfc6f1be8"}]: dispatch 2024-04-26T22:50:03.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:03 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3126762236' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95f248ab-e3e8-425c-a0f8-c26cfc6f1be8"}]': finished 2024-04-26T22:50:03.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:03 smithi027 ceph-mon[28034]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T22:50:03.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:03 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:50:04.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:04 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1688663076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:50:04.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:04 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1688663076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:50:04.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:04 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1688663076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:50:05.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:05 smithi080 ceph-mon[32324]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:05.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:05 smithi094 ceph-mon[32889]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:05.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:05 smithi027 ceph-mon[28034]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:07.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:07 smithi080 ceph-mon[32324]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:07.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:07 smithi094 ceph-mon[32889]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:07.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:07 smithi027 ceph-mon[28034]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:09.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:09 smithi080 ceph-mon[32324]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:09.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:09 smithi094 ceph-mon[32889]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:09.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:09 smithi027 ceph-mon[28034]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:11.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:11 smithi080 ceph-mon[32324]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:11.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:11 smithi094 ceph-mon[32889]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:11.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:11 smithi027 ceph-mon[28034]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:13.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:13 smithi080 ceph-mon[32324]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:13.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:13 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T22:50:13.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:13 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:13.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:13 smithi080 ceph-mon[32324]: Deploying daemon osd.3 on smithi027 2024-04-26T22:50:13.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:13 smithi094 ceph-mon[32889]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:13.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:13 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T22:50:13.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:13 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:13.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:13 smithi094 ceph-mon[32889]: Deploying daemon osd.3 on smithi027 2024-04-26T22:50:13.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:13 smithi027 ceph-mon[28034]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:13.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:13 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T22:50:13.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:13 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:13.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:13 smithi027 ceph-mon[28034]: Deploying daemon osd.3 on smithi027 2024-04-26T22:50:15.595 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:15 smithi027 ceph-mon[28034]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:15.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:15 smithi094 ceph-mon[32889]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:16.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:15 smithi080 ceph-mon[32324]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:17.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:17 smithi027 ceph-mon[28034]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:17.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:17 smithi094 ceph-mon[32889]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:18.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:17 smithi080 ceph-mon[32324]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:19.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:19 smithi094 ceph-mon[32889]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:19.795 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:19 smithi027 ceph-mon[28034]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:20.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:19 smithi080 ceph-mon[32324]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:21.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:21 smithi027 ceph-mon[28034]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:21 smithi080 ceph-mon[32324]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:22.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:21 smithi094 ceph-mon[32889]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:22.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:22.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:22.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:50:23.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:23.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:23.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:50:23.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:23.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:23.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:50:23.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:23 smithi027 ceph-mon[28034]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:24.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:23 smithi080 ceph-mon[32324]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:24.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:23 smithi094 ceph-mon[32889]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:26.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:25 smithi080 ceph-mon[32324]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:26.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:25 smithi080 ceph-mon[32324]: from='osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T22:50:26.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:25 smithi094 ceph-mon[32889]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:26.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:25 smithi094 ceph-mon[32889]: from='osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T22:50:26.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:25 smithi027 ceph-mon[28034]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:26.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:25 smithi027 ceph-mon[28034]: from='osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T22:50:26.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:26 smithi027 ceph-mon[28034]: from='osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T22:50:26.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:26 smithi027 ceph-mon[28034]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T22:50:26.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:26 smithi027 ceph-mon[28034]: from='osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T22:50:26.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:50:26.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:26.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:26.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:26 smithi027 ceph-mon[28034]: from='osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T22:50:26.999 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 3 on host 'smithi027' 2024-04-26T22:50:27.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:50:26.997+0000 7f4a3be00700 1 -- 172.21.15.27:0/998397835 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4a44063860 con 0x7f4a280a3c30 2024-04-26T22:50:27.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:50:27.001+0000 7f4a4a000700 1 -- 172.21.15.27:0/998397835 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f4a280a3c30 msgr2=0x7f4a280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:50:27.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:50:27.001+0000 7f4a4a000700 1 --2- 172.21.15.27:0/998397835 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f4a280a3c30 0x7f4a280a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c011490 tx=0x7f4a2c009260 comp rx=0 tx=0).stop 2024-04-26T22:50:27.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:50:27.001+0000 7f4a4a000700 1 -- 172.21.15.27:0/998397835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44075e40 msgr2=0x7f4a4419b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:50:27.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:50:27.001+0000 7f4a4a000700 1 --2- 172.21.15.27:0/998397835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44075e40 0x7f4a4419b810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4a340047e0 tx=0x7f4a34009510 comp rx=0 tx=0).stop 2024-04-26T22:50:27.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:50:27.002+0000 7f4a4a000700 1 -- 172.21.15.27:0/998397835 shutdown_connections 2024-04-26T22:50:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:50:27.002+0000 7f4a4a000700 1 --2- 172.21.15.27:0/998397835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a4410b390 0x7f4a441a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:50:27.002+0000 7f4a4a000700 1 --2- 172.21.15.27:0/998397835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44075e40 0x7f4a4419b810 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:50:27.002+0000 7f4a4a000700 1 --2- 172.21.15.27:0/998397835 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f4a280a3c30 0x7f4a280a60f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:50:27.002+0000 7f4a4a000700 1 --2- 172.21.15.27:0/998397835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a44077410 0x7f4a4419bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:50:27.002+0000 7f4a4a000700 1 -- 172.21.15.27:0/998397835 >> 172.21.15.27:0/998397835 conn(0x7f4a440fea70 msgr2=0x7f4a441072e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:50:27.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:50:27.002+0000 7f4a4a000700 1 -- 172.21.15.27:0/998397835 shutdown_connections 2024-04-26T22:50:27.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:50:27.002+0000 7f4a4a000700 1 -- 172.21.15.27:0/998397835 wait complete. 2024-04-26T22:50:27.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:26 smithi080 ceph-mon[32324]: from='osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T22:50:27.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:26 smithi080 ceph-mon[32324]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T22:50:27.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:26 smithi080 ceph-mon[32324]: from='osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T22:50:27.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:50:27.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:27.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:27.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:26 smithi080 ceph-mon[32324]: from='osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T22:50:27.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:26 smithi094 ceph-mon[32889]: from='osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T22:50:27.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:26 smithi094 ceph-mon[32889]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T22:50:27.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:26 smithi094 ceph-mon[32889]: from='osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T22:50:27.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:50:27.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:27.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:27.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:26 smithi094 ceph-mon[32889]: from='osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T22:50:28.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:27 smithi080 ceph-mon[32324]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:28.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:27 smithi080 ceph-mon[32324]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T22:50:28.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:50:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:50:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:27 smithi080 ceph-mon[32324]: from='osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849]' entity='osd.3' 2024-04-26T22:50:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:27 smithi080 ceph-mon[32324]: osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849] boot 2024-04-26T22:50:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:27 smithi080 ceph-mon[32324]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T22:50:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:50:28.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:27 smithi094 ceph-mon[32889]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:28.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:27 smithi094 ceph-mon[32889]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T22:50:28.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:50:28.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:50:28.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:28.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:28.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:27 smithi094 ceph-mon[32889]: from='osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849]' entity='osd.3' 2024-04-26T22:50:28.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:27 smithi094 ceph-mon[32889]: osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849] boot 2024-04-26T22:50:28.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:27 smithi094 ceph-mon[32889]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T22:50:28.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:50:28.107 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:27 smithi027 ceph-mon[28034]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T22:50:28.107 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:27 smithi027 ceph-mon[28034]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T22:50:28.107 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:50:28.107 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:50:28.107 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:28.107 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:28.107 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:27 smithi027 ceph-mon[28034]: from='osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849]' entity='osd.3' 2024-04-26T22:50:28.108 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:27 smithi027 ceph-mon[28034]: osd.3 [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849] boot 2024-04-26T22:50:28.108 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:27 smithi027 ceph-mon[28034]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T22:50:28.108 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:50:28.174 DEBUG:teuthology.orchestra.run.smithi027:osd.3> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T22:50:28.179 INFO:tasks.cephadm:Deploying osd.4 on smithi080 with /dev/vg_nvme/lv_3... 2024-04-26T22:50:28.179 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T22:50:28.419 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T22:50:28.857 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:28 smithi080 ceph-mon[32324]: purged_snaps scrub starts 2024-04-26T22:50:28.857 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:28 smithi080 ceph-mon[32324]: purged_snaps scrub ok 2024-04-26T22:50:28.857 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:28 smithi080 ceph-mon[32324]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T22:50:29.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:28 smithi094 ceph-mon[32889]: purged_snaps scrub starts 2024-04-26T22:50:29.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:28 smithi094 ceph-mon[32889]: purged_snaps scrub ok 2024-04-26T22:50:29.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:28 smithi094 ceph-mon[32889]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T22:50:29.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:28 smithi027 ceph-mon[28034]: purged_snaps scrub starts 2024-04-26T22:50:29.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:28 smithi027 ceph-mon[28034]: purged_snaps scrub ok 2024-04-26T22:50:29.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:28 smithi027 ceph-mon[28034]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T22:50:29.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:29 smithi080 ceph-mon[32324]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:29.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:29 smithi080 ceph-mon[32324]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T22:50:30.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:29 smithi094 ceph-mon[32889]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:30.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:29 smithi094 ceph-mon[32889]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T22:50:30.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:29 smithi027 ceph-mon[28034]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:30.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:29 smithi027 ceph-mon[28034]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T22:50:30.373 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T22:50:30.374 INFO:teuthology.orchestra.run.smithi080.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T22:50:30.374 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10+0 records in 2024-04-26T22:50:30.374 INFO:teuthology.orchestra.run.smithi080.stderr:10+0 records out 2024-04-26T22:50:30.374 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115661 s, 907 MB/s 2024-04-26T22:50:30.374 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping successful for: 2024-04-26T22:50:30.894 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi080:vg_nvme/lv_3 2024-04-26T22:50:31.134 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T22:50:31.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:31 smithi080 ceph-mon[32324]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:31.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:31 smithi080 ceph-mon[32324]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T22:50:32.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:31 smithi094 ceph-mon[32889]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:32.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:31 smithi094 ceph-mon[32889]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T22:50:32.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:31 smithi027 ceph-mon[28034]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:32.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:31 smithi027 ceph-mon[28034]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T22:50:32.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.713+0000 7f18a09fb700 1 -- 172.21.15.80:0/653361537 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f189c103f40 msgr2=0x7f189c10a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:50:32.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.713+0000 7f18a09fb700 1 --2- 172.21.15.80:0/653361537 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f189c103f40 0x7f189c10a7f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f189400b0a0 tx=0x7f189400b3b0 comp rx=0 tx=0).stop 2024-04-26T22:50:32.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.713+0000 7f18a09fb700 1 -- 172.21.15.80:0/653361537 shutdown_connections 2024-04-26T22:50:32.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.713+0000 7f18a09fb700 1 --2- 172.21.15.80:0/653361537 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f189c103f40 0x7f189c10a7f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:32.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.713+0000 7f18a09fb700 1 --2- 172.21.15.80:0/653361537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f189c1023a0 0x7f189c1027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:32.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.713+0000 7f18a09fb700 1 --2- 172.21.15.80:0/653361537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c1035a0 0x7f189c103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:32.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.713+0000 7f18a09fb700 1 -- 172.21.15.80:0/653361537 >> 172.21.15.80:0/653361537 conn(0x7f189c0fd940 msgr2=0x7f189c0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:50:32.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.713+0000 7f18a09fb700 1 -- 172.21.15.80:0/653361537 shutdown_connections 2024-04-26T22:50:32.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.713+0000 7f18a09fb700 1 -- 172.21.15.80:0/653361537 wait complete. 2024-04-26T22:50:32.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.713+0000 7f18a09fb700 1 Processor -- start 2024-04-26T22:50:32.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f18a09fb700 1 -- start start 2024-04-26T22:50:32.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f18a09fb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f189c1023a0 0x7f189c10cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:50:32.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f18a09fb700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f189c1035a0 0x7f189c10b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:50:32.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f18a09fb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c103f40 0x7f189c10b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:50:32.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f18a09fb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f189c10f020 con 0x7f189c103f40 2024-04-26T22:50:32.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f18a09fb700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f189c10eea0 con 0x7f189c1035a0 2024-04-26T22:50:32.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f18a09fb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f189c10f1a0 con 0x7f189c1023a0 2024-04-26T22:50:32.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f1899d9b700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f189c1035a0 0x7f189c10b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:50:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f1899d9b700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f189c1035a0 0x7f189c10b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59994/0 (socket says 172.21.15.80:59994) 2024-04-26T22:50:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f1899d9b700 1 -- 172.21.15.80:0/2354184569 learned_addr learned my addr 172.21.15.80:0/2354184569 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-26T22:50:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f189a59c700 1 --2- 172.21.15.80:0/2354184569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f189c1023a0 0x7f189c10cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:50:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f1899d9b700 1 -- 172.21.15.80:0/2354184569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f189c1023a0 msgr2=0x7f189c10cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:50:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f1899d9b700 1 --2- 172.21.15.80:0/2354184569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f189c1023a0 0x7f189c10cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f1899d9b700 1 -- 172.21.15.80:0/2354184569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c103f40 msgr2=0x7f189c10b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:50:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f1899d9b700 1 --2- 172.21.15.80:0/2354184569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c103f40 0x7f189c10b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f1899d9b700 1 -- 172.21.15.80:0/2354184569 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c10bfe0 con 0x7f189c1035a0 2024-04-26T22:50:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f1899d9b700 1 --2- 172.21.15.80:0/2354184569 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f189c1035a0 0x7f189c10b1f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f189000b530 tx=0x7f189000b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:50:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f188b7fe700 1 -- 172.21.15.80:0/2354184569 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890015020 con 0x7f189c1035a0 2024-04-26T22:50:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f188b7fe700 1 -- 172.21.15.80:0/2354184569 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1890011ce0 con 0x7f189c1035a0 2024-04-26T22:50:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f188b7fe700 1 -- 172.21.15.80:0/2354184569 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890010d20 con 0x7f189c1035a0 2024-04-26T22:50:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f18a09fb700 1 -- 172.21.15.80:0/2354184569 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189c1a3830 con 0x7f189c1035a0 2024-04-26T22:50:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.714+0000 7f18a09fb700 1 -- 172.21.15.80:0/2354184569 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189c1a3dc0 con 0x7f189c1035a0 2024-04-26T22:50:32.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.715+0000 7f18a09fb700 1 -- 172.21.15.80:0/2354184569 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f189c06a110 con 0x7f189c1035a0 2024-04-26T22:50:32.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.716+0000 7f188b7fe700 1 -- 172.21.15.80:0/2354184569 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1890018020 con 0x7f189c1035a0 2024-04-26T22:50:32.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.717+0000 7f188b7fe700 1 --2- 172.21.15.80:0/2354184569 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f18840a3c40 0x7f18840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:50:32.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.717+0000 7f188b7fe700 1 -- 172.21.15.80:0/2354184569 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f18900cea20 con 0x7f189c1035a0 2024-04-26T22:50:32.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.718+0000 7f189a59c700 1 --2- 172.21.15.80:0/2354184569 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f18840a3c40 0x7f18840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:50:32.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.718+0000 7f189a59c700 1 --2- 172.21.15.80:0/2354184569 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f18840a3c40 0x7f18840a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f188c006fd0 tx=0x7f188c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:50:32.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.722+0000 7f188b7fe700 1 -- 172.21.15.80:0/2354184569 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f189001f020 con 0x7f189c1035a0 2024-04-26T22:50:32.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:32.932+0000 7f18a09fb700 1 -- 172.21.15.80:0/2354184569 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f189c063860 con 0x7f18840a3c40 2024-04-26T22:50:32.932 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:32 smithi080 ceph-mon[32324]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T22:50:33.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:32 smithi094 ceph-mon[32889]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T22:50:33.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:32 smithi027 ceph-mon[28034]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T22:50:33.897 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:33 smithi080 ceph-mon[32324]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:33.897 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:33 smithi080 ceph-mon[32324]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:50:33.897 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:50:33.897 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:50:33.897 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:33.897 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:33.897 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:33.897 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:50:33.897 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:50:33.897 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:33.897 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:33.898 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:50:33.898 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:34.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:33 smithi094 ceph-mon[32889]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:34.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:33 smithi094 ceph-mon[32889]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:50:34.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:50:34.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:50:34.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:34.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:34.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:34.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:50:34.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:50:34.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:34.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:34.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:50:34.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:34.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:33 smithi027 ceph-mon[28034]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:34.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:33 smithi027 ceph-mon[28034]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:50:34.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:50:34.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:50:34.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:34.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:34.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:34.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:50:34.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:50:34.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:34.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:34.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:50:34.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:35.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:34 smithi080 ceph-mon[32324]: Detected new or changed devices on smithi027 2024-04-26T22:50:35.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:34 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi027 to 8544M 2024-04-26T22:50:35.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:34 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:50:35.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:34 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:35.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:34 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:50:35.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:34 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:35.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:34 smithi094 ceph-mon[32889]: Detected new or changed devices on smithi027 2024-04-26T22:50:35.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:34 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi027 to 8544M 2024-04-26T22:50:35.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:34 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:50:35.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:34 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:35.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:34 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:50:35.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:34 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:35.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:34 smithi027 ceph-mon[28034]: Detected new or changed devices on smithi027 2024-04-26T22:50:35.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:34 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi027 to 8544M 2024-04-26T22:50:35.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:34 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:50:35.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:34 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:35.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:34 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:50:35.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:34 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:36.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:35 smithi080 ceph-mon[32324]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:36.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:35 smithi080 ceph-mon[32324]: from='client.? 172.21.15.80:0/3775888509' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d4f0826-5221-41cc-a658-fc518acf4c98"}]: dispatch 2024-04-26T22:50:36.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:35 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d4f0826-5221-41cc-a658-fc518acf4c98"}]: dispatch 2024-04-26T22:50:36.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:35 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d4f0826-5221-41cc-a658-fc518acf4c98"}]': finished 2024-04-26T22:50:36.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:35 smithi080 ceph-mon[32324]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T22:50:36.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:50:36.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:35 smithi094 ceph-mon[32889]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:36.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:35 smithi094 ceph-mon[32889]: from='client.? 172.21.15.80:0/3775888509' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d4f0826-5221-41cc-a658-fc518acf4c98"}]: dispatch 2024-04-26T22:50:36.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:35 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d4f0826-5221-41cc-a658-fc518acf4c98"}]: dispatch 2024-04-26T22:50:36.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:35 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d4f0826-5221-41cc-a658-fc518acf4c98"}]': finished 2024-04-26T22:50:36.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:35 smithi094 ceph-mon[32889]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T22:50:36.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:50:36.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:35 smithi027 ceph-mon[28034]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:36.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:35 smithi027 ceph-mon[28034]: from='client.? 172.21.15.80:0/3775888509' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d4f0826-5221-41cc-a658-fc518acf4c98"}]: dispatch 2024-04-26T22:50:36.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:35 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d4f0826-5221-41cc-a658-fc518acf4c98"}]: dispatch 2024-04-26T22:50:36.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:35 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d4f0826-5221-41cc-a658-fc518acf4c98"}]': finished 2024-04-26T22:50:36.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:35 smithi027 ceph-mon[28034]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T22:50:36.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:50:37.266 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:36 smithi027 ceph-mon[28034]: from='client.? 172.21.15.80:0/4071515756' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:50:37.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:36 smithi080 ceph-mon[32324]: from='client.? 172.21.15.80:0/4071515756' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:50:37.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:36 smithi094 ceph-mon[32889]: from='client.? 172.21.15.80:0/4071515756' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:50:38.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:37 smithi027 ceph-mon[28034]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T22:50:38.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:37 smithi080 ceph-mon[32324]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T22:50:38.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:37 smithi094 ceph-mon[32889]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T22:50:40.000 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:39 smithi080 ceph-mon[32324]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T22:50:40.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:39 smithi094 ceph-mon[32889]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T22:50:40.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:39 smithi027 ceph-mon[28034]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T22:50:42.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:41 smithi080 ceph-mon[32324]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-26T22:50:42.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:41 smithi094 ceph-mon[32889]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-26T22:50:42.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:41 smithi027 ceph-mon[28034]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-26T22:50:44.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:43 smithi080 ceph-mon[32324]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T22:50:44.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:43 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T22:50:44.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:43 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:44.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:43 smithi080 ceph-mon[32324]: Deploying daemon osd.4 on smithi080 2024-04-26T22:50:44.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:43 smithi094 ceph-mon[32889]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T22:50:44.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:43 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T22:50:44.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:43 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:44.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:43 smithi094 ceph-mon[32889]: Deploying daemon osd.4 on smithi080 2024-04-26T22:50:44.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:43 smithi027 ceph-mon[28034]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T22:50:44.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:43 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T22:50:44.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:43 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:44.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:43 smithi027 ceph-mon[28034]: Deploying daemon osd.4 on smithi080 2024-04-26T22:50:46.259 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:45 smithi080 ceph-mon[32324]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T22:50:46.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:45 smithi094 ceph-mon[32889]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T22:50:46.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:45 smithi027 ceph-mon[28034]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T22:50:48.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:47 smithi080 ceph-mon[32324]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:48.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:47 smithi094 ceph-mon[32889]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:48.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:47 smithi027 ceph-mon[28034]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:50.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:49 smithi080 ceph-mon[32324]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:50.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:49 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:50.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:49 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:50:50.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:49 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:50.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:49 smithi094 ceph-mon[32889]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:50.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:49 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:50.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:49 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:50:50.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:49 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:50.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:49 smithi027 ceph-mon[28034]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:50.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:49 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:50.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:49 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:50:50.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:49 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:52.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:51 smithi080 ceph-mon[32324]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:52.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:51 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:52.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:51 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:52.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:51 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:52.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:51 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:50:52.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:51 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:52.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:51 smithi094 ceph-mon[32889]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:52.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:51 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:52.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:51 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:52.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:51 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:52.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:51 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:50:52.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:51 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:52.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:51 smithi027 ceph-mon[28034]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:52.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:51 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:52.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:51 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:52.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:51 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:52.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:51 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:50:52.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:51 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:52.469 INFO:teuthology.orchestra.run.smithi080.stdout:Created osd(s) 4 on host 'smithi080' 2024-04-26T22:50:52.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:52.468+0000 7f188b7fe700 1 -- 172.21.15.80:0/2354184569 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f189c063860 con 0x7f18840a3c40 2024-04-26T22:50:52.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:52.471+0000 7f18a09fb700 1 -- 172.21.15.80:0/2354184569 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f18840a3c40 msgr2=0x7f18840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:50:52.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:52.471+0000 7f18a09fb700 1 --2- 172.21.15.80:0/2354184569 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f18840a3c40 0x7f18840a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f188c006fd0 tx=0x7f188c008040 comp rx=0 tx=0).stop 2024-04-26T22:50:52.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:52.471+0000 7f18a09fb700 1 -- 172.21.15.80:0/2354184569 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f189c1035a0 msgr2=0x7f189c10b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:50:52.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:52.471+0000 7f18a09fb700 1 --2- 172.21.15.80:0/2354184569 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f189c1035a0 0x7f189c10b1f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f189000b530 tx=0x7f189000b8f0 comp rx=0 tx=0).stop 2024-04-26T22:50:52.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:52.471+0000 7f18a09fb700 1 -- 172.21.15.80:0/2354184569 shutdown_connections 2024-04-26T22:50:52.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:52.471+0000 7f18a09fb700 1 --2- 172.21.15.80:0/2354184569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c103f40 0x7f189c10b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:52.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:52.471+0000 7f18a09fb700 1 --2- 172.21.15.80:0/2354184569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f189c1023a0 0x7f189c10cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:52.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:52.471+0000 7f18a09fb700 1 --2- 172.21.15.80:0/2354184569 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f18840a3c40 0x7f18840a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:52.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:52.471+0000 7f18a09fb700 1 --2- 172.21.15.80:0/2354184569 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f189c1035a0 0x7f189c10b1f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:52.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:52.471+0000 7f18a09fb700 1 -- 172.21.15.80:0/2354184569 >> 172.21.15.80:0/2354184569 conn(0x7f189c0fd940 msgr2=0x7f189c1068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:50:52.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:52.471+0000 7f18a09fb700 1 -- 172.21.15.80:0/2354184569 shutdown_connections 2024-04-26T22:50:52.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:50:52.471+0000 7f18a09fb700 1 -- 172.21.15.80:0/2354184569 wait complete. 2024-04-26T22:50:53.107 DEBUG:teuthology.orchestra.run.smithi080:osd.4> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T22:50:53.109 INFO:tasks.cephadm:Deploying osd.5 on smithi094 with /dev/vg_nvme/lv_3... 2024-04-26T22:50:53.109 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T22:50:53.351 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:50:53.705 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:53 smithi094 ceph-mon[32889]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:53.706 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:53.706 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:53.706 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:50:53.706 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:53 smithi094 ceph-mon[32889]: from='osd.4 [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T22:50:53.706 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:53 smithi094 ceph-mon[32889]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T22:50:53.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:53 smithi080 ceph-mon[32324]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:53.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:53.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:53.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:50:53.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:53 smithi080 ceph-mon[32324]: from='osd.4 [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T22:50:53.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:53 smithi080 ceph-mon[32324]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T22:50:53.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:53 smithi027 ceph-mon[28034]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:53.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:53.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:53.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:50:53.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:53 smithi027 ceph-mon[28034]: from='osd.4 [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T22:50:53.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:53 smithi027 ceph-mon[28034]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T22:50:54.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:54 smithi080 ceph-mon[32324]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T22:50:54.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:54 smithi080 ceph-mon[32324]: from='osd.4 [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:50:54.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:54 smithi080 ceph-mon[32324]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T22:50:54.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:54 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:50:54.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:54 smithi080 ceph-mon[32324]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:50:54.522 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 22:50:54 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4[43920]: 2024-04-26T22:50:54.442+0000 7f87aeb5e700 -1 osd.4 0 waiting for initial osdmap 2024-04-26T22:50:54.522 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 22:50:54 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4[43920]: 2024-04-26T22:50:54.448+0000 7f87aa951700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:50:54.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:54 smithi094 ceph-mon[32889]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T22:50:54.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:54 smithi094 ceph-mon[32889]: from='osd.4 [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:50:54.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:54 smithi094 ceph-mon[32889]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T22:50:54.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:54 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:50:54.696 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:54 smithi094 ceph-mon[32889]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:50:54.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:54 smithi027 ceph-mon[28034]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T22:50:54.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:54 smithi027 ceph-mon[28034]: from='osd.4 [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:50:54.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:54 smithi027 ceph-mon[28034]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T22:50:54.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:50:54.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:54 smithi027 ceph-mon[28034]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:50:55.394 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T22:50:55.394 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T22:50:55.394 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-26T22:50:55.394 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-26T22:50:55.394 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00851339 s, 1.2 GB/s 2024-04-26T22:50:55.394 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-26T22:50:55.721 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:55 smithi080 ceph-mon[32324]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:55.721 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:55 smithi080 ceph-mon[32324]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-26T22:50:55.721 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:55 smithi080 ceph-mon[32324]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T22:50:55.721 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:55 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:50:55.721 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:55 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:50:55.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:55 smithi094 ceph-mon[32889]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:55.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:55 smithi094 ceph-mon[32889]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-26T22:50:55.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:55 smithi094 ceph-mon[32889]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T22:50:55.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:55 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:50:55.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:55 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:50:55.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:55 smithi027 ceph-mon[28034]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:55.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:55 smithi027 ceph-mon[28034]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-26T22:50:55.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:55 smithi027 ceph-mon[28034]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T22:50:55.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:55 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:50:55.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:55 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:50:56.136 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_3 2024-04-26T22:50:56.391 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:50:56.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:56 smithi080 ceph-mon[32324]: purged_snaps scrub starts 2024-04-26T22:50:56.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:56 smithi080 ceph-mon[32324]: purged_snaps scrub ok 2024-04-26T22:50:56.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:56 smithi080 ceph-mon[32324]: osd.4 [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588] boot 2024-04-26T22:50:56.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:56 smithi080 ceph-mon[32324]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T22:50:56.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:50:56.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:56 smithi080 ceph-mon[32324]: Detected new or changed devices on smithi080 2024-04-26T22:50:56.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:56.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:56.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:50:56.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:50:56.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:56 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi080 to 8544M 2024-04-26T22:50:56.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:56.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:56.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:50:56.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:56.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:56 smithi094 ceph-mon[32889]: purged_snaps scrub starts 2024-04-26T22:50:56.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:56 smithi094 ceph-mon[32889]: purged_snaps scrub ok 2024-04-26T22:50:56.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:56 smithi094 ceph-mon[32889]: osd.4 [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588] boot 2024-04-26T22:50:56.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:56 smithi094 ceph-mon[32889]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T22:50:56.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:50:56.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:56 smithi094 ceph-mon[32889]: Detected new or changed devices on smithi080 2024-04-26T22:50:56.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:56.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:56.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:50:56.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:50:56.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:56 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi080 to 8544M 2024-04-26T22:50:56.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:56.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:56.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:50:56.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:56.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:56 smithi027 ceph-mon[28034]: purged_snaps scrub starts 2024-04-26T22:50:56.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:56 smithi027 ceph-mon[28034]: purged_snaps scrub ok 2024-04-26T22:50:56.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:56 smithi027 ceph-mon[28034]: osd.4 [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588] boot 2024-04-26T22:50:56.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:56 smithi027 ceph-mon[28034]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T22:50:56.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:50:56.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:56 smithi027 ceph-mon[28034]: Detected new or changed devices on smithi080 2024-04-26T22:50:56.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:56.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:56.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:50:56.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:50:56.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:56 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi080 to 8544M 2024-04-26T22:50:56.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:56.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:56.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:50:56.867 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:50:57.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:57 smithi094 ceph-mon[32889]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:57.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:57 smithi094 ceph-mon[32889]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T22:50:57.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:57 smithi080 ceph-mon[32324]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:57.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:57 smithi080 ceph-mon[32324]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T22:50:57.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:57 smithi027 ceph-mon[28034]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-26T22:50:57.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:57 smithi027 ceph-mon[28034]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T22:50:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.111+0000 7f2666a9a700 1 -- 172.21.15.94:0/3467579983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2660102380 msgr2=0x7f26601027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:50:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.111+0000 7f2666a9a700 1 --2- 172.21.15.94:0/3467579983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2660102380 0x7f26601027a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2654009990 tx=0x7f2654009ca0 comp rx=0 tx=0).stop 2024-04-26T22:50:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.111+0000 7f2666a9a700 1 -- 172.21.15.94:0/3467579983 shutdown_connections 2024-04-26T22:50:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.111+0000 7f2666a9a700 1 --2- 172.21.15.94:0/3467579983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2660103f20 0x7f266010a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.111+0000 7f2666a9a700 1 --2- 172.21.15.94:0/3467579983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2660102380 0x7f26601027a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.111+0000 7f2666a9a700 1 --2- 172.21.15.94:0/3467579983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2660103580 0x7f26601039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.111+0000 7f2666a9a700 1 -- 172.21.15.94:0/3467579983 >> 172.21.15.94:0/3467579983 conn(0x7f26600fd920 msgr2=0x7f26600ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:50:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.111+0000 7f2666a9a700 1 -- 172.21.15.94:0/3467579983 shutdown_connections 2024-04-26T22:50:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.111+0000 7f2666a9a700 1 -- 172.21.15.94:0/3467579983 wait complete. 2024-04-26T22:50:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.111+0000 7f2666a9a700 1 Processor -- start 2024-04-26T22:50:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.111+0000 7f2666a9a700 1 -- start start 2024-04-26T22:50:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f2666a9a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2660102380 0x7f266006bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:50:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f2666a9a700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2660103580 0x7f266019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:50:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f2666a9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2660103f20 0x7f26601a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:50:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f2666a9a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f266010de70 con 0x7f2660103f20 2024-04-26T22:50:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f2666a9a700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f266010dcf0 con 0x7f2660103580 2024-04-26T22:50:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f2666a9a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f266010dff0 con 0x7f2660102380 2024-04-26T22:50:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f2664836700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2660102380 0x7f266006bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:50:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f2664836700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2660102380 0x7f266006bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40510/0 (socket says 172.21.15.94:40510) 2024-04-26T22:50:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f2664836700 1 -- 172.21.15.94:0/482107891 learned_addr learned my addr 172.21.15.94:0/482107891 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T22:50:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f265ffff700 1 --2- 172.21.15.94:0/482107891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2660103580 0x7f266019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:50:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f2664836700 1 -- 172.21.15.94:0/482107891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2660103580 msgr2=0x7f266019e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:50:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f2664836700 1 --2- 172.21.15.94:0/482107891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2660103580 0x7f266019e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f2664836700 1 -- 172.21.15.94:0/482107891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2660103f20 msgr2=0x7f26601a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:50:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f2664836700 1 --2- 172.21.15.94:0/482107891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2660103f20 0x7f26601a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:50:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f2664836700 1 -- 172.21.15.94:0/482107891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26601a1310 con 0x7f2660102380 2024-04-26T22:50:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f2664836700 1 --2- 172.21.15.94:0/482107891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2660102380 0x7f266006bc60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f26540059d0 tx=0x7f26540047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:50:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f265dffb700 1 -- 172.21.15.94:0/482107891 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654027070 con 0x7f2660102380 2024-04-26T22:50:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f265dffb700 1 -- 172.21.15.94:0/482107891 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2654022470 con 0x7f2660102380 2024-04-26T22:50:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f2666a9a700 1 -- 172.21.15.94:0/482107891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26601a15a0 con 0x7f2660102380 2024-04-26T22:50:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.112+0000 7f265dffb700 1 -- 172.21.15.94:0/482107891 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654004230 con 0x7f2660102380 2024-04-26T22:50:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.113+0000 7f2666a9a700 1 -- 172.21.15.94:0/482107891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26601a19f0 con 0x7f2660102380 2024-04-26T22:50:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.114+0000 7f2666a9a700 1 -- 172.21.15.94:0/482107891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f266002e490 con 0x7f2660102380 2024-04-26T22:50:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.114+0000 7f265dffb700 1 -- 172.21.15.94:0/482107891 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f265402a070 con 0x7f2660102380 2024-04-26T22:50:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.115+0000 7f265dffb700 1 --2- 172.21.15.94:0/482107891 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f26500a3c40 0x7f26500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:50:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.115+0000 7f265dffb700 1 -- 172.21.15.94:0/482107891 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f26540d0250 con 0x7f2660102380 2024-04-26T22:50:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.116+0000 7f265ffff700 1 --2- 172.21.15.94:0/482107891 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f26500a3c40 0x7f26500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:50:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.116+0000 7f265ffff700 1 --2- 172.21.15.94:0/482107891 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f26500a3c40 0x7f26500a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f266006c1e0 tx=0x7f264c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:50:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.120+0000 7f265dffb700 1 -- 172.21.15.94:0/482107891 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2654013070 con 0x7f2660102380 2024-04-26T22:50:58.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:50:58.326+0000 7f2666a9a700 1 -- 172.21.15.94:0/482107891 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2660063860 con 0x7f26500a3c40 2024-04-26T22:50:58.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:50:58.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:50:58.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:58.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:50:58.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:50:58.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:58.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:50:58.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:50:58.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:59.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:59 smithi080 ceph-mon[32324]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:50:59.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:50:59 smithi080 ceph-mon[32324]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:50:59.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:59 smithi094 ceph-mon[32889]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:50:59.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:50:59 smithi094 ceph-mon[32889]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:50:59.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:59 smithi027 ceph-mon[28034]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:50:59.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:50:59 smithi027 ceph-mon[28034]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:01.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:01 smithi080 ceph-mon[32324]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:01.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:01 smithi080 ceph-mon[32324]: from='client.? 172.21.15.94:0/1549580093' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7369cc9a-e487-46bb-8f93-42a025150669"}]: dispatch 2024-04-26T22:51:01.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:01 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7369cc9a-e487-46bb-8f93-42a025150669"}]: dispatch 2024-04-26T22:51:01.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:01 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7369cc9a-e487-46bb-8f93-42a025150669"}]': finished 2024-04-26T22:51:01.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:01 smithi080 ceph-mon[32324]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T22:51:01.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:01 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:51:01.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:01 smithi094 ceph-mon[32889]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:01.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:01 smithi094 ceph-mon[32889]: from='client.? 172.21.15.94:0/1549580093' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7369cc9a-e487-46bb-8f93-42a025150669"}]: dispatch 2024-04-26T22:51:01.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:01 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7369cc9a-e487-46bb-8f93-42a025150669"}]: dispatch 2024-04-26T22:51:01.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:01 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7369cc9a-e487-46bb-8f93-42a025150669"}]': finished 2024-04-26T22:51:01.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:01 smithi094 ceph-mon[32889]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T22:51:01.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:51:01.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:01 smithi027 ceph-mon[28034]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:01.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:01 smithi027 ceph-mon[28034]: from='client.? 172.21.15.94:0/1549580093' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7369cc9a-e487-46bb-8f93-42a025150669"}]: dispatch 2024-04-26T22:51:01.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:01 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7369cc9a-e487-46bb-8f93-42a025150669"}]: dispatch 2024-04-26T22:51:01.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:01 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7369cc9a-e487-46bb-8f93-42a025150669"}]': finished 2024-04-26T22:51:01.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:01 smithi027 ceph-mon[28034]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T22:51:01.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:01 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:51:02.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:02 smithi080 ceph-mon[32324]: from='client.? 172.21.15.94:0/3646629014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:51:02.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:02 smithi094 ceph-mon[32889]: from='client.? 172.21.15.94:0/3646629014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:51:02.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:02 smithi027 ceph-mon[28034]: from='client.? 172.21.15.94:0/3646629014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:51:03.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:03 smithi080 ceph-mon[32324]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:03.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:03 smithi094 ceph-mon[32889]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:03.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:03 smithi027 ceph-mon[28034]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:05.724 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:05 smithi094 ceph-mon[32889]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:05.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:05 smithi080 ceph-mon[32324]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:05.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:05 smithi027 ceph-mon[28034]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:07.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:07 smithi080 ceph-mon[32324]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:07.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:07 smithi094 ceph-mon[32889]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:07.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:07 smithi027 ceph-mon[28034]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:09.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:09 smithi080 ceph-mon[32324]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:09.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:09 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T22:51:09.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:09 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:09.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:09 smithi080 ceph-mon[32324]: Deploying daemon osd.5 on smithi094 2024-04-26T22:51:09.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:09 smithi094 ceph-mon[32889]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:09.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:09 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T22:51:09.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:09 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:09.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:09 smithi094 ceph-mon[32889]: Deploying daemon osd.5 on smithi094 2024-04-26T22:51:09.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:09 smithi027 ceph-mon[28034]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:09.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:09 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T22:51:09.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:09 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:09.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:09 smithi027 ceph-mon[28034]: Deploying daemon osd.5 on smithi094 2024-04-26T22:51:11.667 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:11 smithi094 ceph-mon[32889]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:11.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:11 smithi027 ceph-mon[28034]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:12.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:11 smithi080 ceph-mon[32324]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:13.767 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:13 smithi094 ceph-mon[32889]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:13.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:13 smithi027 ceph-mon[28034]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:14.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:13 smithi080 ceph-mon[32324]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:15.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:15 smithi080 ceph-mon[32324]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:15.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:15 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:15.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:15 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:15.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:15 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:51:15.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:15 smithi094 ceph-mon[32889]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:15.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:15 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:15.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:15 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:15.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:15 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:51:15.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:15 smithi027 ceph-mon[28034]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:15.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:15 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:15.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:15 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:15.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:15 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:51:17.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:17 smithi094 ceph-mon[32889]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:17.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:17 smithi080 ceph-mon[32324]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:17.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:17 smithi027 ceph-mon[28034]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:18.417 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 5 on host 'smithi094' 2024-04-26T22:51:18.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:51:18.417+0000 7f265dffb700 1 -- 172.21.15.94:0/482107891 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2660063860 con 0x7f26500a3c40 2024-04-26T22:51:18.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:51:18.419+0000 7f2666a9a700 1 -- 172.21.15.94:0/482107891 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f26500a3c40 msgr2=0x7f26500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:51:18.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:51:18.419+0000 7f2666a9a700 1 --2- 172.21.15.94:0/482107891 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f26500a3c40 0x7f26500a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f266006c1e0 tx=0x7f264c009190 comp rx=0 tx=0).stop 2024-04-26T22:51:18.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:51:18.419+0000 7f2666a9a700 1 -- 172.21.15.94:0/482107891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2660102380 msgr2=0x7f266006bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:51:18.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:51:18.419+0000 7f2666a9a700 1 --2- 172.21.15.94:0/482107891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2660102380 0x7f266006bc60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f26540059d0 tx=0x7f26540047b0 comp rx=0 tx=0).stop 2024-04-26T22:51:18.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:51:18.419+0000 7f2666a9a700 1 -- 172.21.15.94:0/482107891 shutdown_connections 2024-04-26T22:51:18.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:51:18.419+0000 7f2666a9a700 1 --2- 172.21.15.94:0/482107891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2660103f20 0x7f26601a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:51:18.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:51:18.419+0000 7f2666a9a700 1 --2- 172.21.15.94:0/482107891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2660102380 0x7f266006bc60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:51:18.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:51:18.419+0000 7f2666a9a700 1 --2- 172.21.15.94:0/482107891 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f26500a3c40 0x7f26500a6100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:51:18.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:51:18.419+0000 7f2666a9a700 1 --2- 172.21.15.94:0/482107891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2660103580 0x7f266019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:51:18.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:51:18.419+0000 7f2666a9a700 1 -- 172.21.15.94:0/482107891 >> 172.21.15.94:0/482107891 conn(0x7f26600fd920 msgr2=0x7f26601069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:51:18.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:51:18.419+0000 7f2666a9a700 1 -- 172.21.15.94:0/482107891 shutdown_connections 2024-04-26T22:51:18.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:51:18.419+0000 7f2666a9a700 1 -- 172.21.15.94:0/482107891 wait complete. 2024-04-26T22:51:18.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:18 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:18.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:18 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:18.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:18 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:18.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:18 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:51:18.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:18 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:18.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:18 smithi027 ceph-mon[28034]: from='osd.5 [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T22:51:18.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:18 smithi027 ceph-mon[28034]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T22:51:18.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:18 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:18.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:18 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:51:18.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:18 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:19.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:19.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:19.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:19.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:51:19.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:19.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:18 smithi080 ceph-mon[32324]: from='osd.5 [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T22:51:19.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:18 smithi080 ceph-mon[32324]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T22:51:19.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:19.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:51:19.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:19.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:18 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:19.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:18 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:19.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:18 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:19.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:18 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:51:19.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:18 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:19.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:18 smithi094 ceph-mon[32889]: from='osd.5 [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T22:51:19.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:18 smithi094 ceph-mon[32889]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T22:51:19.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:18 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:19.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:18 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:51:19.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:18 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:19.107 DEBUG:teuthology.orchestra.run.smithi094:osd.5> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T22:51:19.109 INFO:tasks.cephadm:Deploying osd.6 on smithi027 with /dev/vg_nvme/lv_2... 2024-04-26T22:51:19.109 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T22:51:19.585 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:51:19.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:19 smithi094 ceph-mon[32889]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:19.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:19 smithi094 ceph-mon[32889]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T22:51:19.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:19 smithi094 ceph-mon[32889]: from='osd.5 [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:51:19.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:19 smithi094 ceph-mon[32889]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T22:51:19.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:19 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:51:19.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:19 smithi094 ceph-mon[32889]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:51:19.977 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:19 smithi027 ceph-mon[28034]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:19.978 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:19 smithi027 ceph-mon[28034]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T22:51:19.978 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:19 smithi027 ceph-mon[28034]: from='osd.5 [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:51:19.978 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:19 smithi027 ceph-mon[28034]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T22:51:19.978 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:19 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:51:19.978 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:19 smithi027 ceph-mon[28034]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:51:20.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:19 smithi080 ceph-mon[32324]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:20.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:19 smithi080 ceph-mon[32324]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T22:51:20.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:19 smithi080 ceph-mon[32324]: from='osd.5 [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:51:20.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:19 smithi080 ceph-mon[32324]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T22:51:20.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:19 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:51:20.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:19 smithi080 ceph-mon[32324]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:51:20.257 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 22:51:20 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T22:51:20.046+0000 7f6d7da07700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T22:51:20.257 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 22:51:20 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T22:51:20.052+0000 7f6d757f2700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:51:20.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:20 smithi094 ceph-mon[32889]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T22:51:20.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:20 smithi094 ceph-mon[32889]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T22:51:20.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:20 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:51:20.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:20 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:51:20.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:20 smithi094 ceph-mon[32889]: from='osd.5 ' entity='osd.5' 2024-04-26T22:51:21.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:20 smithi080 ceph-mon[32324]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T22:51:21.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:20 smithi080 ceph-mon[32324]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T22:51:21.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:20 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:51:21.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:20 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:51:21.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:20 smithi080 ceph-mon[32324]: from='osd.5 ' entity='osd.5' 2024-04-26T22:51:21.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:20 smithi027 ceph-mon[28034]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T22:51:21.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:20 smithi027 ceph-mon[28034]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T22:51:21.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:20 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:51:21.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:20 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:51:21.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:20 smithi027 ceph-mon[28034]: from='osd.5 ' entity='osd.5' 2024-04-26T22:51:21.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:21 smithi027 ceph-mon[28034]: purged_snaps scrub starts 2024-04-26T22:51:21.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:21 smithi027 ceph-mon[28034]: purged_snaps scrub ok 2024-04-26T22:51:21.793 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:21 smithi027 ceph-mon[28034]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:21.793 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:21 smithi027 ceph-mon[28034]: osd.5 [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302] boot 2024-04-26T22:51:21.793 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:21 smithi027 ceph-mon[28034]: osdmap e38: 6 total, 6 up, 6 in 2024-04-26T22:51:21.793 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:51:21.793 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:21 smithi027 ceph-mon[28034]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T22:51:22.014 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:21 smithi094 ceph-mon[32889]: purged_snaps scrub starts 2024-04-26T22:51:22.014 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:21 smithi094 ceph-mon[32889]: purged_snaps scrub ok 2024-04-26T22:51:22.014 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:21 smithi094 ceph-mon[32889]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:22.014 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:21 smithi094 ceph-mon[32889]: osd.5 [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302] boot 2024-04-26T22:51:22.014 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:21 smithi094 ceph-mon[32889]: osdmap e38: 6 total, 6 up, 6 in 2024-04-26T22:51:22.015 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:51:22.015 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:21 smithi094 ceph-mon[32889]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T22:51:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:21 smithi080 ceph-mon[32324]: purged_snaps scrub starts 2024-04-26T22:51:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:21 smithi080 ceph-mon[32324]: purged_snaps scrub ok 2024-04-26T22:51:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:21 smithi080 ceph-mon[32324]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-26T22:51:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:21 smithi080 ceph-mon[32324]: osd.5 [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302] boot 2024-04-26T22:51:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:21 smithi080 ceph-mon[32324]: osdmap e38: 6 total, 6 up, 6 in 2024-04-26T22:51:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:51:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:21 smithi080 ceph-mon[32324]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T22:51:22.812 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T22:51:22.813 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T22:51:22.813 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-04-26T22:51:22.813 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-04-26T22:51:22.813 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0374916 s, 280 MB/s 2024-04-26T22:51:22.813 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-04-26T22:51:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:23 smithi080 ceph-mon[32324]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T22:51:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:51:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:51:23.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:23.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:23.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:51:23.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:23.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:23 smithi094 ceph-mon[32889]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T22:51:23.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:23.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:23.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:51:23.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:51:23.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:23.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:23.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:51:23.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:23.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:23 smithi027 ceph-mon[28034]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T22:51:23.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:23.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:23.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:51:23.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:51:23.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:23.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:23.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:51:23.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:23.720 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_2 2024-04-26T22:51:24.170 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:51:24.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:24 smithi027 ceph-mon[28034]: Detected new or changed devices on smithi094 2024-04-26T22:51:24.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:24 smithi027 ceph-mon[28034]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:24.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:24 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi094 to 8544M 2024-04-26T22:51:24.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:24 smithi027 ceph-mon[28034]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T22:51:24.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:24 smithi027 ceph-mon[28034]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T22:51:24.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:24 smithi080 ceph-mon[32324]: Detected new or changed devices on smithi094 2024-04-26T22:51:24.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:24 smithi080 ceph-mon[32324]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:24.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:24 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi094 to 8544M 2024-04-26T22:51:24.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:24 smithi080 ceph-mon[32324]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T22:51:24.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:24 smithi080 ceph-mon[32324]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T22:51:24.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:24 smithi094 ceph-mon[32889]: Detected new or changed devices on smithi094 2024-04-26T22:51:24.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:24 smithi094 ceph-mon[32889]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:24.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:24 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi094 to 8544M 2024-04-26T22:51:24.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:24 smithi094 ceph-mon[32889]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T22:51:24.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:24 smithi094 ceph-mon[32889]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T22:51:26.254 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:26 smithi027 ceph-mon[28034]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:26.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:26 smithi080 ceph-mon[32324]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:26.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:26 smithi094 ceph-mon[32889]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.556+0000 7f2bc8400700 1 -- 172.21.15.27:0/215529788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bc0109ac0 msgr2=0x7f2bc0109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:51:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.556+0000 7f2bc8400700 1 --2- 172.21.15.27:0/215529788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bc0109ac0 0x7f2bc0109ee0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40099c0 tx=0x7f2bb4009cd0 comp rx=0 tx=0).stop 2024-04-26T22:51:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.556+0000 7f2bc8400700 1 -- 172.21.15.27:0/215529788 shutdown_connections 2024-04-26T22:51:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.556+0000 7f2bc8400700 1 --2- 172.21.15.27:0/215529788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc010b660 0x7f2bc0111f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:51:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.556+0000 7f2bc8400700 1 --2- 172.21.15.27:0/215529788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bc0109ac0 0x7f2bc0109ee0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:51:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.556+0000 7f2bc8400700 1 --2- 172.21.15.27:0/215529788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bc010acc0 0x7f2bc010b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:51:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.557+0000 7f2bc8400700 1 -- 172.21.15.27:0/215529788 >> 172.21.15.27:0/215529788 conn(0x7f2bc0078a90 msgr2=0x7f2bc007aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:51:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.557+0000 7f2bc8400700 1 -- 172.21.15.27:0/215529788 shutdown_connections 2024-04-26T22:51:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.557+0000 7f2bc8400700 1 -- 172.21.15.27:0/215529788 wait complete. 2024-04-26T22:51:26.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.558+0000 7f2bc8400700 1 Processor -- start 2024-04-26T22:51:26.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.558+0000 7f2bc8400700 1 -- start start 2024-04-26T22:51:26.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.558+0000 7f2bc8400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bc0109ac0 0x7f2bc019fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:51:26.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.558+0000 7f2bc8400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bc010acc0 0x7f2bc01a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:51:26.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.558+0000 7f2bc8400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc010b660 0x7f2bc01a7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:51:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.558+0000 7f2bc8400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0115580 con 0x7f2bc0109ac0 2024-04-26T22:51:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.558+0000 7f2bc8400700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0115400 con 0x7f2bc010acc0 2024-04-26T22:51:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.558+0000 7f2bc8400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0115700 con 0x7f2bc010b660 2024-04-26T22:51:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.559+0000 7f2bc5a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bc0109ac0 0x7f2bc019fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:51:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.559+0000 7f2bc5000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bc010acc0 0x7f2bc01a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:51:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.559+0000 7f2bc5000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bc010acc0 0x7f2bc01a0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:56002/0 (socket says 172.21.15.27:56002) 2024-04-26T22:51:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.559+0000 7f2bc5000700 1 -- 172.21.15.27:0/2466947375 learned_addr learned my addr 172.21.15.27:0/2466947375 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:51:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.559+0000 7f2bc6400700 1 --2- 172.21.15.27:0/2466947375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc010b660 0x7f2bc01a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:51:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.559+0000 7f2bc5000700 1 -- 172.21.15.27:0/2466947375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc010b660 msgr2=0x7f2bc01a7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:51:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.559+0000 7f2bc5000700 1 --2- 172.21.15.27:0/2466947375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc010b660 0x7f2bc01a7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:51:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.559+0000 7f2bc5000700 1 -- 172.21.15.27:0/2466947375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bc0109ac0 msgr2=0x7f2bc019fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:51:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.559+0000 7f2bc5000700 1 --2- 172.21.15.27:0/2466947375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bc0109ac0 0x7f2bc019fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:51:26.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.559+0000 7f2bc5000700 1 -- 172.21.15.27:0/2466947375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc01a7830 con 0x7f2bc010acc0 2024-04-26T22:51:26.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.559+0000 7f2bc5000700 1 --2- 172.21.15.27:0/2466947375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bc010acc0 0x7f2bc01a0110 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00e9a0 tx=0x7f2bac00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:51:26.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.560+0000 7f2bbe000700 1 -- 172.21.15.27:0/2466947375 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00ca10 con 0x7f2bc010acc0 2024-04-26T22:51:26.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.560+0000 7f2bbe000700 1 -- 172.21.15.27:0/2466947375 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bac00cbb0 con 0x7f2bc010acc0 2024-04-26T22:51:26.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.560+0000 7f2bc8400700 1 -- 172.21.15.27:0/2466947375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc01a7b20 con 0x7f2bc010acc0 2024-04-26T22:51:26.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.560+0000 7f2bbe000700 1 -- 172.21.15.27:0/2466947375 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac018680 con 0x7f2bc010acc0 2024-04-26T22:51:26.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.560+0000 7f2bc8400700 1 -- 172.21.15.27:0/2466947375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc01a80b0 con 0x7f2bc010acc0 2024-04-26T22:51:26.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.561+0000 7f2bc8400700 1 -- 172.21.15.27:0/2466947375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc006a110 con 0x7f2bc010acc0 2024-04-26T22:51:26.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.563+0000 7f2bbe000700 1 -- 172.21.15.27:0/2466947375 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2bac010430 con 0x7f2bc010acc0 2024-04-26T22:51:26.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.564+0000 7f2bbe000700 1 --2- 172.21.15.27:0/2466947375 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2ba80a3cc0 0x7f2ba80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:51:26.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.565+0000 7f2bbe000700 1 -- 172.21.15.27:0/2466947375 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f2bac0cf6c0 con 0x7f2bc010acc0 2024-04-26T22:51:26.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.565+0000 7f2bc5a00700 1 --2- 172.21.15.27:0/2466947375 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2ba80a3cc0 0x7f2ba80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:51:26.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.565+0000 7f2bc5a00700 1 --2- 172.21.15.27:0/2466947375 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2ba80a3cc0 0x7f2ba80a6180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400b580 tx=0x7f2bb4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:51:26.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.568+0000 7f2bbe000700 1 -- 172.21.15.27:0/2466947375 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2bac09cec0 con 0x7f2bc010acc0 2024-04-26T22:51:26.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:26.815+0000 7f2bc8400700 1 -- 172.21.15.27:0/2466947375 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2bc0063860 con 0x7f2ba80a3cc0 2024-04-26T22:51:27.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:51:27.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:51:27.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:27.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:51:27.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:51:27.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:27.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:51:27.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:51:27.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:28.422 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:28 smithi080 ceph-mon[32324]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-04-26T22:51:28.423 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:28 smithi080 ceph-mon[32324]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:51:28.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:28 smithi094 ceph-mon[32889]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-04-26T22:51:28.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:28 smithi094 ceph-mon[32889]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:51:28.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:28 smithi027 ceph-mon[28034]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-04-26T22:51:28.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:28 smithi027 ceph-mon[28034]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:51:29.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:29 smithi080 ceph-mon[32324]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T22:51:29.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:29 smithi094 ceph-mon[32889]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T22:51:29.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:29 smithi027 ceph-mon[28034]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T22:51:31.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:31 smithi080 ceph-mon[32324]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T22:51:31.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:31 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/671098247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "675deb21-5cf8-457d-acdb-188d3372dd8a"}]: dispatch 2024-04-26T22:51:31.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:31 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/671098247' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "675deb21-5cf8-457d-acdb-188d3372dd8a"}]': finished 2024-04-26T22:51:31.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:31 smithi080 ceph-mon[32324]: osdmap e43: 7 total, 6 up, 7 in 2024-04-26T22:51:31.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:31 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:51:31.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:31 smithi094 ceph-mon[32889]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T22:51:31.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:31 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/671098247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "675deb21-5cf8-457d-acdb-188d3372dd8a"}]: dispatch 2024-04-26T22:51:31.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:31 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/671098247' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "675deb21-5cf8-457d-acdb-188d3372dd8a"}]': finished 2024-04-26T22:51:31.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:31 smithi094 ceph-mon[32889]: osdmap e43: 7 total, 6 up, 7 in 2024-04-26T22:51:31.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:31 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:51:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:31 smithi027 ceph-mon[28034]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T22:51:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:31 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/671098247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "675deb21-5cf8-457d-acdb-188d3372dd8a"}]: dispatch 2024-04-26T22:51:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:31 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/671098247' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "675deb21-5cf8-457d-acdb-188d3372dd8a"}]': finished 2024-04-26T22:51:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:31 smithi027 ceph-mon[28034]: osdmap e43: 7 total, 6 up, 7 in 2024-04-26T22:51:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:31 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:51:32.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:32 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1464288951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:51:32.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:32 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1464288951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:51:32.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:32 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1464288951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:51:33.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:33 smithi080 ceph-mon[32324]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T22:51:33.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:33 smithi094 ceph-mon[32889]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T22:51:33.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:33 smithi027 ceph-mon[28034]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T22:51:35.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:35 smithi080 ceph-mon[32324]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T22:51:35.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:35 smithi094 ceph-mon[32889]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T22:51:35.819 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:35 smithi027 ceph-mon[28034]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T22:51:37.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:37 smithi080 ceph-mon[32324]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:37.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:37 smithi094 ceph-mon[32889]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:37.828 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:37 smithi027 ceph-mon[28034]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:39.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:39 smithi027 ceph-mon[28034]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:39.708 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:39 smithi080 ceph-mon[32324]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:39.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:39 smithi094 ceph-mon[32889]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:41.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:41 smithi080 ceph-mon[32324]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:41.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:41 smithi094 ceph-mon[32889]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:41.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:41 smithi027 ceph-mon[28034]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:42.732 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:42 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T22:51:42.732 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:42 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:42.732 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:42 smithi027 ceph-mon[28034]: Deploying daemon osd.6 on smithi027 2024-04-26T22:51:42.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:42 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T22:51:42.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:42 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:42.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:42 smithi080 ceph-mon[32324]: Deploying daemon osd.6 on smithi027 2024-04-26T22:51:42.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:42 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T22:51:42.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:42 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:42.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:42 smithi094 ceph-mon[32889]: Deploying daemon osd.6 on smithi027 2024-04-26T22:51:43.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:43 smithi027 ceph-mon[28034]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:43.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:43 smithi080 ceph-mon[32324]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:43.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:43 smithi094 ceph-mon[32889]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:45.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:45 smithi027 ceph-mon[28034]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:45.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:45 smithi080 ceph-mon[32324]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:45.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:45 smithi094 ceph-mon[32889]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:47.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:47 smithi080 ceph-mon[32324]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:47.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:47 smithi094 ceph-mon[32889]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:47.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:47 smithi027 ceph-mon[28034]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:49.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:49 smithi027 ceph-mon[28034]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:49.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:49 smithi080 ceph-mon[32324]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:49.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:49 smithi094 ceph-mon[32889]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:51.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:51 smithi080 ceph-mon[32324]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:51.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:51 smithi094 ceph-mon[32889]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:51.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:51 smithi027 ceph-mon[28034]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:53.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:52 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:53.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:52 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:53.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:52 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:51:53.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:52 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:53.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:52 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:53.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:52 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:51:53.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:52 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:53.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:52 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:53.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:52 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:51:54.021 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:53 smithi027 ceph-mon[28034]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:54.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:53 smithi080 ceph-mon[32324]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:54.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:53 smithi094 ceph-mon[32889]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:56.064 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:55 smithi027 ceph-mon[28034]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:56.065 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:55 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:56.065 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:55 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:56.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:55 smithi080 ceph-mon[32324]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:56.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:55 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:56.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:55 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:56.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:55 smithi094 ceph-mon[32889]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:56.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:55 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:56.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:55 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:57.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:56 smithi027 ceph-mon[28034]: from='osd.6 [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T22:51:57.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:57.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:51:57.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:56 smithi080 ceph-mon[32324]: from='osd.6 [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T22:51:57.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:57.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:51:57.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:56 smithi094 ceph-mon[32889]: from='osd.6 [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T22:51:57.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:57.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:51:57.539 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 6 on host 'smithi027' 2024-04-26T22:51:57.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:57.537+0000 7f2bbe000700 1 -- 172.21.15.27:0/2466947375 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2bc0063860 con 0x7f2ba80a3cc0 2024-04-26T22:51:57.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:57.541+0000 7f2bc8400700 1 -- 172.21.15.27:0/2466947375 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2ba80a3cc0 msgr2=0x7f2ba80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:51:57.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:57.541+0000 7f2bc8400700 1 --2- 172.21.15.27:0/2466947375 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2ba80a3cc0 0x7f2ba80a6180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400b580 tx=0x7f2bb4005fb0 comp rx=0 tx=0).stop 2024-04-26T22:51:57.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:57.542+0000 7f2bc8400700 1 -- 172.21.15.27:0/2466947375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bc010acc0 msgr2=0x7f2bc01a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:51:57.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:57.542+0000 7f2bc8400700 1 --2- 172.21.15.27:0/2466947375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bc010acc0 0x7f2bc01a0110 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00e9a0 tx=0x7f2bac00ecb0 comp rx=0 tx=0).stop 2024-04-26T22:51:57.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:57.542+0000 7f2bc8400700 1 -- 172.21.15.27:0/2466947375 shutdown_connections 2024-04-26T22:51:57.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:57.542+0000 7f2bc8400700 1 --2- 172.21.15.27:0/2466947375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc010b660 0x7f2bc01a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:51:57.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:57.542+0000 7f2bc8400700 1 --2- 172.21.15.27:0/2466947375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bc0109ac0 0x7f2bc019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:51:57.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:57.542+0000 7f2bc8400700 1 --2- 172.21.15.27:0/2466947375 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2ba80a3cc0 0x7f2ba80a6180 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:51:57.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:57.542+0000 7f2bc8400700 1 --2- 172.21.15.27:0/2466947375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bc010acc0 0x7f2bc01a0110 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:51:57.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:57.542+0000 7f2bc8400700 1 -- 172.21.15.27:0/2466947375 >> 172.21.15.27:0/2466947375 conn(0x7f2bc0078a90 msgr2=0x7f2bc010e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:51:57.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:57.542+0000 7f2bc8400700 1 -- 172.21.15.27:0/2466947375 shutdown_connections 2024-04-26T22:51:57.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:51:57.543+0000 7f2bc8400700 1 -- 172.21.15.27:0/2466947375 wait complete. 2024-04-26T22:51:58.173 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:57 smithi080 ceph-mon[32324]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:58.173 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:57 smithi080 ceph-mon[32324]: from='osd.6 [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T22:51:58.173 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:57 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:58.173 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:57 smithi080 ceph-mon[32324]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T22:51:58.173 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:57 smithi080 ceph-mon[32324]: from='osd.6 [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T22:51:58.173 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:57 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:51:58.173 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:57 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:58.173 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:57 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:51:58.173 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:57 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:58.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:57 smithi094 ceph-mon[32889]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:58.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:57 smithi094 ceph-mon[32889]: from='osd.6 [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T22:51:58.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:57 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:58.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:57 smithi094 ceph-mon[32889]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T22:51:58.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:57 smithi094 ceph-mon[32889]: from='osd.6 [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T22:51:58.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:57 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:51:58.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:57 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:58.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:57 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:51:58.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:57 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:58.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:57 smithi027 ceph-mon[28034]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:51:58.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:57 smithi027 ceph-mon[28034]: from='osd.6 [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T22:51:58.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:58.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:57 smithi027 ceph-mon[28034]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T22:51:58.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:57 smithi027 ceph-mon[28034]: from='osd.6 [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T22:51:58.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:51:58.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:58.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:51:58.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:57 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:51:58.436 DEBUG:teuthology.orchestra.run.smithi027:osd.6> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.6.service 2024-04-26T22:51:58.441 INFO:tasks.cephadm:Deploying osd.7 on smithi080 with /dev/vg_nvme/lv_2... 2024-04-26T22:51:58.441 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T22:51:58.685 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T22:51:58.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:58 smithi027 ceph-mon[28034]: from='osd.6 [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T22:51:58.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:58 smithi027 ceph-mon[28034]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T22:51:58.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:51:58.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:51:59.144 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:58 smithi080 ceph-mon[32324]: from='osd.6 [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T22:51:59.144 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:58 smithi080 ceph-mon[32324]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T22:51:59.144 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:51:59.144 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:51:59.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:58 smithi094 ceph-mon[32889]: from='osd.6 [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T22:51:59.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:58 smithi094 ceph-mon[32889]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T22:51:59.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:51:59.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:52:00.086 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:59 smithi027 ceph-mon[28034]: purged_snaps scrub starts 2024-04-26T22:52:00.086 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:59 smithi027 ceph-mon[28034]: purged_snaps scrub ok 2024-04-26T22:52:00.086 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:59 smithi027 ceph-mon[28034]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:52:00.087 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:59 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:52:00.087 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:59 smithi027 ceph-mon[28034]: osd.6 [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655] boot 2024-04-26T22:52:00.087 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:59 smithi027 ceph-mon[28034]: osdmap e46: 7 total, 7 up, 7 in 2024-04-26T22:52:00.087 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:51:59 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:52:00.193 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:59 smithi080 ceph-mon[32324]: purged_snaps scrub starts 2024-04-26T22:52:00.193 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:59 smithi080 ceph-mon[32324]: purged_snaps scrub ok 2024-04-26T22:52:00.193 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:59 smithi080 ceph-mon[32324]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:52:00.193 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:59 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:52:00.193 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:59 smithi080 ceph-mon[32324]: osd.6 [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655] boot 2024-04-26T22:52:00.193 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:59 smithi080 ceph-mon[32324]: osdmap e46: 7 total, 7 up, 7 in 2024-04-26T22:52:00.194 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:51:59 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:52:00.274 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:59 smithi094 ceph-mon[32889]: purged_snaps scrub starts 2024-04-26T22:52:00.274 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:59 smithi094 ceph-mon[32889]: purged_snaps scrub ok 2024-04-26T22:52:00.274 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:59 smithi094 ceph-mon[32889]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-04-26T22:52:00.274 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:59 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:52:00.274 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:59 smithi094 ceph-mon[32889]: osd.6 [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655] boot 2024-04-26T22:52:00.274 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:59 smithi094 ceph-mon[32889]: osdmap e46: 7 total, 7 up, 7 in 2024-04-26T22:52:00.274 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:51:59 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:52:00.583 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T22:52:00.584 INFO:teuthology.orchestra.run.smithi080.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T22:52:00.584 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10+0 records in 2024-04-26T22:52:00.584 INFO:teuthology.orchestra.run.smithi080.stderr:10+0 records out 2024-04-26T22:52:00.584 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115524 s, 908 MB/s 2024-04-26T22:52:00.584 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping successful for: 2024-04-26T22:52:00.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:00 smithi027 ceph-mon[28034]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T22:52:01.181 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:00 smithi080 ceph-mon[32324]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T22:52:01.211 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi080:vg_nvme/lv_2 2024-04-26T22:52:01.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:00 smithi094 ceph-mon[32889]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T22:52:01.462 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T22:52:02.263 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:01 smithi080 ceph-mon[32324]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:02.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:01 smithi094 ceph-mon[32889]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:02.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:01 smithi027 ceph-mon[28034]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:03.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.113+0000 7f68a0b57700 1 -- 172.21.15.80:0/1653925785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c105010 msgr2=0x7f689c10b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:52:03.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.113+0000 7f68a0b57700 1 --2- 172.21.15.80:0/1653925785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c105010 0x7f689c10b8c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f689400b0d0 tx=0x7f689400b3e0 comp rx=0 tx=0).stop 2024-04-26T22:52:03.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.113+0000 7f68a0b57700 1 -- 172.21.15.80:0/1653925785 shutdown_connections 2024-04-26T22:52:03.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.113+0000 7f68a0b57700 1 --2- 172.21.15.80:0/1653925785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c105010 0x7f689c10b8c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:03.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.113+0000 7f68a0b57700 1 --2- 172.21.15.80:0/1653925785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689c103470 0x7f689c103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:03.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.113+0000 7f68a0b57700 1 --2- 172.21.15.80:0/1653925785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f689c104670 0x7f689c104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:03.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.113+0000 7f68a0b57700 1 -- 172.21.15.80:0/1653925785 >> 172.21.15.80:0/1653925785 conn(0x7f689c0fea10 msgr2=0x7f689c100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:52:03.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.113+0000 7f68a0b57700 1 -- 172.21.15.80:0/1653925785 shutdown_connections 2024-04-26T22:52:03.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.113+0000 7f68a0b57700 1 -- 172.21.15.80:0/1653925785 wait complete. 2024-04-26T22:52:03.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.113+0000 7f68a0b57700 1 Processor -- start 2024-04-26T22:52:03.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.113+0000 7f68a0b57700 1 -- start start 2024-04-26T22:52:03.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f68a0b57700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f689c103470 0x7f689c1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:52:03.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f68a0b57700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689c104670 0x7f689c199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:52:03.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f68a0b57700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c105010 0x7f689c1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:52:03.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f68a0b57700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f689c10f000 con 0x7f689c104670 2024-04-26T22:52:03.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f68a0b57700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f689c10ee80 con 0x7f689c105010 2024-04-26T22:52:03.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f68a0b57700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f689c10f180 con 0x7f689c103470 2024-04-26T22:52:03.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f689ad9d700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c105010 0x7f689c1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:52:03.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f689ad9d700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c105010 0x7f689c1a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37166/0 (socket says 172.21.15.80:37166) 2024-04-26T22:52:03.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f689ad9d700 1 -- 172.21.15.80:0/1211821185 learned_addr learned my addr 172.21.15.80:0/1211821185 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-26T22:52:03.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f689a59c700 1 --2- 172.21.15.80:0/1211821185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f689c103470 0x7f689c1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:52:03.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f689ad9d700 1 -- 172.21.15.80:0/1211821185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f689c103470 msgr2=0x7f689c1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:52:03.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f689ad9d700 1 --2- 172.21.15.80:0/1211821185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f689c103470 0x7f689c1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:03.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f689ad9d700 1 -- 172.21.15.80:0/1211821185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689c104670 msgr2=0x7f689c199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:52:03.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f689ad9d700 1 --2- 172.21.15.80:0/1211821185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689c104670 0x7f689c199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:03.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f689ad9d700 1 -- 172.21.15.80:0/1211821185 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c1a1330 con 0x7f689c105010 2024-04-26T22:52:03.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f689a59c700 1 --2- 172.21.15.80:0/1211821185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f689c103470 0x7f689c1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:52:03.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f689ad9d700 1 --2- 172.21.15.80:0/1211821185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c105010 0x7f689c1a0c90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6894009fd0 tx=0x7f6894004ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:52:03.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f688b7fe700 1 -- 172.21.15.80:0/1211821185 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689401f030 con 0x7f689c105010 2024-04-26T22:52:03.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f688b7fe700 1 -- 172.21.15.80:0/1211821185 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6894007bb0 con 0x7f689c105010 2024-04-26T22:52:03.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f68a0b57700 1 -- 172.21.15.80:0/1211821185 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689c1a15c0 con 0x7f689c105010 2024-04-26T22:52:03.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.114+0000 7f688b7fe700 1 -- 172.21.15.80:0/1211821185 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894004810 con 0x7f689c105010 2024-04-26T22:52:03.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.115+0000 7f68a0b57700 1 -- 172.21.15.80:0/1211821185 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f689c1a1a90 con 0x7f689c105010 2024-04-26T22:52:03.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.116+0000 7f68a0b57700 1 -- 172.21.15.80:0/1211821185 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f689c02e490 con 0x7f689c105010 2024-04-26T22:52:03.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.116+0000 7f688b7fe700 1 -- 172.21.15.80:0/1211821185 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6894003a00 con 0x7f689c105010 2024-04-26T22:52:03.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.117+0000 7f688b7fe700 1 --2- 172.21.15.80:0/1211821185 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f68840a3cc0 0x7f68840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:52:03.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.117+0000 7f688b7fe700 1 -- 172.21.15.80:0/1211821185 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f68940d05f0 con 0x7f689c105010 2024-04-26T22:52:03.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.118+0000 7f689a59c700 1 --2- 172.21.15.80:0/1211821185 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f68840a3cc0 0x7f68840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:52:03.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.118+0000 7f689a59c700 1 --2- 172.21.15.80:0/1211821185 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f68840a3cc0 0x7f68840a6180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f689c1044d0 tx=0x7f688c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:52:03.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.122+0000 7f688b7fe700 1 -- 172.21.15.80:0/1211821185 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f689409dd70 con 0x7f689c105010 2024-04-26T22:52:03.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:03.329+0000 7f68a0b57700 1 -- 172.21.15.80:0/1211821185 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f689c063860 con 0x7f68840a3cc0 2024-04-26T22:52:04.196 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:03 smithi080 ceph-mon[32324]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:04.197 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:03 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:52:04.197 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:03 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:52:04.197 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:03 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:04.197 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:03 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:04.197 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:03 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:04.197 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:03 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:04.197 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:03 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:04.197 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:03 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:04.197 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:03 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:04.197 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:03 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:04.197 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:03 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:04.197 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:03 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:04.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:03 smithi094 ceph-mon[32889]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:04.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:03 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:52:04.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:03 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:52:04.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:03 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:04.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:03 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:04.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:03 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:04.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:03 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:04.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:03 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:04.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:03 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:04.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:03 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:04.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:03 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:04.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:03 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:04.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:03 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:04.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:03 smithi027 ceph-mon[28034]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:04.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:03 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:52:04.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:03 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:52:04.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:03 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:04.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:03 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:04.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:03 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:04.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:03 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:04.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:03 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:04.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:03 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:04.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:03 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:04.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:03 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:04.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:03 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:04.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:03 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:05.209 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:04 smithi080 ceph-mon[32324]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:52:05.209 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:04 smithi080 ceph-mon[32324]: Detected new or changed devices on smithi027 2024-04-26T22:52:05.209 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:04 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi027 to 5696M 2024-04-26T22:52:05.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:04 smithi094 ceph-mon[32889]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:52:05.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:04 smithi094 ceph-mon[32889]: Detected new or changed devices on smithi027 2024-04-26T22:52:05.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:04 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi027 to 5696M 2024-04-26T22:52:05.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:04 smithi027 ceph-mon[28034]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:52:05.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:04 smithi027 ceph-mon[28034]: Detected new or changed devices on smithi027 2024-04-26T22:52:05.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:04 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi027 to 5696M 2024-04-26T22:52:06.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:05 smithi027 ceph-mon[28034]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:06.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:05 smithi080 ceph-mon[32324]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:06.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:05 smithi094 ceph-mon[32889]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:07.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:06 smithi080 ceph-mon[32324]: from='client.? 172.21.15.80:0/2316580670' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e9aa483-e206-4ba2-b39b-9f7547da1459"}]: dispatch 2024-04-26T22:52:07.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:06 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e9aa483-e206-4ba2-b39b-9f7547da1459"}]: dispatch 2024-04-26T22:52:07.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:06 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e9aa483-e206-4ba2-b39b-9f7547da1459"}]': finished 2024-04-26T22:52:07.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:06 smithi080 ceph-mon[32324]: osdmap e48: 8 total, 7 up, 8 in 2024-04-26T22:52:07.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:06 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:52:07.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:06 smithi080 ceph-mon[32324]: from='client.? 172.21.15.80:0/701046835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:52:07.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:06 smithi094 ceph-mon[32889]: from='client.? 172.21.15.80:0/2316580670' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e9aa483-e206-4ba2-b39b-9f7547da1459"}]: dispatch 2024-04-26T22:52:07.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:06 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e9aa483-e206-4ba2-b39b-9f7547da1459"}]: dispatch 2024-04-26T22:52:07.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:06 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e9aa483-e206-4ba2-b39b-9f7547da1459"}]': finished 2024-04-26T22:52:07.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:06 smithi094 ceph-mon[32889]: osdmap e48: 8 total, 7 up, 8 in 2024-04-26T22:52:07.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:06 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:52:07.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:06 smithi094 ceph-mon[32889]: from='client.? 172.21.15.80:0/701046835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:52:07.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:06 smithi027 ceph-mon[28034]: from='client.? 172.21.15.80:0/2316580670' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e9aa483-e206-4ba2-b39b-9f7547da1459"}]: dispatch 2024-04-26T22:52:07.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:06 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e9aa483-e206-4ba2-b39b-9f7547da1459"}]: dispatch 2024-04-26T22:52:07.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:06 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e9aa483-e206-4ba2-b39b-9f7547da1459"}]': finished 2024-04-26T22:52:07.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:06 smithi027 ceph-mon[28034]: osdmap e48: 8 total, 7 up, 8 in 2024-04-26T22:52:07.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:06 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:52:07.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:06 smithi027 ceph-mon[28034]: from='client.? 172.21.15.80:0/701046835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:52:08.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:08 smithi027 ceph-mon[28034]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:08.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:08 smithi080 ceph-mon[32324]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:08.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:08 smithi094 ceph-mon[32889]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:10.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:10 smithi080 ceph-mon[32324]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:10.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:10 smithi027 ceph-mon[28034]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:10.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:10 smithi094 ceph-mon[32889]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:12.188 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:12 smithi080 ceph-mon[32324]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:12.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:12 smithi027 ceph-mon[28034]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:12.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:12 smithi094 ceph-mon[32889]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:14.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:14 smithi080 ceph-mon[32324]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:14.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:14 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T22:52:14.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:14 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:14.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:14 smithi027 ceph-mon[28034]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:14.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:14 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T22:52:14.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:14 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:14.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:14 smithi094 ceph-mon[32889]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:14.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:14 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T22:52:14.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:14 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:15.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:15 smithi080 ceph-mon[32324]: Deploying daemon osd.7 on smithi080 2024-04-26T22:52:15.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:15 smithi027 ceph-mon[28034]: Deploying daemon osd.7 on smithi080 2024-04-26T22:52:15.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:15 smithi094 ceph-mon[32889]: Deploying daemon osd.7 on smithi080 2024-04-26T22:52:16.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:16 smithi027 ceph-mon[28034]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:16.494 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:16 smithi080 ceph-mon[32324]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:16.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:16 smithi094 ceph-mon[32889]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:18.262 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:18 smithi080 ceph-mon[32324]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:18.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:18 smithi027 ceph-mon[28034]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:18.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:18 smithi094 ceph-mon[32889]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:20.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:19 smithi027 ceph-mon[28034]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:20.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:19 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:20.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:19 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:20.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:19 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:52:20.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:19 smithi080 ceph-mon[32324]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:20.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:19 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:20.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:19 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:20.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:19 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:52:20.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:19 smithi094 ceph-mon[32889]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:20.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:19 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:20.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:19 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:20.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:19 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:52:21.936 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:21 smithi080 ceph-mon[32324]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:22.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:21 smithi027 ceph-mon[28034]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:22.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:22.270+0000 7f688b7fe700 1 -- 172.21.15.80:0/1211821185 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f689c063860 con 0x7f68840a3cc0 2024-04-26T22:52:22.271 INFO:teuthology.orchestra.run.smithi080.stdout:Created osd(s) 7 on host 'smithi080' 2024-04-26T22:52:22.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:22.272+0000 7f68a0b57700 1 -- 172.21.15.80:0/1211821185 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f68840a3cc0 msgr2=0x7f68840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:52:22.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:22.272+0000 7f68a0b57700 1 --2- 172.21.15.80:0/1211821185 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f68840a3cc0 0x7f68840a6180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f689c1044d0 tx=0x7f688c008040 comp rx=0 tx=0).stop 2024-04-26T22:52:22.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:22.272+0000 7f68a0b57700 1 -- 172.21.15.80:0/1211821185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c105010 msgr2=0x7f689c1a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:52:22.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:22.272+0000 7f68a0b57700 1 --2- 172.21.15.80:0/1211821185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c105010 0x7f689c1a0c90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6894009fd0 tx=0x7f6894004ef0 comp rx=0 tx=0).stop 2024-04-26T22:52:22.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:22.273+0000 7f68a0b57700 1 -- 172.21.15.80:0/1211821185 shutdown_connections 2024-04-26T22:52:22.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:22.273+0000 7f68a0b57700 1 --2- 172.21.15.80:0/1211821185 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f68840a3cc0 0x7f68840a6180 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:22.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:22.273+0000 7f68a0b57700 1 --2- 172.21.15.80:0/1211821185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c105010 0x7f689c1a0c90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:22.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:22.273+0000 7f68a0b57700 1 --2- 172.21.15.80:0/1211821185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f689c103470 0x7f689c1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:22.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:22.273+0000 7f68a0b57700 1 --2- 172.21.15.80:0/1211821185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689c104670 0x7f689c199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:22.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:22.273+0000 7f68a0b57700 1 -- 172.21.15.80:0/1211821185 >> 172.21.15.80:0/1211821185 conn(0x7f689c0fea10 msgr2=0x7f689c107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:52:22.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:22.273+0000 7f68a0b57700 1 -- 172.21.15.80:0/1211821185 shutdown_connections 2024-04-26T22:52:22.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:52:22.273+0000 7f68a0b57700 1 -- 172.21.15.80:0/1211821185 wait complete. 2024-04-26T22:52:22.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:21 smithi094 ceph-mon[32889]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:22.893 DEBUG:teuthology.orchestra.run.smithi080:osd.7> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.7.service 2024-04-26T22:52:22.896 INFO:tasks.cephadm:Deploying osd.8 on smithi094 with /dev/vg_nvme/lv_2... 2024-04-26T22:52:22.896 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T22:52:23.130 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:52:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:23 smithi080 ceph-mon[32324]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:23 smithi080 ceph-mon[32324]: from='osd.7 [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T22:52:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:23 smithi080 ceph-mon[32324]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T22:52:23.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:23.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:23.524 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:23.524 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:23.524 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:23 smithi094 ceph-mon[32889]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:23.524 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:23 smithi094 ceph-mon[32889]: from='osd.7 [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T22:52:23.524 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:23 smithi094 ceph-mon[32889]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T22:52:23.524 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:23.524 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:23.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:23.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:23.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:23 smithi027 ceph-mon[28034]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:23.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:23 smithi027 ceph-mon[28034]: from='osd.7 [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T22:52:23.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:23 smithi027 ceph-mon[28034]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T22:52:23.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:23.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:24.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:24 smithi080 ceph-mon[32324]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T22:52:24.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:24 smithi080 ceph-mon[32324]: from='osd.7 [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:52:24.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:24 smithi080 ceph-mon[32324]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T22:52:24.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:24 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:52:24.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:24 smithi080 ceph-mon[32324]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:52:24.522 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 22:52:24 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T22:52:24.267+0000 7fe16f836700 -1 osd.7 0 waiting for initial osdmap 2024-04-26T22:52:24.522 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 22:52:24 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T22:52:24.278+0000 7fe16b629700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:52:24.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:24 smithi094 ceph-mon[32889]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T22:52:24.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:24 smithi094 ceph-mon[32889]: from='osd.7 [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:52:24.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:24 smithi094 ceph-mon[32889]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T22:52:24.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:24 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:52:24.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:24 smithi094 ceph-mon[32889]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:52:24.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:24 smithi027 ceph-mon[28034]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T22:52:24.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:24 smithi027 ceph-mon[28034]: from='osd.7 [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:52:24.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:24 smithi027 ceph-mon[28034]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T22:52:24.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:24 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:52:24.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:24 smithi027 ceph-mon[28034]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:52:25.185 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T22:52:25.185 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T22:52:25.186 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-26T22:52:25.186 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-26T22:52:25.186 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00829404 s, 1.3 GB/s 2024-04-26T22:52:25.186 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-26T22:52:25.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:25 smithi094 ceph-mon[32889]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-26T22:52:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:25 smithi094 ceph-mon[32889]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T22:52:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:52:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:25 smithi094 ceph-mon[32889]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:25 smithi094 ceph-mon[32889]: osd.7 [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127] boot 2024-04-26T22:52:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:25 smithi094 ceph-mon[32889]: osdmap e51: 8 total, 8 up, 8 in 2024-04-26T22:52:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:52:25.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:25 smithi027 ceph-mon[28034]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-26T22:52:25.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:25 smithi027 ceph-mon[28034]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T22:52:25.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:52:25.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:25 smithi027 ceph-mon[28034]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:25.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:25 smithi027 ceph-mon[28034]: osd.7 [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127] boot 2024-04-26T22:52:25.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:25 smithi027 ceph-mon[28034]: osdmap e51: 8 total, 8 up, 8 in 2024-04-26T22:52:25.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:52:25.759 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:25 smithi080 ceph-mon[32324]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-26T22:52:25.759 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:25 smithi080 ceph-mon[32324]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T22:52:25.759 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:52:25.759 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:25 smithi080 ceph-mon[32324]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-26T22:52:25.759 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:25 smithi080 ceph-mon[32324]: osd.7 [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127] boot 2024-04-26T22:52:25.759 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:25 smithi080 ceph-mon[32324]: osdmap e51: 8 total, 8 up, 8 in 2024-04-26T22:52:25.759 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:52:25.896 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_2 2024-04-26T22:52:26.129 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:52:26.467 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:26 smithi080 ceph-mon[32324]: purged_snaps scrub starts 2024-04-26T22:52:26.467 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:26 smithi080 ceph-mon[32324]: purged_snaps scrub ok 2024-04-26T22:52:26.467 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:26 smithi080 ceph-mon[32324]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T22:52:26.521 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:26 smithi094 ceph-mon[32889]: purged_snaps scrub starts 2024-04-26T22:52:26.521 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:26 smithi094 ceph-mon[32889]: purged_snaps scrub ok 2024-04-26T22:52:26.521 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:26 smithi094 ceph-mon[32889]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T22:52:26.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:26 smithi027 ceph-mon[28034]: purged_snaps scrub starts 2024-04-26T22:52:26.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:26 smithi027 ceph-mon[28034]: purged_snaps scrub ok 2024-04-26T22:52:26.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:26 smithi027 ceph-mon[28034]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T22:52:27.620 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: pgmap v203: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:27.620 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: Detected new or changed devices on smithi080 2024-04-26T22:52:27.620 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:27.620 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:27.620 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:27.620 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:27.620 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:27.620 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi080 to 5696M 2024-04-26T22:52:27.620 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:27.620 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:27.620 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:27.620 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:27.620 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:52:27.621 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:27.621 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:27.621 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:27.621 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T22:52:27.621 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:27 smithi094 ceph-mon[32889]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-26T22:52:27.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: pgmap v203: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:27.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: Detected new or changed devices on smithi080 2024-04-26T22:52:27.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:27.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:27.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:27.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:27.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:27.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi080 to 5696M 2024-04-26T22:52:27.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:27.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:27.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:27.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:27.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:52:27.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:27.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:27.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:27.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T22:52:27.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:27 smithi027 ceph-mon[28034]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-26T22:52:27.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.873+0000 7fb960d9e700 1 -- 172.21.15.94:0/1515492187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb95c1091e0 msgr2=0x7fb95c10b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:52:27.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.873+0000 7fb960d9e700 1 --2- 172.21.15.94:0/1515492187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb95c1091e0 0x7fb95c10b5f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb95400b0a0 tx=0x7fb95400b3b0 comp rx=0 tx=0).stop 2024-04-26T22:52:27.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.874+0000 7fb960d9e700 1 -- 172.21.15.94:0/1515492187 shutdown_connections 2024-04-26T22:52:27.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.874+0000 7fb960d9e700 1 --2- 172.21.15.94:0/1515492187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb95c1091e0 0x7fb95c10b5f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:27.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.874+0000 7fb960d9e700 1 --2- 172.21.15.94:0/1515492187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c103f80 0x7fb95c106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:27.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.874+0000 7fb960d9e700 1 --2- 172.21.15.94:0/1515492187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb95c1068b0 0x7fb95c108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:27.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.874+0000 7fb960d9e700 1 -- 172.21.15.94:0/1515492187 >> 172.21.15.94:0/1515492187 conn(0x7fb95c0fd950 msgr2=0x7fb95c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:52:27.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.874+0000 7fb960d9e700 1 -- 172.21.15.94:0/1515492187 shutdown_connections 2024-04-26T22:52:27.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.874+0000 7fb960d9e700 1 -- 172.21.15.94:0/1515492187 wait complete. 2024-04-26T22:52:27.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.874+0000 7fb960d9e700 1 Processor -- start 2024-04-26T22:52:27.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.874+0000 7fb960d9e700 1 -- start start 2024-04-26T22:52:27.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb960d9e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb95c103f80 0x7fb95c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:52:27.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb960d9e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb95c1068b0 0x7fb95c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:52:27.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb960d9e700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c1091e0 0x7fb95c1a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:52:27.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb960d9e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c10efe0 con 0x7fb95c1068b0 2024-04-26T22:52:27.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb960d9e700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c10ee60 con 0x7fb95c1091e0 2024-04-26T22:52:27.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb960d9e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c10f160 con 0x7fb95c103f80 2024-04-26T22:52:27.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb95a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb95c103f80 0x7fb95c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:52:27.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb95a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb95c103f80 0x7fb95c199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51862/0 (socket says 172.21.15.94:51862) 2024-04-26T22:52:27.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb95a59c700 1 -- 172.21.15.94:0/3717251039 learned_addr learned my addr 172.21.15.94:0/3717251039 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T22:52:27.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb95ad9d700 1 --2- 172.21.15.94:0/3717251039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c1091e0 0x7fb95c1a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:52:27.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb95a59c700 1 -- 172.21.15.94:0/3717251039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c1091e0 msgr2=0x7fb95c1a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:52:27.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb95a59c700 1 --2- 172.21.15.94:0/3717251039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c1091e0 0x7fb95c1a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:27.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb95a59c700 1 -- 172.21.15.94:0/3717251039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb95c1068b0 msgr2=0x7fb95c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:52:27.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb95a59c700 1 --2- 172.21.15.94:0/3717251039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb95c1068b0 0x7fb95c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:27.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb95a59c700 1 -- 172.21.15.94:0/3717251039 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95c1a12e0 con 0x7fb95c103f80 2024-04-26T22:52:27.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb95a59c700 1 --2- 172.21.15.94:0/3717251039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb95c103f80 0x7fb95c199680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00d9b0 tx=0x7fb94c00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:52:27.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb94b7fe700 1 -- 172.21.15.94:0/3717251039 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c017070 con 0x7fb95c103f80 2024-04-26T22:52:27.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb960d9e700 1 -- 172.21.15.94:0/3717251039 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95c1a15d0 con 0x7fb95c103f80 2024-04-26T22:52:27.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb94b7fe700 1 -- 172.21.15.94:0/3717251039 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb94c005020 con 0x7fb95c103f80 2024-04-26T22:52:27.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb94b7fe700 1 -- 172.21.15.94:0/3717251039 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c009b50 con 0x7fb95c103f80 2024-04-26T22:52:27.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.875+0000 7fb960d9e700 1 -- 172.21.15.94:0/3717251039 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb95c1a1b40 con 0x7fb95c103f80 2024-04-26T22:52:27.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.876+0000 7fb960d9e700 1 -- 172.21.15.94:0/3717251039 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb95c06a110 con 0x7fb95c103f80 2024-04-26T22:52:27.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.877+0000 7fb94b7fe700 1 -- 172.21.15.94:0/3717251039 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb94c0051d0 con 0x7fb95c103f80 2024-04-26T22:52:27.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.878+0000 7fb94b7fe700 1 --2- 172.21.15.94:0/3717251039 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb9440a3c40 0x7fb9440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:52:27.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.878+0000 7fb94b7fe700 1 -- 172.21.15.94:0/3717251039 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7fb94c0cfbc0 con 0x7fb95c103f80 2024-04-26T22:52:27.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.878+0000 7fb959d9b700 1 --2- 172.21.15.94:0/3717251039 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb9440a3c40 0x7fb9440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:52:27.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.879+0000 7fb959d9b700 1 --2- 172.21.15.94:0/3717251039 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb9440a3c40 0x7fb9440a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb9500097b0 tx=0x7fb950006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:52:27.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:27.882+0000 7fb94b7fe700 1 -- 172.21.15.94:0/3717251039 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb94c020020 con 0x7fb95c103f80 2024-04-26T22:52:28.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: pgmap v203: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: Detected new or changed devices on smithi080 2024-04-26T22:52:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi080 to 5696M 2024-04-26T22:52:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:52:28.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:28.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:28.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:28.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T22:52:28.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:27 smithi080 ceph-mon[32324]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-26T22:52:28.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:28.087+0000 7fb960d9e700 1 -- 172.21.15.94:0/3717251039 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb95c063860 con 0x7fb9440a3c40 2024-04-26T22:52:28.844 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:28 smithi094 ceph-mon[32889]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:52:28.844 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:28 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:52:28.844 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:28 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:52:28.844 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:28 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:28.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:28 smithi027 ceph-mon[28034]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:52:28.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:28 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:52:28.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:28 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:52:28.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:28 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:28.923 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:28 smithi080 ceph-mon[32324]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:52:28.923 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:28 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:52:28.923 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:28 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:52:28.923 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:28 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:29.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:29 smithi027 ceph-mon[28034]: pgmap v205: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:29.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:29 smithi027 ceph-mon[28034]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T22:52:29.919 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:29 smithi094 ceph-mon[32889]: pgmap v205: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:29.919 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:29 smithi094 ceph-mon[32889]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T22:52:30.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:29 smithi080 ceph-mon[32324]: pgmap v205: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:30.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:29 smithi080 ceph-mon[32324]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T22:52:30.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:30 smithi027 ceph-mon[28034]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T22:52:31.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:30 smithi080 ceph-mon[32324]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T22:52:31.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:30 smithi094 ceph-mon[32889]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T22:52:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:31 smithi027 ceph-mon[28034]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:31 smithi027 ceph-mon[28034]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-26T22:52:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:31 smithi027 ceph-mon[28034]: Cluster is now healthy 2024-04-26T22:52:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:31 smithi027 ceph-mon[28034]: from='client.? 172.21.15.94:0/4260877038' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "effb8164-0b8e-425f-af66-624c2909eb96"}]: dispatch 2024-04-26T22:52:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:31 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "effb8164-0b8e-425f-af66-624c2909eb96"}]: dispatch 2024-04-26T22:52:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:31 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "effb8164-0b8e-425f-af66-624c2909eb96"}]': finished 2024-04-26T22:52:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:31 smithi027 ceph-mon[28034]: osdmap e56: 9 total, 8 up, 9 in 2024-04-26T22:52:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:31 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:52:32.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:31 smithi080 ceph-mon[32324]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:32.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:31 smithi080 ceph-mon[32324]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-26T22:52:32.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:31 smithi080 ceph-mon[32324]: Cluster is now healthy 2024-04-26T22:52:32.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:31 smithi080 ceph-mon[32324]: from='client.? 172.21.15.94:0/4260877038' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "effb8164-0b8e-425f-af66-624c2909eb96"}]: dispatch 2024-04-26T22:52:32.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:31 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "effb8164-0b8e-425f-af66-624c2909eb96"}]: dispatch 2024-04-26T22:52:32.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:31 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "effb8164-0b8e-425f-af66-624c2909eb96"}]': finished 2024-04-26T22:52:32.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:31 smithi080 ceph-mon[32324]: osdmap e56: 9 total, 8 up, 9 in 2024-04-26T22:52:32.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:31 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:52:32.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:31 smithi094 ceph-mon[32889]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:32.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:31 smithi094 ceph-mon[32889]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-26T22:52:32.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:31 smithi094 ceph-mon[32889]: Cluster is now healthy 2024-04-26T22:52:32.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:31 smithi094 ceph-mon[32889]: from='client.? 172.21.15.94:0/4260877038' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "effb8164-0b8e-425f-af66-624c2909eb96"}]: dispatch 2024-04-26T22:52:32.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:31 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "effb8164-0b8e-425f-af66-624c2909eb96"}]: dispatch 2024-04-26T22:52:32.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:31 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "effb8164-0b8e-425f-af66-624c2909eb96"}]': finished 2024-04-26T22:52:32.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:31 smithi094 ceph-mon[32889]: osdmap e56: 9 total, 8 up, 9 in 2024-04-26T22:52:32.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:31 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:52:32.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:32 smithi027 ceph-mon[28034]: from='client.? 172.21.15.94:0/3557334324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:52:33.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:32 smithi080 ceph-mon[32324]: from='client.? 172.21.15.94:0/3557334324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:52:33.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:32 smithi094 ceph-mon[32889]: from='client.? 172.21.15.94:0/3557334324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:52:33.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:33 smithi027 ceph-mon[28034]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:34.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:33 smithi080 ceph-mon[32324]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:34.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:33 smithi094 ceph-mon[32889]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:35.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:35 smithi027 ceph-mon[28034]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:35.985 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:35 smithi094 ceph-mon[32889]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:36.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:35 smithi080 ceph-mon[32324]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:37.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:37 smithi027 ceph-mon[28034]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:38.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:37 smithi080 ceph-mon[32324]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:38.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:37 smithi094 ceph-mon[32889]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:38.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:38 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T22:52:38.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:38 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:39.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:38 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T22:52:39.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:38 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:39.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:38 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T22:52:39.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:38 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:39.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:39 smithi094 ceph-mon[32889]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:39.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:39 smithi094 ceph-mon[32889]: Deploying daemon osd.8 on smithi094 2024-04-26T22:52:40.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:39 smithi080 ceph-mon[32324]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:40.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:39 smithi080 ceph-mon[32324]: Deploying daemon osd.8 on smithi094 2024-04-26T22:52:40.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:39 smithi027 ceph-mon[28034]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:40.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:39 smithi027 ceph-mon[28034]: Deploying daemon osd.8 on smithi094 2024-04-26T22:52:41.985 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:41 smithi094 ceph-mon[32889]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:42.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:41 smithi080 ceph-mon[32324]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:42.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:41 smithi027 ceph-mon[28034]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:43.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:43 smithi094 ceph-mon[32889]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:44.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:43 smithi080 ceph-mon[32324]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:44.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:43 smithi027 ceph-mon[28034]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:45.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:45 smithi080 ceph-mon[32324]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:45.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:45 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:45.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:45 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:45.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:45 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:52:45.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:45 smithi094 ceph-mon[32889]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:45.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:45 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:45.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:45 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:45.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:45 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:52:45.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:45 smithi027 ceph-mon[28034]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:45.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:45 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:45.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:45 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:45.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:45 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:52:47.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:47 smithi080 ceph-mon[32324]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:47.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:47 smithi094 ceph-mon[32889]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:47.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:47 smithi027 ceph-mon[28034]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:48.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:48.629+0000 7fb94b7fe700 1 -- 172.21.15.94:0/3717251039 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb95c063860 con 0x7fb9440a3c40 2024-04-26T22:52:48.629 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 8 on host 'smithi094' 2024-04-26T22:52:48.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:48.631+0000 7fb960d9e700 1 -- 172.21.15.94:0/3717251039 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb9440a3c40 msgr2=0x7fb9440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:52:48.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:48.631+0000 7fb960d9e700 1 --2- 172.21.15.94:0/3717251039 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb9440a3c40 0x7fb9440a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb9500097b0 tx=0x7fb950006d20 comp rx=0 tx=0).stop 2024-04-26T22:52:48.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:48.631+0000 7fb960d9e700 1 -- 172.21.15.94:0/3717251039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb95c103f80 msgr2=0x7fb95c199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:52:48.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:48.631+0000 7fb960d9e700 1 --2- 172.21.15.94:0/3717251039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb95c103f80 0x7fb95c199680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00d9b0 tx=0x7fb94c00dcc0 comp rx=0 tx=0).stop 2024-04-26T22:52:48.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:48.631+0000 7fb960d9e700 1 -- 172.21.15.94:0/3717251039 shutdown_connections 2024-04-26T22:52:48.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:48.631+0000 7fb960d9e700 1 --2- 172.21.15.94:0/3717251039 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb9440a3c40 0x7fb9440a6100 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:48.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:48.631+0000 7fb960d9e700 1 --2- 172.21.15.94:0/3717251039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c1091e0 0x7fb95c1a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:48.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:48.631+0000 7fb960d9e700 1 --2- 172.21.15.94:0/3717251039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb95c103f80 0x7fb95c199680 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:48.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:48.631+0000 7fb960d9e700 1 --2- 172.21.15.94:0/3717251039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb95c1068b0 0x7fb95c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:48.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:48.631+0000 7fb960d9e700 1 -- 172.21.15.94:0/3717251039 >> 172.21.15.94:0/3717251039 conn(0x7fb95c0fd950 msgr2=0x7fb95c104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:52:48.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:48.631+0000 7fb960d9e700 1 -- 172.21.15.94:0/3717251039 shutdown_connections 2024-04-26T22:52:48.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:52:48.631+0000 7fb960d9e700 1 -- 172.21.15.94:0/3717251039 wait complete. 2024-04-26T22:52:49.000 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:49.000 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:49.000 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:49.000 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:49.000 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:49.001 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:48 smithi094 ceph-mon[32889]: from='osd.8 [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T22:52:49.001 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:48 smithi094 ceph-mon[32889]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T22:52:49.001 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:49.001 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:49.001 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:52:49.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:49.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:49.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:49.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:49.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:49.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:48 smithi080 ceph-mon[32324]: from='osd.8 [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T22:52:49.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:48 smithi080 ceph-mon[32324]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T22:52:49.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:49.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:49.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:52:49.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:49.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:49.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:49.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:49.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:49.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:48 smithi027 ceph-mon[28034]: from='osd.8 [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T22:52:49.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:48 smithi027 ceph-mon[28034]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T22:52:49.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:49.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:49.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:52:49.460 DEBUG:teuthology.orchestra.run.smithi094:osd.8> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.8.service 2024-04-26T22:52:49.463 INFO:tasks.cephadm:Deploying osd.9 on smithi027 with /dev/vg_nvme/lv_1... 2024-04-26T22:52:49.463 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T22:52:49.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:49 smithi027 ceph-mon[28034]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:49.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:49 smithi027 ceph-mon[28034]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T22:52:49.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:49 smithi027 ceph-mon[28034]: from='osd.8 [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:52:49.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:49 smithi027 ceph-mon[28034]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T22:52:49.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:49 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:52:49.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:49 smithi027 ceph-mon[28034]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:52:49.936 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:52:49.963 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:49 smithi094 ceph-mon[32889]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:49.963 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:49 smithi094 ceph-mon[32889]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T22:52:49.963 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:49 smithi094 ceph-mon[32889]: from='osd.8 [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:52:49.963 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:49 smithi094 ceph-mon[32889]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T22:52:49.963 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:49 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:52:49.963 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:49 smithi094 ceph-mon[32889]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:52:50.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:49 smithi080 ceph-mon[32324]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:50.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:49 smithi080 ceph-mon[32324]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T22:52:50.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:49 smithi080 ceph-mon[32324]: from='osd.8 [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:52:50.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:49 smithi080 ceph-mon[32324]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T22:52:50.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:49 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:52:50.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:49 smithi080 ceph-mon[32324]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:52:50.285 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 22:52:50 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T22:52:50.181+0000 7f87b4149700 -1 osd.8 0 waiting for initial osdmap 2024-04-26T22:52:50.285 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 22:52:50 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T22:52:50.192+0000 7f87abf34700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:52:51.009 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:50 smithi094 ceph-mon[32889]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T22:52:51.010 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:50 smithi094 ceph-mon[32889]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T22:52:51.010 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:50 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:52:51.010 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:50 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:52:51.010 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:50 smithi094 ceph-mon[32889]: from='osd.8 ' entity='osd.8' 2024-04-26T22:52:51.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:50 smithi080 ceph-mon[32324]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T22:52:51.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:50 smithi080 ceph-mon[32324]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T22:52:51.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:50 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:52:51.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:50 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:52:51.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:50 smithi080 ceph-mon[32324]: from='osd.8 ' entity='osd.8' 2024-04-26T22:52:51.023 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:50 smithi027 ceph-mon[28034]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T22:52:51.024 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:50 smithi027 ceph-mon[28034]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T22:52:51.024 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:50 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:52:51.024 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:50 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:52:51.024 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:50 smithi027 ceph-mon[28034]: from='osd.8 ' entity='osd.8' 2024-04-26T22:52:51.862 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:51 smithi027 ceph-mon[28034]: purged_snaps scrub starts 2024-04-26T22:52:51.862 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:51 smithi027 ceph-mon[28034]: purged_snaps scrub ok 2024-04-26T22:52:51.862 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:51 smithi027 ceph-mon[28034]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:51.862 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:51 smithi027 ceph-mon[28034]: osd.8 [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649] boot 2024-04-26T22:52:51.862 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:51 smithi027 ceph-mon[28034]: osdmap e59: 9 total, 9 up, 9 in 2024-04-26T22:52:51.863 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:51 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:52:51.863 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:51 smithi027 ceph-mon[28034]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T22:52:52.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:51 smithi080 ceph-mon[32324]: purged_snaps scrub starts 2024-04-26T22:52:52.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:51 smithi080 ceph-mon[32324]: purged_snaps scrub ok 2024-04-26T22:52:52.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:51 smithi080 ceph-mon[32324]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:52.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:51 smithi080 ceph-mon[32324]: osd.8 [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649] boot 2024-04-26T22:52:52.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:51 smithi080 ceph-mon[32324]: osdmap e59: 9 total, 9 up, 9 in 2024-04-26T22:52:52.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:51 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:52:52.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:51 smithi080 ceph-mon[32324]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T22:52:52.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:51 smithi094 ceph-mon[32889]: purged_snaps scrub starts 2024-04-26T22:52:52.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:51 smithi094 ceph-mon[32889]: purged_snaps scrub ok 2024-04-26T22:52:52.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:51 smithi094 ceph-mon[32889]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-26T22:52:52.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:51 smithi094 ceph-mon[32889]: osd.8 [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649] boot 2024-04-26T22:52:52.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:51 smithi094 ceph-mon[32889]: osdmap e59: 9 total, 9 up, 9 in 2024-04-26T22:52:52.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:51 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:52:52.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:51 smithi094 ceph-mon[32889]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T22:52:53.222 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T22:52:53.223 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T22:52:53.223 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-04-26T22:52:53.223 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-04-26T22:52:53.223 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0354253 s, 296 MB/s 2024-04-26T22:52:53.223 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-04-26T22:52:53.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:53 smithi080 ceph-mon[32324]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T22:52:53.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:53.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:53.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:53.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:53.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:53.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:53.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:53.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:53.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:53.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:53 smithi094 ceph-mon[32889]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T22:52:53.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:53.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:53.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:53.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:53.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:53.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:53.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:53.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:53.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:53.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:53 smithi027 ceph-mon[28034]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T22:52:53.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:53.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:53.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:53.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:53.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:52:53.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:53.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:53.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:53.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:52:54.171 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_1 2024-04-26T22:52:54.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:54 smithi027 ceph-mon[28034]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:52:54.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:54 smithi027 ceph-mon[28034]: Detected new or changed devices on smithi094 2024-04-26T22:52:54.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:54 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi094 to 5696M 2024-04-26T22:52:54.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:54 smithi027 ceph-mon[28034]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T22:52:54.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:54 smithi080 ceph-mon[32324]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:52:54.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:54 smithi080 ceph-mon[32324]: Detected new or changed devices on smithi094 2024-04-26T22:52:54.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:54 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi094 to 5696M 2024-04-26T22:52:54.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:54 smithi080 ceph-mon[32324]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T22:52:54.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:54 smithi094 ceph-mon[32889]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:52:54.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:54 smithi094 ceph-mon[32889]: Detected new or changed devices on smithi094 2024-04-26T22:52:54.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:54 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi094 to 5696M 2024-04-26T22:52:54.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:54 smithi094 ceph-mon[32889]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T22:52:54.632 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:52:55.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:55 smithi027 ceph-mon[28034]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T22:52:55.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:55 smithi027 ceph-mon[28034]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:52:55.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:55 smithi080 ceph-mon[32324]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T22:52:55.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:55 smithi080 ceph-mon[32324]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:52:55.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:55 smithi094 ceph-mon[32889]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T22:52:55.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:55 smithi094 ceph-mon[32889]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:52:57.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.413+0000 7f02db600700 1 -- 172.21.15.27:0/3026237472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d4107260 msgr2=0x7f02d410db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:52:57.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.413+0000 7f02db600700 1 --2- 172.21.15.27:0/3026237472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d4107260 0x7f02d410db10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00b0d0 tx=0x7f02cc00b3e0 comp rx=0 tx=0).stop 2024-04-26T22:52:57.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.413+0000 7f02db600700 1 -- 172.21.15.27:0/3026237472 shutdown_connections 2024-04-26T22:52:57.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.413+0000 7f02db600700 1 --2- 172.21.15.27:0/3026237472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d4107260 0x7f02d410db10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:57.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.413+0000 7f02db600700 1 --2- 172.21.15.27:0/3026237472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02d41056c0 0x7f02d4105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:57.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.413+0000 7f02db600700 1 --2- 172.21.15.27:0/3026237472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02d41068c0 0x7f02d4106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:57.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.413+0000 7f02db600700 1 -- 172.21.15.27:0/3026237472 >> 172.21.15.27:0/3026237472 conn(0x7f02d4100c80 msgr2=0x7f02d41030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:52:57.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.413+0000 7f02db600700 1 -- 172.21.15.27:0/3026237472 shutdown_connections 2024-04-26T22:52:57.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.413+0000 7f02db600700 1 -- 172.21.15.27:0/3026237472 wait complete. 2024-04-26T22:52:57.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.414+0000 7f02db600700 1 Processor -- start 2024-04-26T22:52:57.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.414+0000 7f02db600700 1 -- start start 2024-04-26T22:52:57.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.414+0000 7f02db600700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02d41056c0 0x7f02d419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:52:57.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.414+0000 7f02db600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d41068c0 0x7f02d419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:52:57.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.414+0000 7f02db600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02d4107260 0x7f02d41a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:52:57.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.414+0000 7f02db600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f02d41111b0 con 0x7f02d41068c0 2024-04-26T22:52:57.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.414+0000 7f02db600700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f02d4111030 con 0x7f02d41056c0 2024-04-26T22:52:57.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.414+0000 7f02db600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f02d4111330 con 0x7f02d4107260 2024-04-26T22:52:57.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.415+0000 7f02d8c00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02d41056c0 0x7f02d419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:52:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.415+0000 7f02d3e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d41068c0 0x7f02d419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:52:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.415+0000 7f02d8c00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02d41056c0 0x7f02d419b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:59490/0 (socket says 172.21.15.27:59490) 2024-04-26T22:52:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.415+0000 7f02d9600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02d4107260 0x7f02d41a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:52:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.415+0000 7f02d8c00700 1 -- 172.21.15.27:0/1675049250 learned_addr learned my addr 172.21.15.27:0/1675049250 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:52:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.415+0000 7f02d8c00700 1 -- 172.21.15.27:0/1675049250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02d4107260 msgr2=0x7f02d41a2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:52:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.415+0000 7f02d8c00700 1 --2- 172.21.15.27:0/1675049250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02d4107260 0x7f02d41a2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.415+0000 7f02d8c00700 1 -- 172.21.15.27:0/1675049250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d41068c0 msgr2=0x7f02d419bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:52:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.415+0000 7f02d8c00700 1 --2- 172.21.15.27:0/1675049250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d41068c0 0x7f02d419bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:52:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.415+0000 7f02d8c00700 1 -- 172.21.15.27:0/1675049250 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d41a33f0 con 0x7f02d41056c0 2024-04-26T22:52:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.416+0000 7f02d8c00700 1 --2- 172.21.15.27:0/1675049250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02d41056c0 0x7f02d419b790 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f02c800d980 tx=0x7f02c800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:52:57.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.416+0000 7f02d1600700 1 -- 172.21.15.27:0/1675049250 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8017070 con 0x7f02d41056c0 2024-04-26T22:52:57.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.416+0000 7f02db600700 1 -- 172.21.15.27:0/1675049250 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d41a36e0 con 0x7f02d41056c0 2024-04-26T22:52:57.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.416+0000 7f02d1600700 1 -- 172.21.15.27:0/1675049250 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c800fd40 con 0x7f02d41056c0 2024-04-26T22:52:57.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.416+0000 7f02d1600700 1 -- 172.21.15.27:0/1675049250 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c800e3b0 con 0x7f02d41056c0 2024-04-26T22:52:57.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.416+0000 7f02db600700 1 -- 172.21.15.27:0/1675049250 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d41a3bd0 con 0x7f02d41056c0 2024-04-26T22:52:57.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.417+0000 7f02db600700 1 -- 172.21.15.27:0/1675049250 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d406a110 con 0x7f02d41056c0 2024-04-26T22:52:57.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.419+0000 7f02d1600700 1 -- 172.21.15.27:0/1675049250 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f02c800f890 con 0x7f02d41056c0 2024-04-26T22:52:57.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.421+0000 7f02d1600700 1 --2- 172.21.15.27:0/1675049250 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f02bc0a3c40 0x7f02bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:52:57.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.421+0000 7f02d1600700 1 -- 172.21.15.27:0/1675049250 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f02c80cf520 con 0x7f02d41056c0 2024-04-26T22:52:57.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.421+0000 7f02d3e00700 1 --2- 172.21.15.27:0/1675049250 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f02bc0a3c40 0x7f02bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:52:57.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.422+0000 7f02d3e00700 1 --2- 172.21.15.27:0/1675049250 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f02bc0a3c40 0x7f02bc0a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f02c0007be0 tx=0x7f02c00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:52:57.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.423+0000 7f02d1600700 1 -- 172.21.15.27:0/1675049250 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f02c809c810 con 0x7f02d41056c0 2024-04-26T22:52:57.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:57 smithi027 ceph-mon[28034]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-04-26T22:52:57.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:52:57.678+0000 7f02db600700 1 -- 172.21.15.27:0/1675049250 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f02d4063860 con 0x7f02bc0a3c40 2024-04-26T22:52:57.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:57 smithi080 ceph-mon[32324]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-04-26T22:52:57.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:57 smithi094 ceph-mon[32889]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-04-26T22:52:58.749 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:58 smithi027 ceph-mon[28034]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:52:58.750 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:52:58.750 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:52:58.750 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:58.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:58 smithi080 ceph-mon[32324]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:52:58.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:52:58.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:52:58.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:58.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:58 smithi094 ceph-mon[32889]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:52:58.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:52:58.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:52:58.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:59.761 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:52:59 smithi027 ceph-mon[28034]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T22:52:59.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:52:59 smithi080 ceph-mon[32324]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T22:52:59.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:52:59 smithi094 ceph-mon[32889]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T22:53:01.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:01 smithi080 ceph-mon[32324]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T22:53:01.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:01 smithi094 ceph-mon[32889]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T22:53:01.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:01 smithi027 ceph-mon[28034]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T22:53:02.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:02 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3732569937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c52e6ee-8407-498e-a0fe-579588406f2f"}]: dispatch 2024-04-26T22:53:02.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:02 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3732569937' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c52e6ee-8407-498e-a0fe-579588406f2f"}]': finished 2024-04-26T22:53:02.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:02 smithi080 ceph-mon[32324]: osdmap e64: 10 total, 9 up, 10 in 2024-04-26T22:53:02.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:02 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:02.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:02 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3732569937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c52e6ee-8407-498e-a0fe-579588406f2f"}]: dispatch 2024-04-26T22:53:02.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:02 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3732569937' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c52e6ee-8407-498e-a0fe-579588406f2f"}]': finished 2024-04-26T22:53:02.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:02 smithi094 ceph-mon[32889]: osdmap e64: 10 total, 9 up, 10 in 2024-04-26T22:53:02.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:02 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:02.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:02 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3732569937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c52e6ee-8407-498e-a0fe-579588406f2f"}]: dispatch 2024-04-26T22:53:02.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:02 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3732569937' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c52e6ee-8407-498e-a0fe-579588406f2f"}]': finished 2024-04-26T22:53:02.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:02 smithi027 ceph-mon[28034]: osdmap e64: 10 total, 9 up, 10 in 2024-04-26T22:53:02.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:02 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:03.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:03 smithi080 ceph-mon[32324]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T22:53:03.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:03 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/520246481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:53:03.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:03 smithi094 ceph-mon[32889]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T22:53:03.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:03 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/520246481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:53:03.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:03 smithi027 ceph-mon[28034]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T22:53:03.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:03 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/520246481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:53:05.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:05 smithi080 ceph-mon[32324]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T22:53:05.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:05 smithi094 ceph-mon[32889]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T22:53:05.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:05 smithi027 ceph-mon[28034]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T22:53:07.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:07 smithi027 ceph-mon[28034]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:07.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:07 smithi080 ceph-mon[32324]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:07.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:07 smithi094 ceph-mon[32889]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:09.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:09 smithi027 ceph-mon[28034]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:09.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:09 smithi080 ceph-mon[32324]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:09.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:09 smithi094 ceph-mon[32889]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:11.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:11 smithi080 ceph-mon[32324]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:11.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:11 smithi094 ceph-mon[32889]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:11.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:11 smithi027 ceph-mon[28034]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:12.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:12 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T22:53:12.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:12 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:12.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:12 smithi080 ceph-mon[32324]: Deploying daemon osd.9 on smithi027 2024-04-26T22:53:12.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:12 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T22:53:12.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:12 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:12.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:12 smithi094 ceph-mon[32889]: Deploying daemon osd.9 on smithi027 2024-04-26T22:53:12.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:12 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T22:53:12.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:12 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:12.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:12 smithi027 ceph-mon[28034]: Deploying daemon osd.9 on smithi027 2024-04-26T22:53:13.634 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:13 smithi027 ceph-mon[28034]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:13.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:13 smithi094 ceph-mon[32889]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:14.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:13 smithi080 ceph-mon[32324]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:15.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:15 smithi094 ceph-mon[32889]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:15.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:15 smithi027 ceph-mon[28034]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:16.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:15 smithi080 ceph-mon[32324]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:17.774 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:17 smithi027 ceph-mon[28034]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:18.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:17 smithi080 ceph-mon[32324]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:18.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:17 smithi094 ceph-mon[32889]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:19.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:19 smithi027 ceph-mon[28034]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:20.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:19 smithi080 ceph-mon[32324]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:20.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:19 smithi094 ceph-mon[32889]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:21.836 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:21 smithi094 ceph-mon[32889]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:21.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:21 smithi027 ceph-mon[28034]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:21 smithi080 ceph-mon[32324]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:23.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:23.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:23.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:53:23.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:23.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:23.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:53:23.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:23.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:23.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:53:24.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:23 smithi080 ceph-mon[32324]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:24.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:23 smithi094 ceph-mon[32889]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:24.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:23 smithi027 ceph-mon[28034]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:26.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:25 smithi080 ceph-mon[32324]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:26.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:25 smithi094 ceph-mon[32889]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:26.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:25 smithi027 ceph-mon[28034]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:27.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:26 smithi080 ceph-mon[32324]: from='osd.9 [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T22:53:27.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:27.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:26 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:27.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:26 smithi094 ceph-mon[32889]: from='osd.9 [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T22:53:27.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:27.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:26 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:27.104 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:26 smithi027 ceph-mon[28034]: from='osd.9 [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T22:53:27.104 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:27.104 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:26 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:27.290 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 9 on host 'smithi027' 2024-04-26T22:53:27.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:53:27.291+0000 7f02d1600700 1 -- 172.21.15.27:0/1675049250 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f02d4063860 con 0x7f02bc0a3c40 2024-04-26T22:53:27.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:53:27.294+0000 7f02db600700 1 -- 172.21.15.27:0/1675049250 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f02bc0a3c40 msgr2=0x7f02bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:53:27.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:53:27.294+0000 7f02db600700 1 --2- 172.21.15.27:0/1675049250 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f02bc0a3c40 0x7f02bc0a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f02c0007be0 tx=0x7f02c00073d0 comp rx=0 tx=0).stop 2024-04-26T22:53:27.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:53:27.294+0000 7f02db600700 1 -- 172.21.15.27:0/1675049250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02d41056c0 msgr2=0x7f02d419b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:53:27.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:53:27.294+0000 7f02db600700 1 --2- 172.21.15.27:0/1675049250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02d41056c0 0x7f02d419b790 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f02c800d980 tx=0x7f02c800dc90 comp rx=0 tx=0).stop 2024-04-26T22:53:27.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:53:27.294+0000 7f02db600700 1 -- 172.21.15.27:0/1675049250 shutdown_connections 2024-04-26T22:53:27.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:53:27.294+0000 7f02db600700 1 --2- 172.21.15.27:0/1675049250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02d4107260 0x7f02d41a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:27.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:53:27.294+0000 7f02db600700 1 --2- 172.21.15.27:0/1675049250 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f02bc0a3c40 0x7f02bc0a6100 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:27.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:53:27.294+0000 7f02db600700 1 --2- 172.21.15.27:0/1675049250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02d41056c0 0x7f02d419b790 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:27.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:53:27.294+0000 7f02db600700 1 --2- 172.21.15.27:0/1675049250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d41068c0 0x7f02d419bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:27.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:53:27.294+0000 7f02db600700 1 -- 172.21.15.27:0/1675049250 >> 172.21.15.27:0/1675049250 conn(0x7f02d4100c80 msgr2=0x7f02d4109b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:53:27.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:53:27.294+0000 7f02db600700 1 -- 172.21.15.27:0/1675049250 shutdown_connections 2024-04-26T22:53:27.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:53:27.294+0000 7f02db600700 1 -- 172.21.15.27:0/1675049250 wait complete. 2024-04-26T22:53:28.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:27 smithi080 ceph-mon[32324]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:28.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:27 smithi080 ceph-mon[32324]: from='osd.9 [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T22:53:28.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:27 smithi080 ceph-mon[32324]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T22:53:28.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:27 smithi080 ceph-mon[32324]: from='osd.9 [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T22:53:28.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:53:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:28.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:53:28.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:27 smithi094 ceph-mon[32889]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:28.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:27 smithi094 ceph-mon[32889]: from='osd.9 [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T22:53:28.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:27 smithi094 ceph-mon[32889]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T22:53:28.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:27 smithi094 ceph-mon[32889]: from='osd.9 [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T22:53:28.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:28.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:28.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:53:28.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:28.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:28.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:28.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:53:28.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:27 smithi027 ceph-mon[28034]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:28.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:27 smithi027 ceph-mon[28034]: from='osd.9 [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T22:53:28.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:27 smithi027 ceph-mon[28034]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T22:53:28.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:27 smithi027 ceph-mon[28034]: from='osd.9 [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T22:53:28.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:28.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:28.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:53:28.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:28.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:28.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:28.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:53:28.318 DEBUG:teuthology.orchestra.run.smithi027:osd.9> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.9.service 2024-04-26T22:53:28.326 INFO:tasks.cephadm:Deploying osd.10 on smithi080 with /dev/vg_nvme/lv_1... 2024-04-26T22:53:28.326 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T22:53:28.574 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T22:53:28.835 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:28 smithi027 ceph-mon[28034]: from='osd.9 [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T22:53:28.835 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:28 smithi027 ceph-mon[28034]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T22:53:28.835 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:28 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:28.836 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:28 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:28.956 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:28 smithi080 ceph-mon[32324]: from='osd.9 [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T22:53:28.956 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:28 smithi080 ceph-mon[32324]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T22:53:28.956 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:28 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:28.956 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:28 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:29.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:28 smithi094 ceph-mon[32889]: from='osd.9 [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T22:53:29.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:28 smithi094 ceph-mon[32889]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T22:53:29.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:28 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:29.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:28 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:30.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:29 smithi080 ceph-mon[32324]: purged_snaps scrub starts 2024-04-26T22:53:30.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:29 smithi080 ceph-mon[32324]: purged_snaps scrub ok 2024-04-26T22:53:30.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:29 smithi080 ceph-mon[32324]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:30.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:30.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:29 smithi080 ceph-mon[32324]: osd.9 [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976] boot 2024-04-26T22:53:30.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:29 smithi080 ceph-mon[32324]: osdmap e67: 10 total, 10 up, 10 in 2024-04-26T22:53:30.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:29 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:30.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:29 smithi094 ceph-mon[32889]: purged_snaps scrub starts 2024-04-26T22:53:30.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:29 smithi094 ceph-mon[32889]: purged_snaps scrub ok 2024-04-26T22:53:30.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:29 smithi094 ceph-mon[32889]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:30.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:30.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:29 smithi094 ceph-mon[32889]: osd.9 [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976] boot 2024-04-26T22:53:30.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:29 smithi094 ceph-mon[32889]: osdmap e67: 10 total, 10 up, 10 in 2024-04-26T22:53:30.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:29 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:30.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:29 smithi027 ceph-mon[28034]: purged_snaps scrub starts 2024-04-26T22:53:30.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:29 smithi027 ceph-mon[28034]: purged_snaps scrub ok 2024-04-26T22:53:30.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:29 smithi027 ceph-mon[28034]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-26T22:53:30.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:30.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:29 smithi027 ceph-mon[28034]: osd.9 [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976] boot 2024-04-26T22:53:30.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:29 smithi027 ceph-mon[28034]: osdmap e67: 10 total, 10 up, 10 in 2024-04-26T22:53:30.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:29 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:53:30.645 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T22:53:30.645 INFO:teuthology.orchestra.run.smithi080.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T22:53:30.645 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10+0 records in 2024-04-26T22:53:30.645 INFO:teuthology.orchestra.run.smithi080.stderr:10+0 records out 2024-04-26T22:53:30.645 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011275 s, 930 MB/s 2024-04-26T22:53:30.645 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping successful for: 2024-04-26T22:53:31.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:30 smithi027 ceph-mon[28034]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T22:53:31.269 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:30 smithi080 ceph-mon[32324]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T22:53:31.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:30 smithi094 ceph-mon[32889]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T22:53:31.298 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi080:vg_nvme/lv_1 2024-04-26T22:53:31.536 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T22:53:31.923 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:31 smithi080 ceph-mon[32324]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:32.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:31 smithi027 ceph-mon[28034]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:32.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:31 smithi094 ceph-mon[32889]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:33.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.253+0000 7f220fbe1700 1 -- 172.21.15.80:0/368144806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22081023a0 msgr2=0x7f22081027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:53:33.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.253+0000 7f220fbe1700 1 --2- 172.21.15.80:0/368144806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22081023a0 0x7f22081027c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f21f80099c0 tx=0x7f21f8009cd0 comp rx=0 tx=0).stop 2024-04-26T22:53:33.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.253+0000 7f220fbe1700 1 -- 172.21.15.80:0/368144806 shutdown_connections 2024-04-26T22:53:33.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.253+0000 7f220fbe1700 1 --2- 172.21.15.80:0/368144806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2208103f40 0x7f220810a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:33.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.253+0000 7f220fbe1700 1 --2- 172.21.15.80:0/368144806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22081023a0 0x7f22081027c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:33.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.253+0000 7f220fbe1700 1 --2- 172.21.15.80:0/368144806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22081035a0 0x7f2208103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:33.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.253+0000 7f220fbe1700 1 -- 172.21.15.80:0/368144806 >> 172.21.15.80:0/368144806 conn(0x7f22080fd980 msgr2=0x7f22080ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:53:33.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.253+0000 7f220fbe1700 1 -- 172.21.15.80:0/368144806 shutdown_connections 2024-04-26T22:53:33.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.253+0000 7f220fbe1700 1 -- 172.21.15.80:0/368144806 wait complete. 2024-04-26T22:53:33.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.254+0000 7f220fbe1700 1 Processor -- start 2024-04-26T22:53:33.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.254+0000 7f220fbe1700 1 -- start start 2024-04-26T22:53:33.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.254+0000 7f220fbe1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22081023a0 0x7f22081996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:53:33.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.254+0000 7f220fbe1700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22081035a0 0x7f2208199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:53:33.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.254+0000 7f220fbe1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2208103f40 0x7f22081a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:53:33.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.254+0000 7f220fbe1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f220810deb0 con 0x7f2208103f40 2024-04-26T22:53:33.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.254+0000 7f220fbe1700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f220810dd30 con 0x7f22081035a0 2024-04-26T22:53:33.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.254+0000 7f220fbe1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f220810e030 con 0x7f22081023a0 2024-04-26T22:53:33.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.254+0000 7f220d17c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22081035a0 0x7f2208199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:53:33.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.254+0000 7f220d97d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22081023a0 0x7f22081996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:53:33.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.254+0000 7f220d17c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22081035a0 0x7f2208199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54056/0 (socket says 172.21.15.80:54056) 2024-04-26T22:53:33.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.254+0000 7f220d17c700 1 -- 172.21.15.80:0/1995417792 learned_addr learned my addr 172.21.15.80:0/1995417792 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-26T22:53:33.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.255+0000 7f220d17c700 1 -- 172.21.15.80:0/1995417792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22081023a0 msgr2=0x7f22081996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:53:33.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.255+0000 7f220d17c700 1 --2- 172.21.15.80:0/1995417792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22081023a0 0x7f22081996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:33.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.255+0000 7f220d17c700 1 -- 172.21.15.80:0/1995417792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2208103f40 msgr2=0x7f22081a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:53:33.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.255+0000 7f220d17c700 1 --2- 172.21.15.80:0/1995417792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2208103f40 0x7f22081a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:33.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.255+0000 7f220d17c700 1 -- 172.21.15.80:0/1995417792 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22081a1300 con 0x7f22081035a0 2024-04-26T22:53:33.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.255+0000 7f220d17c700 1 --2- 172.21.15.80:0/1995417792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22081035a0 0x7f2208199be0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f220400b530 tx=0x7f220400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:53:33.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.255+0000 7f21feffd700 1 -- 172.21.15.80:0/1995417792 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204015020 con 0x7f22081035a0 2024-04-26T22:53:33.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.255+0000 7f21feffd700 1 -- 172.21.15.80:0/1995417792 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2204011ce0 con 0x7f22081035a0 2024-04-26T22:53:33.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.255+0000 7f21feffd700 1 -- 172.21.15.80:0/1995417792 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220400f2e0 con 0x7f22081035a0 2024-04-26T22:53:33.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.255+0000 7f220fbe1700 1 -- 172.21.15.80:0/1995417792 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22081a15f0 con 0x7f22081035a0 2024-04-26T22:53:33.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.255+0000 7f220fbe1700 1 -- 172.21.15.80:0/1995417792 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22081a1b50 con 0x7f22081035a0 2024-04-26T22:53:33.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.256+0000 7f220fbe1700 1 -- 172.21.15.80:0/1995417792 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22081a1e70 con 0x7f22081035a0 2024-04-26T22:53:33.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.257+0000 7f21feffd700 1 -- 172.21.15.80:0/1995417792 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2204018020 con 0x7f22081035a0 2024-04-26T22:53:33.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.258+0000 7f21feffd700 1 --2- 172.21.15.80:0/1995417792 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f21f40a3c40 0x7f21f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:53:33.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.258+0000 7f21feffd700 1 -- 172.21.15.80:0/1995417792 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f22040cf9b0 con 0x7f22081035a0 2024-04-26T22:53:33.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.258+0000 7f220d97d700 1 --2- 172.21.15.80:0/1995417792 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f21f40a3c40 0x7f21f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:53:33.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.259+0000 7f220d97d700 1 --2- 172.21.15.80:0/1995417792 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f21f40a3c40 0x7f21f40a6100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f21f8006010 tx=0x7f21f8003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:53:33.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.262+0000 7f21feffd700 1 -- 172.21.15.80:0/1995417792 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f220401f020 con 0x7f22081035a0 2024-04-26T22:53:33.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:33.466+0000 7f220fbe1700 1 -- 172.21.15.80:0/1995417792 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f22080ffba0 con 0x7f21f40a3c40 2024-04-26T22:53:34.095 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:33 smithi080 ceph-mon[32324]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:34.095 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:53:34.095 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:53:34.096 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:33 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:34.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:33 smithi027 ceph-mon[28034]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:34.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:53:34.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:53:34.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:33 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:34.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:33 smithi094 ceph-mon[32889]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:34.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:53:34.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:53:34.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:33 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:35.369 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:35 smithi080 ceph-mon[32324]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:53:35.370 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:35 smithi080 ceph-mon[32324]: Detected new or changed devices on smithi027 2024-04-26T22:53:35.370 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:35.370 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:35.370 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:35.370 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:35.370 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:35.370 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:35.370 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:35.370 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:35.370 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:53:35.370 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:35 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:35.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:35 smithi094 ceph-mon[32889]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:53:35.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:35 smithi094 ceph-mon[32889]: Detected new or changed devices on smithi027 2024-04-26T22:53:35.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:35.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:35.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:35.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:35.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:35.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:35.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:35.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:35.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:53:35.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:35 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:35.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:35 smithi027 ceph-mon[28034]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:53:35.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:35 smithi027 ceph-mon[28034]: Detected new or changed devices on smithi027 2024-04-26T22:53:35.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:35.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:35.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:35.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:35.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:35.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:35.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:35.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:35.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:53:35.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:35 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:36.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:36 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi027 to 4272M 2024-04-26T22:53:36.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:36 smithi080 ceph-mon[32324]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:36.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:36 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi027 to 4272M 2024-04-26T22:53:36.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:36 smithi094 ceph-mon[32889]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:36.541 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:36 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi027 to 4272M 2024-04-26T22:53:36.542 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:36 smithi027 ceph-mon[28034]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:37.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:37 smithi080 ceph-mon[32324]: from='client.? 172.21.15.80:0/1801211551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33af8d56-6def-4c77-9fb2-8d2837c18c95"}]: dispatch 2024-04-26T22:53:37.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:37 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33af8d56-6def-4c77-9fb2-8d2837c18c95"}]: dispatch 2024-04-26T22:53:37.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:37 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33af8d56-6def-4c77-9fb2-8d2837c18c95"}]': finished 2024-04-26T22:53:37.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:37 smithi080 ceph-mon[32324]: osdmap e69: 11 total, 10 up, 11 in 2024-04-26T22:53:37.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:37 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:53:37.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:37 smithi080 ceph-mon[32324]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:37.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:37 smithi080 ceph-mon[32324]: from='client.? 172.21.15.80:0/468290441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:53:37.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:37 smithi094 ceph-mon[32889]: from='client.? 172.21.15.80:0/1801211551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33af8d56-6def-4c77-9fb2-8d2837c18c95"}]: dispatch 2024-04-26T22:53:37.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:37 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33af8d56-6def-4c77-9fb2-8d2837c18c95"}]: dispatch 2024-04-26T22:53:37.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:37 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33af8d56-6def-4c77-9fb2-8d2837c18c95"}]': finished 2024-04-26T22:53:37.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:37 smithi094 ceph-mon[32889]: osdmap e69: 11 total, 10 up, 11 in 2024-04-26T22:53:37.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:37 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:53:37.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:37 smithi094 ceph-mon[32889]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:37.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:37 smithi094 ceph-mon[32889]: from='client.? 172.21.15.80:0/468290441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:53:37.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:37 smithi027 ceph-mon[28034]: from='client.? 172.21.15.80:0/1801211551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33af8d56-6def-4c77-9fb2-8d2837c18c95"}]: dispatch 2024-04-26T22:53:37.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:37 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33af8d56-6def-4c77-9fb2-8d2837c18c95"}]: dispatch 2024-04-26T22:53:37.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:37 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33af8d56-6def-4c77-9fb2-8d2837c18c95"}]': finished 2024-04-26T22:53:37.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:37 smithi027 ceph-mon[28034]: osdmap e69: 11 total, 10 up, 11 in 2024-04-26T22:53:37.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:37 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:53:37.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:37 smithi027 ceph-mon[28034]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:37.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:37 smithi027 ceph-mon[28034]: from='client.? 172.21.15.80:0/468290441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:53:39.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:39 smithi027 ceph-mon[28034]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:39.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:39 smithi080 ceph-mon[32324]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:39.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:39 smithi094 ceph-mon[32889]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:41.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:41 smithi080 ceph-mon[32324]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:41.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:41 smithi094 ceph-mon[32889]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:41.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:41 smithi027 ceph-mon[28034]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:43.586 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:43 smithi080 ceph-mon[32324]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:43.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:43 smithi094 ceph-mon[32889]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:43.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:43 smithi027 ceph-mon[28034]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:44.510 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:44 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T22:53:44.510 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:44 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:44.510 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:44 smithi080 ceph-mon[32324]: Deploying daemon osd.10 on smithi080 2024-04-26T22:53:44.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:44 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T22:53:44.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:44 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:44.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:44 smithi094 ceph-mon[32889]: Deploying daemon osd.10 on smithi080 2024-04-26T22:53:44.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:44 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T22:53:44.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:44 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:44.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:44 smithi027 ceph-mon[28034]: Deploying daemon osd.10 on smithi080 2024-04-26T22:53:45.588 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:45 smithi080 ceph-mon[32324]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:45.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:45 smithi094 ceph-mon[32889]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:45.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:45 smithi027 ceph-mon[28034]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:47.719 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:47 smithi080 ceph-mon[32324]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:47.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:47 smithi094 ceph-mon[32889]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:47.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:47 smithi027 ceph-mon[28034]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:49.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:49 smithi080 ceph-mon[32324]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:49.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:49 smithi094 ceph-mon[32889]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:49.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:49 smithi027 ceph-mon[28034]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:50.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:50 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:50.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:50 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:50.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:50 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:53:50.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:50 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:50.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:50 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:50.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:50 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:53:50.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:50 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:50.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:50 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:50.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:50 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:53:51.689 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:51 smithi080 ceph-mon[32324]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:51.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:51 smithi094 ceph-mon[32889]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:51.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:51 smithi027 ceph-mon[28034]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:53.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:53.654+0000 7f21feffd700 1 -- 172.21.15.80:0/1995417792 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f22080ffba0 con 0x7f21f40a3c40 2024-04-26T22:53:53.655 INFO:teuthology.orchestra.run.smithi080.stdout:Created osd(s) 10 on host 'smithi080' 2024-04-26T22:53:53.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:53.656+0000 7f220fbe1700 1 -- 172.21.15.80:0/1995417792 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f21f40a3c40 msgr2=0x7f21f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:53:53.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:53.656+0000 7f220fbe1700 1 --2- 172.21.15.80:0/1995417792 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f21f40a3c40 0x7f21f40a6100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f21f8006010 tx=0x7f21f8003960 comp rx=0 tx=0).stop 2024-04-26T22:53:53.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:53.656+0000 7f220fbe1700 1 -- 172.21.15.80:0/1995417792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22081035a0 msgr2=0x7f2208199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:53:53.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:53.656+0000 7f220fbe1700 1 --2- 172.21.15.80:0/1995417792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22081035a0 0x7f2208199be0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f220400b530 tx=0x7f220400b840 comp rx=0 tx=0).stop 2024-04-26T22:53:53.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:53.656+0000 7f220fbe1700 1 -- 172.21.15.80:0/1995417792 shutdown_connections 2024-04-26T22:53:53.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:53.656+0000 7f220fbe1700 1 --2- 172.21.15.80:0/1995417792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2208103f40 0x7f22081a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:53.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:53.656+0000 7f220fbe1700 1 --2- 172.21.15.80:0/1995417792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22081023a0 0x7f22081996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:53.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:53.656+0000 7f220fbe1700 1 --2- 172.21.15.80:0/1995417792 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f21f40a3c40 0x7f21f40a6100 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:53.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:53.656+0000 7f220fbe1700 1 --2- 172.21.15.80:0/1995417792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22081035a0 0x7f2208199be0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:53.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:53.656+0000 7f220fbe1700 1 -- 172.21.15.80:0/1995417792 >> 172.21.15.80:0/1995417792 conn(0x7f22080fd980 msgr2=0x7f22080ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:53:53.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:53.656+0000 7f220fbe1700 1 -- 172.21.15.80:0/1995417792 shutdown_connections 2024-04-26T22:53:53.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-26T22:53:53.656+0000 7f220fbe1700 1 -- 172.21.15.80:0/1995417792 wait complete. 2024-04-26T22:53:53.973 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:53 smithi080 ceph-mon[32324]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:53.974 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:53.974 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:53.974 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:53.974 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:53:53.974 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:53.974 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:53 smithi080 ceph-mon[32324]: from='osd.10 [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T22:53:53.974 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:53 smithi080 ceph-mon[32324]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T22:53:53.974 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:53.974 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:53.974 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:53:54.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:53 smithi094 ceph-mon[32889]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:54.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:54.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:54.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:54.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:53:54.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:54.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:53 smithi094 ceph-mon[32889]: from='osd.10 [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T22:53:54.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:53 smithi094 ceph-mon[32889]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T22:53:54.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:54.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:54.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:53:54.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:53 smithi027 ceph-mon[28034]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:54.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:54.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:54.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:54.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:53:54.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:54.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:53 smithi027 ceph-mon[28034]: from='osd.10 [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T22:53:54.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:53 smithi027 ceph-mon[28034]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T22:53:54.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:54.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:54.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:53:54.305 DEBUG:teuthology.orchestra.run.smithi080:osd.10> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.10.service 2024-04-26T22:53:54.307 INFO:tasks.cephadm:Deploying osd.11 on smithi094 with /dev/vg_nvme/lv_1... 2024-04-26T22:53:54.307 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T22:53:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:53:54.948 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:54 smithi094 ceph-mon[32889]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T22:53:54.948 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:54 smithi094 ceph-mon[32889]: from='osd.10 [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:53:54.948 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:54 smithi094 ceph-mon[32889]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T22:53:54.948 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:54 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:53:54.948 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:54 smithi094 ceph-mon[32889]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:53:55.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:54 smithi080 ceph-mon[32324]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T22:53:55.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:54 smithi080 ceph-mon[32324]: from='osd.10 [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:53:55.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:54 smithi080 ceph-mon[32324]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T22:53:55.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:54 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:53:55.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:54 smithi080 ceph-mon[32324]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:53:55.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 22:53:54 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T22:53:54.784+0000 7fc1abe8b700 -1 osd.10 0 waiting for initial osdmap 2024-04-26T22:53:55.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 22:53:54 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T22:53:54.797+0000 7fc1a7c7e700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:53:55.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:54 smithi027 ceph-mon[28034]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T22:53:55.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:54 smithi027 ceph-mon[28034]: from='osd.10 [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:53:55.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:54 smithi027 ceph-mon[28034]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T22:53:55.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:54 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:53:55.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:54 smithi027 ceph-mon[28034]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-26T22:53:55.994 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:55 smithi094 ceph-mon[32889]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:55.995 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:55 smithi094 ceph-mon[32889]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-26T22:53:55.995 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:55 smithi094 ceph-mon[32889]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T22:53:55.995 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:55 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:53:55.995 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:55 smithi094 ceph-mon[32889]: osd.10 [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391] boot 2024-04-26T22:53:55.995 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:55 smithi094 ceph-mon[32889]: osdmap e72: 11 total, 11 up, 11 in 2024-04-26T22:53:55.995 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:55 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:53:56.112 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:55 smithi080 ceph-mon[32324]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:56.112 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:55 smithi080 ceph-mon[32324]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-26T22:53:56.112 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:55 smithi080 ceph-mon[32324]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T22:53:56.112 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:55 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:53:56.113 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:55 smithi080 ceph-mon[32324]: osd.10 [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391] boot 2024-04-26T22:53:56.113 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:55 smithi080 ceph-mon[32324]: osdmap e72: 11 total, 11 up, 11 in 2024-04-26T22:53:56.113 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:55 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:53:56.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:55 smithi027 ceph-mon[28034]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-26T22:53:56.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:55 smithi027 ceph-mon[28034]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-26T22:53:56.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:55 smithi027 ceph-mon[28034]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T22:53:56.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:55 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:53:56.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:55 smithi027 ceph-mon[28034]: osd.10 [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391] boot 2024-04-26T22:53:56.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:55 smithi027 ceph-mon[28034]: osdmap e72: 11 total, 11 up, 11 in 2024-04-26T22:53:56.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:55 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:53:56.726 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T22:53:56.726 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T22:53:56.727 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-26T22:53:56.727 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-26T22:53:56.727 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00856909 s, 1.2 GB/s 2024-04-26T22:53:56.727 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-26T22:53:57.388 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:57 smithi094 ceph-mon[32889]: purged_snaps scrub starts 2024-04-26T22:53:57.388 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:57 smithi094 ceph-mon[32889]: purged_snaps scrub ok 2024-04-26T22:53:57.389 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:57 smithi094 ceph-mon[32889]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T22:53:57.415 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_1 2024-04-26T22:53:57.434 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:57 smithi080 ceph-mon[32324]: purged_snaps scrub starts 2024-04-26T22:53:57.434 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:57 smithi080 ceph-mon[32324]: purged_snaps scrub ok 2024-04-26T22:53:57.434 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:57 smithi080 ceph-mon[32324]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T22:53:57.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:57 smithi027 ceph-mon[28034]: purged_snaps scrub starts 2024-04-26T22:53:57.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:57 smithi027 ceph-mon[28034]: purged_snaps scrub ok 2024-04-26T22:53:57.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:57 smithi027 ceph-mon[28034]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T22:53:57.655 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:53:58.505 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:58 smithi094 ceph-mon[32889]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:53:58.505 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:58 smithi094 ceph-mon[32889]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T22:53:58.505 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:58 smithi094 ceph-mon[32889]: Detected new or changed devices on smithi080 2024-04-26T22:53:58.505 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:58.505 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:58.505 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:58.505 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:58.505 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:58.506 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:58.506 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:58 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi080 to 4272M 2024-04-26T22:53:58.506 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:58.506 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:58.506 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:53:58.506 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:58 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:58.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:58 smithi080 ceph-mon[32324]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:53:58.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:58 smithi080 ceph-mon[32324]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T22:53:58.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:58 smithi080 ceph-mon[32324]: Detected new or changed devices on smithi080 2024-04-26T22:53:58.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:58.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:58.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:58.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:58.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:58.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:58.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:58 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi080 to 4272M 2024-04-26T22:53:58.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:58.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:58.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:53:58.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:58 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:58.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:58 smithi027 ceph-mon[28034]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:53:58.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:58 smithi027 ceph-mon[28034]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T22:53:58.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:58 smithi027 ceph-mon[28034]: Detected new or changed devices on smithi080 2024-04-26T22:53:58.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:58.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:58.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:58.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:58.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:53:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:58 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi080 to 4272M 2024-04-26T22:53:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:53:58.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:58 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:53:59.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.432+0000 7fa2dfe6b700 1 -- 172.21.15.94:0/2054572476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2d8104710 msgr2=0x7fa2d8104b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:53:59.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.432+0000 7fa2dfe6b700 1 --2- 172.21.15.94:0/2054572476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2d8104710 0x7fa2d8104b70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009990 tx=0x7fa2cc009ca0 comp rx=0 tx=0).stop 2024-04-26T22:53:59.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.432+0000 7fa2dfe6b700 1 -- 172.21.15.94:0/2054572476 shutdown_connections 2024-04-26T22:53:59.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.432+0000 7fa2dfe6b700 1 --2- 172.21.15.94:0/2054572476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2d81050b0 0x7fa2d810b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:59.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.432+0000 7fa2dfe6b700 1 --2- 172.21.15.94:0/2054572476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2d8101730 0x7fa2d8101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:59.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.432+0000 7fa2dfe6b700 1 --2- 172.21.15.94:0/2054572476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2d8104710 0x7fa2d8104b70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:59.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.432+0000 7fa2dfe6b700 1 -- 172.21.15.94:0/2054572476 >> 172.21.15.94:0/2054572476 conn(0x7fa2d80fd0b0 msgr2=0x7fa2d80ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:53:59.433 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:59 smithi094 ceph-mon[32889]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T22:53:59.433 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:53:59 smithi094 ceph-mon[32889]: pgmap v272: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:53:59.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.432+0000 7fa2dfe6b700 1 -- 172.21.15.94:0/2054572476 shutdown_connections 2024-04-26T22:53:59.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.432+0000 7fa2dfe6b700 1 -- 172.21.15.94:0/2054572476 wait complete. 2024-04-26T22:53:59.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.433+0000 7fa2dfe6b700 1 Processor -- start 2024-04-26T22:53:59.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.433+0000 7fa2dfe6b700 1 -- start start 2024-04-26T22:53:59.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.433+0000 7fa2dfe6b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2d8101730 0x7fa2d819b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:53:59.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.433+0000 7fa2dfe6b700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2d8104710 0x7fa2d819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:53:59.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.433+0000 7fa2dfe6b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2d81050b0 0x7fa2d81a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:53:59.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.433+0000 7fa2dfe6b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d810efa0 con 0x7fa2d8101730 2024-04-26T22:53:59.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.433+0000 7fa2dfe6b700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d810ee20 con 0x7fa2d8104710 2024-04-26T22:53:59.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.433+0000 7fa2dfe6b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d810f120 con 0x7fa2d81050b0 2024-04-26T22:53:59.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.433+0000 7fa2de408700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2d81050b0 0x7fa2d81a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:53:59.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.433+0000 7fa2dd406700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2d8104710 0x7fa2d819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:53:59.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.433+0000 7fa2dd406700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2d8104710 0x7fa2d819bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.94:37598/0 (socket says 172.21.15.94:37598) 2024-04-26T22:53:59.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.433+0000 7fa2dd406700 1 -- 172.21.15.94:0/2410663491 learned_addr learned my addr 172.21.15.94:0/2410663491 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T22:53:59.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.434+0000 7fa2dd406700 1 -- 172.21.15.94:0/2410663491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2d81050b0 msgr2=0x7fa2d81a2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:53:59.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.434+0000 7fa2dd406700 1 --2- 172.21.15.94:0/2410663491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2d81050b0 0x7fa2d81a2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:59.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.434+0000 7fa2dd406700 1 -- 172.21.15.94:0/2410663491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2d8101730 msgr2=0x7fa2d819b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:53:59.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.434+0000 7fa2dd406700 1 --2- 172.21.15.94:0/2410663491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2d8101730 0x7fa2d819b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:53:59.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.434+0000 7fa2dd406700 1 -- 172.21.15.94:0/2410663491 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d81a3450 con 0x7fa2d8104710 2024-04-26T22:53:59.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.434+0000 7fa2de408700 1 --2- 172.21.15.94:0/2410663491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2d81050b0 0x7fa2d81a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:53:59.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.434+0000 7fa2dd406700 1 --2- 172.21.15.94:0/2410663491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2d8104710 0x7fa2d819bd30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc006010 tx=0x7fa2cc0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:53:59.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.434+0000 7fa2d6ffd700 1 -- 172.21.15.94:0/2410663491 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc027070 con 0x7fa2d8104710 2024-04-26T22:53:59.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.434+0000 7fa2d6ffd700 1 -- 172.21.15.94:0/2410663491 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc00fbc0 con 0x7fa2d8104710 2024-04-26T22:53:59.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.434+0000 7fa2dfe6b700 1 -- 172.21.15.94:0/2410663491 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d81a36e0 con 0x7fa2d8104710 2024-04-26T22:53:59.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.434+0000 7fa2d6ffd700 1 -- 172.21.15.94:0/2410663491 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc017650 con 0x7fa2d8104710 2024-04-26T22:53:59.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.434+0000 7fa2dfe6b700 1 -- 172.21.15.94:0/2410663491 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d81a3bb0 con 0x7fa2d8104710 2024-04-26T22:53:59.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.435+0000 7fa2dfe6b700 1 -- 172.21.15.94:0/2410663491 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d806a110 con 0x7fa2d8104710 2024-04-26T22:53:59.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.436+0000 7fa2d6ffd700 1 -- 172.21.15.94:0/2410663491 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa2cc009d10 con 0x7fa2d8104710 2024-04-26T22:53:59.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.437+0000 7fa2d6ffd700 1 --2- 172.21.15.94:0/2410663491 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2c00a3c40 0x7fa2c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:53:59.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.437+0000 7fa2d6ffd700 1 -- 172.21.15.94:0/2410663491 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7fa2cc0d0a30 con 0x7fa2d8104710 2024-04-26T22:53:59.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.437+0000 7fa2ddc07700 1 --2- 172.21.15.94:0/2410663491 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2c00a3c40 0x7fa2c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:53:59.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.438+0000 7fa2ddc07700 1 --2- 172.21.15.94:0/2410663491 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2c00a3c40 0x7fa2c00a6100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa2d806af00 tx=0x7fa2d0009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:53:59.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.441+0000 7fa2d6ffd700 1 -- 172.21.15.94:0/2410663491 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa2cc013070 con 0x7fa2d8104710 2024-04-26T22:53:59.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:59 smithi080 ceph-mon[32324]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T22:53:59.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:53:59 smithi080 ceph-mon[32324]: pgmap v272: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:53:59.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:59 smithi027 ceph-mon[28034]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T22:53:59.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:53:59 smithi027 ceph-mon[28034]: pgmap v272: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:53:59.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:53:59.650+0000 7fa2dfe6b700 1 -- 172.21.15.94:0/2410663491 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa2d8103c20 con 0x7fa2c00a3c40 2024-04-26T22:54:00.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:00 smithi080 ceph-mon[32324]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T22:54:00.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:00 smithi080 ceph-mon[32324]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:54:00.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:00 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:54:00.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:00 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:54:00.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:00 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:00.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:00 smithi094 ceph-mon[32889]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T22:54:00.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:00 smithi094 ceph-mon[32889]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:54:00.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:00 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:54:00.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:00 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:54:00.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:00 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:00.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:00 smithi027 ceph-mon[28034]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T22:54:00.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:00 smithi027 ceph-mon[28034]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:54:00.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:00 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T22:54:00.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:00 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T22:54:00.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:00 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:01.495 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:01 smithi094 ceph-mon[32889]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-04-26T22:54:01.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:01 smithi080 ceph-mon[32324]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-04-26T22:54:01.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:01 smithi027 ceph-mon[28034]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-04-26T22:54:03.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:03 smithi080 ceph-mon[32324]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T22:54:03.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:03 smithi080 ceph-mon[32324]: from='client.? 172.21.15.94:0/801402351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffee2993-b4e0-4309-9cd8-83ac32df51c8"}]: dispatch 2024-04-26T22:54:03.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:03 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffee2993-b4e0-4309-9cd8-83ac32df51c8"}]: dispatch 2024-04-26T22:54:03.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:03 smithi080 ceph-mon[32324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffee2993-b4e0-4309-9cd8-83ac32df51c8"}]': finished 2024-04-26T22:54:03.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:03 smithi080 ceph-mon[32324]: osdmap e77: 12 total, 11 up, 12 in 2024-04-26T22:54:03.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:03 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:03.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:03 smithi080 ceph-mon[32324]: from='client.? 172.21.15.94:0/352446458' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:54:03.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:03 smithi094 ceph-mon[32889]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T22:54:03.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:03 smithi094 ceph-mon[32889]: from='client.? 172.21.15.94:0/801402351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffee2993-b4e0-4309-9cd8-83ac32df51c8"}]: dispatch 2024-04-26T22:54:03.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:03 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffee2993-b4e0-4309-9cd8-83ac32df51c8"}]: dispatch 2024-04-26T22:54:03.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:03 smithi094 ceph-mon[32889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffee2993-b4e0-4309-9cd8-83ac32df51c8"}]': finished 2024-04-26T22:54:03.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:03 smithi094 ceph-mon[32889]: osdmap e77: 12 total, 11 up, 12 in 2024-04-26T22:54:03.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:03 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:03.786 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:03 smithi094 ceph-mon[32889]: from='client.? 172.21.15.94:0/352446458' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:54:03.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:03 smithi027 ceph-mon[28034]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T22:54:03.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:03 smithi027 ceph-mon[28034]: from='client.? 172.21.15.94:0/801402351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffee2993-b4e0-4309-9cd8-83ac32df51c8"}]: dispatch 2024-04-26T22:54:03.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:03 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffee2993-b4e0-4309-9cd8-83ac32df51c8"}]: dispatch 2024-04-26T22:54:03.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:03 smithi027 ceph-mon[28034]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffee2993-b4e0-4309-9cd8-83ac32df51c8"}]': finished 2024-04-26T22:54:03.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:03 smithi027 ceph-mon[28034]: osdmap e77: 12 total, 11 up, 12 in 2024-04-26T22:54:03.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:03 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:03.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:03 smithi027 ceph-mon[28034]: from='client.? 172.21.15.94:0/352446458' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T22:54:05.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:05 smithi080 ceph-mon[32324]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-26T22:54:05.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:05 smithi094 ceph-mon[32889]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-26T22:54:05.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:05 smithi027 ceph-mon[28034]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-26T22:54:07.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:07 smithi080 ceph-mon[32324]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T22:54:07.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:07 smithi094 ceph-mon[32889]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T22:54:07.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:07 smithi027 ceph-mon[28034]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T22:54:09.765 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:09 smithi094 ceph-mon[32889]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-26T22:54:09.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:09 smithi080 ceph-mon[32324]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-26T22:54:09.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:09 smithi027 ceph-mon[28034]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-26T22:54:10.759 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:10 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T22:54:10.759 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:10 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:10.759 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:10 smithi094 ceph-mon[32889]: Deploying daemon osd.11 on smithi094 2024-04-26T22:54:10.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:10 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T22:54:10.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:10 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:10.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:10 smithi080 ceph-mon[32324]: Deploying daemon osd.11 on smithi094 2024-04-26T22:54:10.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:10 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T22:54:10.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:10 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:10.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:10 smithi027 ceph-mon[28034]: Deploying daemon osd.11 on smithi094 2024-04-26T22:54:11.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:11 smithi080 ceph-mon[32324]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:11.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:11 smithi094 ceph-mon[32889]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:11.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:11 smithi027 ceph-mon[28034]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:13.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:13 smithi080 ceph-mon[32324]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:13.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:13 smithi094 ceph-mon[32889]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:13.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:13 smithi027 ceph-mon[28034]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:15.718 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:15 smithi094 ceph-mon[32889]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:15.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:15 smithi080 ceph-mon[32324]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:15.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:15 smithi027 ceph-mon[28034]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:17.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:16 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:17.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:16 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:54:17.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:16 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:17.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:17.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:54:17.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:17.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:17.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:54:17.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:18.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:17 smithi080 ceph-mon[32324]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:18.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:17 smithi094 ceph-mon[32889]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:18.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:17 smithi027 ceph-mon[28034]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:20.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:20.098+0000 7fa2d6ffd700 1 -- 172.21.15.94:0/2410663491 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa2d8103c20 con 0x7fa2c00a3c40 2024-04-26T22:54:20.099 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 11 on host 'smithi094' 2024-04-26T22:54:20.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:20.100+0000 7fa2dfe6b700 1 -- 172.21.15.94:0/2410663491 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2c00a3c40 msgr2=0x7fa2c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:20.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:20.100+0000 7fa2dfe6b700 1 --2- 172.21.15.94:0/2410663491 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2c00a3c40 0x7fa2c00a6100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa2d806af00 tx=0x7fa2d0009250 comp rx=0 tx=0).stop 2024-04-26T22:54:20.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:20.100+0000 7fa2dfe6b700 1 -- 172.21.15.94:0/2410663491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2d8104710 msgr2=0x7fa2d819bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:20.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:20.100+0000 7fa2dfe6b700 1 --2- 172.21.15.94:0/2410663491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2d8104710 0x7fa2d819bd30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc006010 tx=0x7fa2cc0047b0 comp rx=0 tx=0).stop 2024-04-26T22:54:20.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:20.100+0000 7fa2dfe6b700 1 -- 172.21.15.94:0/2410663491 shutdown_connections 2024-04-26T22:54:20.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:20.100+0000 7fa2dfe6b700 1 --2- 172.21.15.94:0/2410663491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2d81050b0 0x7fa2d81a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:20.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:20.100+0000 7fa2dfe6b700 1 --2- 172.21.15.94:0/2410663491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2d8101730 0x7fa2d819b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:20.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:20.100+0000 7fa2dfe6b700 1 --2- 172.21.15.94:0/2410663491 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2c00a3c40 0x7fa2c00a6100 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:20.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:20.100+0000 7fa2dfe6b700 1 --2- 172.21.15.94:0/2410663491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2d8104710 0x7fa2d819bd30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:20.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:20.100+0000 7fa2dfe6b700 1 -- 172.21.15.94:0/2410663491 >> 172.21.15.94:0/2410663491 conn(0x7fa2d80fd0b0 msgr2=0x7fa2d8102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:20.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:20.101+0000 7fa2dfe6b700 1 -- 172.21.15.94:0/2410663491 shutdown_connections 2024-04-26T22:54:20.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:20.101+0000 7fa2dfe6b700 1 -- 172.21.15.94:0/2410663491 wait complete. 2024-04-26T22:54:20.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:19 smithi080 ceph-mon[32324]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:20.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:19 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:20.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:19 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:20.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:19 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:20.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:19 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:54:20.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:19 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:20.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:19 smithi080 ceph-mon[32324]: from='osd.11 [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T22:54:20.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:19 smithi080 ceph-mon[32324]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T22:54:20.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:19 smithi094 ceph-mon[32889]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:20.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:19 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:20.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:19 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:20.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:19 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:20.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:19 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:54:20.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:19 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:20.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:19 smithi094 ceph-mon[32889]: from='osd.11 [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T22:54:20.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:19 smithi094 ceph-mon[32889]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T22:54:20.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:19 smithi027 ceph-mon[28034]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:20.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:19 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:20.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:19 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:20.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:19 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:20.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:19 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:54:20.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:19 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:20.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:19 smithi027 ceph-mon[28034]: from='osd.11 [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T22:54:20.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:19 smithi027 ceph-mon[28034]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T22:54:20.823 DEBUG:teuthology.orchestra.run.smithi094:osd.11> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.11.service 2024-04-26T22:54:20.826 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-26T22:54:20.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T22:54:21.046 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:21.047 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:21.047 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:54:21.047 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:21 smithi094 ceph-mon[32889]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T22:54:21.047 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:21 smithi094 ceph-mon[32889]: from='osd.11 [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:54:21.047 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:21 smithi094 ceph-mon[32889]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T22:54:21.047 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:21.047 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:21 smithi094 ceph-mon[32889]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:54:21.312 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:54:21.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:21.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:21.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:54:21.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:21 smithi027 ceph-mon[28034]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T22:54:21.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:21 smithi027 ceph-mon[28034]: from='osd.11 [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:54:21.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:21 smithi027 ceph-mon[28034]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T22:54:21.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:21.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:21 smithi027 ceph-mon[28034]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:54:21.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:21.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:21.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:54:21.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:21 smithi080 ceph-mon[32324]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T22:54:21.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:21 smithi080 ceph-mon[32324]: from='osd.11 [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:54:21.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:21 smithi080 ceph-mon[32324]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T22:54:21.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:21.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:21 smithi080 ceph-mon[32324]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T22:54:22.034 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 22:54:21 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T22:54:21.771+0000 7efde0eea700 -1 osd.11 0 waiting for initial osdmap 2024-04-26T22:54:22.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 22:54:21 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T22:54:21.787+0000 7efdd8cd5700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:54:22.325 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:22 smithi094 ceph-mon[32889]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:22.325 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:22 smithi094 ceph-mon[32889]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T22:54:22.325 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:22 smithi094 ceph-mon[32889]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T22:54:22.325 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:22.326 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:22.326 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:22 smithi094 ceph-mon[32889]: from='osd.11 ' entity='osd.11' 2024-04-26T22:54:22.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:22 smithi027 ceph-mon[28034]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:22.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:22 smithi027 ceph-mon[28034]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T22:54:22.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:22 smithi027 ceph-mon[28034]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T22:54:22.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:22.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:22.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:22 smithi027 ceph-mon[28034]: from='osd.11 ' entity='osd.11' 2024-04-26T22:54:22.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:22 smithi080 ceph-mon[32324]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:22.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:22 smithi080 ceph-mon[32324]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T22:54:22.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:22 smithi080 ceph-mon[32324]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T22:54:22.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:22.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:22.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:22 smithi080 ceph-mon[32324]: from='osd.11 ' entity='osd.11' 2024-04-26T22:54:23.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:23 smithi027 ceph-mon[28034]: purged_snaps scrub starts 2024-04-26T22:54:23.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:23 smithi027 ceph-mon[28034]: purged_snaps scrub ok 2024-04-26T22:54:23.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:23.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:23 smithi027 ceph-mon[28034]: osd.11 [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925] boot 2024-04-26T22:54:23.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:23 smithi027 ceph-mon[28034]: osdmap e80: 12 total, 12 up, 12 in 2024-04-26T22:54:23.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:23.303 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:23 smithi094 ceph-mon[32889]: purged_snaps scrub starts 2024-04-26T22:54:23.303 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:23 smithi094 ceph-mon[32889]: purged_snaps scrub ok 2024-04-26T22:54:23.303 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:23.303 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:23 smithi094 ceph-mon[32889]: osd.11 [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925] boot 2024-04-26T22:54:23.303 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:23 smithi094 ceph-mon[32889]: osdmap e80: 12 total, 12 up, 12 in 2024-04-26T22:54:23.303 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:23 smithi080 ceph-mon[32324]: purged_snaps scrub starts 2024-04-26T22:54:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:23 smithi080 ceph-mon[32324]: purged_snaps scrub ok 2024-04-26T22:54:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:23 smithi080 ceph-mon[32324]: osd.11 [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925] boot 2024-04-26T22:54:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:23 smithi080 ceph-mon[32324]: osdmap e80: 12 total, 12 up, 12 in 2024-04-26T22:54:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:54:23.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.767+0000 7fe0a2a00700 1 -- 172.21.15.27:0/944030853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c1046d0 msgr2=0x7fe09c104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:23.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.767+0000 7fe0a2a00700 1 --2- 172.21.15.27:0/944030853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c1046d0 0x7fe09c104b30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe0880099c0 tx=0x7fe088009cd0 comp rx=0 tx=0).stop 2024-04-26T22:54:23.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.767+0000 7fe0a2a00700 1 -- 172.21.15.27:0/944030853 shutdown_connections 2024-04-26T22:54:23.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.767+0000 7fe0a2a00700 1 --2- 172.21.15.27:0/944030853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe09c105070 0x7fe09c10b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:23.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.767+0000 7fe0a2a00700 1 --2- 172.21.15.27:0/944030853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe09c1034d0 0x7fe09c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:23.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.767+0000 7fe0a2a00700 1 --2- 172.21.15.27:0/944030853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c1046d0 0x7fe09c104b30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:23.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.767+0000 7fe0a2a00700 1 -- 172.21.15.27:0/944030853 >> 172.21.15.27:0/944030853 conn(0x7fe09c0fea70 msgr2=0x7fe09c100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:23.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.768+0000 7fe0a2a00700 1 -- 172.21.15.27:0/944030853 shutdown_connections 2024-04-26T22:54:23.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.768+0000 7fe0a2a00700 1 -- 172.21.15.27:0/944030853 wait complete. 2024-04-26T22:54:23.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.769+0000 7fe0a2a00700 1 Processor -- start 2024-04-26T22:54:23.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.769+0000 7fe0a2a00700 1 -- start start 2024-04-26T22:54:23.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.769+0000 7fe0a2a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe09c1034d0 0x7fe09c1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:23.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.769+0000 7fe0a2a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe09c1046d0 0x7fe09c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:23.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.770+0000 7fe0a2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c105070 0x7fe09c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:23.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.770+0000 7fe0a2a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c10efa0 con 0x7fe09c105070 2024-04-26T22:54:23.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.770+0000 7fe0a2a00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c10ee20 con 0x7fe09c1034d0 2024-04-26T22:54:23.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.770+0000 7fe0a2a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c10f120 con 0x7fe09c1046d0 2024-04-26T22:54:23.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.770+0000 7fe09be00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe09c1034d0 0x7fe09c1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:23.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.770+0000 7fe09be00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe09c1034d0 0x7fe09c1995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:41992/0 (socket says 172.21.15.27:41992) 2024-04-26T22:54:23.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.770+0000 7fe09be00700 1 -- 172.21.15.27:0/43481956 learned_addr learned my addr 172.21.15.27:0/43481956 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:54:23.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.770+0000 7fe0a0a00700 1 --2- 172.21.15.27:0/43481956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c105070 0x7fe09c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:23.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.770+0000 7fe09b400700 1 --2- 172.21.15.27:0/43481956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe09c1046d0 0x7fe09c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:23.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.770+0000 7fe0a0a00700 1 -- 172.21.15.27:0/43481956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe09c1046d0 msgr2=0x7fe09c199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:23.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.770+0000 7fe0a0a00700 1 --2- 172.21.15.27:0/43481956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe09c1046d0 0x7fe09c199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:23.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.770+0000 7fe0a0a00700 1 -- 172.21.15.27:0/43481956 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe09c1034d0 msgr2=0x7fe09c1995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:23.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.770+0000 7fe0a0a00700 1 --2- 172.21.15.27:0/43481956 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe09c1034d0 0x7fe09c1995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:23.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.770+0000 7fe0a0a00700 1 -- 172.21.15.27:0/43481956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09c1a1250 con 0x7fe09c105070 2024-04-26T22:54:23.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.770+0000 7fe09be00700 1 --2- 172.21.15.27:0/43481956 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe09c1034d0 0x7fe09c1995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:54:23.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.771+0000 7fe0a0a00700 1 --2- 172.21.15.27:0/43481956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c105070 0x7fe09c1a0bb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe094009f60 tx=0x7fe094009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:23.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.771+0000 7fe098c00700 1 -- 172.21.15.27:0/43481956 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0940092e0 con 0x7fe09c105070 2024-04-26T22:54:23.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.771+0000 7fe0a2a00700 1 -- 172.21.15.27:0/43481956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09c1a14e0 con 0x7fe09c105070 2024-04-26T22:54:23.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.771+0000 7fe098c00700 1 -- 172.21.15.27:0/43481956 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe094004bb0 con 0x7fe09c105070 2024-04-26T22:54:23.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.771+0000 7fe098c00700 1 -- 172.21.15.27:0/43481956 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094004020 con 0x7fe09c105070 2024-04-26T22:54:23.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.771+0000 7fe0a2a00700 1 -- 172.21.15.27:0/43481956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe09c1a19d0 con 0x7fe09c105070 2024-04-26T22:54:23.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.772+0000 7fe0a2a00700 1 -- 172.21.15.27:0/43481956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe09c06a110 con 0x7fe09c105070 2024-04-26T22:54:23.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.775+0000 7fe098c00700 1 -- 172.21.15.27:0/43481956 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe09401b030 con 0x7fe09c105070 2024-04-26T22:54:23.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.777+0000 7fe098c00700 1 --2- 172.21.15.27:0/43481956 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe0800a3c40 0x7fe0800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:23.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.777+0000 7fe098c00700 1 -- 172.21.15.27:0/43481956 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fe0940cfa00 con 0x7fe09c105070 2024-04-26T22:54:23.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.777+0000 7fe09be00700 1 --2- 172.21.15.27:0/43481956 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe0800a3c40 0x7fe0800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:23.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.778+0000 7fe09be00700 1 --2- 172.21.15.27:0/43481956 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe0800a3c40 0x7fe0800a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe08c005950 tx=0x7fe08c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:23.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:23.782+0000 7fe098c00700 1 -- 172.21.15.27:0/43481956 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe094022020 con 0x7fe09c105070 2024-04-26T22:54:24.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:24.019+0000 7fe0a2a00700 1 -- 172.21.15.27:0/43481956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe09c064030 con 0x7fe09c105070 2024-04-26T22:54:24.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:24.020+0000 7fe098c00700 1 -- 172.21.15.27:0/43481956 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7fe094010040 con 0x7fe09c105070 2024-04-26T22:54:24.021 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:54:24.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:24.023+0000 7fe0a2a00700 1 -- 172.21.15.27:0/43481956 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe0800a3c40 msgr2=0x7fe0800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:24.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:24.023+0000 7fe0a2a00700 1 --2- 172.21.15.27:0/43481956 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe0800a3c40 0x7fe0800a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe08c005950 tx=0x7fe08c009260 comp rx=0 tx=0).stop 2024-04-26T22:54:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:24.023+0000 7fe0a2a00700 1 -- 172.21.15.27:0/43481956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c105070 msgr2=0x7fe09c1a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:24.023+0000 7fe0a2a00700 1 --2- 172.21.15.27:0/43481956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c105070 0x7fe09c1a0bb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe094009f60 tx=0x7fe094009f90 comp rx=0 tx=0).stop 2024-04-26T22:54:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:24.024+0000 7fe0a2a00700 1 -- 172.21.15.27:0/43481956 shutdown_connections 2024-04-26T22:54:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:24.024+0000 7fe0a2a00700 1 --2- 172.21.15.27:0/43481956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c105070 0x7fe09c1a0bb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:24.024+0000 7fe0a2a00700 1 --2- 172.21.15.27:0/43481956 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe0800a3c40 0x7fe0800a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:24.024+0000 7fe0a2a00700 1 --2- 172.21.15.27:0/43481956 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe09c1034d0 0x7fe09c1995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:24.024+0000 7fe0a2a00700 1 --2- 172.21.15.27:0/43481956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe09c1046d0 0x7fe09c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:24.024+0000 7fe0a2a00700 1 -- 172.21.15.27:0/43481956 >> 172.21.15.27:0/43481956 conn(0x7fe09c0fea70 msgr2=0x7fe09c107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:24.024+0000 7fe0a2a00700 1 -- 172.21.15.27:0/43481956 shutdown_connections 2024-04-26T22:54:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:24.024+0000 7fe0a2a00700 1 -- 172.21.15.27:0/43481956 wait complete. 2024-04-26T22:54:24.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:24 smithi094 ceph-mon[32889]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:24.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:24 smithi094 ceph-mon[32889]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T22:54:24.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:24 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/43481956' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T22:54:24.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:24 smithi027 ceph-mon[28034]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:24.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:24 smithi027 ceph-mon[28034]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T22:54:24.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:24 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/43481956' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T22:54:24.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:24 smithi080 ceph-mon[32324]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-26T22:54:24.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:24 smithi080 ceph-mon[32324]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T22:54:24.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:24 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/43481956' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T22:54:24.876 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714172062,"num_in_osds":12,"osd_in_since":1714172042,"num_remapped_pgs":0} 2024-04-26T22:54:24.876 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T22:54:25.352 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:54:25.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:25 smithi027 ceph-mon[28034]: Detected new or changed devices on smithi094 2024-04-26T22:54:25.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:25.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:25.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:54:25.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:54:25.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:54:25.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:54:25.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:25 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi094 to 4272M 2024-04-26T22:54:25.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:25.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:25.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:54:25.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:25.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:25 smithi080 ceph-mon[32324]: Detected new or changed devices on smithi094 2024-04-26T22:54:25.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:25.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:25.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:54:25.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:54:25.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:54:25.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:54:25.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:25 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi094 to 4272M 2024-04-26T22:54:25.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:25.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:25.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:54:25.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:25.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:25 smithi094 ceph-mon[32889]: Detected new or changed devices on smithi094 2024-04-26T22:54:25.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:25.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:54:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:54:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:54:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:54:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:25 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi094 to 4272M 2024-04-26T22:54:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:54:25.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:26.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:26 smithi027 ceph-mon[28034]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:26.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:26 smithi080 ceph-mon[32324]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:26.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:26 smithi094 ceph-mon[32889]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:27.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.838+0000 7f5bdfa00700 1 -- 172.21.15.27:0/80138921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bd806ba50 msgr2=0x7f5bd810f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:27.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.838+0000 7f5bdfa00700 1 --2- 172.21.15.27:0/80138921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bd806ba50 0x7f5bd810f990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009990 tx=0x7f5bc8009ca0 comp rx=0 tx=0).stop 2024-04-26T22:54:27.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.838+0000 7f5bdfa00700 1 -- 172.21.15.27:0/80138921 shutdown_connections 2024-04-26T22:54:27.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.838+0000 7f5bdfa00700 1 --2- 172.21.15.27:0/80138921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd810fed0 0x7f5bd81122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:27.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.838+0000 7f5bdfa00700 1 --2- 172.21.15.27:0/80138921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5bd806b130 0x7f5bd806b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:27.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.838+0000 7f5bdfa00700 1 --2- 172.21.15.27:0/80138921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bd806ba50 0x7f5bd810f990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:27.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.838+0000 7f5bdfa00700 1 -- 172.21.15.27:0/80138921 >> 172.21.15.27:0/80138921 conn(0x7f5bd80792d0 msgr2=0x7f5bd80796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:27.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.838+0000 7f5bdfa00700 1 -- 172.21.15.27:0/80138921 shutdown_connections 2024-04-26T22:54:27.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.838+0000 7f5bdfa00700 1 -- 172.21.15.27:0/80138921 wait complete. 2024-04-26T22:54:27.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.839+0000 7f5bdfa00700 1 Processor -- start 2024-04-26T22:54:27.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.839+0000 7f5bdfa00700 1 -- start start 2024-04-26T22:54:27.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.840+0000 7f5bdfa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bd806b130 0x7f5bd81a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:27.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.840+0000 7f5bdfa00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5bd806ba50 0x7f5bd81a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:27.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.840+0000 7f5bdfa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd810fed0 0x7f5bd819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:27.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.840+0000 7f5bdfa00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd81157e0 con 0x7f5bd806b130 2024-04-26T22:54:27.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.840+0000 7f5bdfa00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd8115660 con 0x7f5bd806ba50 2024-04-26T22:54:27.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.840+0000 7f5bdfa00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd8115960 con 0x7f5bd810fed0 2024-04-26T22:54:27.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.840+0000 7f5bdd000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bd806b130 0x7f5bd81a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:27.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.840+0000 7f5bdd000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bd806b130 0x7f5bd81a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42398/0 (socket says 172.21.15.27:42398) 2024-04-26T22:54:27.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.840+0000 7f5bdd000700 1 -- 172.21.15.27:0/1002793247 learned_addr learned my addr 172.21.15.27:0/1002793247 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:54:27.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.840+0000 7f5bcfe00700 1 --2- 172.21.15.27:0/1002793247 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5bd806ba50 0x7f5bd81a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:27.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.840+0000 7f5bdd000700 1 -- 172.21.15.27:0/1002793247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd810fed0 msgr2=0x7f5bd819bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:27.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.840+0000 7f5bdd000700 1 --2- 172.21.15.27:0/1002793247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd810fed0 0x7f5bd819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:27.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.840+0000 7f5bdd000700 1 -- 172.21.15.27:0/1002793247 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5bd806ba50 msgr2=0x7f5bd81a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:27.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.840+0000 7f5bdd000700 1 --2- 172.21.15.27:0/1002793247 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5bd806ba50 0x7f5bd81a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:27.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.840+0000 7f5bdd000700 1 -- 172.21.15.27:0/1002793247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd819c2a0 con 0x7f5bd806b130 2024-04-26T22:54:27.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.841+0000 7f5bdd000700 1 --2- 172.21.15.27:0/1002793247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bd806b130 0x7f5bd81a1930 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000e9a0 tx=0x7f5bc000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:27.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.841+0000 7f5bd6000700 1 -- 172.21.15.27:0/1002793247 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc000cb50 con 0x7f5bd806b130 2024-04-26T22:54:27.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.841+0000 7f5bd6000700 1 -- 172.21.15.27:0/1002793247 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bc000ccf0 con 0x7f5bd806b130 2024-04-26T22:54:27.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.841+0000 7f5bd6000700 1 -- 172.21.15.27:0/1002793247 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0018680 con 0x7f5bd806b130 2024-04-26T22:54:27.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.841+0000 7f5bdfa00700 1 -- 172.21.15.27:0/1002793247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd819c590 con 0x7f5bd806b130 2024-04-26T22:54:27.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.841+0000 7f5bdfa00700 1 -- 172.21.15.27:0/1002793247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd81a8800 con 0x7f5bd806b130 2024-04-26T22:54:27.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.843+0000 7f5bdfa00700 1 -- 172.21.15.27:0/1002793247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd8102940 con 0x7f5bd806b130 2024-04-26T22:54:27.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.846+0000 7f5bd6000700 1 -- 172.21.15.27:0/1002793247 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5bc0010430 con 0x7f5bd806b130 2024-04-26T22:54:27.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.848+0000 7f5bd6000700 1 --2- 172.21.15.27:0/1002793247 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5bb80a3c40 0x7f5bb80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:27.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.848+0000 7f5bd6000700 1 -- 172.21.15.27:0/1002793247 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5bc00d0430 con 0x7f5bd806b130 2024-04-26T22:54:27.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.848+0000 7f5bcfe00700 1 --2- 172.21.15.27:0/1002793247 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5bb80a3c40 0x7f5bb80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:27.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.849+0000 7f5bcfe00700 1 --2- 172.21.15.27:0/1002793247 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5bb80a3c40 0x7f5bb80a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800b2e0 tx=0x7f5bc8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:27.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:27.851+0000 7f5bd6000700 1 -- 172.21.15.27:0/1002793247 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5bc009d210 con 0x7f5bd806b130 2024-04-26T22:54:28.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:28.086+0000 7f5bdfa00700 1 -- 172.21.15.27:0/1002793247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5bd802e490 con 0x7f5bd806b130 2024-04-26T22:54:28.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:28.089+0000 7f5bd6000700 1 -- 172.21.15.27:0/1002793247 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18337 (secure 0 0 0) 0x7f5bc009cb10 con 0x7f5bd806b130 2024-04-26T22:54:28.090 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:54:28.090 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":81,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:54:23.783142+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:54:28.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:28.092+0000 7f5bdfa00700 1 -- 172.21.15.27:0/1002793247 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5bb80a3c40 msgr2=0x7f5bb80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:28.092+0000 7f5bdfa00700 1 --2- 172.21.15.27:0/1002793247 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5bb80a3c40 0x7f5bb80a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800b2e0 tx=0x7f5bc8005960 comp rx=0 tx=0).stop 2024-04-26T22:54:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:28.092+0000 7f5bdfa00700 1 -- 172.21.15.27:0/1002793247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bd806b130 msgr2=0x7f5bd81a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:28.092+0000 7f5bdfa00700 1 --2- 172.21.15.27:0/1002793247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bd806b130 0x7f5bd81a1930 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000e9a0 tx=0x7f5bc000ecb0 comp rx=0 tx=0).stop 2024-04-26T22:54:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:28.093+0000 7f5bdfa00700 1 -- 172.21.15.27:0/1002793247 shutdown_connections 2024-04-26T22:54:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:28.093+0000 7f5bdfa00700 1 --2- 172.21.15.27:0/1002793247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd810fed0 0x7f5bd819bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:28.093+0000 7f5bdfa00700 1 --2- 172.21.15.27:0/1002793247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bd806b130 0x7f5bd81a1930 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:28.093+0000 7f5bdfa00700 1 --2- 172.21.15.27:0/1002793247 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5bb80a3c40 0x7f5bb80a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:28.093+0000 7f5bdfa00700 1 --2- 172.21.15.27:0/1002793247 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5bd806ba50 0x7f5bd81a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:28.093+0000 7f5bdfa00700 1 -- 172.21.15.27:0/1002793247 >> 172.21.15.27:0/1002793247 conn(0x7f5bd80792d0 msgr2=0x7f5bd81113c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:28.093+0000 7f5bdfa00700 1 -- 172.21.15.27:0/1002793247 shutdown_connections 2024-04-26T22:54:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:28.093+0000 7f5bdfa00700 1 -- 172.21.15.27:0/1002793247 wait complete. 2024-04-26T22:54:28.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:28 smithi027 ceph-mon[28034]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:28.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:28 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1002793247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:54:28.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:28 smithi080 ceph-mon[32324]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:28.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:28 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1002793247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:54:28.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:28 smithi094 ceph-mon[32889]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:28.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:28 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1002793247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:54:29.004 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:49:54.482679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:54:29.005 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T22:54:29.476 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:54:30.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:30 smithi027 ceph-mon[28034]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:30.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:30 smithi080 ceph-mon[32324]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:30.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:30 smithi094 ceph-mon[32889]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:32.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.062+0000 7f019ae00700 1 -- 172.21.15.27:0/885447225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0194104560 msgr2=0x7f0194104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:32.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.062+0000 7f019ae00700 1 --2- 172.21.15.27:0/885447225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0194104560 0x7f0194104940 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f018800b0a0 tx=0x7f018800b3b0 comp rx=0 tx=0).stop 2024-04-26T22:54:32.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.062+0000 7f019ae00700 1 -- 172.21.15.27:0/885447225 shutdown_connections 2024-04-26T22:54:32.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.062+0000 7f019ae00700 1 --2- 172.21.15.27:0/885447225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f019410d990 0x7f019410fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:32.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.062+0000 7f019ae00700 1 --2- 172.21.15.27:0/885447225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0194104560 0x7f0194104940 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:32.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.062+0000 7f019ae00700 1 --2- 172.21.15.27:0/885447225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0194104e80 0x7f019410d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:32.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.062+0000 7f019ae00700 1 -- 172.21.15.27:0/885447225 >> 172.21.15.27:0/885447225 conn(0x7f01940fddb0 msgr2=0x7f01941001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:32.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.062+0000 7f019ae00700 1 -- 172.21.15.27:0/885447225 shutdown_connections 2024-04-26T22:54:32.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.062+0000 7f019ae00700 1 -- 172.21.15.27:0/885447225 wait complete. 2024-04-26T22:54:32.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.063+0000 7f019ae00700 1 Processor -- start 2024-04-26T22:54:32.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.063+0000 7f019ae00700 1 -- start start 2024-04-26T22:54:32.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.064+0000 7f019ae00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0194104560 0x7f019419bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:32.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.064+0000 7f019ae00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0194104e80 0x7f019419c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:32.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.064+0000 7f019ae00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019410d990 0x7f01941a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:32.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.064+0000 7f019ae00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f01941135d0 con 0x7f019410d990 2024-04-26T22:54:32.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.064+0000 7f019ae00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0194113450 con 0x7f0194104560 2024-04-26T22:54:32.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.064+0000 7f019ae00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0194113750 con 0x7f0194104e80 2024-04-26T22:54:32.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.064+0000 7f0199a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0194104560 0x7f019419bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:32.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.064+0000 7f0199a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0194104560 0x7f019419bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:38252/0 (socket says 172.21.15.27:38252) 2024-04-26T22:54:32.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.064+0000 7f0199a00700 1 -- 172.21.15.27:0/427759740 learned_addr learned my addr 172.21.15.27:0/427759740 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:54:32.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.064+0000 7f0199000700 1 --2- 172.21.15.27:0/427759740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0194104e80 0x7f019419c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:32.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.064+0000 7f019a400700 1 --2- 172.21.15.27:0/427759740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019410d990 0x7f01941a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:32.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.065+0000 7f0199a00700 1 -- 172.21.15.27:0/427759740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0194104e80 msgr2=0x7f019419c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:32.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.065+0000 7f0199a00700 1 --2- 172.21.15.27:0/427759740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0194104e80 0x7f019419c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:32.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.065+0000 7f0199a00700 1 -- 172.21.15.27:0/427759740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019410d990 msgr2=0x7f01941a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:32.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.065+0000 7f0199a00700 1 --2- 172.21.15.27:0/427759740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019410d990 0x7f01941a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:32.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.065+0000 7f0199a00700 1 -- 172.21.15.27:0/427759740 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01941a0d50 con 0x7f0194104560 2024-04-26T22:54:32.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.065+0000 7f0199a00700 1 --2- 172.21.15.27:0/427759740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0194104560 0x7f019419bd40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f018800d010 tx=0x7f0188004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:32.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.065+0000 7f0192000700 1 -- 172.21.15.27:0/427759740 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188021070 con 0x7f0194104560 2024-04-26T22:54:32.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.065+0000 7f019ae00700 1 -- 172.21.15.27:0/427759740 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01941a0fe0 con 0x7f0194104560 2024-04-26T22:54:32.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.065+0000 7f0192000700 1 -- 172.21.15.27:0/427759740 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0188007c40 con 0x7f0194104560 2024-04-26T22:54:32.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.065+0000 7f0192000700 1 -- 172.21.15.27:0/427759740 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188004710 con 0x7f0194104560 2024-04-26T22:54:32.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.065+0000 7f019ae00700 1 -- 172.21.15.27:0/427759740 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01941a88c0 con 0x7f0194104560 2024-04-26T22:54:32.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.067+0000 7f019ae00700 1 -- 172.21.15.27:0/427759740 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f019406a110 con 0x7f0194104560 2024-04-26T22:54:32.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.069+0000 7f0192000700 1 -- 172.21.15.27:0/427759740 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0188003a00 con 0x7f0194104560 2024-04-26T22:54:32.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.070+0000 7f0192000700 1 --2- 172.21.15.27:0/427759740 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f017c0a3cc0 0x7f017c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:32.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.071+0000 7f0192000700 1 -- 172.21.15.27:0/427759740 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f018800f020 con 0x7f0194104560 2024-04-26T22:54:32.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.071+0000 7f0199000700 1 --2- 172.21.15.27:0/427759740 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f017c0a3cc0 0x7f017c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:32.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.071+0000 7f0199000700 1 --2- 172.21.15.27:0/427759740 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f017c0a3cc0 0x7f017c0a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0194104150 tx=0x7f0180009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:32.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.075+0000 7f0192000700 1 -- 172.21.15.27:0/427759740 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f018802a020 con 0x7f0194104560 2024-04-26T22:54:32.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.304+0000 7f019ae00700 1 -- 172.21.15.27:0/427759740 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f019402e490 con 0x7f0194104560 2024-04-26T22:54:32.305 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:32 smithi027 ceph-mon[28034]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.305+0000 7f0192000700 1 -- 172.21.15.27:0/427759740 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f018809e360 con 0x7f0194104560 2024-04-26T22:54:32.305 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T22:54:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.309+0000 7f019ae00700 1 -- 172.21.15.27:0/427759740 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f017c0a3cc0 msgr2=0x7f017c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.309+0000 7f019ae00700 1 --2- 172.21.15.27:0/427759740 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f017c0a3cc0 0x7f017c0a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0194104150 tx=0x7f0180009260 comp rx=0 tx=0).stop 2024-04-26T22:54:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.309+0000 7f019ae00700 1 -- 172.21.15.27:0/427759740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0194104560 msgr2=0x7f019419bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.309+0000 7f019ae00700 1 --2- 172.21.15.27:0/427759740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0194104560 0x7f019419bd40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f018800d010 tx=0x7f0188004d00 comp rx=0 tx=0).stop 2024-04-26T22:54:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.309+0000 7f019ae00700 1 -- 172.21.15.27:0/427759740 shutdown_connections 2024-04-26T22:54:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.310+0000 7f019ae00700 1 --2- 172.21.15.27:0/427759740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019410d990 0x7f01941a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.310+0000 7f019ae00700 1 --2- 172.21.15.27:0/427759740 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f017c0a3cc0 0x7f017c0a6180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:32.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.310+0000 7f019ae00700 1 --2- 172.21.15.27:0/427759740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0194104560 0x7f019419bd40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:32.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.310+0000 7f019ae00700 1 --2- 172.21.15.27:0/427759740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0194104e80 0x7f019419c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:32.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.310+0000 7f019ae00700 1 -- 172.21.15.27:0/427759740 >> 172.21.15.27:0/427759740 conn(0x7f01940fddb0 msgr2=0x7f01940fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:32.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.310+0000 7f019ae00700 1 -- 172.21.15.27:0/427759740 shutdown_connections 2024-04-26T22:54:32.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:32.310+0000 7f019ae00700 1 -- 172.21.15.27:0/427759740 wait complete. 2024-04-26T22:54:32.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:32 smithi080 ceph-mon[32324]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:32.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:32 smithi094 ceph-mon[32889]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:33.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:33 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/427759740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:54:33.311 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:33 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/427759740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:54:33.360 INFO:tasks.cephadm:Adding mds.a on smithi027 2024-04-26T22:54:33.360 INFO:tasks.cephadm:Adding mds.d on smithi027 2024-04-26T22:54:33.360 INFO:tasks.cephadm:Adding mds.g on smithi027 2024-04-26T22:54:33.360 INFO:tasks.cephadm:Adding mds.j on smithi027 2024-04-26T22:54:33.361 INFO:tasks.cephadm:Adding mds.b on smithi080 2024-04-26T22:54:33.361 INFO:tasks.cephadm:Adding mds.e on smithi080 2024-04-26T22:54:33.361 INFO:tasks.cephadm:Adding mds.h on smithi080 2024-04-26T22:54:33.361 INFO:tasks.cephadm:Adding mds.k on smithi080 2024-04-26T22:54:33.361 INFO:tasks.cephadm:Adding mds.c on smithi094 2024-04-26T22:54:33.361 INFO:tasks.cephadm:Adding mds.f on smithi094 2024-04-26T22:54:33.361 INFO:tasks.cephadm:Adding mds.i on smithi094 2024-04-26T22:54:33.361 INFO:tasks.cephadm:Adding mds.l on smithi094 2024-04-26T22:54:33.361 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi094=c;smithi094=f;smithi094=i;smithi094=l' 2024-04-26T22:54:33.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:33 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/427759740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:54:33.627 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:54:34.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:34 smithi094 ceph-mon[32889]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:34.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:34 smithi080 ceph-mon[32324]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:34.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:34 smithi027 ceph-mon[28034]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.476+0000 7f964faed700 1 -- 172.21.15.94:0/2711779653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648102370 msgr2=0x7f9648102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.476+0000 7f964faed700 1 --2- 172.21.15.94:0/2711779653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648102370 0x7f9648102790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9638009990 tx=0x7f9638009ca0 comp rx=0 tx=0).stop 2024-04-26T22:54:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.476+0000 7f964faed700 1 -- 172.21.15.94:0/2711779653 shutdown_connections 2024-04-26T22:54:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.476+0000 7f964faed700 1 --2- 172.21.15.94:0/2711779653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9648103f10 0x7f964810a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.476+0000 7f964faed700 1 --2- 172.21.15.94:0/2711779653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648102370 0x7f9648102790 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.476+0000 7f964faed700 1 --2- 172.21.15.94:0/2711779653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9648103570 0x7f96481039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.476+0000 7f964faed700 1 -- 172.21.15.94:0/2711779653 >> 172.21.15.94:0/2711779653 conn(0x7f96480fd910 msgr2=0x7f96480ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.476+0000 7f964faed700 1 -- 172.21.15.94:0/2711779653 shutdown_connections 2024-04-26T22:54:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.476+0000 7f964faed700 1 -- 172.21.15.94:0/2711779653 wait complete. 2024-04-26T22:54:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964faed700 1 Processor -- start 2024-04-26T22:54:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964faed700 1 -- start start 2024-04-26T22:54:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964faed700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9648102370 0x7f96481996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964faed700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648103570 0x7f9648199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964faed700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9648103f10 0x7f96481a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964faed700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f964810dee0 con 0x7f9648102370 2024-04-26T22:54:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964faed700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f964810dd60 con 0x7f9648103f10 2024-04-26T22:54:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964faed700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f964810e060 con 0x7f9648103570 2024-04-26T22:54:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964d088700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648103570 0x7f9648199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964d088700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648103570 0x7f9648199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36994/0 (socket says 172.21.15.94:36994) 2024-04-26T22:54:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964d088700 1 -- 172.21.15.94:0/247694762 learned_addr learned my addr 172.21.15.94:0/247694762 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T22:54:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964e08a700 1 --2- 172.21.15.94:0/247694762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9648103f10 0x7f96481a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964d088700 1 -- 172.21.15.94:0/247694762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9648103f10 msgr2=0x7f96481a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964d088700 1 --2- 172.21.15.94:0/247694762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9648103f10 0x7f96481a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964d088700 1 -- 172.21.15.94:0/247694762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9648102370 msgr2=0x7f96481996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964d088700 1 --2- 172.21.15.94:0/247694762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9648102370 0x7f96481996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.477+0000 7f964d088700 1 -- 172.21.15.94:0/247694762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96481a1300 con 0x7f9648103570 2024-04-26T22:54:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.478+0000 7f964d088700 1 --2- 172.21.15.94:0/247694762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648103570 0x7f9648199be0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f964400b560 tx=0x7f964400b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.478+0000 7f963effd700 1 -- 172.21.15.94:0/247694762 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644015020 con 0x7f9648103570 2024-04-26T22:54:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.478+0000 7f963effd700 1 -- 172.21.15.94:0/247694762 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9644011ce0 con 0x7f9648103570 2024-04-26T22:54:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.478+0000 7f963effd700 1 -- 172.21.15.94:0/247694762 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644010da0 con 0x7f9648103570 2024-04-26T22:54:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.478+0000 7f964faed700 1 -- 172.21.15.94:0/247694762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96481a15f0 con 0x7f9648103570 2024-04-26T22:54:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.478+0000 7f964faed700 1 -- 172.21.15.94:0/247694762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96481a1bb0 con 0x7f9648103570 2024-04-26T22:54:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.479+0000 7f964faed700 1 -- 172.21.15.94:0/247694762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f964806a110 con 0x7f9648103570 2024-04-26T22:54:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.480+0000 7f963effd700 1 -- 172.21.15.94:0/247694762 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9644018020 con 0x7f9648103570 2024-04-26T22:54:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.480+0000 7f963effd700 1 --2- 172.21.15.94:0/247694762 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f96340a3c40 0x7f96340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.480+0000 7f963effd700 1 -- 172.21.15.94:0/247694762 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f96440cf790 con 0x7f9648103570 2024-04-26T22:54:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.481+0000 7f964d889700 1 --2- 172.21.15.94:0/247694762 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f96340a3c40 0x7f96340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.482+0000 7f964d889700 1 --2- 172.21.15.94:0/247694762 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f96340a3c40 0x7f96340a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f963800b5c0 tx=0x7f9638003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.485+0000 7f963effd700 1 -- 172.21.15.94:0/247694762 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f964401f020 con 0x7f9648103570 2024-04-26T22:54:35.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.694+0000 7f964faed700 1 -- 172.21.15.94:0/247694762 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi094=c;smithi094=f;smithi094=i;smithi094=l", "target": ["mon-mgr", ""]}) -- 0x7f96480ffc60 con 0x7f96340a3c40 2024-04-26T22:54:35.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.711+0000 7f963effd700 1 -- 172.21.15.94:0/247694762 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f96480ffc60 con 0x7f96340a3c40 2024-04-26T22:54:35.711 INFO:teuthology.orchestra.run.smithi094.stdout:Scheduled mds.all update... 2024-04-26T22:54:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.713+0000 7f964faed700 1 -- 172.21.15.94:0/247694762 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f96340a3c40 msgr2=0x7f96340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.713+0000 7f964faed700 1 --2- 172.21.15.94:0/247694762 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f96340a3c40 0x7f96340a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f963800b5c0 tx=0x7f9638003860 comp rx=0 tx=0).stop 2024-04-26T22:54:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.713+0000 7f964faed700 1 -- 172.21.15.94:0/247694762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648103570 msgr2=0x7f9648199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.713+0000 7f964faed700 1 --2- 172.21.15.94:0/247694762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648103570 0x7f9648199be0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f964400b560 tx=0x7f964400b920 comp rx=0 tx=0).stop 2024-04-26T22:54:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.713+0000 7f964faed700 1 -- 172.21.15.94:0/247694762 shutdown_connections 2024-04-26T22:54:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.713+0000 7f964faed700 1 --2- 172.21.15.94:0/247694762 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f96340a3c40 0x7f96340a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.713+0000 7f964faed700 1 --2- 172.21.15.94:0/247694762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9648103f10 0x7f96481a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.713+0000 7f964faed700 1 --2- 172.21.15.94:0/247694762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9648102370 0x7f96481996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.713+0000 7f964faed700 1 --2- 172.21.15.94:0/247694762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648103570 0x7f9648199be0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.713+0000 7f964faed700 1 -- 172.21.15.94:0/247694762 >> 172.21.15.94:0/247694762 conn(0x7f96480fd910 msgr2=0x7f96480ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.713+0000 7f964faed700 1 -- 172.21.15.94:0/247694762 shutdown_connections 2024-04-26T22:54:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T22:54:35.713+0000 7f964faed700 1 -- 172.21.15.94:0/247694762 wait complete. 2024-04-26T22:54:36.405 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:36 smithi094 ceph-mon[32889]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:36.405 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:36 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:36.406 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:36 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:54:36.431 DEBUG:teuthology.orchestra.run.smithi027:mds.a> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.a.service 2024-04-26T22:54:36.438 DEBUG:teuthology.orchestra.run.smithi027:mds.d> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.d.service 2024-04-26T22:54:36.483 DEBUG:teuthology.orchestra.run.smithi027:mds.g> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.g.service 2024-04-26T22:54:36.496 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:36 smithi027 ceph-mon[28034]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:36.496 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:36 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:36.496 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:36 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:54:36.500 DEBUG:teuthology.orchestra.run.smithi027:mds.j> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.j.service 2024-04-26T22:54:36.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:36 smithi080 ceph-mon[32324]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:36.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:36 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:36.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:36 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:54:36.547 DEBUG:teuthology.orchestra.run.smithi080:mds.b> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.b.service 2024-04-26T22:54:36.550 DEBUG:teuthology.orchestra.run.smithi080:mds.e> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.e.service 2024-04-26T22:54:36.594 DEBUG:teuthology.orchestra.run.smithi080:mds.h> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.h.service 2024-04-26T22:54:36.638 DEBUG:teuthology.orchestra.run.smithi080:mds.k> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.k.service 2024-04-26T22:54:36.681 DEBUG:teuthology.orchestra.run.smithi094:mds.c> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.c.service 2024-04-26T22:54:36.684 DEBUG:teuthology.orchestra.run.smithi094:mds.f> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.f.service 2024-04-26T22:54:36.727 DEBUG:teuthology.orchestra.run.smithi094:mds.i> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.i.service 2024-04-26T22:54:36.771 DEBUG:teuthology.orchestra.run.smithi094:mds.l> sudo journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.l.service 2024-04-26T22:54:36.814 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-26T22:54:36.814 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:54:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.064+0000 7fcccd800640 1 -- 172.21.15.27:0/1911484006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc8105b80 msgr2=0x7fccc8106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.064+0000 7fcccd800640 1 --2- 172.21.15.27:0/1911484006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc8105b80 0x7fccc8106000 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009900 tx=0x7fccb002a410 comp rx=0 tx=0).stop 2024-04-26T22:54:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.065+0000 7fcccd800640 1 -- 172.21.15.27:0/1911484006 shutdown_connections 2024-04-26T22:54:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.065+0000 7fcccd800640 1 --2- 172.21.15.27:0/1911484006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccc8106540 0x7fccc810cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.065+0000 7fcccd800640 1 --2- 172.21.15.27:0/1911484006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc8105b80 0x7fccc8106000 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.065+0000 7fcccd800640 1 --2- 172.21.15.27:0/1911484006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fccc8104990 0x7fccc8104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.065+0000 7fcccd800640 1 -- 172.21.15.27:0/1911484006 >> 172.21.15.27:0/1911484006 conn(0x7fccc80fff00 msgr2=0x7fccc8102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.065+0000 7fcccd800640 1 -- 172.21.15.27:0/1911484006 shutdown_connections 2024-04-26T22:54:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.065+0000 7fcccd800640 1 -- 172.21.15.27:0/1911484006 wait complete. 2024-04-26T22:54:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.066+0000 7fcccd800640 1 Processor -- start 2024-04-26T22:54:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.066+0000 7fcccd800640 1 -- start start 2024-04-26T22:54:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.066+0000 7fcccd800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc8104990 0x7fccc819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.066+0000 7fcccd800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccc8105b80 0x7fccc819b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.066+0000 7fcccd800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fccc8106540 0x7fccc81a2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.066+0000 7fcccd800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81104f0 con 0x7fccc8104990 2024-04-26T22:54:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.066+0000 7fcccd800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fccc8110370 con 0x7fccc8106540 2024-04-26T22:54:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.066+0000 7fcccd800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fccc8110670 con 0x7fccc8105b80 2024-04-26T22:54:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.066+0000 7fccbf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccc8105b80 0x7fccc819b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.066+0000 7fccc7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc8104990 0x7fccc819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.066+0000 7fccbf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccc8105b80 0x7fccc819b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:41194/0 (socket says 172.21.15.27:41194) 2024-04-26T22:54:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.066+0000 7fccbf400640 1 -- 172.21.15.27:0/362134081 learned_addr learned my addr 172.21.15.27:0/362134081 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:54:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.067+0000 7fcccce00640 1 --2- 172.21.15.27:0/362134081 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fccc8106540 0x7fccc81a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.067+0000 7fccbf400640 1 -- 172.21.15.27:0/362134081 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fccc8106540 msgr2=0x7fccc81a2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.067+0000 7fccbf400640 1 --2- 172.21.15.27:0/362134081 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fccc8106540 0x7fccc81a2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.067+0000 7fccbf400640 1 -- 172.21.15.27:0/362134081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc8104990 msgr2=0x7fccc819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.067+0000 7fccbf400640 1 --2- 172.21.15.27:0/362134081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc8104990 0x7fccc819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.067+0000 7fccbf400640 1 -- 172.21.15.27:0/362134081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc81a26f0 con 0x7fccc8105b80 2024-04-26T22:54:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.067+0000 7fccc7e00640 1 --2- 172.21.15.27:0/362134081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc8104990 0x7fccc819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:54:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.067+0000 7fccbf400640 1 --2- 172.21.15.27:0/362134081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccc8105b80 0x7fccc819b010 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fccb0005ab0 tx=0x7fccb0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.067+0000 7fccc5600640 1 -- 172.21.15.27:0/362134081 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb003f070 con 0x7fccc8105b80 2024-04-26T22:54:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.068+0000 7fccc5600640 1 -- 172.21.15.27:0/362134081 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccb0031c50 con 0x7fccc8105b80 2024-04-26T22:54:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.068+0000 7fcccd800640 1 -- 172.21.15.27:0/362134081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc81a2980 con 0x7fccc8105b80 2024-04-26T22:54:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.068+0000 7fccc5600640 1 -- 172.21.15.27:0/362134081 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0035030 con 0x7fccc8105b80 2024-04-26T22:54:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.068+0000 7fcccd800640 1 -- 172.21.15.27:0/362134081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc81a2ec0 con 0x7fccc8105b80 2024-04-26T22:54:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.069+0000 7fcccd800640 1 -- 172.21.15.27:0/362134081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc810bc50 con 0x7fccc8105b80 2024-04-26T22:54:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.071+0000 7fccc5600640 1 -- 172.21.15.27:0/362134081 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fccb0042070 con 0x7fccc8105b80 2024-04-26T22:54:37.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.073+0000 7fccc5600640 1 --2- 172.21.15.27:0/362134081 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fccac0a3cd0 0x7fccac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:37.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.073+0000 7fccc5600640 1 -- 172.21.15.27:0/362134081 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fccb00f0d90 con 0x7fccc8105b80 2024-04-26T22:54:37.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.073+0000 7fccc7e00640 1 --2- 172.21.15.27:0/362134081 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fccac0a3cd0 0x7fccac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.074+0000 7fccc7e00640 1 --2- 172.21.15.27:0/362134081 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fccac0a3cd0 0x7fccac0a6190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fccc81059e0 tx=0x7fccb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.075+0000 7fccc5600640 1 -- 172.21.15.27:0/362134081 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fccb00bd520 con 0x7fccc8105b80 2024-04-26T22:54:37.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.284+0000 7fcccd800640 1 -- 172.21.15.27:0/362134081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fccc802dd20 con 0x7fccc8105b80 2024-04-26T22:54:37.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.285+0000 7fccc5600640 1 -- 172.21.15.27:0/362134081 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18337 (secure 0 0 0) 0x7fccb00c1820 con 0x7fccc8105b80 2024-04-26T22:54:37.286 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:54:37.286 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":81,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:54:23.783142+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:54:37.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.288+0000 7fcccd800640 1 -- 172.21.15.27:0/362134081 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fccac0a3cd0 msgr2=0x7fccac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:37.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.288+0000 7fcccd800640 1 --2- 172.21.15.27:0/362134081 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fccac0a3cd0 0x7fccac0a6190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fccc81059e0 tx=0x7fccb8005f90 comp rx=0 tx=0).stop 2024-04-26T22:54:37.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.288+0000 7fcccd800640 1 -- 172.21.15.27:0/362134081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccc8105b80 msgr2=0x7fccc819b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:37.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.288+0000 7fcccd800640 1 --2- 172.21.15.27:0/362134081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccc8105b80 0x7fccc819b010 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fccb0005ab0 tx=0x7fccb0003b90 comp rx=0 tx=0).stop 2024-04-26T22:54:37.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.288+0000 7fcccd800640 1 -- 172.21.15.27:0/362134081 shutdown_connections 2024-04-26T22:54:37.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.288+0000 7fcccd800640 1 --2- 172.21.15.27:0/362134081 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fccac0a3cd0 0x7fccac0a6190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.288+0000 7fcccd800640 1 --2- 172.21.15.27:0/362134081 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fccc8106540 0x7fccc81a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.288+0000 7fcccd800640 1 --2- 172.21.15.27:0/362134081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccc8105b80 0x7fccc819b010 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.288+0000 7fcccd800640 1 --2- 172.21.15.27:0/362134081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc8104990 0x7fccc819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.288+0000 7fcccd800640 1 -- 172.21.15.27:0/362134081 >> 172.21.15.27:0/362134081 conn(0x7fccc80fff00 msgr2=0x7fccc8108e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:37.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.288+0000 7fcccd800640 1 -- 172.21.15.27:0/362134081 shutdown_connections 2024-04-26T22:54:37.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.288+0000 7fcccd800640 1 -- 172.21.15.27:0/362134081 wait complete. 2024-04-26T22:54:37.307 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:37 smithi027 ceph-mon[28034]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi094=c;smithi094=f;smithi094=i;smithi094=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:54:37.307 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:37 smithi027 ceph-mon[28034]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi094=c;smithi094=f;smithi094=i;smithi094=l;count:12 2024-04-26T22:54:37.307 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:37 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:37.307 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:37 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:54:37.307 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:37 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:37.307 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:37 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:37.307 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:37 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:37.307 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:37 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:37.310 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:49:54.482679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:54:37.311 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:54:37.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.519+0000 7f71b6000640 1 -- 172.21.15.27:0/2933530987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71b0105c10 msgr2=0x7f71b0108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:37.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.519+0000 7f71b6000640 1 --2- 172.21.15.27:0/2933530987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71b0105c10 0x7f71b0108000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f719c009860 tx=0x7f719c02a340 comp rx=0 tx=0).stop 2024-04-26T22:54:37.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.520+0000 7f71b6000640 1 -- 172.21.15.27:0/2933530987 shutdown_connections 2024-04-26T22:54:37.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.520+0000 7f71b6000640 1 --2- 172.21.15.27:0/2933530987 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71b0108540 0x7f71b010a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.520+0000 7f71b6000640 1 --2- 172.21.15.27:0/2933530987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71b0105c10 0x7f71b0108000 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.520+0000 7f71b6000640 1 --2- 172.21.15.27:0/2933530987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b01032e0 0x7f71b01056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.520+0000 7f71b6000640 1 -- 172.21.15.27:0/2933530987 >> 172.21.15.27:0/2933530987 conn(0x7f71b00fcc10 msgr2=0x7f71b00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:37.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.520+0000 7f71b6000640 1 -- 172.21.15.27:0/2933530987 shutdown_connections 2024-04-26T22:54:37.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.520+0000 7f71b6000640 1 -- 172.21.15.27:0/2933530987 wait complete. 2024-04-26T22:54:37.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:37 smithi080 ceph-mon[32324]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi094=c;smithi094=f;smithi094=i;smithi094=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:54:37.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:37 smithi080 ceph-mon[32324]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi094=c;smithi094=f;smithi094=i;smithi094=l;count:12 2024-04-26T22:54:37.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:37 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:37.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:37 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:54:37.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:37 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:37.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:37 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:37.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:37 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:37.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:37 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:37.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.521+0000 7f71b6000640 1 Processor -- start 2024-04-26T22:54:37.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.521+0000 7f71b6000640 1 -- start start 2024-04-26T22:54:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.521+0000 7f71b6000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71b01032e0 0x7f71b0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.521+0000 7f71b6000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b0105c10 0x7f71b0198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.521+0000 7f71b6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71b0108540 0x7f71b019ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.521+0000 7f71b6000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f71b010e2b0 con 0x7f71b0105c10 2024-04-26T22:54:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.521+0000 7f71b6000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f71b010e130 con 0x7f71b01032e0 2024-04-26T22:54:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.522+0000 7f71b6000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71b010e430 con 0x7f71b0108540 2024-04-26T22:54:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.522+0000 7f71b4c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71b01032e0 0x7f71b0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.522+0000 7f71b4c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71b01032e0 0x7f71b0198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:38288/0 (socket says 172.21.15.27:38288) 2024-04-26T22:54:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.522+0000 7f71afe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b0105c10 0x7f71b0198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.522+0000 7f71b4c00640 1 -- 172.21.15.27:0/3390506327 learned_addr learned my addr 172.21.15.27:0/3390506327 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:54:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.522+0000 7f71b5600640 1 --2- 172.21.15.27:0/3390506327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71b0108540 0x7f71b019ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.522+0000 7f71b4c00640 1 -- 172.21.15.27:0/3390506327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71b0108540 msgr2=0x7f71b019ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.522+0000 7f71b4c00640 1 --2- 172.21.15.27:0/3390506327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71b0108540 0x7f71b019ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.522+0000 7f71b4c00640 1 -- 172.21.15.27:0/3390506327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b0105c10 msgr2=0x7f71b0198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.522+0000 7f71b4c00640 1 --2- 172.21.15.27:0/3390506327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b0105c10 0x7f71b0198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.522+0000 7f71b4c00640 1 -- 172.21.15.27:0/3390506327 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b01a0630 con 0x7f71b01032e0 2024-04-26T22:54:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.523+0000 7f71b4c00640 1 --2- 172.21.15.27:0/3390506327 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71b01032e0 0x7f71b0198a10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f71a400e9c0 tx=0x7f71a400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.523+0000 7f71ad600640 1 -- 172.21.15.27:0/3390506327 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a400ccc0 con 0x7f71b01032e0 2024-04-26T22:54:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.523+0000 7f71ad600640 1 -- 172.21.15.27:0/3390506327 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71a40054d0 con 0x7f71b01032e0 2024-04-26T22:54:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.523+0000 7f71b6000640 1 -- 172.21.15.27:0/3390506327 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b01a0920 con 0x7f71b01032e0 2024-04-26T22:54:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.523+0000 7f71ad600640 1 -- 172.21.15.27:0/3390506327 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4010430 con 0x7f71b01032e0 2024-04-26T22:54:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.523+0000 7f71b6000640 1 -- 172.21.15.27:0/3390506327 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b01a0e40 con 0x7f71b01032e0 2024-04-26T22:54:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.524+0000 7f71b6000640 1 -- 172.21.15.27:0/3390506327 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b00fe700 con 0x7f71b01032e0 2024-04-26T22:54:37.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.526+0000 7f71ad600640 1 -- 172.21.15.27:0/3390506327 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f71a4004f80 con 0x7f71b01032e0 2024-04-26T22:54:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.527+0000 7f71ad600640 1 --2- 172.21.15.27:0/3390506327 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f71980a3bb0 0x7f71980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.528+0000 7f71ad600640 1 -- 172.21.15.27:0/3390506327 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f71a40d0420 con 0x7f71b01032e0 2024-04-26T22:54:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.528+0000 7f71afe00640 1 --2- 172.21.15.27:0/3390506327 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f71980a3bb0 0x7f71980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.528+0000 7f71afe00640 1 --2- 172.21.15.27:0/3390506327 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f71980a3bb0 0x7f71980a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f719c009830 tx=0x7f719c030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:37.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.532+0000 7f71ad600640 1 -- 172.21.15.27:0/3390506327 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f71a409d200 con 0x7f71b01032e0 2024-04-26T22:54:37.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:37 smithi094 ceph-mon[32889]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi094=c;smithi094=f;smithi094=i;smithi094=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:54:37.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:37 smithi094 ceph-mon[32889]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi094=c;smithi094=f;smithi094=i;smithi094=l;count:12 2024-04-26T22:54:37.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:37 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:37.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:37 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:54:37.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:37 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:37.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:37 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:37.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:37 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:37.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:37 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.739+0000 7f71b6000640 1 -- 172.21.15.27:0/3390506327 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f71b00635f0 con 0x7f71b01032e0 2024-04-26T22:54:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.740+0000 7f71ad600640 1 -- 172.21.15.27:0/3390506327 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f71a409cb00 con 0x7f71b01032e0 2024-04-26T22:54:37.742 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T22:54:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.742+0000 7f71b6000640 1 -- 172.21.15.27:0/3390506327 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f71980a3bb0 msgr2=0x7f71980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.742+0000 7f71b6000640 1 --2- 172.21.15.27:0/3390506327 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f71980a3bb0 0x7f71980a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f719c009830 tx=0x7f719c030450 comp rx=0 tx=0).stop 2024-04-26T22:54:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.742+0000 7f71b6000640 1 -- 172.21.15.27:0/3390506327 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71b01032e0 msgr2=0x7f71b0198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.742+0000 7f71b6000640 1 --2- 172.21.15.27:0/3390506327 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71b01032e0 0x7f71b0198a10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f71a400e9c0 tx=0x7f71a400ee80 comp rx=0 tx=0).stop 2024-04-26T22:54:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.743+0000 7f71b6000640 1 -- 172.21.15.27:0/3390506327 shutdown_connections 2024-04-26T22:54:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.743+0000 7f71b6000640 1 --2- 172.21.15.27:0/3390506327 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f71980a3bb0 0x7f71980a6070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.743+0000 7f71b6000640 1 --2- 172.21.15.27:0/3390506327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71b0108540 0x7f71b019ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.743+0000 7f71b6000640 1 --2- 172.21.15.27:0/3390506327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b0105c10 0x7f71b0198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.743+0000 7f71b6000640 1 --2- 172.21.15.27:0/3390506327 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71b01032e0 0x7f71b0198a10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.743+0000 7f71b6000640 1 -- 172.21.15.27:0/3390506327 >> 172.21.15.27:0/3390506327 conn(0x7f71b00fcc10 msgr2=0x7f71b0103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:37.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.743+0000 7f71b6000640 1 -- 172.21.15.27:0/3390506327 shutdown_connections 2024-04-26T22:54:37.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.743+0000 7f71b6000640 1 -- 172.21.15.27:0/3390506327 wait complete. 2024-04-26T22:54:37.762 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:54:37.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.945+0000 7fa067e00640 1 -- 172.21.15.27:0/930398117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0681027a0 msgr2=0x7fa068102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:37.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.945+0000 7fa067e00640 1 --2- 172.21.15.27:0/930398117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0681027a0 0x7fa068102bc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa058009900 tx=0x7fa05802a410 comp rx=0 tx=0).stop 2024-04-26T22:54:37.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.945+0000 7fa067e00640 1 -- 172.21.15.27:0/930398117 shutdown_connections 2024-04-26T22:54:37.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.945+0000 7fa067e00640 1 --2- 172.21.15.27:0/930398117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa068104350 0x7fa06810abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.945+0000 7fa067e00640 1 --2- 172.21.15.27:0/930398117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa068103990 0x7fa068103e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.945+0000 7fa067e00640 1 --2- 172.21.15.27:0/930398117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0681027a0 0x7fa068102bc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.945+0000 7fa067e00640 1 -- 172.21.15.27:0/930398117 >> 172.21.15.27:0/930398117 conn(0x7fa0680fdd10 msgr2=0x7fa068100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:37.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.945+0000 7fa067e00640 1 -- 172.21.15.27:0/930398117 shutdown_connections 2024-04-26T22:54:37.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.945+0000 7fa067e00640 1 -- 172.21.15.27:0/930398117 wait complete. 2024-04-26T22:54:37.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.946+0000 7fa067e00640 1 Processor -- start 2024-04-26T22:54:37.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.946+0000 7fa067e00640 1 -- start start 2024-04-26T22:54:37.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.947+0000 7fa067e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0681027a0 0x7fa0680766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:37.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.947+0000 7fa067e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa068103990 0x7fa068074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:37.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.947+0000 7fa067e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa068104350 0x7fa068075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:37.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.947+0000 7fa067e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa06810e2f0 con 0x7fa068104350 2024-04-26T22:54:37.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.947+0000 7fa067e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa06810e170 con 0x7fa0681027a0 2024-04-26T22:54:37.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.947+0000 7fa067e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa06810e470 con 0x7fa068103990 2024-04-26T22:54:37.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.947+0000 7fa066a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0681027a0 0x7fa0680766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:37.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.947+0000 7fa066a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0681027a0 0x7fa0680766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:44482/0 (socket says 172.21.15.27:44482) 2024-04-26T22:54:37.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.947+0000 7fa066a00640 1 -- 172.21.15.27:0/61918123 learned_addr learned my addr 172.21.15.27:0/61918123 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:54:37.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.947+0000 7fa05fe00640 1 --2- 172.21.15.27:0/61918123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa068103990 0x7fa068074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:37.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.947+0000 7fa067400640 1 --2- 172.21.15.27:0/61918123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa068104350 0x7fa068075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:37.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.947+0000 7fa066a00640 1 -- 172.21.15.27:0/61918123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa068103990 msgr2=0x7fa068074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:37.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.947+0000 7fa066a00640 1 --2- 172.21.15.27:0/61918123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa068103990 0x7fa068074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.947+0000 7fa066a00640 1 -- 172.21.15.27:0/61918123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa068104350 msgr2=0x7fa068075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:37.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.947+0000 7fa066a00640 1 --2- 172.21.15.27:0/61918123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa068104350 0x7fa068075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:37.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.947+0000 7fa066a00640 1 -- 172.21.15.27:0/61918123 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa068075af0 con 0x7fa0681027a0 2024-04-26T22:54:37.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.948+0000 7fa066a00640 1 --2- 172.21.15.27:0/61918123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0681027a0 0x7fa0680766e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa058031500 tx=0x7fa058003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:37.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.948+0000 7fa05f400640 1 -- 172.21.15.27:0/61918123 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05803f070 con 0x7fa0681027a0 2024-04-26T22:54:37.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.948+0000 7fa067e00640 1 -- 172.21.15.27:0/61918123 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0681a29f0 con 0x7fa0681027a0 2024-04-26T22:54:37.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.948+0000 7fa05f400640 1 -- 172.21.15.27:0/61918123 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa058031c50 con 0x7fa0681027a0 2024-04-26T22:54:37.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.948+0000 7fa05f400640 1 -- 172.21.15.27:0/61918123 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058035030 con 0x7fa0681027a0 2024-04-26T22:54:37.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.948+0000 7fa067e00640 1 -- 172.21.15.27:0/61918123 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0681a2f90 con 0x7fa0681027a0 2024-04-26T22:54:37.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.949+0000 7fa067e00640 1 -- 172.21.15.27:0/61918123 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa068109ac0 con 0x7fa0681027a0 2024-04-26T22:54:37.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.951+0000 7fa05f400640 1 -- 172.21.15.27:0/61918123 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa058042070 con 0x7fa0681027a0 2024-04-26T22:54:37.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.953+0000 7fa05f400640 1 --2- 172.21.15.27:0/61918123 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa04c0a3cd0 0x7fa04c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:37.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.953+0000 7fa05f400640 1 -- 172.21.15.27:0/61918123 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa0580f0d70 con 0x7fa0681027a0 2024-04-26T22:54:37.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.953+0000 7fa05fe00640 1 --2- 172.21.15.27:0/61918123 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa04c0a3cd0 0x7fa04c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:37.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.954+0000 7fa05fe00640 1 --2- 172.21.15.27:0/61918123 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa04c0a3cd0 0x7fa04c0a6190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa068076450 tx=0x7fa050005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:37.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:37.955+0000 7fa05f400640 1 -- 172.21.15.27:0/61918123 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa0580bd500 con 0x7fa0681027a0 2024-04-26T22:54:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:38.200+0000 7fa067e00640 1 -- 172.21.15.27:0/61918123 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa06802dad0 con 0x7fa0681027a0 2024-04-26T22:54:38.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:38.201+0000 7fa05f400640 1 -- 172.21.15.27:0/61918123 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fa0580c1800 con 0x7fa0681027a0 2024-04-26T22:54:38.201 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:54:38.201 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-26T22:54:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 1 2024-04-26T22:54:38.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:38.203+0000 7fa067e00640 1 -- 172.21.15.27:0/61918123 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa04c0a3cd0 msgr2=0x7fa04c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:38.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:38.203+0000 7fa067e00640 1 --2- 172.21.15.27:0/61918123 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa04c0a3cd0 0x7fa04c0a6190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa068076450 tx=0x7fa050005f90 comp rx=0 tx=0).stop 2024-04-26T22:54:38.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:38.203+0000 7fa067e00640 1 -- 172.21.15.27:0/61918123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0681027a0 msgr2=0x7fa0680766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:38.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:38.203+0000 7fa067e00640 1 --2- 172.21.15.27:0/61918123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0681027a0 0x7fa0680766e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa058031500 tx=0x7fa058003c10 comp rx=0 tx=0).stop 2024-04-26T22:54:38.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:38.203+0000 7fa067e00640 1 -- 172.21.15.27:0/61918123 shutdown_connections 2024-04-26T22:54:38.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:38.203+0000 7fa067e00640 1 --2- 172.21.15.27:0/61918123 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa04c0a3cd0 0x7fa04c0a6190 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:38.203+0000 7fa067e00640 1 --2- 172.21.15.27:0/61918123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa068104350 0x7fa068075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:38.203+0000 7fa067e00640 1 --2- 172.21.15.27:0/61918123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa068103990 0x7fa068074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:38.203+0000 7fa067e00640 1 --2- 172.21.15.27:0/61918123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0681027a0 0x7fa0680766e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:38.203+0000 7fa067e00640 1 -- 172.21.15.27:0/61918123 >> 172.21.15.27:0/61918123 conn(0x7fa0680fdd10 msgr2=0x7fa068106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:38.204+0000 7fa067e00640 1 -- 172.21.15.27:0/61918123 shutdown_connections 2024-04-26T22:54:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:38.204+0000 7fa067e00640 1 -- 172.21.15.27:0/61918123 wait complete. 2024-04-26T22:54:38.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:38 smithi094 ceph-mon[32889]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:38.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:38 smithi094 ceph-mon[32889]: Deploying daemon mds.l on smithi094 2024-04-26T22:54:38.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:38 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/362134081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:54:38.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:38 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3390506327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:54:38.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:38 smithi027 ceph-mon[28034]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:38.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:38 smithi027 ceph-mon[28034]: Deploying daemon mds.l on smithi094 2024-04-26T22:54:38.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:38 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/362134081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:54:38.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:38 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3390506327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:54:38.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:38 smithi080 ceph-mon[32324]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:38.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:38 smithi080 ceph-mon[32324]: Deploying daemon mds.l on smithi094 2024-04-26T22:54:38.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:38 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/362134081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:54:38.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:38 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3390506327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:54:39.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:39 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/61918123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:39.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:39 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/61918123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:39.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:39 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/61918123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:39.688 INFO:journalctl@ceph.mds.l.smithi094.stdout:Apr 26 22:54:39 smithi094 systemd[1]: Starting Ceph mds.l for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T22:54:39.949 INFO:journalctl@ceph.mds.l.smithi094.stdout:Apr 26 22:54:39 smithi094 podman[62866]: 2024-04-26 22:54:39.591785338 +0000 UTC m=+0.017924825 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:54:39.949 INFO:journalctl@ceph.mds.l.smithi094.stdout:Apr 26 22:54:39 smithi094 podman[62866]: 2024-04-26 22:54:39.721301847 +0000 UTC m=+0.147441312 container create c369057cfe5b86b5227fd45dcef707ebdce8c257dbd4c026e9a2db8fbde23eb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:54:39.949 INFO:journalctl@ceph.mds.l.smithi094.stdout:Apr 26 22:54:39 smithi094 podman[62866]: 2024-04-26 22:54:39.949233309 +0000 UTC m=+0.375372773 container init c369057cfe5b86b5227fd45dcef707ebdce8c257dbd4c026e9a2db8fbde23eb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-26T22:54:40.222 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:54:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:40 smithi094 ceph-mon[32889]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:40 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.94:6832/828002248,v1:172.21.15.94:6833/828002248] up:boot 2024-04-26T22:54:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:40 smithi094 ceph-mon[32889]: fsmap 1 up:standby 2024-04-26T22:54:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:40 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T22:54:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:40 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:40 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:40.285 INFO:journalctl@ceph.mds.l.smithi094.stdout:Apr 26 22:54:39 smithi094 podman[62866]: 2024-04-26 22:54:39.952081073 +0000 UTC m=+0.378220546 container start c369057cfe5b86b5227fd45dcef707ebdce8c257dbd4c026e9a2db8fbde23eb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:54:40.285 INFO:journalctl@ceph.mds.l.smithi094.stdout:Apr 26 22:54:39 smithi094 ceph-mds[62885]: -- 172.21.15.94:0/3583209082 <== mon.1 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560e159d3a40 con 0x560e159a6c00 2024-04-26T22:54:40.285 INFO:journalctl@ceph.mds.l.smithi094.stdout:Apr 26 22:54:39 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-l[62881]: starting mds.l at 2024-04-26T22:54:40.285 INFO:journalctl@ceph.mds.l.smithi094.stdout:Apr 26 22:54:40 smithi094 bash[62866]: c369057cfe5b86b5227fd45dcef707ebdce8c257dbd4c026e9a2db8fbde23eb1 2024-04-26T22:54:40.285 INFO:journalctl@ceph.mds.l.smithi094.stdout:Apr 26 22:54:40 smithi094 systemd[1]: Started Ceph mds.l for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:54:40.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.452+0000 7fb55f000640 1 -- 172.21.15.27:0/428311083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb558106720 msgr2=0x7fb55810cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:40.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.452+0000 7fb55f000640 1 --2- 172.21.15.27:0/428311083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb558106720 0x7fb55810cfb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb550009ca0 tx=0x7fb55002a8c0 comp rx=0 tx=0).stop 2024-04-26T22:54:40.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.452+0000 7fb55f000640 1 -- 172.21.15.27:0/428311083 shutdown_connections 2024-04-26T22:54:40.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.452+0000 7fb55f000640 1 --2- 172.21.15.27:0/428311083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb558106720 0x7fb55810cfb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:40.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.452+0000 7fb55f000640 1 --2- 172.21.15.27:0/428311083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb558105d60 0x7fb5581061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:40.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.452+0000 7fb55f000640 1 --2- 172.21.15.27:0/428311083 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558104b70 0x7fb558104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:40.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.452+0000 7fb55f000640 1 -- 172.21.15.27:0/428311083 >> 172.21.15.27:0/428311083 conn(0x7fb5581000e0 msgr2=0x7fb558102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:40.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.452+0000 7fb55f000640 1 -- 172.21.15.27:0/428311083 shutdown_connections 2024-04-26T22:54:40.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.452+0000 7fb55f000640 1 -- 172.21.15.27:0/428311083 wait complete. 2024-04-26T22:54:40.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.453+0000 7fb55f000640 1 Processor -- start 2024-04-26T22:54:40.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.453+0000 7fb55f000640 1 -- start start 2024-04-26T22:54:40.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.453+0000 7fb55f000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558104b70 0x7fb55819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:40.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.453+0000 7fb55f000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb558105d60 0x7fb55819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:40.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.453+0000 7fb55f000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb558106720 0x7fb5581a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:40.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.453+0000 7fb55f000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb558110680 con 0x7fb558105d60 2024-04-26T22:54:40.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.453+0000 7fb55f000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb558110500 con 0x7fb558104b70 2024-04-26T22:54:40.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.453+0000 7fb55f000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb558110800 con 0x7fb558106720 2024-04-26T22:54:40.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.454+0000 7fb557e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558104b70 0x7fb55819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:40.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.454+0000 7fb557e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558104b70 0x7fb55819ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:44494/0 (socket says 172.21.15.27:44494) 2024-04-26T22:54:40.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.454+0000 7fb557e00640 1 -- 172.21.15.27:0/2869653816 learned_addr learned my addr 172.21.15.27:0/2869653816 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:54:40.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.454+0000 7fb55d000640 1 --2- 172.21.15.27:0/2869653816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb558106720 0x7fb5581a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:40.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.454+0000 7fb557400640 1 --2- 172.21.15.27:0/2869653816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb558105d60 0x7fb55819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:40.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.454+0000 7fb557e00640 1 -- 172.21.15.27:0/2869653816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb558106720 msgr2=0x7fb5581a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:40.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.454+0000 7fb557e00640 1 --2- 172.21.15.27:0/2869653816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb558106720 0x7fb5581a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:40.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.454+0000 7fb557e00640 1 -- 172.21.15.27:0/2869653816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb558105d60 msgr2=0x7fb55819b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:40.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.454+0000 7fb557e00640 1 --2- 172.21.15.27:0/2869653816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb558105d60 0x7fb55819b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:40.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.454+0000 7fb557e00640 1 -- 172.21.15.27:0/2869653816 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5581a2920 con 0x7fb558104b70 2024-04-26T22:54:40.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.454+0000 7fb557e00640 1 --2- 172.21.15.27:0/2869653816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558104b70 0x7fb55819ad10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb54800e9c0 tx=0x7fb54800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:40.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.455+0000 7fb554c00640 1 -- 172.21.15.27:0/2869653816 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54800cd80 con 0x7fb558104b70 2024-04-26T22:54:40.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.455+0000 7fb554c00640 1 -- 172.21.15.27:0/2869653816 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5480035f0 con 0x7fb558104b70 2024-04-26T22:54:40.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.455+0000 7fb554c00640 1 -- 172.21.15.27:0/2869653816 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548010690 con 0x7fb558104b70 2024-04-26T22:54:40.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.455+0000 7fb55f000640 1 -- 172.21.15.27:0/2869653816 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5581a2c10 con 0x7fb558104b70 2024-04-26T22:54:40.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.455+0000 7fb55f000640 1 -- 172.21.15.27:0/2869653816 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5581a3110 con 0x7fb558104b70 2024-04-26T22:54:40.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.456+0000 7fb55f000640 1 -- 172.21.15.27:0/2869653816 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb55810be90 con 0x7fb558104b70 2024-04-26T22:54:40.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.458+0000 7fb554c00640 1 -- 172.21.15.27:0/2869653816 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb5480037a0 con 0x7fb558104b70 2024-04-26T22:54:40.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.459+0000 7fb554c00640 1 --2- 172.21.15.27:0/2869653816 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb53c0a3bb0 0x7fb53c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:40.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.459+0000 7fb554c00640 1 -- 172.21.15.27:0/2869653816 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb5480d04a0 con 0x7fb558104b70 2024-04-26T22:54:40.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.460+0000 7fb557400640 1 --2- 172.21.15.27:0/2869653816 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb53c0a3bb0 0x7fb53c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:40.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.460+0000 7fb557400640 1 --2- 172.21.15.27:0/2869653816 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb53c0a3bb0 0x7fb53c0a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb5440095f0 tx=0x7fb544005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:40.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.463+0000 7fb554c00640 1 -- 172.21.15.27:0/2869653816 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb54809d280 con 0x7fb558104b70 2024-04-26T22:54:40.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:40 smithi080 ceph-mon[32324]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:40.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:40 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.94:6832/828002248,v1:172.21.15.94:6833/828002248] up:boot 2024-04-26T22:54:40.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:40 smithi080 ceph-mon[32324]: fsmap 1 up:standby 2024-04-26T22:54:40.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:40 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T22:54:40.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:40 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:40.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:40 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:40.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:40 smithi027 ceph-mon[28034]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:40.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:40 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.94:6832/828002248,v1:172.21.15.94:6833/828002248] up:boot 2024-04-26T22:54:40.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:40 smithi027 ceph-mon[28034]: fsmap 1 up:standby 2024-04-26T22:54:40.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:40 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T22:54:40.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:40 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:40.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:40 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:40.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.712+0000 7fb55f000640 1 -- 172.21.15.27:0/2869653816 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb55802dad0 con 0x7fb558104b70 2024-04-26T22:54:40.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.713+0000 7fb554c00640 1 -- 172.21.15.27:0/2869653816 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1390 (secure 0 0 0) 0x7fb54809cb80 con 0x7fb558104b70 2024-04-26T22:54:40.714 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:54:40.714 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T22:54:40.714 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 2 2024-04-26T22:54:40.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.716+0000 7fb55f000640 1 -- 172.21.15.27:0/2869653816 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb53c0a3bb0 msgr2=0x7fb53c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:40.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.716+0000 7fb55f000640 1 --2- 172.21.15.27:0/2869653816 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb53c0a3bb0 0x7fb53c0a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb5440095f0 tx=0x7fb544005f90 comp rx=0 tx=0).stop 2024-04-26T22:54:40.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.716+0000 7fb55f000640 1 -- 172.21.15.27:0/2869653816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558104b70 msgr2=0x7fb55819ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:40.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.716+0000 7fb55f000640 1 --2- 172.21.15.27:0/2869653816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558104b70 0x7fb55819ad10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb54800e9c0 tx=0x7fb54800ee80 comp rx=0 tx=0).stop 2024-04-26T22:54:40.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.716+0000 7fb55f000640 1 -- 172.21.15.27:0/2869653816 shutdown_connections 2024-04-26T22:54:40.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.716+0000 7fb55f000640 1 --2- 172.21.15.27:0/2869653816 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb53c0a3bb0 0x7fb53c0a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:40.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.716+0000 7fb55f000640 1 --2- 172.21.15.27:0/2869653816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb558106720 0x7fb5581a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:40.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.716+0000 7fb55f000640 1 --2- 172.21.15.27:0/2869653816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb558105d60 0x7fb55819b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:40.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.716+0000 7fb55f000640 1 --2- 172.21.15.27:0/2869653816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558104b70 0x7fb55819ad10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:40.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.716+0000 7fb55f000640 1 -- 172.21.15.27:0/2869653816 >> 172.21.15.27:0/2869653816 conn(0x7fb5581000e0 msgr2=0x7fb558109060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:40.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.716+0000 7fb55f000640 1 -- 172.21.15.27:0/2869653816 shutdown_connections 2024-04-26T22:54:40.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:40.716+0000 7fb55f000640 1 -- 172.21.15.27:0/2869653816 wait complete. 2024-04-26T22:54:41.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:41 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:41.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:41 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:41.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:41 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:41.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:41 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:41.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:41 smithi080 ceph-mon[32324]: Deploying daemon mds.k on smithi080 2024-04-26T22:54:41.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:41 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2869653816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:41.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:41 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:41.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:41 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:41.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:41 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:41.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:41 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:41.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:41 smithi094 ceph-mon[32889]: Deploying daemon mds.k on smithi080 2024-04-26T22:54:41.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:41 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2869653816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:41.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:41 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:41.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:41 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:41.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:41 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:41.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:41 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:41.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:41 smithi027 ceph-mon[28034]: Deploying daemon mds.k on smithi080 2024-04-26T22:54:41.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:41 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2869653816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:42.243 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:42 smithi080 ceph-mon[32324]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:42.521 INFO:journalctl@ceph.mds.k.smithi080.stdout:Apr 26 22:54:42 smithi080 systemd[1]: Starting Ceph mds.k for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T22:54:42.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:42 smithi094 ceph-mon[32889]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:42.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:42 smithi027 ceph-mon[28034]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:42.737 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:54:42.891 INFO:journalctl@ceph.mds.k.smithi080.stdout:Apr 26 22:54:42 smithi080 podman[62055]: 2024-04-26 22:54:42.543955889 +0000 UTC m=+0.017600168 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:54:42.891 INFO:journalctl@ceph.mds.k.smithi080.stdout:Apr 26 22:54:42 smithi080 podman[62055]: 2024-04-26 22:54:42.652913578 +0000 UTC m=+0.126557882 container create 7012fecda154b31978e2fa53777cca52d14620043bd7dba9abcd8f5eb5c0fcaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True) 2024-04-26T22:54:42.891 INFO:journalctl@ceph.mds.k.smithi080.stdout:Apr 26 22:54:42 smithi080 podman[62055]: 2024-04-26 22:54:42.888885601 +0000 UTC m=+0.362529862 container init 7012fecda154b31978e2fa53777cca52d14620043bd7dba9abcd8f5eb5c0fcaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-26T22:54:42.891 INFO:journalctl@ceph.mds.k.smithi080.stdout:Apr 26 22:54:42 smithi080 podman[62055]: 2024-04-26 22:54:42.891465667 +0000 UTC m=+0.365109937 container start 7012fecda154b31978e2fa53777cca52d14620043bd7dba9abcd8f5eb5c0fcaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-26T22:54:42.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.962+0000 7fa8c4800640 1 -- 172.21.15.27:0/2202581350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8bc078ed0 msgr2=0x7fa8bc079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:42.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.962+0000 7fa8c4800640 1 --2- 172.21.15.27:0/2202581350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8bc078ed0 0x7fa8bc079770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009ca0 tx=0x7fa8b802a8c0 comp rx=0 tx=0).stop 2024-04-26T22:54:42.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.962+0000 7fa8c4800640 1 -- 172.21.15.27:0/2202581350 shutdown_connections 2024-04-26T22:54:42.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.962+0000 7fa8c4800640 1 --2- 172.21.15.27:0/2202581350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8bc078ed0 0x7fa8bc079770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:42.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.962+0000 7fa8c4800640 1 --2- 172.21.15.27:0/2202581350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8bc078530 0x7fa8bc078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:42.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.962+0000 7fa8c4800640 1 --2- 172.21.15.27:0/2202581350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8bc0772f0 0x7fa8bc077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:42.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.962+0000 7fa8c4800640 1 -- 172.21.15.27:0/2202581350 >> 172.21.15.27:0/2202581350 conn(0x7fa8bc1000e0 msgr2=0x7fa8bc102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:42.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.962+0000 7fa8c4800640 1 -- 172.21.15.27:0/2202581350 shutdown_connections 2024-04-26T22:54:42.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.962+0000 7fa8c4800640 1 -- 172.21.15.27:0/2202581350 wait complete. 2024-04-26T22:54:42.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.963+0000 7fa8c4800640 1 Processor -- start 2024-04-26T22:54:42.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.963+0000 7fa8c4800640 1 -- start start 2024-04-26T22:54:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.964+0000 7fa8c4800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8bc0772f0 0x7fa8bc19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.964+0000 7fa8c4800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8bc078530 0x7fa8bc19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.964+0000 7fa8c4800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8bc078ed0 0x7fa8bc1a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.964+0000 7fa8c4800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc114af0 con 0x7fa8bc0772f0 2024-04-26T22:54:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.964+0000 7fa8c4800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc114970 con 0x7fa8bc078ed0 2024-04-26T22:54:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.964+0000 7fa8c4800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc114c70 con 0x7fa8bc078530 2024-04-26T22:54:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.964+0000 7fa8c1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8bc0772f0 0x7fa8bc19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.964+0000 7fa8c2800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8bc078ed0 0x7fa8bc1a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.964+0000 7fa8c1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8bc078530 0x7fa8bc19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.964+0000 7fa8c1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8bc0772f0 0x7fa8bc19f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58230/0 (socket says 172.21.15.27:58230) 2024-04-26T22:54:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.964+0000 7fa8c1e00640 1 -- 172.21.15.27:0/726459358 learned_addr learned my addr 172.21.15.27:0/726459358 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:54:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.964+0000 7fa8c2800640 1 -- 172.21.15.27:0/726459358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8bc078530 msgr2=0x7fa8bc19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.964+0000 7fa8c2800640 1 --2- 172.21.15.27:0/726459358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8bc078530 0x7fa8bc19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.964+0000 7fa8c2800640 1 -- 172.21.15.27:0/726459358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8bc0772f0 msgr2=0x7fa8bc19f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.964+0000 7fa8c2800640 1 --2- 172.21.15.27:0/726459358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8bc0772f0 0x7fa8bc19f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.964+0000 7fa8c2800640 1 -- 172.21.15.27:0/726459358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc1a6d90 con 0x7fa8bc078ed0 2024-04-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.965+0000 7fa8c2800640 1 --2- 172.21.15.27:0/726459358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8bc078ed0 0x7fa8bc1a6720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8031a20 tx=0x7fa8b80316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.965+0000 7fa8b6a00640 1 -- 172.21.15.27:0/726459358 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8003b70 con 0x7fa8bc078ed0 2024-04-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.965+0000 7fa8c4800640 1 -- 172.21.15.27:0/726459358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8bc1a7020 con 0x7fa8bc078ed0 2024-04-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.965+0000 7fa8b6a00640 1 -- 172.21.15.27:0/726459358 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8b8003d10 con 0x7fa8bc078ed0 2024-04-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.965+0000 7fa8b6a00640 1 -- 172.21.15.27:0/726459358 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b802fae0 con 0x7fa8bc078ed0 2024-04-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.965+0000 7fa8c4800640 1 -- 172.21.15.27:0/726459358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8bc1a7570 con 0x7fa8bc078ed0 2024-04-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.966+0000 7fa8c4800640 1 -- 172.21.15.27:0/726459358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8bc1114e0 con 0x7fa8bc078ed0 2024-04-26T22:54:42.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.968+0000 7fa8b6a00640 1 -- 172.21.15.27:0/726459358 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa8b8004610 con 0x7fa8bc078ed0 2024-04-26T22:54:42.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.970+0000 7fa8b6a00640 1 --2- 172.21.15.27:0/726459358 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa8a40a3bb0 0x7fa8a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:42.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.970+0000 7fa8b6a00640 1 -- 172.21.15.27:0/726459358 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa8b80f1340 con 0x7fa8bc078ed0 2024-04-26T22:54:42.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.970+0000 7fa8c1e00640 1 --2- 172.21.15.27:0/726459358 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa8a40a3bb0 0x7fa8a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:42.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.971+0000 7fa8c1e00640 1 --2- 172.21.15.27:0/726459358 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa8a40a3bb0 0x7fa8a40a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009c90 tx=0x7fa8b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:42.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:42.974+0000 7fa8b6a00640 1 -- 172.21.15.27:0/726459358 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa8b80be120 con 0x7fa8bc078ed0 2024-04-26T22:54:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:43.242+0000 7fa8c4800640 1 -- 172.21.15.27:0/726459358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8bc063530 con 0x7fa8bc078ed0 2024-04-26T22:54:43.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:43.242+0000 7fa8b6a00640 1 -- 172.21.15.27:0/726459358 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2212 (secure 0 0 0) 0x7fa8b80bda20 con 0x7fa8bc078ed0 2024-04-26T22:54:43.243 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:54:43.243 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T22:54:43.243 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 3 2024-04-26T22:54:43.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:43.245+0000 7fa8c4800640 1 -- 172.21.15.27:0/726459358 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa8a40a3bb0 msgr2=0x7fa8a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:43.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:43.245+0000 7fa8c4800640 1 --2- 172.21.15.27:0/726459358 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa8a40a3bb0 0x7fa8a40a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009c90 tx=0x7fa8b0005f90 comp rx=0 tx=0).stop 2024-04-26T22:54:43.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:43.245+0000 7fa8c4800640 1 -- 172.21.15.27:0/726459358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8bc078ed0 msgr2=0x7fa8bc1a6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:43.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:43.245+0000 7fa8c4800640 1 --2- 172.21.15.27:0/726459358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8bc078ed0 0x7fa8bc1a6720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8031a20 tx=0x7fa8b80316f0 comp rx=0 tx=0).stop 2024-04-26T22:54:43.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:43.245+0000 7fa8c4800640 1 -- 172.21.15.27:0/726459358 shutdown_connections 2024-04-26T22:54:43.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:43.245+0000 7fa8c4800640 1 --2- 172.21.15.27:0/726459358 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa8a40a3bb0 0x7fa8a40a6070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:43.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:43.245+0000 7fa8c4800640 1 --2- 172.21.15.27:0/726459358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8bc078ed0 0x7fa8bc1a6720 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:43.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:43.245+0000 7fa8c4800640 1 --2- 172.21.15.27:0/726459358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8bc078530 0x7fa8bc19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:43.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:43.245+0000 7fa8c4800640 1 --2- 172.21.15.27:0/726459358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8bc0772f0 0x7fa8bc19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:43.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:43.245+0000 7fa8c4800640 1 -- 172.21.15.27:0/726459358 >> 172.21.15.27:0/726459358 conn(0x7fa8bc1000e0 msgr2=0x7fa8bc101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:43.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:43.246+0000 7fa8c4800640 1 -- 172.21.15.27:0/726459358 shutdown_connections 2024-04-26T22:54:43.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:43.246+0000 7fa8c4800640 1 -- 172.21.15.27:0/726459358 wait complete. 2024-04-26T22:54:43.271 INFO:journalctl@ceph.mds.k.smithi080.stdout:Apr 26 22:54:42 smithi080 ceph-mds[62074]: -- 172.21.15.80:0/2097560275 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555e81d75a40 con 0x555e81d48800 2024-04-26T22:54:43.271 INFO:journalctl@ceph.mds.k.smithi080.stdout:Apr 26 22:54:42 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-k[62070]: starting mds.k at 2024-04-26T22:54:43.271 INFO:journalctl@ceph.mds.k.smithi080.stdout:Apr 26 22:54:42 smithi080 bash[62055]: 7012fecda154b31978e2fa53777cca52d14620043bd7dba9abcd8f5eb5c0fcaa 2024-04-26T22:54:43.271 INFO:journalctl@ceph.mds.k.smithi080.stdout:Apr 26 22:54:43 smithi080 systemd[1]: Started Ceph mds.k for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:54:44.322 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:44 smithi080 ceph-mon[32324]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:44.322 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:44 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:44.322 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:44 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:44.322 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:44 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:44.322 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:44 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:44.322 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:44 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:44.323 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:44 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:44.323 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:44 smithi080 ceph-mon[32324]: Deploying daemon mds.j on smithi027 2024-04-26T22:54:44.323 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:44 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.80:6832/899705990,v1:172.21.15.80:6833/899705990] up:boot 2024-04-26T22:54:44.323 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:44 smithi080 ceph-mon[32324]: fsmap 2 up:standby 2024-04-26T22:54:44.323 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:44 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T22:54:44.323 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:44 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/726459358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:44.338 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:44 smithi027 ceph-mon[28034]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:44.338 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:44 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:44.338 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:44 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:44.338 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:44 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:44.338 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:44 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:44.338 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:44 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:44.339 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:44 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:44.339 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:44 smithi027 ceph-mon[28034]: Deploying daemon mds.j on smithi027 2024-04-26T22:54:44.339 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:44 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.80:6832/899705990,v1:172.21.15.80:6833/899705990] up:boot 2024-04-26T22:54:44.339 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:44 smithi027 ceph-mon[28034]: fsmap 2 up:standby 2024-04-26T22:54:44.339 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:44 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T22:54:44.339 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:44 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/726459358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:44.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:44 smithi094 ceph-mon[32889]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:44.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:44 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:44.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:44 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:44.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:44 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:44.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:44 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:44.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:44 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:44.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:44 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:44.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:44 smithi094 ceph-mon[32889]: Deploying daemon mds.j on smithi027 2024-04-26T22:54:44.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:44 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.80:6832/899705990,v1:172.21.15.80:6833/899705990] up:boot 2024-04-26T22:54:44.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:44 smithi094 ceph-mon[32889]: fsmap 2 up:standby 2024-04-26T22:54:44.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:44 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T22:54:44.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:44 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/726459358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:45.266 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:54:45.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.495+0000 7fc817e00640 1 -- 172.21.15.27:0/524811058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc818104350 msgr2=0x7fc81810abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:45.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.495+0000 7fc817e00640 1 --2- 172.21.15.27:0/524811058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc818104350 0x7fc81810abe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc810009be0 tx=0x7fc81002a6c0 comp rx=0 tx=0).stop 2024-04-26T22:54:45.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.496+0000 7fc817e00640 1 -- 172.21.15.27:0/524811058 shutdown_connections 2024-04-26T22:54:45.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.496+0000 7fc817e00640 1 --2- 172.21.15.27:0/524811058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc818104350 0x7fc81810abe0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:45.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.496+0000 7fc817e00640 1 --2- 172.21.15.27:0/524811058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc818103990 0x7fc818103e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:45.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.496+0000 7fc817e00640 1 --2- 172.21.15.27:0/524811058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8181027a0 0x7fc818102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:45.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.496+0000 7fc817e00640 1 -- 172.21.15.27:0/524811058 >> 172.21.15.27:0/524811058 conn(0x7fc8180fdd10 msgr2=0x7fc818100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:45.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.496+0000 7fc817e00640 1 -- 172.21.15.27:0/524811058 shutdown_connections 2024-04-26T22:54:45.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.496+0000 7fc817e00640 1 -- 172.21.15.27:0/524811058 wait complete. 2024-04-26T22:54:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.497+0000 7fc817e00640 1 Processor -- start 2024-04-26T22:54:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.497+0000 7fc817e00640 1 -- start start 2024-04-26T22:54:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.497+0000 7fc817e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8181027a0 0x7fc8181988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.497+0000 7fc817e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc818103990 0x7fc818198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.497+0000 7fc817e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc818104350 0x7fc81819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.497+0000 7fc817e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc81810e2d0 con 0x7fc818104350 2024-04-26T22:54:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.497+0000 7fc817e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc81810e150 con 0x7fc818103990 2024-04-26T22:54:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.497+0000 7fc817e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc81810e450 con 0x7fc8181027a0 2024-04-26T22:54:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.497+0000 7fc817400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc818104350 0x7fc81819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.497+0000 7fc816000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc818103990 0x7fc818198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.498+0000 7fc817400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc818104350 0x7fc81819fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58258/0 (socket says 172.21.15.27:58258) 2024-04-26T22:54:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.498+0000 7fc817400640 1 -- 172.21.15.27:0/2277182273 learned_addr learned my addr 172.21.15.27:0/2277182273 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:54:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.498+0000 7fc816a00640 1 --2- 172.21.15.27:0/2277182273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8181027a0 0x7fc8181988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.498+0000 7fc817400640 1 -- 172.21.15.27:0/2277182273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8181027a0 msgr2=0x7fc8181988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.498+0000 7fc817400640 1 --2- 172.21.15.27:0/2277182273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8181027a0 0x7fc8181988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.498+0000 7fc817400640 1 -- 172.21.15.27:0/2277182273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc818103990 msgr2=0x7fc818198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.498+0000 7fc817400640 1 --2- 172.21.15.27:0/2277182273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc818103990 0x7fc818198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.498+0000 7fc817400640 1 -- 172.21.15.27:0/2277182273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8181a04b0 con 0x7fc818104350 2024-04-26T22:54:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.498+0000 7fc816000640 1 --2- 172.21.15.27:0/2277182273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc818103990 0x7fc818198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:54:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.498+0000 7fc817400640 1 --2- 172.21.15.27:0/2277182273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc818104350 0x7fc81819fe40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc810034040 tx=0x7fc810030d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.499+0000 7fc80f400640 1 -- 172.21.15.27:0/2277182273 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc810041020 con 0x7fc818104350 2024-04-26T22:54:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.499+0000 7fc80f400640 1 -- 172.21.15.27:0/2277182273 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8100318f0 con 0x7fc818104350 2024-04-26T22:54:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.499+0000 7fc817e00640 1 -- 172.21.15.27:0/2277182273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc818074b60 con 0x7fc818104350 2024-04-26T22:54:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.499+0000 7fc80f400640 1 -- 172.21.15.27:0/2277182273 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc810036040 con 0x7fc818104350 2024-04-26T22:54:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.499+0000 7fc817e00640 1 -- 172.21.15.27:0/2277182273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc818075050 con 0x7fc818104350 2024-04-26T22:54:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.500+0000 7fc817e00640 1 -- 172.21.15.27:0/2277182273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc818109a60 con 0x7fc818104350 2024-04-26T22:54:45.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.502+0000 7fc80f400640 1 -- 172.21.15.27:0/2277182273 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc810044020 con 0x7fc818104350 2024-04-26T22:54:45.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.504+0000 7fc80f400640 1 --2- 172.21.15.27:0/2277182273 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc7fc0a3c50 0x7fc7fc0a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:45.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.504+0000 7fc80f400640 1 -- 172.21.15.27:0/2277182273 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc8100f1250 con 0x7fc818104350 2024-04-26T22:54:45.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.504+0000 7fc816a00640 1 --2- 172.21.15.27:0/2277182273 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc7fc0a3c50 0x7fc7fc0a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:45.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.505+0000 7fc816a00640 1 --2- 172.21.15.27:0/2277182273 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc7fc0a3c50 0x7fc7fc0a6110 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc8181037f0 tx=0x7fc808009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:45.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.507+0000 7fc80f400640 1 -- 172.21.15.27:0/2277182273 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc81004b030 con 0x7fc818104350 2024-04-26T22:54:45.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.783+0000 7fc817e00640 1 -- 172.21.15.27:0/2277182273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc81802dad0 con 0x7fc818104350 2024-04-26T22:54:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.785+0000 7fc80f400640 1 -- 172.21.15.27:0/2277182273 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2212 (secure 0 0 0) 0x7fc8100be030 con 0x7fc818104350 2024-04-26T22:54:45.784 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:54:45.784 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T22:54:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 3 2024-04-26T22:54:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.787+0000 7fc817e00640 1 -- 172.21.15.27:0/2277182273 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc7fc0a3c50 msgr2=0x7fc7fc0a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.787+0000 7fc817e00640 1 --2- 172.21.15.27:0/2277182273 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc7fc0a3c50 0x7fc7fc0a6110 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc8181037f0 tx=0x7fc808009260 comp rx=0 tx=0).stop 2024-04-26T22:54:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.787+0000 7fc817e00640 1 -- 172.21.15.27:0/2277182273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc818104350 msgr2=0x7fc81819fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.787+0000 7fc817e00640 1 --2- 172.21.15.27:0/2277182273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc818104350 0x7fc81819fe40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc810034040 tx=0x7fc810030d70 comp rx=0 tx=0).stop 2024-04-26T22:54:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.788+0000 7fc817e00640 1 -- 172.21.15.27:0/2277182273 shutdown_connections 2024-04-26T22:54:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.788+0000 7fc817e00640 1 --2- 172.21.15.27:0/2277182273 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc7fc0a3c50 0x7fc7fc0a6110 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.788+0000 7fc817e00640 1 --2- 172.21.15.27:0/2277182273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc818103990 0x7fc818198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.788+0000 7fc817e00640 1 --2- 172.21.15.27:0/2277182273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc818104350 0x7fc81819fe40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.788+0000 7fc817e00640 1 --2- 172.21.15.27:0/2277182273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8181027a0 0x7fc8181988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.788+0000 7fc817e00640 1 -- 172.21.15.27:0/2277182273 >> 172.21.15.27:0/2277182273 conn(0x7fc8180fdd10 msgr2=0x7fc818106c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.788+0000 7fc817e00640 1 -- 172.21.15.27:0/2277182273 shutdown_connections 2024-04-26T22:54:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:45.788+0000 7fc817e00640 1 -- 172.21.15.27:0/2277182273 wait complete. 2024-04-26T22:54:46.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:46 smithi027 ceph-mon[28034]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:46.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:46 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2277182273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:46.365 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 22:54:46 smithi027 systemd[1]: Starting Ceph mds.j for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T22:54:46.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:46 smithi080 ceph-mon[32324]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:46.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:46 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2277182273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:46.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:46 smithi094 ceph-mon[32889]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:46.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:46 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2277182273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:46.986 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 22:54:46 smithi027 podman[67895]: 2024-04-26 22:54:46.574653519 +0000 UTC m=+0.036723729 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:54:46.986 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 22:54:46 smithi027 podman[67895]: 2024-04-26 22:54:46.689095029 +0000 UTC m=+0.151165260 container create 03423de96aae6e19beb0db2b8722345e047482e8168932ed2c53c9ccf4ee08ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T22:54:46.986 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 22:54:46 smithi027 podman[67895]: 2024-04-26 22:54:46.98504912 +0000 UTC m=+0.447119395 container init 03423de96aae6e19beb0db2b8722345e047482e8168932ed2c53c9ccf4ee08ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-26T22:54:47.365 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 22:54:46 smithi027 podman[67895]: 2024-04-26 22:54:46.995961078 +0000 UTC m=+0.458031291 container start 03423de96aae6e19beb0db2b8722345e047482e8168932ed2c53c9ccf4ee08ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:54:47.365 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 22:54:47 smithi027 ceph-mds[67914]: -- 172.21.15.27:0/2751361429 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d8ef45a40 con 0x556d8ef18800 2024-04-26T22:54:47.365 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 22:54:47 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-j[67910]: starting mds.j at 2024-04-26T22:54:47.365 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 22:54:47 smithi027 bash[67895]: 03423de96aae6e19beb0db2b8722345e047482e8168932ed2c53c9ccf4ee08ba 2024-04-26T22:54:47.365 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 22:54:47 smithi027 systemd[1]: Started Ceph mds.j for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:54:47.808 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:54:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.028+0000 7f8438e00640 1 -- 172.21.15.27:0/1543842299 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84341027a0 msgr2=0x7f8434102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.028+0000 7f8438e00640 1 --2- 172.21.15.27:0/1543842299 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84341027a0 0x7f8434102bc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8424009860 tx=0x7f842402a340 comp rx=0 tx=0).stop 2024-04-26T22:54:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.028+0000 7f8438e00640 1 -- 172.21.15.27:0/1543842299 shutdown_connections 2024-04-26T22:54:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.028+0000 7f8438e00640 1 --2- 172.21.15.27:0/1543842299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8434104350 0x7f843410abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.028+0000 7f8438e00640 1 --2- 172.21.15.27:0/1543842299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434103990 0x7f8434103e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.028+0000 7f8438e00640 1 --2- 172.21.15.27:0/1543842299 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84341027a0 0x7f8434102bc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.028+0000 7f8438e00640 1 -- 172.21.15.27:0/1543842299 >> 172.21.15.27:0/1543842299 conn(0x7f84340fdd10 msgr2=0x7f8434100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.028+0000 7f8438e00640 1 -- 172.21.15.27:0/1543842299 shutdown_connections 2024-04-26T22:54:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.028+0000 7f8438e00640 1 -- 172.21.15.27:0/1543842299 wait complete. 2024-04-26T22:54:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.029+0000 7f8438e00640 1 Processor -- start 2024-04-26T22:54:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.029+0000 7f8438e00640 1 -- start start 2024-04-26T22:54:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.030+0000 7f8438e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84341027a0 0x7f843419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.030+0000 7f8438e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8434103990 0x7f843419b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.030+0000 7f8438e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434104350 0x7f84341a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.030+0000 7f8438e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f843410e2a0 con 0x7f8434103990 2024-04-26T22:54:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.030+0000 7f8438e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f843410e120 con 0x7f84341027a0 2024-04-26T22:54:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.030+0000 7f8438e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f843410e420 con 0x7f8434104350 2024-04-26T22:54:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.030+0000 7f8433400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84341027a0 0x7f843419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.030+0000 7f8433400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84341027a0 0x7f843419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:42700/0 (socket says 172.21.15.27:42700) 2024-04-26T22:54:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.030+0000 7f8433400640 1 -- 172.21.15.27:0/4291983372 learned_addr learned my addr 172.21.15.27:0/4291983372 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:54:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.030+0000 7f8433e00640 1 --2- 172.21.15.27:0/4291983372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434104350 0x7f84341a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.030+0000 7f8432a00640 1 --2- 172.21.15.27:0/4291983372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8434103990 0x7f843419b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.031+0000 7f8433400640 1 -- 172.21.15.27:0/4291983372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434104350 msgr2=0x7f84341a20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.031+0000 7f8433400640 1 --2- 172.21.15.27:0/4291983372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434104350 0x7f84341a20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.031+0000 7f8433400640 1 -- 172.21.15.27:0/4291983372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8434103990 msgr2=0x7f843419b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.031+0000 7f8433400640 1 --2- 172.21.15.27:0/4291983372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8434103990 0x7f843419b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.031+0000 7f8433400640 1 -- 172.21.15.27:0/4291983372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84341a2720 con 0x7f84341027a0 2024-04-26T22:54:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.031+0000 7f8433e00640 1 --2- 172.21.15.27:0/4291983372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434104350 0x7f84341a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:54:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.031+0000 7f8433400640 1 --2- 172.21.15.27:0/4291983372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84341027a0 0x7f843419ab00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f84240057c0 tx=0x7f8424032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.031+0000 7f842be00640 1 -- 172.21.15.27:0/4291983372 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424037040 con 0x7f84341027a0 2024-04-26T22:54:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.031+0000 7f8438e00640 1 -- 172.21.15.27:0/4291983372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84341a29b0 con 0x7f84341027a0 2024-04-26T22:54:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.031+0000 7f842be00640 1 -- 172.21.15.27:0/4291983372 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8424030bd0 con 0x7f84341027a0 2024-04-26T22:54:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.031+0000 7f842be00640 1 -- 172.21.15.27:0/4291983372 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424005280 con 0x7f84341027a0 2024-04-26T22:54:48.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.032+0000 7f8438e00640 1 -- 172.21.15.27:0/4291983372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84341a2e90 con 0x7f84341027a0 2024-04-26T22:54:48.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.034+0000 7f8438e00640 1 -- 172.21.15.27:0/4291983372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84341a30c0 con 0x7f84341027a0 2024-04-26T22:54:48.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.035+0000 7f842be00640 1 -- 172.21.15.27:0/4291983372 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8424043020 con 0x7f84341027a0 2024-04-26T22:54:48.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.036+0000 7f842be00640 1 --2- 172.21.15.27:0/4291983372 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f84180a3c50 0x7f84180a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.037+0000 7f842be00640 1 -- 172.21.15.27:0/4291983372 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f84240ef510 con 0x7f84341027a0 2024-04-26T22:54:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.037+0000 7f8432a00640 1 --2- 172.21.15.27:0/4291983372 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f84180a3c50 0x7f84180a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.038+0000 7f8432a00640 1 --2- 172.21.15.27:0/4291983372 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f84180a3c50 0x7f84180a6110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f843419c010 tx=0x7f8420009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:48.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.041+0000 7f842be00640 1 -- 172.21.15.27:0/4291983372 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f84240bc2f0 con 0x7f84341027a0 2024-04-26T22:54:48.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.315+0000 7f8438e00640 1 -- 172.21.15.27:0/4291983372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84341a33b0 con 0x7f84341027a0 2024-04-26T22:54:48.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.316+0000 7f842be00640 1 -- 172.21.15.27:0/4291983372 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3037 (secure 0 0 0) 0x7f8424030d80 con 0x7f84341027a0 2024-04-26T22:54:48.316 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:54:48.316 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T22:54:48.316 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 4 2024-04-26T22:54:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.319+0000 7f8438e00640 1 -- 172.21.15.27:0/4291983372 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f84180a3c50 msgr2=0x7f84180a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.319+0000 7f8438e00640 1 --2- 172.21.15.27:0/4291983372 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f84180a3c50 0x7f84180a6110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f843419c010 tx=0x7f8420009260 comp rx=0 tx=0).stop 2024-04-26T22:54:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.319+0000 7f8438e00640 1 -- 172.21.15.27:0/4291983372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84341027a0 msgr2=0x7f843419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.319+0000 7f8438e00640 1 --2- 172.21.15.27:0/4291983372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84341027a0 0x7f843419ab00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f84240057c0 tx=0x7f8424032da0 comp rx=0 tx=0).stop 2024-04-26T22:54:48.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.319+0000 7f8438e00640 1 -- 172.21.15.27:0/4291983372 shutdown_connections 2024-04-26T22:54:48.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.319+0000 7f8438e00640 1 --2- 172.21.15.27:0/4291983372 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f84180a3c50 0x7f84180a6110 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:48.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.319+0000 7f8438e00640 1 --2- 172.21.15.27:0/4291983372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434104350 0x7f84341a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:48.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.319+0000 7f8438e00640 1 --2- 172.21.15.27:0/4291983372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8434103990 0x7f843419b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:48.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.319+0000 7f8438e00640 1 --2- 172.21.15.27:0/4291983372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84341027a0 0x7f843419ab00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:48.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.319+0000 7f8438e00640 1 -- 172.21.15.27:0/4291983372 >> 172.21.15.27:0/4291983372 conn(0x7f84340fdd10 msgr2=0x7f8434074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:48.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.319+0000 7f8438e00640 1 -- 172.21.15.27:0/4291983372 shutdown_connections 2024-04-26T22:54:48.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:48.319+0000 7f8438e00640 1 -- 172.21.15.27:0/4291983372 wait complete. 2024-04-26T22:54:48.338 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:48 smithi027 ceph-mon[28034]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:48.338 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:48 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.27:6834/3122709125,v1:172.21.15.27:6835/3122709125] up:boot 2024-04-26T22:54:48.339 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:48 smithi027 ceph-mon[28034]: fsmap 3 up:standby 2024-04-26T22:54:48.339 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T22:54:48.339 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:48.339 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:48.339 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:48.339 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:48.339 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:48.339 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:48 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:48.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:48 smithi080 ceph-mon[32324]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:48.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:48 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.27:6834/3122709125,v1:172.21.15.27:6835/3122709125] up:boot 2024-04-26T22:54:48.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:48 smithi080 ceph-mon[32324]: fsmap 3 up:standby 2024-04-26T22:54:48.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T22:54:48.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:48.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:48.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:48.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:48.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:48.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:48 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:48.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:48 smithi094 ceph-mon[32889]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:48.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:48 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.27:6834/3122709125,v1:172.21.15.27:6835/3122709125] up:boot 2024-04-26T22:54:48.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:48 smithi094 ceph-mon[32889]: fsmap 3 up:standby 2024-04-26T22:54:48.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T22:54:48.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:48.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:48.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:48.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:48.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:48.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:48 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:49.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:49 smithi027 ceph-mon[28034]: Deploying daemon mds.d on smithi027 2024-04-26T22:54:49.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:49 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/4291983372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:49.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:49 smithi080 ceph-mon[32324]: Deploying daemon mds.d on smithi027 2024-04-26T22:54:49.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:49 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/4291983372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:49.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:49 smithi094 ceph-mon[32889]: Deploying daemon mds.d on smithi027 2024-04-26T22:54:49.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:49 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/4291983372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:50.343 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:54:50.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:50 smithi027 ceph-mon[28034]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:50.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:50 smithi080 ceph-mon[32324]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:50.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:50 smithi094 ceph-mon[32889]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.569+0000 7f9d7ee00640 1 -- 172.21.15.27:0/1471899922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d78105d10 msgr2=0x7f9d78108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.570+0000 7f9d7ee00640 1 --2- 172.21.15.27:0/1471899922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d78105d10 0x7f9d78108100 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400fb60 tx=0x7f9d6402e4b0 comp rx=0 tx=0).stop 2024-04-26T22:54:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.570+0000 7f9d7ee00640 1 -- 172.21.15.27:0/1471899922 shutdown_connections 2024-04-26T22:54:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.570+0000 7f9d7ee00640 1 --2- 172.21.15.27:0/1471899922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d78108640 0x7f9d7810aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.570+0000 7f9d7ee00640 1 --2- 172.21.15.27:0/1471899922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d78105d10 0x7f9d78108100 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.570+0000 7f9d7ee00640 1 --2- 172.21.15.27:0/1471899922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d781033e0 0x7f9d781057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.571+0000 7f9d7ee00640 1 -- 172.21.15.27:0/1471899922 >> 172.21.15.27:0/1471899922 conn(0x7f9d780fcdd0 msgr2=0x7f9d780ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.572+0000 7f9d7ee00640 1 -- 172.21.15.27:0/1471899922 shutdown_connections 2024-04-26T22:54:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.572+0000 7f9d7ee00640 1 -- 172.21.15.27:0/1471899922 wait complete. 2024-04-26T22:54:50.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.575+0000 7f9d7ee00640 1 Processor -- start 2024-04-26T22:54:50.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.575+0000 7f9d7ee00640 1 -- start start 2024-04-26T22:54:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.576+0000 7f9d7ee00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d781033e0 0x7f9d7819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.576+0000 7f9d7ee00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d78105d10 0x7f9d7819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.576+0000 7f9d7ee00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d78108640 0x7f9d781a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.576+0000 7f9d7ee00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d78114b10 con 0x7f9d781033e0 2024-04-26T22:54:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.576+0000 7f9d7ee00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9d78114990 con 0x7f9d78108640 2024-04-26T22:54:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.576+0000 7f9d7ee00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9d78114c90 con 0x7f9d78105d10 2024-04-26T22:54:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.576+0000 7f9d77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d781033e0 0x7f9d7819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.576+0000 7f9d77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d781033e0 0x7f9d7819f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39386/0 (socket says 172.21.15.27:39386) 2024-04-26T22:54:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.576+0000 7f9d77e00640 1 -- 172.21.15.27:0/1572787045 learned_addr learned my addr 172.21.15.27:0/1572787045 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:54:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.576+0000 7f9d77400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d78105d10 0x7f9d7819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.577+0000 7f9d7ce00640 1 --2- 172.21.15.27:0/1572787045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d78108640 0x7f9d781a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.577+0000 7f9d77400640 1 -- 172.21.15.27:0/1572787045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d78108640 msgr2=0x7f9d781a6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.577+0000 7f9d77400640 1 --2- 172.21.15.27:0/1572787045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d78108640 0x7f9d781a6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.577+0000 7f9d77400640 1 -- 172.21.15.27:0/1572787045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d781033e0 msgr2=0x7f9d7819f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.577+0000 7f9d77400640 1 --2- 172.21.15.27:0/1572787045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d781033e0 0x7f9d7819f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.577+0000 7f9d77400640 1 -- 172.21.15.27:0/1572787045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d781a6de0 con 0x7f9d78105d10 2024-04-26T22:54:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.577+0000 7f9d77400640 1 --2- 172.21.15.27:0/1572787045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d78105d10 0x7f9d7819f700 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9d640348b0 tx=0x7f9d64034da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.578+0000 7f9d74c00640 1 -- 172.21.15.27:0/1572787045 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64004320 con 0x7f9d78105d10 2024-04-26T22:54:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.578+0000 7f9d74c00640 1 -- 172.21.15.27:0/1572787045 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d640044c0 con 0x7f9d78105d10 2024-04-26T22:54:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.578+0000 7f9d74c00640 1 -- 172.21.15.27:0/1572787045 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64003aa0 con 0x7f9d78105d10 2024-04-26T22:54:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.578+0000 7f9d7ee00640 1 -- 172.21.15.27:0/1572787045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d781a7070 con 0x7f9d78105d10 2024-04-26T22:54:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.578+0000 7f9d7ee00640 1 -- 172.21.15.27:0/1572787045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d781a7510 con 0x7f9d78105d10 2024-04-26T22:54:50.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.580+0000 7f9d7ee00640 1 -- 172.21.15.27:0/1572787045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d780fe8c0 con 0x7f9d78105d10 2024-04-26T22:54:50.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.583+0000 7f9d74c00640 1 -- 172.21.15.27:0/1572787045 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9d64045020 con 0x7f9d78105d10 2024-04-26T22:54:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.584+0000 7f9d74c00640 1 --2- 172.21.15.27:0/1572787045 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f9d5c0a3c30 0x7f9d5c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.584+0000 7f9d74c00640 1 -- 172.21.15.27:0/1572787045 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9d640f0580 con 0x7f9d78105d10 2024-04-26T22:54:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.585+0000 7f9d77e00640 1 --2- 172.21.15.27:0/1572787045 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f9d5c0a3c30 0x7f9d5c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:50.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.585+0000 7f9d77e00640 1 --2- 172.21.15.27:0/1572787045 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f9d5c0a3c30 0x7f9d5c0a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9d680095f0 tx=0x7f9d68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:50.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.589+0000 7f9d74c00640 1 -- 172.21.15.27:0/1572787045 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9d6404c020 con 0x7f9d78105d10 2024-04-26T22:54:50.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.838+0000 7f9d7ee00640 1 -- 172.21.15.27:0/1572787045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d7802dad0 con 0x7f9d78105d10 2024-04-26T22:54:50.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.839+0000 7f9d74c00640 1 -- 172.21.15.27:0/1572787045 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3037 (secure 0 0 0) 0x7f9d640bd400 con 0x7f9d78105d10 2024-04-26T22:54:50.839 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:54:50.840 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T22:54:50.840 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 4 2024-04-26T22:54:50.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.841+0000 7f9d7ee00640 1 -- 172.21.15.27:0/1572787045 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f9d5c0a3c30 msgr2=0x7f9d5c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:50.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.841+0000 7f9d7ee00640 1 --2- 172.21.15.27:0/1572787045 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f9d5c0a3c30 0x7f9d5c0a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9d680095f0 tx=0x7f9d68005f90 comp rx=0 tx=0).stop 2024-04-26T22:54:50.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.841+0000 7f9d7ee00640 1 -- 172.21.15.27:0/1572787045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d78105d10 msgr2=0x7f9d7819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:50.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.841+0000 7f9d7ee00640 1 --2- 172.21.15.27:0/1572787045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d78105d10 0x7f9d7819f700 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9d640348b0 tx=0x7f9d64034da0 comp rx=0 tx=0).stop 2024-04-26T22:54:50.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.842+0000 7f9d7ee00640 1 -- 172.21.15.27:0/1572787045 shutdown_connections 2024-04-26T22:54:50.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.842+0000 7f9d7ee00640 1 --2- 172.21.15.27:0/1572787045 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f9d5c0a3c30 0x7f9d5c0a60f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:50.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.842+0000 7f9d7ee00640 1 --2- 172.21.15.27:0/1572787045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d78108640 0x7f9d781a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:50.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.842+0000 7f9d7ee00640 1 --2- 172.21.15.27:0/1572787045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d78105d10 0x7f9d7819f700 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:50.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.842+0000 7f9d7ee00640 1 --2- 172.21.15.27:0/1572787045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d781033e0 0x7f9d7819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:50.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.842+0000 7f9d7ee00640 1 -- 172.21.15.27:0/1572787045 >> 172.21.15.27:0/1572787045 conn(0x7f9d780fcdd0 msgr2=0x7f9d78103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:50.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.842+0000 7f9d7ee00640 1 -- 172.21.15.27:0/1572787045 shutdown_connections 2024-04-26T22:54:50.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:50.842+0000 7f9d7ee00640 1 -- 172.21.15.27:0/1572787045 wait complete. 2024-04-26T22:54:51.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:51 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1572787045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:51.365 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 22:54:51 smithi027 podman[68273]: 2024-04-26 22:54:51.229775742 +0000 UTC m=+0.167156600 container create abcc565af192780581507ea15f3919480fab12162f49dd51e1280f834b6a7f7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:54:51.436 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:51 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1572787045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:51.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:51 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1572787045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:51.865 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 22:54:51 smithi027 podman[68273]: 2024-04-26 22:54:51.567224158 +0000 UTC m=+0.504605029 container init abcc565af192780581507ea15f3919480fab12162f49dd51e1280f834b6a7f7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:54:51.866 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 22:54:51 smithi027 podman[68273]: 2024-04-26 22:54:51.578599003 +0000 UTC m=+0.515979839 container start abcc565af192780581507ea15f3919480fab12162f49dd51e1280f834b6a7f7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:54:51.866 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 22:54:51 smithi027 ceph-mds[68291]: -- 172.21.15.27:0/2414565003 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561642547a40 con 0x56164251a800 2024-04-26T22:54:51.866 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 22:54:51 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-d[68287]: starting mds.d at 2024-04-26T22:54:51.866 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 22:54:51 smithi027 bash[68273]: abcc565af192780581507ea15f3919480fab12162f49dd51e1280f834b6a7f7c 2024-04-26T22:54:51.866 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 22:54:51 smithi027 systemd[1]: Started Ceph mds.d for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:54:52.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:52 smithi027 ceph-mon[28034]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:52.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:52 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:52.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:52 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:52.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:52 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:52.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:52 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:52.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:52 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:52.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:52 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:52.451 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:52 smithi094 ceph-mon[32889]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:52.451 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:52 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:52.452 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:52 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:52.452 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:52 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:52.452 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:52 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:52.452 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:52 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:52.452 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:52 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:52.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:52 smithi080 ceph-mon[32324]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:52.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:52 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:52.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:52 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:52.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:52 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:52.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:52 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:52.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:52 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:52.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:52 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:52.861 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:54:53.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.087+0000 7fabcc200640 1 -- 172.21.15.27:0/994094339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabc4106720 msgr2=0x7fabc410cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:53.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.087+0000 7fabcc200640 1 --2- 172.21.15.27:0/994094339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabc4106720 0x7fabc410cfb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009ca0 tx=0x7fabbc02a8c0 comp rx=0 tx=0).stop 2024-04-26T22:54:53.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.087+0000 7fabcc200640 1 -- 172.21.15.27:0/994094339 shutdown_connections 2024-04-26T22:54:53.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.087+0000 7fabcc200640 1 --2- 172.21.15.27:0/994094339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabc4106720 0x7fabc410cfb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:53.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.087+0000 7fabcc200640 1 --2- 172.21.15.27:0/994094339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc4105d60 0x7fabc41061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:53.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.087+0000 7fabcc200640 1 --2- 172.21.15.27:0/994094339 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc4104b70 0x7fabc4104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:53.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.087+0000 7fabcc200640 1 -- 172.21.15.27:0/994094339 >> 172.21.15.27:0/994094339 conn(0x7fabc41000e0 msgr2=0x7fabc4102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:53.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.088+0000 7fabcc200640 1 -- 172.21.15.27:0/994094339 shutdown_connections 2024-04-26T22:54:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.088+0000 7fabcc200640 1 -- 172.21.15.27:0/994094339 wait complete. 2024-04-26T22:54:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.089+0000 7fabcc200640 1 Processor -- start 2024-04-26T22:54:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.089+0000 7fabcc200640 1 -- start start 2024-04-26T22:54:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.090+0000 7fabcc200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc4104b70 0x7fabc419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.090+0000 7fabcc200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabc4105d60 0x7fabc419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.090+0000 7fabcc200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc4106720 0x7fabc41a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.090+0000 7fabcc200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41106a0 con 0x7fabc4104b70 2024-04-26T22:54:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.090+0000 7fabcc200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fabc4110520 con 0x7fabc4106720 2024-04-26T22:54:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.090+0000 7fabcc200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fabc4110820 con 0x7fabc4105d60 2024-04-26T22:54:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.090+0000 7fabc9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc4104b70 0x7fabc419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.090+0000 7fabc8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabc4105d60 0x7fabc419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.090+0000 7fabc8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabc4105d60 0x7fabc419b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:50942/0 (socket says 172.21.15.27:50942) 2024-04-26T22:54:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.090+0000 7fabc8e00640 1 -- 172.21.15.27:0/3343091445 learned_addr learned my addr 172.21.15.27:0/3343091445 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:54:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.090+0000 7fabc9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc4104b70 0x7fabc419ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39422/0 (socket says 172.21.15.27:39422) 2024-04-26T22:54:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.090+0000 7fabca200640 1 --2- 172.21.15.27:0/3343091445 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc4106720 0x7fabc41a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.090+0000 7fabc8e00640 1 -- 172.21.15.27:0/3343091445 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc4106720 msgr2=0x7fabc41a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.090+0000 7fabc8e00640 1 --2- 172.21.15.27:0/3343091445 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc4106720 0x7fabc41a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.091+0000 7fabc8e00640 1 -- 172.21.15.27:0/3343091445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc4104b70 msgr2=0x7fabc419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.091+0000 7fabc8e00640 1 --2- 172.21.15.27:0/3343091445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc4104b70 0x7fabc419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.091+0000 7fabc8e00640 1 -- 172.21.15.27:0/3343091445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc41a2920 con 0x7fabc4105d60 2024-04-26T22:54:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.091+0000 7fabc8e00640 1 --2- 172.21.15.27:0/3343091445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabc4105d60 0x7fabc419b250 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fabb000e9f0 tx=0x7fabb000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:53.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.091+0000 7fabc2000640 1 -- 172.21.15.27:0/3343091445 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb000cd80 con 0x7fabc4105d60 2024-04-26T22:54:53.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.091+0000 7fabcc200640 1 -- 172.21.15.27:0/3343091445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc41a2c10 con 0x7fabc4105d60 2024-04-26T22:54:53.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.091+0000 7fabc2000640 1 -- 172.21.15.27:0/3343091445 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb00108e0 con 0x7fabc4105d60 2024-04-26T22:54:53.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.091+0000 7fabc2000640 1 -- 172.21.15.27:0/3343091445 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0018980 con 0x7fabc4105d60 2024-04-26T22:54:53.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.091+0000 7fabcc200640 1 -- 172.21.15.27:0/3343091445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc41a3160 con 0x7fabc4105d60 2024-04-26T22:54:53.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.092+0000 7fabcc200640 1 -- 172.21.15.27:0/3343091445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc410be90 con 0x7fabc4105d60 2024-04-26T22:54:53.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.099+0000 7fabc2000640 1 -- 172.21.15.27:0/3343091445 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fabb0010430 con 0x7fabc4105d60 2024-04-26T22:54:53.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.100+0000 7fabc2000640 1 --2- 172.21.15.27:0/3343091445 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faba80a3bb0 0x7faba80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:53.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.101+0000 7fabc2000640 1 -- 172.21.15.27:0/3343091445 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fabb00d0820 con 0x7fabc4105d60 2024-04-26T22:54:53.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.101+0000 7fabc9800640 1 --2- 172.21.15.27:0/3343091445 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faba80a3bb0 0x7faba80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:53.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.102+0000 7fabc9800640 1 --2- 172.21.15.27:0/3343091445 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faba80a3bb0 0x7faba80a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fabc419c1e0 tx=0x7fabb400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:53.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.105+0000 7fabc2000640 1 -- 172.21.15.27:0/3343091445 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fabb009cff0 con 0x7fabc4105d60 2024-04-26T22:54:53.147 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:53 smithi080 ceph-mon[32324]: Deploying daemon mds.e on smithi080 2024-04-26T22:54:53.147 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:53 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.27:6836/1639451555,v1:172.21.15.27:6837/1639451555] up:boot 2024-04-26T22:54:53.148 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:53 smithi080 ceph-mon[32324]: fsmap 4 up:standby 2024-04-26T22:54:53.148 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:53 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T22:54:53.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.362+0000 7fabcc200640 1 -- 172.21.15.27:0/3343091445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabc402dc90 con 0x7fabc4105d60 2024-04-26T22:54:53.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.363+0000 7fabc2000640 1 -- 172.21.15.27:0/3343091445 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3862 (secure 0 0 0) 0x7fabb00a12f0 con 0x7fabc4105d60 2024-04-26T22:54:53.364 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:54:53.364 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-26T22:54:53.364 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 5 2024-04-26T22:54:53.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:53 smithi027 ceph-mon[28034]: Deploying daemon mds.e on smithi080 2024-04-26T22:54:53.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:53 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.27:6836/1639451555,v1:172.21.15.27:6837/1639451555] up:boot 2024-04-26T22:54:53.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:53 smithi027 ceph-mon[28034]: fsmap 4 up:standby 2024-04-26T22:54:53.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:53 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T22:54:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.366+0000 7fabcc200640 1 -- 172.21.15.27:0/3343091445 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faba80a3bb0 msgr2=0x7faba80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.366+0000 7fabcc200640 1 --2- 172.21.15.27:0/3343091445 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faba80a3bb0 0x7faba80a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fabc419c1e0 tx=0x7fabb400b390 comp rx=0 tx=0).stop 2024-04-26T22:54:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.367+0000 7fabcc200640 1 -- 172.21.15.27:0/3343091445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabc4105d60 msgr2=0x7fabc419b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.367+0000 7fabcc200640 1 --2- 172.21.15.27:0/3343091445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabc4105d60 0x7fabc419b250 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fabb000e9f0 tx=0x7fabb000eeb0 comp rx=0 tx=0).stop 2024-04-26T22:54:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.367+0000 7fabcc200640 1 -- 172.21.15.27:0/3343091445 shutdown_connections 2024-04-26T22:54:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.367+0000 7fabcc200640 1 --2- 172.21.15.27:0/3343091445 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faba80a3bb0 0x7faba80a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.367+0000 7fabcc200640 1 --2- 172.21.15.27:0/3343091445 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc4106720 0x7fabc41a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.367+0000 7fabcc200640 1 --2- 172.21.15.27:0/3343091445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabc4105d60 0x7fabc419b250 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:53.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.367+0000 7fabcc200640 1 --2- 172.21.15.27:0/3343091445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc4104b70 0x7fabc419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:53.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.367+0000 7fabcc200640 1 -- 172.21.15.27:0/3343091445 >> 172.21.15.27:0/3343091445 conn(0x7fabc41000e0 msgr2=0x7fabc4109060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:53.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.367+0000 7fabcc200640 1 -- 172.21.15.27:0/3343091445 shutdown_connections 2024-04-26T22:54:53.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:53.367+0000 7fabcc200640 1 -- 172.21.15.27:0/3343091445 wait complete. 2024-04-26T22:54:53.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:53 smithi094 ceph-mon[32889]: Deploying daemon mds.e on smithi080 2024-04-26T22:54:53.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:53 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.27:6836/1639451555,v1:172.21.15.27:6837/1639451555] up:boot 2024-04-26T22:54:53.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:53 smithi094 ceph-mon[32889]: fsmap 4 up:standby 2024-04-26T22:54:53.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:53 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T22:54:54.163 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:54 smithi080 ceph-mon[32324]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:54.163 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:54 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3343091445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:54.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:54 smithi027 ceph-mon[28034]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:54.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:54 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3343091445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:54.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:54 smithi094 ceph-mon[32889]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:54.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:54 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3343091445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:54.833 INFO:journalctl@ceph.mds.e.smithi080.stdout:Apr 26 22:54:54 smithi080 podman[62373]: 2024-04-26 22:54:54.680497097 +0000 UTC m=+0.017835811 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:54:54.833 INFO:journalctl@ceph.mds.e.smithi080.stdout:Apr 26 22:54:54 smithi080 podman[62373]: 2024-04-26 22:54:54.83326487 +0000 UTC m=+0.170603584 container create cb6cb4468c2c475b34d90ab5446dfa1835c1e16e4b038d6077ff11589319e634 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:54:55.145 INFO:journalctl@ceph.mds.e.smithi080.stdout:Apr 26 22:54:55 smithi080 podman[62373]: 2024-04-26 22:54:55.052674709 +0000 UTC m=+0.390013423 container init cb6cb4468c2c475b34d90ab5446dfa1835c1e16e4b038d6077ff11589319e634 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-26T22:54:55.146 INFO:journalctl@ceph.mds.e.smithi080.stdout:Apr 26 22:54:55 smithi080 podman[62373]: 2024-04-26 22:54:55.055249172 +0000 UTC m=+0.392587887 container start cb6cb4468c2c475b34d90ab5446dfa1835c1e16e4b038d6077ff11589319e634 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T22:54:55.146 INFO:journalctl@ceph.mds.e.smithi080.stdout:Apr 26 22:54:55 smithi080 ceph-mds[62393]: -- 172.21.15.80:0/3084116654 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563f19c5ba40 con 0x563f19c2f000 2024-04-26T22:54:55.146 INFO:journalctl@ceph.mds.e.smithi080.stdout:Apr 26 22:54:55 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-e[62389]: starting mds.e at 2024-04-26T22:54:55.387 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:54:55.521 INFO:journalctl@ceph.mds.e.smithi080.stdout:Apr 26 22:54:55 smithi080 bash[62373]: cb6cb4468c2c475b34d90ab5446dfa1835c1e16e4b038d6077ff11589319e634 2024-04-26T22:54:55.521 INFO:journalctl@ceph.mds.e.smithi080.stdout:Apr 26 22:54:55 smithi080 systemd[1]: Started Ceph mds.e for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:54:55.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.604+0000 7efdaca00640 1 -- 172.21.15.27:0/3276713146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda8103990 msgr2=0x7efda8103e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:55.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.604+0000 7efdaca00640 1 --2- 172.21.15.27:0/3276713146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda8103990 0x7efda8103e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efd90009900 tx=0x7efd9002a410 comp rx=0 tx=0).stop 2024-04-26T22:54:55.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.605+0000 7efdaca00640 1 -- 172.21.15.27:0/3276713146 shutdown_connections 2024-04-26T22:54:55.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.605+0000 7efdaca00640 1 --2- 172.21.15.27:0/3276713146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efda8104350 0x7efda810abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:55.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.605+0000 7efdaca00640 1 --2- 172.21.15.27:0/3276713146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda8103990 0x7efda8103e10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:55.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.605+0000 7efdaca00640 1 --2- 172.21.15.27:0/3276713146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efda81027a0 0x7efda8102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:55.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.605+0000 7efdaca00640 1 -- 172.21.15.27:0/3276713146 >> 172.21.15.27:0/3276713146 conn(0x7efda80fdd10 msgr2=0x7efda8100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:55.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.605+0000 7efdaca00640 1 -- 172.21.15.27:0/3276713146 shutdown_connections 2024-04-26T22:54:55.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.605+0000 7efdaca00640 1 -- 172.21.15.27:0/3276713146 wait complete. 2024-04-26T22:54:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.606+0000 7efdaca00640 1 Processor -- start 2024-04-26T22:54:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.606+0000 7efdaca00640 1 -- start start 2024-04-26T22:54:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.606+0000 7efdaca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda81027a0 0x7efda819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.606+0000 7efdaca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efda8103990 0x7efda819aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.606+0000 7efdaca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efda8104350 0x7efda81a2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.606+0000 7efdaca00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efda810e2b0 con 0x7efda81027a0 2024-04-26T22:54:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.606+0000 7efdaca00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efda810e130 con 0x7efda8103990 2024-04-26T22:54:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.606+0000 7efdaca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efda810e430 con 0x7efda8104350 2024-04-26T22:54:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.606+0000 7efda7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda81027a0 0x7efda819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.607+0000 7efda7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda81027a0 0x7efda819aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39442/0 (socket says 172.21.15.27:39442) 2024-04-26T22:54:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.607+0000 7efda7400640 1 -- 172.21.15.27:0/337616618 learned_addr learned my addr 172.21.15.27:0/337616618 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:54:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.607+0000 7efd9ea00640 1 --2- 172.21.15.27:0/337616618 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efda8103990 0x7efda819aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.607+0000 7efda7e00640 1 --2- 172.21.15.27:0/337616618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efda8104350 0x7efda81a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.607+0000 7efda7400640 1 -- 172.21.15.27:0/337616618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efda8104350 msgr2=0x7efda81a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.607+0000 7efda7400640 1 --2- 172.21.15.27:0/337616618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efda8104350 0x7efda81a2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.607+0000 7efda7400640 1 -- 172.21.15.27:0/337616618 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efda8103990 msgr2=0x7efda819aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.607+0000 7efda7400640 1 --2- 172.21.15.27:0/337616618 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efda8103990 0x7efda819aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.607+0000 7efda7400640 1 -- 172.21.15.27:0/337616618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda81a26d0 con 0x7efda81027a0 2024-04-26T22:54:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.607+0000 7efda7400640 1 --2- 172.21.15.27:0/337616618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda81027a0 0x7efda819aab0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd9800ebc0 tx=0x7efd9800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.607+0000 7efda4c00640 1 -- 172.21.15.27:0/337616618 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98003b70 con 0x7efda81027a0 2024-04-26T22:54:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.608+0000 7efda4c00640 1 -- 172.21.15.27:0/337616618 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd98003d10 con 0x7efda81027a0 2024-04-26T22:54:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.608+0000 7efda4c00640 1 -- 172.21.15.27:0/337616618 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98019800 con 0x7efda81027a0 2024-04-26T22:54:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.608+0000 7efdaca00640 1 -- 172.21.15.27:0/337616618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda81a29c0 con 0x7efda81027a0 2024-04-26T22:54:55.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.608+0000 7efdaca00640 1 -- 172.21.15.27:0/337616618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda81a2e90 con 0x7efda81027a0 2024-04-26T22:54:55.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.609+0000 7efdaca00640 1 -- 172.21.15.27:0/337616618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda81a3120 con 0x7efda81027a0 2024-04-26T22:54:55.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.611+0000 7efda4c00640 1 -- 172.21.15.27:0/337616618 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efd980199a0 con 0x7efda81027a0 2024-04-26T22:54:55.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.612+0000 7efda4c00640 1 --2- 172.21.15.27:0/337616618 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7efd8c0a3cd0 0x7efd8c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:55.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.613+0000 7efda4c00640 1 -- 172.21.15.27:0/337616618 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7efd980d0760 con 0x7efda81027a0 2024-04-26T22:54:55.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.613+0000 7efd9ea00640 1 --2- 172.21.15.27:0/337616618 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7efd8c0a3cd0 0x7efd8c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:55.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.614+0000 7efd9ea00640 1 --2- 172.21.15.27:0/337616618 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7efd8c0a3cd0 0x7efd8c0a6190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efd90000c00 tx=0x7efd900314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:55.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.618+0000 7efda4c00640 1 -- 172.21.15.27:0/337616618 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efd9809d5e0 con 0x7efda81027a0 2024-04-26T22:54:55.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.873+0000 7efdaca00640 1 -- 172.21.15.27:0/337616618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efda81a3420 con 0x7efda81027a0 2024-04-26T22:54:55.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.874+0000 7efda4c00640 1 -- 172.21.15.27:0/337616618 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4687 (secure 0 0 0) 0x7efd9809cee0 con 0x7efda81027a0 2024-04-26T22:54:55.875 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:54:55.875 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T22:54:55.875 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 6 2024-04-26T22:54:55.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.877+0000 7efdaca00640 1 -- 172.21.15.27:0/337616618 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7efd8c0a3cd0 msgr2=0x7efd8c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:55.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.877+0000 7efdaca00640 1 --2- 172.21.15.27:0/337616618 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7efd8c0a3cd0 0x7efd8c0a6190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efd90000c00 tx=0x7efd900314a0 comp rx=0 tx=0).stop 2024-04-26T22:54:55.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.877+0000 7efdaca00640 1 -- 172.21.15.27:0/337616618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda81027a0 msgr2=0x7efda819aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:55.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.877+0000 7efdaca00640 1 --2- 172.21.15.27:0/337616618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda81027a0 0x7efda819aab0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd9800ebc0 tx=0x7efd9800c6a0 comp rx=0 tx=0).stop 2024-04-26T22:54:55.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.877+0000 7efdaca00640 1 -- 172.21.15.27:0/337616618 shutdown_connections 2024-04-26T22:54:55.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.877+0000 7efdaca00640 1 --2- 172.21.15.27:0/337616618 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7efd8c0a3cd0 0x7efd8c0a6190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:55.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.877+0000 7efdaca00640 1 --2- 172.21.15.27:0/337616618 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efda8103990 0x7efda819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:55.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.877+0000 7efdaca00640 1 --2- 172.21.15.27:0/337616618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efda8104350 0x7efda81a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:55.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.877+0000 7efdaca00640 1 --2- 172.21.15.27:0/337616618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda81027a0 0x7efda819aab0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:55.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.877+0000 7efdaca00640 1 -- 172.21.15.27:0/337616618 >> 172.21.15.27:0/337616618 conn(0x7efda80fdd10 msgr2=0x7efda8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:55.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.877+0000 7efdaca00640 1 -- 172.21.15.27:0/337616618 shutdown_connections 2024-04-26T22:54:55.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:55.877+0000 7efdaca00640 1 -- 172.21.15.27:0/337616618 wait complete. 2024-04-26T22:54:56.170 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:56 smithi094 ceph-mon[32889]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:56.170 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:56 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.80:6834/3792288301,v1:172.21.15.80:6835/3792288301] up:boot 2024-04-26T22:54:56.170 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:56 smithi094 ceph-mon[32889]: fsmap 5 up:standby 2024-04-26T22:54:56.170 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T22:54:56.170 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:56.170 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:56.170 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:56.171 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:56.171 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:56.171 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:56 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:56.171 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:56 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/337616618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:56.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:56 smithi027 ceph-mon[28034]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:56.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:56 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.80:6834/3792288301,v1:172.21.15.80:6835/3792288301] up:boot 2024-04-26T22:54:56.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:56 smithi027 ceph-mon[28034]: fsmap 5 up:standby 2024-04-26T22:54:56.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T22:54:56.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:56.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:56.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:56.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:56.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:56.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:56 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:56.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:56 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/337616618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:56.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:56 smithi080 ceph-mon[32324]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:56.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:56 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.80:6834/3792288301,v1:172.21.15.80:6835/3792288301] up:boot 2024-04-26T22:54:56.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:56 smithi080 ceph-mon[32324]: fsmap 5 up:standby 2024-04-26T22:54:56.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T22:54:56.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:56.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:56.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:56.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:56.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:56.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:56 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:56.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:56 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/337616618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:57.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:57 smithi094 ceph-mon[32889]: Deploying daemon mds.f on smithi094 2024-04-26T22:54:57.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:57 smithi080 ceph-mon[32324]: Deploying daemon mds.f on smithi094 2024-04-26T22:54:57.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:57 smithi027 ceph-mon[28034]: Deploying daemon mds.f on smithi094 2024-04-26T22:54:57.727 INFO:journalctl@ceph.mds.f.smithi094.stdout:Apr 26 22:54:57 smithi094 systemd[1]: Starting Ceph mds.f for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T22:54:57.899 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:54:57.998 INFO:journalctl@ceph.mds.f.smithi094.stdout:Apr 26 22:54:57 smithi094 podman[63188]: 2024-04-26 22:54:57.630964863 +0000 UTC m=+0.018171306 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:54:57.998 INFO:journalctl@ceph.mds.f.smithi094.stdout:Apr 26 22:54:57 smithi094 podman[63188]: 2024-04-26 22:54:57.754285471 +0000 UTC m=+0.141491955 container create 885bd0a555de9a32fc8c768ec439f5b034c5e72e22e6b194faf09cb138a25668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-26T22:54:57.998 INFO:journalctl@ceph.mds.f.smithi094.stdout:Apr 26 22:54:57 smithi094 podman[63188]: 2024-04-26 22:54:57.998571999 +0000 UTC m=+0.385778441 container init 885bd0a555de9a32fc8c768ec439f5b034c5e72e22e6b194faf09cb138a25668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:54:58.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.131+0000 7f3b9a000640 1 -- 172.21.15.27:0/710330361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b94104b70 msgr2=0x7f3b94104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:58.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.131+0000 7f3b9a000640 1 --2- 172.21.15.27:0/710330361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b94104b70 0x7f3b94104f90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009900 tx=0x7f3b8402a3f0 comp rx=0 tx=0).stop 2024-04-26T22:54:58.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.131+0000 7f3b9a000640 1 -- 172.21.15.27:0/710330361 shutdown_connections 2024-04-26T22:54:58.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.131+0000 7f3b9a000640 1 --2- 172.21.15.27:0/710330361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b94106720 0x7f3b9410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:58.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.131+0000 7f3b9a000640 1 --2- 172.21.15.27:0/710330361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94105d60 0x7f3b941061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:58.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.131+0000 7f3b9a000640 1 --2- 172.21.15.27:0/710330361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b94104b70 0x7f3b94104f90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:58.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.131+0000 7f3b9a000640 1 -- 172.21.15.27:0/710330361 >> 172.21.15.27:0/710330361 conn(0x7f3b941000e0 msgr2=0x7f3b94102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:58.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.131+0000 7f3b9a000640 1 -- 172.21.15.27:0/710330361 shutdown_connections 2024-04-26T22:54:58.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.131+0000 7f3b9a000640 1 -- 172.21.15.27:0/710330361 wait complete. 2024-04-26T22:54:58.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.132+0000 7f3b9a000640 1 Processor -- start 2024-04-26T22:54:58.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.132+0000 7f3b9a000640 1 -- start start 2024-04-26T22:54:58.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.133+0000 7f3b9a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b94104b70 0x7f3b9419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:58.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.133+0000 7f3b9a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94105d60 0x7f3b9419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:58.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.133+0000 7f3b9a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b94106720 0x7f3b941a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:58.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.133+0000 7f3b9a000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3b941106c0 con 0x7f3b94104b70 2024-04-26T22:54:58.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.133+0000 7f3b9a000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3b94110540 con 0x7f3b94106720 2024-04-26T22:54:58.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.133+0000 7f3b9a000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3b94110840 con 0x7f3b94105d60 2024-04-26T22:54:58.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.133+0000 7f3b92a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94105d60 0x7f3b9419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:58.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.133+0000 7f3b93400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b94104b70 0x7f3b9419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:58.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.133+0000 7f3b92a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94105d60 0x7f3b9419b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:32802/0 (socket says 172.21.15.27:32802) 2024-04-26T22:54:58.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.133+0000 7f3b92a00640 1 -- 172.21.15.27:0/3322568742 learned_addr learned my addr 172.21.15.27:0/3322568742 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:54:58.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.133+0000 7f3b93e00640 1 --2- 172.21.15.27:0/3322568742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b94106720 0x7f3b941a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:58.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.133+0000 7f3b92a00640 1 -- 172.21.15.27:0/3322568742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b94106720 msgr2=0x7f3b941a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:58.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.133+0000 7f3b92a00640 1 --2- 172.21.15.27:0/3322568742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b94106720 0x7f3b941a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:58.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.133+0000 7f3b92a00640 1 -- 172.21.15.27:0/3322568742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b94104b70 msgr2=0x7f3b9419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:58.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.133+0000 7f3b92a00640 1 --2- 172.21.15.27:0/3322568742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b94104b70 0x7f3b9419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:58.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.133+0000 7f3b92a00640 1 -- 172.21.15.27:0/3322568742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b941a29c0 con 0x7f3b94105d60 2024-04-26T22:54:58.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.134+0000 7f3b92a00640 1 --2- 172.21.15.27:0/3322568742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94105d60 0x7f3b9419b2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000e9f0 tx=0x7f3b8000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:58.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.134+0000 7f3b8be00640 1 -- 172.21.15.27:0/3322568742 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8000cd80 con 0x7f3b94105d60 2024-04-26T22:54:58.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.134+0000 7f3b8be00640 1 -- 172.21.15.27:0/3322568742 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b800108e0 con 0x7f3b94105d60 2024-04-26T22:54:58.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.134+0000 7f3b9a000640 1 -- 172.21.15.27:0/3322568742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b941a2cb0 con 0x7f3b94105d60 2024-04-26T22:54:58.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.134+0000 7f3b8be00640 1 -- 172.21.15.27:0/3322568742 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80018980 con 0x7f3b94105d60 2024-04-26T22:54:58.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.134+0000 7f3b9a000640 1 -- 172.21.15.27:0/3322568742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b941a31d0 con 0x7f3b94105d60 2024-04-26T22:54:58.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.136+0000 7f3b9a000640 1 -- 172.21.15.27:0/3322568742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b9410bf10 con 0x7f3b94105d60 2024-04-26T22:54:58.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.138+0000 7f3b8be00640 1 -- 172.21.15.27:0/3322568742 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3b80010430 con 0x7f3b94105d60 2024-04-26T22:54:58.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.139+0000 7f3b8be00640 1 --2- 172.21.15.27:0/3322568742 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3b780a3bb0 0x7f3b780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:54:58.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.139+0000 7f3b8be00640 1 -- 172.21.15.27:0/3322568742 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3b800d03d0 con 0x7f3b94105d60 2024-04-26T22:54:58.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.139+0000 7f3b93400640 1 --2- 172.21.15.27:0/3322568742 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3b780a3bb0 0x7f3b780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:54:58.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.140+0000 7f3b93400640 1 --2- 172.21.15.27:0/3322568742 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3b780a3bb0 0x7f3b780a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3b84005ab0 tx=0x7f3b84005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:54:58.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.143+0000 7f3b8be00640 1 -- 172.21.15.27:0/3322568742 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3b8009cb60 con 0x7f3b94105d60 2024-04-26T22:54:58.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:58 smithi094 ceph-mon[32889]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:58.285 INFO:journalctl@ceph.mds.f.smithi094.stdout:Apr 26 22:54:58 smithi094 podman[63188]: 2024-04-26 22:54:58.001162382 +0000 UTC m=+0.388368829 container start 885bd0a555de9a32fc8c768ec439f5b034c5e72e22e6b194faf09cb138a25668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:54:58.285 INFO:journalctl@ceph.mds.f.smithi094.stdout:Apr 26 22:54:58 smithi094 ceph-mds[63207]: -- 172.21.15.94:0/2469838654 <== mon.1 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d84a95ba40 con 0x55d84a92f000 2024-04-26T22:54:58.285 INFO:journalctl@ceph.mds.f.smithi094.stdout:Apr 26 22:54:58 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-f[63203]: starting mds.f at 2024-04-26T22:54:58.285 INFO:journalctl@ceph.mds.f.smithi094.stdout:Apr 26 22:54:58 smithi094 bash[63188]: 885bd0a555de9a32fc8c768ec439f5b034c5e72e22e6b194faf09cb138a25668 2024-04-26T22:54:58.285 INFO:journalctl@ceph.mds.f.smithi094.stdout:Apr 26 22:54:58 smithi094 systemd[1]: Started Ceph mds.f for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:54:58.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:58 smithi027 ceph-mon[28034]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:58.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.397+0000 7f3b9a000640 1 -- 172.21.15.27:0/3322568742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b94063530 con 0x7f3b94105d60 2024-04-26T22:54:58.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.398+0000 7f3b8be00640 1 -- 172.21.15.27:0/3322568742 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5512 (secure 0 0 0) 0x7f3b800a0e60 con 0x7f3b94105d60 2024-04-26T22:54:58.399 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:54:58.399 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-26T22:54:58.399 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2024-04-26T22:54:58.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.401+0000 7f3b9a000640 1 -- 172.21.15.27:0/3322568742 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3b780a3bb0 msgr2=0x7f3b780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.401+0000 7f3b9a000640 1 --2- 172.21.15.27:0/3322568742 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3b780a3bb0 0x7f3b780a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3b84005ab0 tx=0x7f3b84005a20 comp rx=0 tx=0).stop 2024-04-26T22:54:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.401+0000 7f3b9a000640 1 -- 172.21.15.27:0/3322568742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94105d60 msgr2=0x7f3b9419b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:54:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.401+0000 7f3b9a000640 1 --2- 172.21.15.27:0/3322568742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94105d60 0x7f3b9419b2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000e9f0 tx=0x7f3b8000eeb0 comp rx=0 tx=0).stop 2024-04-26T22:54:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.402+0000 7f3b9a000640 1 -- 172.21.15.27:0/3322568742 shutdown_connections 2024-04-26T22:54:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.402+0000 7f3b9a000640 1 --2- 172.21.15.27:0/3322568742 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3b780a3bb0 0x7f3b780a6070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.402+0000 7f3b9a000640 1 --2- 172.21.15.27:0/3322568742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b94106720 0x7f3b941a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.402+0000 7f3b9a000640 1 --2- 172.21.15.27:0/3322568742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94105d60 0x7f3b9419b2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.402+0000 7f3b9a000640 1 --2- 172.21.15.27:0/3322568742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b94104b70 0x7f3b9419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:54:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.402+0000 7f3b9a000640 1 -- 172.21.15.27:0/3322568742 >> 172.21.15.27:0/3322568742 conn(0x7f3b941000e0 msgr2=0x7f3b941090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:54:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.402+0000 7f3b9a000640 1 -- 172.21.15.27:0/3322568742 shutdown_connections 2024-04-26T22:54:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:54:58.402+0000 7f3b9a000640 1 -- 172.21.15.27:0/3322568742 wait complete. 2024-04-26T22:54:58.423 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:58 smithi080 ceph-mon[32324]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:54:59.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:59 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.94:6834/3513669668,v1:172.21.15.94:6835/3513669668] up:boot 2024-04-26T22:54:59.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:59 smithi080 ceph-mon[32324]: fsmap 6 up:standby 2024-04-26T22:54:59.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:59 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T22:54:59.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:59 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:59.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:59 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:59.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:59 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:59.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:59 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:59.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:59 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:59.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:59 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:59.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:59 smithi080 ceph-mon[32324]: Deploying daemon mds.c on smithi094 2024-04-26T22:54:59.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:54:59 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3322568742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:59.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:59 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.94:6834/3513669668,v1:172.21.15.94:6835/3513669668] up:boot 2024-04-26T22:54:59.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:59 smithi094 ceph-mon[32889]: fsmap 6 up:standby 2024-04-26T22:54:59.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:59 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T22:54:59.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:59 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:59.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:59 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:59.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:59 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:59.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:59 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:59.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:59 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:59.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:59 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:59.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:59 smithi094 ceph-mon[32889]: Deploying daemon mds.c on smithi094 2024-04-26T22:54:59.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:54:59 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3322568742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:54:59.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:59 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.94:6834/3513669668,v1:172.21.15.94:6835/3513669668] up:boot 2024-04-26T22:54:59.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:59 smithi027 ceph-mon[28034]: fsmap 6 up:standby 2024-04-26T22:54:59.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:59 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T22:54:59.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:59 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:59.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:59 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:59.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:59 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:54:59.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:59 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:54:59.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:59 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:54:59.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:59 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:59.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:59 smithi027 ceph-mon[28034]: Deploying daemon mds.c on smithi094 2024-04-26T22:54:59.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:54:59 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3322568742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:00.424 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:55:00.504 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:00 smithi027 ceph-mon[28034]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:00.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:00 smithi080 ceph-mon[32324]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:00.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:00 smithi094 ceph-mon[32889]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:00.535 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 22:55:00 smithi094 systemd[1]: Starting Ceph mds.c for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T22:55:00.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.650+0000 7f79a9e00640 1 -- 172.21.15.27:0/781335411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a41031f0 msgr2=0x7f79a41055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:00.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.650+0000 7f79a9e00640 1 --2- 172.21.15.27:0/781335411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a41031f0 0x7f79a41055e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7998009860 tx=0x7f799802a340 comp rx=0 tx=0).stop 2024-04-26T22:55:00.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.650+0000 7f79a9e00640 1 -- 172.21.15.27:0/781335411 shutdown_connections 2024-04-26T22:55:00.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.650+0000 7f79a9e00640 1 --2- 172.21.15.27:0/781335411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a4108450 0x7f79a410a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:00.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.650+0000 7f79a9e00640 1 --2- 172.21.15.27:0/781335411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a4105b20 0x7f79a4107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:00.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.650+0000 7f79a9e00640 1 --2- 172.21.15.27:0/781335411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a41031f0 0x7f79a41055e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:00.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.650+0000 7f79a9e00640 1 -- 172.21.15.27:0/781335411 >> 172.21.15.27:0/781335411 conn(0x7f79a40fcc00 msgr2=0x7f79a40ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:00.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.650+0000 7f79a9e00640 1 -- 172.21.15.27:0/781335411 shutdown_connections 2024-04-26T22:55:00.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.650+0000 7f79a9e00640 1 -- 172.21.15.27:0/781335411 wait complete. 2024-04-26T22:55:00.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.652+0000 7f79a9e00640 1 Processor -- start 2024-04-26T22:55:00.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.652+0000 7f79a9e00640 1 -- start start 2024-04-26T22:55:00.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.652+0000 7f79a9e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a41031f0 0x7f79a419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:00.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.652+0000 7f79a9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a4105b20 0x7f79a419aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:00.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.652+0000 7f79a9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a4108450 0x7f79a41a2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:00.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.652+0000 7f79a9e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f79a410e2c0 con 0x7f79a4105b20 2024-04-26T22:55:00.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.652+0000 7f79a9e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f79a410e140 con 0x7f79a41031f0 2024-04-26T22:55:00.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.652+0000 7f79a9e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f79a410e440 con 0x7f79a4108450 2024-04-26T22:55:00.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.653+0000 7f79a9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a4108450 0x7f79a41a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:00.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.653+0000 7f79a8a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a41031f0 0x7f79a419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:00.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.653+0000 7f79a9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a4108450 0x7f79a41a2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:32826/0 (socket says 172.21.15.27:32826) 2024-04-26T22:55:00.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.653+0000 7f79a9400640 1 -- 172.21.15.27:0/1041095207 learned_addr learned my addr 172.21.15.27:0/1041095207 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:00.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.653+0000 7f79a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a4105b20 0x7f79a419aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:00.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.653+0000 7f79a8a00640 1 -- 172.21.15.27:0/1041095207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a4108450 msgr2=0x7f79a41a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:00.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.653+0000 7f79a8a00640 1 --2- 172.21.15.27:0/1041095207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a4108450 0x7f79a41a2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:00.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.653+0000 7f79a8a00640 1 -- 172.21.15.27:0/1041095207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a4105b20 msgr2=0x7f79a419aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:00.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.653+0000 7f79a8a00640 1 --2- 172.21.15.27:0/1041095207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a4105b20 0x7f79a419aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:00.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.653+0000 7f79a8a00640 1 -- 172.21.15.27:0/1041095207 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a41a26d0 con 0x7f79a41031f0 2024-04-26T22:55:00.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.653+0000 7f79a8a00640 1 --2- 172.21.15.27:0/1041095207 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a41031f0 0x7f79a419aab0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f799802a860 tx=0x7f7998032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:00.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.654+0000 7f79a1600640 1 -- 172.21.15.27:0/1041095207 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998037040 con 0x7f79a41031f0 2024-04-26T22:55:00.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.654+0000 7f79a1600640 1 -- 172.21.15.27:0/1041095207 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7998030bd0 con 0x7f79a41031f0 2024-04-26T22:55:00.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.654+0000 7f79a9e00640 1 -- 172.21.15.27:0/1041095207 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a41a2960 con 0x7f79a41031f0 2024-04-26T22:55:00.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.654+0000 7f79a1600640 1 -- 172.21.15.27:0/1041095207 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998003850 con 0x7f79a41031f0 2024-04-26T22:55:00.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.654+0000 7f79a9e00640 1 -- 172.21.15.27:0/1041095207 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a41a2e40 con 0x7f79a41031f0 2024-04-26T22:55:00.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.655+0000 7f79a9e00640 1 -- 172.21.15.27:0/1041095207 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a4109fa0 con 0x7f79a41031f0 2024-04-26T22:55:00.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.657+0000 7f79a1600640 1 -- 172.21.15.27:0/1041095207 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7998043020 con 0x7f79a41031f0 2024-04-26T22:55:00.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.658+0000 7f79a1600640 1 --2- 172.21.15.27:0/1041095207 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f798c0a3cd0 0x7f798c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:00.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.659+0000 7f79a1600640 1 -- 172.21.15.27:0/1041095207 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f79980ef060 con 0x7f79a41031f0 2024-04-26T22:55:00.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.659+0000 7f79a3e00640 1 --2- 172.21.15.27:0/1041095207 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f798c0a3cd0 0x7f798c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:00.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.659+0000 7f79a3e00640 1 --2- 172.21.15.27:0/1041095207 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f798c0a3cd0 0x7f798c0a6190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f79a419bfc0 tx=0x7f7990009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:00.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.663+0000 7f79a1600640 1 -- 172.21.15.27:0/1041095207 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f79980bbe40 con 0x7f79a41031f0 2024-04-26T22:55:00.804 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 22:55:00 smithi094 podman[63496]: 2024-04-26 22:55:00.441723469 +0000 UTC m=+0.018138464 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:55:00.804 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 22:55:00 smithi094 podman[63496]: 2024-04-26 22:55:00.659517368 +0000 UTC m=+0.235932331 container create fa1e2683a695780eba2cfa1522e5735e4898950d6ba91b8ef9bf02d8337f83fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:55:00.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.910+0000 7f79a9e00640 1 -- 172.21.15.27:0/1041095207 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79a40635d0 con 0x7f79a41031f0 2024-04-26T22:55:00.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.911+0000 7f79a1600640 1 -- 172.21.15.27:0/1041095207 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5512 (secure 0 0 0) 0x7f7998030d80 con 0x7f79a41031f0 2024-04-26T22:55:00.912 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:00.912 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-26T22:55:00.912 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2024-04-26T22:55:00.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.914+0000 7f79a9e00640 1 -- 172.21.15.27:0/1041095207 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f798c0a3cd0 msgr2=0x7f798c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:00.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.914+0000 7f79a9e00640 1 --2- 172.21.15.27:0/1041095207 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f798c0a3cd0 0x7f798c0a6190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f79a419bfc0 tx=0x7f7990009260 comp rx=0 tx=0).stop 2024-04-26T22:55:00.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.914+0000 7f79a9e00640 1 -- 172.21.15.27:0/1041095207 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a41031f0 msgr2=0x7f79a419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:00.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.914+0000 7f79a9e00640 1 --2- 172.21.15.27:0/1041095207 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a41031f0 0x7f79a419aab0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f799802a860 tx=0x7f7998032dc0 comp rx=0 tx=0).stop 2024-04-26T22:55:00.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.914+0000 7f79a9e00640 1 -- 172.21.15.27:0/1041095207 shutdown_connections 2024-04-26T22:55:00.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.914+0000 7f79a9e00640 1 --2- 172.21.15.27:0/1041095207 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f798c0a3cd0 0x7f798c0a6190 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:00.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.914+0000 7f79a9e00640 1 --2- 172.21.15.27:0/1041095207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a4108450 0x7f79a41a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:00.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.914+0000 7f79a9e00640 1 --2- 172.21.15.27:0/1041095207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a4105b20 0x7f79a419aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:00.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.914+0000 7f79a9e00640 1 --2- 172.21.15.27:0/1041095207 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a41031f0 0x7f79a419aab0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:00.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.914+0000 7f79a9e00640 1 -- 172.21.15.27:0/1041095207 >> 172.21.15.27:0/1041095207 conn(0x7f79a40fcc00 msgr2=0x7f79a4102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:00.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.915+0000 7f79a9e00640 1 -- 172.21.15.27:0/1041095207 shutdown_connections 2024-04-26T22:55:00.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:00.915+0000 7f79a9e00640 1 -- 172.21.15.27:0/1041095207 wait complete. 2024-04-26T22:55:01.213 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 22:55:00 smithi094 podman[63496]: 2024-04-26 22:55:00.920659406 +0000 UTC m=+0.497074376 container init fa1e2683a695780eba2cfa1522e5735e4898950d6ba91b8ef9bf02d8337f83fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T22:55:01.213 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 22:55:00 smithi094 podman[63496]: 2024-04-26 22:55:00.923293977 +0000 UTC m=+0.499708947 container start fa1e2683a695780eba2cfa1522e5735e4898950d6ba91b8ef9bf02d8337f83fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T22:55:01.213 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 22:55:00 smithi094 ceph-mds[63515]: -- 172.21.15.94:0/3614220138 <== mon.1 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5621ea215a40 con 0x5621ea1e9000 2024-04-26T22:55:01.213 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 22:55:00 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-c[63511]: starting mds.c at 2024-04-26T22:55:01.214 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 22:55:01 smithi094 bash[63496]: fa1e2683a695780eba2cfa1522e5735e4898950d6ba91b8ef9bf02d8337f83fe 2024-04-26T22:55:01.214 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 22:55:01 smithi094 systemd[1]: Started Ceph mds.c for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:55:01.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:01 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1041095207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:01.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:01 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:01.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:01 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:01.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:01 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:01.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:01 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:55:01.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:01 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:55:01.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:01 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:01.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:01 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1041095207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:01.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:01.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:01.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:01.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:55:01.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:55:01.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:01 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:01.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:01 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1041095207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:01.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:01 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:01.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:01 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:01.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:01 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:01.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:01 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:55:01.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:01 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:55:01.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:01 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:02.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:02 smithi080 ceph-mon[32324]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:02.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:02 smithi080 ceph-mon[32324]: Deploying daemon mds.h on smithi080 2024-04-26T22:55:02.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:02 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.94:6836/3367319222,v1:172.21.15.94:6837/3367319222] up:boot 2024-04-26T22:55:02.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:02 smithi080 ceph-mon[32324]: fsmap 7 up:standby 2024-04-26T22:55:02.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:02 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T22:55:02.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:02 smithi094 ceph-mon[32889]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:02.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:02 smithi094 ceph-mon[32889]: Deploying daemon mds.h on smithi080 2024-04-26T22:55:02.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:02 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.94:6836/3367319222,v1:172.21.15.94:6837/3367319222] up:boot 2024-04-26T22:55:02.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:02 smithi094 ceph-mon[32889]: fsmap 7 up:standby 2024-04-26T22:55:02.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:02 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T22:55:02.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:02 smithi027 ceph-mon[28034]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:02.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:02 smithi027 ceph-mon[28034]: Deploying daemon mds.h on smithi080 2024-04-26T22:55:02.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:02 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.94:6836/3367319222,v1:172.21.15.94:6837/3367319222] up:boot 2024-04-26T22:55:02.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:02 smithi027 ceph-mon[28034]: fsmap 7 up:standby 2024-04-26T22:55:02.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:02 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T22:55:02.936 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:55:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.164+0000 7f6c37200640 1 -- 172.21.15.27:0/3648028618 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c38109f80 msgr2=0x7f6c3810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.164+0000 7f6c37200640 1 --2- 172.21.15.27:0/3648028618 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c38109f80 0x7f6c3810a400 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009900 tx=0x7f6c2002a410 comp rx=0 tx=0).stop 2024-04-26T22:55:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.164+0000 7f6c37200640 1 -- 172.21.15.27:0/3648028618 shutdown_connections 2024-04-26T22:55:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.164+0000 7f6c37200640 1 --2- 172.21.15.27:0/3648028618 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c38109f80 0x7f6c3810a400 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.164+0000 7f6c37200640 1 --2- 172.21.15.27:0/3648028618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c3810a940 0x7f6c381111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.164+0000 7f6c37200640 1 --2- 172.21.15.27:0/3648028618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c38108d90 0x7f6c381091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:03.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.164+0000 7f6c37200640 1 -- 172.21.15.27:0/3648028618 >> 172.21.15.27:0/3648028618 conn(0x7f6c38077d30 msgr2=0x7f6c3807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:03.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.165+0000 7f6c37200640 1 -- 172.21.15.27:0/3648028618 shutdown_connections 2024-04-26T22:55:03.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.165+0000 7f6c37200640 1 -- 172.21.15.27:0/3648028618 wait complete. 2024-04-26T22:55:03.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.166+0000 7f6c37200640 1 Processor -- start 2024-04-26T22:55:03.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.166+0000 7f6c37200640 1 -- start start 2024-04-26T22:55:03.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.166+0000 7f6c37200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c38108d90 0x7f6c3819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:03.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.166+0000 7f6c37200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c38109f80 0x7f6c3819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:03.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.166+0000 7f6c37200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c3810a940 0x7f6c381a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:03.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.167+0000 7f6c37200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6c381148f0 con 0x7f6c38109f80 2024-04-26T22:55:03.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.167+0000 7f6c37200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6c38114770 con 0x7f6c3810a940 2024-04-26T22:55:03.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.167+0000 7f6c37200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c38114a70 con 0x7f6c38108d90 2024-04-26T22:55:03.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.167+0000 7f6c36800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c3810a940 0x7f6c381a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:03.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.167+0000 7f6c35e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c38108d90 0x7f6c3819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:03.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.167+0000 7f6c35e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c38108d90 0x7f6c3819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:32858/0 (socket says 172.21.15.27:32858) 2024-04-26T22:55:03.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.167+0000 7f6c36800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c3810a940 0x7f6c381a6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:54730/0 (socket says 172.21.15.27:54730) 2024-04-26T22:55:03.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.167+0000 7f6c35e00640 1 -- 172.21.15.27:0/178290361 learned_addr learned my addr 172.21.15.27:0/178290361 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:03.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.167+0000 7f6c2fe00640 1 --2- 172.21.15.27:0/178290361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c38109f80 0x7f6c3819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:03.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.167+0000 7f6c36800640 1 -- 172.21.15.27:0/178290361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c38108d90 msgr2=0x7f6c3819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:03.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.167+0000 7f6c36800640 1 --2- 172.21.15.27:0/178290361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c38108d90 0x7f6c3819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:03.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.167+0000 7f6c36800640 1 -- 172.21.15.27:0/178290361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c38109f80 msgr2=0x7f6c3819f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:03.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.167+0000 7f6c36800640 1 --2- 172.21.15.27:0/178290361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c38109f80 0x7f6c3819f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:03.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.167+0000 7f6c36800640 1 -- 172.21.15.27:0/178290361 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c381a6b00 con 0x7f6c3810a940 2024-04-26T22:55:03.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.167+0000 7f6c35e00640 1 --2- 172.21.15.27:0/178290361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c38108d90 0x7f6c3819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:03.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.168+0000 7f6c36800640 1 --2- 172.21.15.27:0/178290361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c3810a940 0x7f6c381a6490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000da10 tx=0x7f6c3000ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:03.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.168+0000 7f6c2ea00640 1 -- 172.21.15.27:0/178290361 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3000bd70 con 0x7f6c3810a940 2024-04-26T22:55:03.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.168+0000 7f6c2ea00640 1 -- 172.21.15.27:0/178290361 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c30003e20 con 0x7f6c3810a940 2024-04-26T22:55:03.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.168+0000 7f6c2ea00640 1 -- 172.21.15.27:0/178290361 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c300106c0 con 0x7f6c3810a940 2024-04-26T22:55:03.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.168+0000 7f6c37200640 1 -- 172.21.15.27:0/178290361 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c381a6df0 con 0x7f6c3810a940 2024-04-26T22:55:03.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.168+0000 7f6c37200640 1 -- 172.21.15.27:0/178290361 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c381a72b0 con 0x7f6c3810a940 2024-04-26T22:55:03.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.169+0000 7f6c37200640 1 -- 172.21.15.27:0/178290361 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c38110050 con 0x7f6c3810a940 2024-04-26T22:55:03.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.171+0000 7f6c2ea00640 1 -- 172.21.15.27:0/178290361 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6c30004270 con 0x7f6c3810a940 2024-04-26T22:55:03.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.172+0000 7f6c2ea00640 1 --2- 172.21.15.27:0/178290361 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6c1c0a3cd0 0x7f6c1c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:03.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.173+0000 7f6c2ea00640 1 -- 172.21.15.27:0/178290361 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6c300d0310 con 0x7f6c3810a940 2024-04-26T22:55:03.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.173+0000 7f6c35e00640 1 --2- 172.21.15.27:0/178290361 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6c1c0a3cd0 0x7f6c1c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:03.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.173+0000 7f6c35e00640 1 --2- 172.21.15.27:0/178290361 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6c1c0a3cd0 0x7f6c1c0a6190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6c280104e0 tx=0x7f6c2800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:03.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.177+0000 7f6c2ea00640 1 -- 172.21.15.27:0/178290361 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6c3009d0f0 con 0x7f6c3810a940 2024-04-26T22:55:03.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:03 smithi027 ceph-mon[28034]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:03.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.431+0000 7f6c37200640 1 -- 172.21.15.27:0/178290361 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c3802dad0 con 0x7f6c3810a940 2024-04-26T22:55:03.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.432+0000 7f6c2ea00640 1 -- 172.21.15.27:0/178290361 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6337 (secure 0 0 0) 0x7f6c3009c9f0 con 0x7f6c3810a940 2024-04-26T22:55:03.433 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:03.433 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T22:55:03.433 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 8 2024-04-26T22:55:03.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.434+0000 7f6c37200640 1 -- 172.21.15.27:0/178290361 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6c1c0a3cd0 msgr2=0x7f6c1c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:03.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.434+0000 7f6c37200640 1 --2- 172.21.15.27:0/178290361 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6c1c0a3cd0 0x7f6c1c0a6190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6c280104e0 tx=0x7f6c2800b410 comp rx=0 tx=0).stop 2024-04-26T22:55:03.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.435+0000 7f6c37200640 1 -- 172.21.15.27:0/178290361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c3810a940 msgr2=0x7f6c381a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:03.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.435+0000 7f6c37200640 1 --2- 172.21.15.27:0/178290361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c3810a940 0x7f6c381a6490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000da10 tx=0x7f6c3000ded0 comp rx=0 tx=0).stop 2024-04-26T22:55:03.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.435+0000 7f6c37200640 1 -- 172.21.15.27:0/178290361 shutdown_connections 2024-04-26T22:55:03.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.435+0000 7f6c37200640 1 --2- 172.21.15.27:0/178290361 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6c1c0a3cd0 0x7f6c1c0a6190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:03.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.435+0000 7f6c37200640 1 --2- 172.21.15.27:0/178290361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c3810a940 0x7f6c381a6490 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:03.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.435+0000 7f6c37200640 1 --2- 172.21.15.27:0/178290361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c38109f80 0x7f6c3819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:03.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.435+0000 7f6c37200640 1 --2- 172.21.15.27:0/178290361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c38108d90 0x7f6c3819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:03.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.435+0000 7f6c37200640 1 -- 172.21.15.27:0/178290361 >> 172.21.15.27:0/178290361 conn(0x7f6c38077d30 msgr2=0x7f6c3810d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:03.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.435+0000 7f6c37200640 1 -- 172.21.15.27:0/178290361 shutdown_connections 2024-04-26T22:55:03.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:03.435+0000 7f6c37200640 1 -- 172.21.15.27:0/178290361 wait complete. 2024-04-26T22:55:03.501 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:03 smithi080 ceph-mon[32324]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:03.501 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 22:55:03 smithi080 systemd[1]: Starting Ceph mds.h for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T22:55:03.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:03 smithi094 ceph-mon[32889]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:03.751 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 22:55:03 smithi080 podman[62694]: 2024-04-26 22:55:03.404334564 +0000 UTC m=+0.017501679 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:55:03.752 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 22:55:03 smithi080 podman[62694]: 2024-04-26 22:55:03.505244062 +0000 UTC m=+0.118411167 container create 52d0925938a37cbfc30f910f269413b4ba60e9403c2f22b09116aceb5075d658 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T22:55:03.752 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 22:55:03 smithi080 podman[62694]: 2024-04-26 22:55:03.749823914 +0000 UTC m=+0.362991016 container init 52d0925938a37cbfc30f910f269413b4ba60e9403c2f22b09116aceb5075d658 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:55:03.752 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 22:55:03 smithi080 podman[62694]: 2024-04-26 22:55:03.752455571 +0000 UTC m=+0.365622674 container start 52d0925938a37cbfc30f910f269413b4ba60e9403c2f22b09116aceb5075d658 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T22:55:04.021 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 22:55:03 smithi080 ceph-mds[62713]: -- 172.21.15.80:0/1368271371 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4b61dda40 con 0x55a4b61b1000 2024-04-26T22:55:04.021 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 22:55:03 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-h[62709]: starting mds.h at 2024-04-26T22:55:04.021 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 22:55:03 smithi080 bash[62694]: 52d0925938a37cbfc30f910f269413b4ba60e9403c2f22b09116aceb5075d658 2024-04-26T22:55:04.022 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 22:55:03 smithi080 systemd[1]: Started Ceph mds.h for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:55:04.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:04 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/178290361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:04.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:04.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:04.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:04.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:55:04.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:55:04.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:04.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:04 smithi080 ceph-mon[32324]: Deploying daemon mds.i on smithi094 2024-04-26T22:55:04.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:04 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/178290361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:04.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:04.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:04.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:04.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:55:04.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:55:04.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:04.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:04 smithi094 ceph-mon[32889]: Deploying daemon mds.i on smithi094 2024-04-26T22:55:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:04 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/178290361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:55:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:55:04.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:04.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:04 smithi027 ceph-mon[28034]: Deploying daemon mds.i on smithi094 2024-04-26T22:55:05.456 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:55:05.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:05 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.80:6836/3963089995,v1:172.21.15.80:6837/3963089995] up:boot 2024-04-26T22:55:05.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:05 smithi080 ceph-mon[32324]: fsmap 8 up:standby 2024-04-26T22:55:05.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T22:55:05.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:05 smithi080 ceph-mon[32324]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:05.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:05 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.80:6836/3963089995,v1:172.21.15.80:6837/3963089995] up:boot 2024-04-26T22:55:05.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:05 smithi094 ceph-mon[32889]: fsmap 8 up:standby 2024-04-26T22:55:05.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T22:55:05.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:05 smithi094 ceph-mon[32889]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:05.539 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:05 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.80:6836/3963089995,v1:172.21.15.80:6837/3963089995] up:boot 2024-04-26T22:55:05.539 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:05 smithi027 ceph-mon[28034]: fsmap 8 up:standby 2024-04-26T22:55:05.539 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T22:55:05.540 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:05 smithi027 ceph-mon[28034]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:05.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.683+0000 7f229a800640 1 -- 172.21.15.27:0/3328566688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2294109f80 msgr2=0x7f229410a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:05.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.683+0000 7f229a800640 1 --2- 172.21.15.27:0/3328566688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2294109f80 0x7f229410a400 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2280009900 tx=0x7f228002a3f0 comp rx=0 tx=0).stop 2024-04-26T22:55:05.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.683+0000 7f229a800640 1 -- 172.21.15.27:0/3328566688 shutdown_connections 2024-04-26T22:55:05.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.683+0000 7f229a800640 1 --2- 172.21.15.27:0/3328566688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f229410a940 0x7f22941111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:05.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.683+0000 7f229a800640 1 --2- 172.21.15.27:0/3328566688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2294109f80 0x7f229410a400 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:05.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.683+0000 7f229a800640 1 --2- 172.21.15.27:0/3328566688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2294108d90 0x7f22941091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:05.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.683+0000 7f229a800640 1 -- 172.21.15.27:0/3328566688 >> 172.21.15.27:0/3328566688 conn(0x7f2294077d30 msgr2=0x7f229407a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:05.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.683+0000 7f229a800640 1 -- 172.21.15.27:0/3328566688 shutdown_connections 2024-04-26T22:55:05.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.683+0000 7f229a800640 1 -- 172.21.15.27:0/3328566688 wait complete. 2024-04-26T22:55:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.684+0000 7f229a800640 1 Processor -- start 2024-04-26T22:55:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.685+0000 7f229a800640 1 -- start start 2024-04-26T22:55:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.685+0000 7f229a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2294108d90 0x7f229419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.685+0000 7f229a800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2294109f80 0x7f229419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.685+0000 7f229a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f229410a940 0x7f22941a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.685+0000 7f229a800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2294114910 con 0x7f2294108d90 2024-04-26T22:55:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.685+0000 7f229a800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2294114790 con 0x7f2294109f80 2024-04-26T22:55:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.685+0000 7f229a800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2294114a90 con 0x7f229410a940 2024-04-26T22:55:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.685+0000 7f2299400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2294108d90 0x7f229419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.685+0000 7f2299400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2294108d90 0x7f229419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38296/0 (socket says 172.21.15.27:38296) 2024-04-26T22:55:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.685+0000 7f2299400640 1 -- 172.21.15.27:0/954894470 learned_addr learned my addr 172.21.15.27:0/954894470 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.685+0000 7f2298a00640 1 --2- 172.21.15.27:0/954894470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2294109f80 0x7f229419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.685+0000 7f2299e00640 1 --2- 172.21.15.27:0/954894470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f229410a940 0x7f22941a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.686+0000 7f2299400640 1 -- 172.21.15.27:0/954894470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f229410a940 msgr2=0x7f22941a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.686+0000 7f2299400640 1 --2- 172.21.15.27:0/954894470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f229410a940 0x7f22941a64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.686+0000 7f2299400640 1 -- 172.21.15.27:0/954894470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2294109f80 msgr2=0x7f229419f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.686+0000 7f2299400640 1 --2- 172.21.15.27:0/954894470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2294109f80 0x7f229419f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.686+0000 7f2299400640 1 -- 172.21.15.27:0/954894470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22941a6b40 con 0x7f2294108d90 2024-04-26T22:55:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.686+0000 7f2298a00640 1 --2- 172.21.15.27:0/954894470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2294109f80 0x7f229419f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:55:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.686+0000 7f2299400640 1 --2- 172.21.15.27:0/954894470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2294108d90 0x7f229419ef20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f228800e9c0 tx=0x7f228800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.686+0000 7f2292000640 1 -- 172.21.15.27:0/954894470 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228800cd80 con 0x7f2294108d90 2024-04-26T22:55:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.686+0000 7f2292000640 1 -- 172.21.15.27:0/954894470 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22880035f0 con 0x7f2294108d90 2024-04-26T22:55:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.686+0000 7f2292000640 1 -- 172.21.15.27:0/954894470 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288010690 con 0x7f2294108d90 2024-04-26T22:55:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.686+0000 7f229a800640 1 -- 172.21.15.27:0/954894470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22941a6e30 con 0x7f2294108d90 2024-04-26T22:55:05.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.686+0000 7f229a800640 1 -- 172.21.15.27:0/954894470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22941a7290 con 0x7f2294108d90 2024-04-26T22:55:05.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.688+0000 7f229a800640 1 -- 172.21.15.27:0/954894470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2294110050 con 0x7f2294108d90 2024-04-26T22:55:05.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.690+0000 7f2292000640 1 -- 172.21.15.27:0/954894470 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2288004000 con 0x7f2294108d90 2024-04-26T22:55:05.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.691+0000 7f2292000640 1 --2- 172.21.15.27:0/954894470 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f227c0a3cd0 0x7f227c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:05.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.691+0000 7f2292000640 1 -- 172.21.15.27:0/954894470 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f22880d0550 con 0x7f2294108d90 2024-04-26T22:55:05.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.691+0000 7f2298a00640 1 --2- 172.21.15.27:0/954894470 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f227c0a3cd0 0x7f227c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:05.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.692+0000 7f2298a00640 1 --2- 172.21.15.27:0/954894470 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f227c0a3cd0 0x7f227c0a6190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2280000c00 tx=0x7f22800314c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:05.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.696+0000 7f2292000640 1 -- 172.21.15.27:0/954894470 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f228809d330 con 0x7f2294108d90 2024-04-26T22:55:05.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.949+0000 7f229a800640 1 -- 172.21.15.27:0/954894470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f229402dad0 con 0x7f2294108d90 2024-04-26T22:55:05.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.951+0000 7f2292000640 1 -- 172.21.15.27:0/954894470 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7162 (secure 0 0 0) 0x7f228809cc30 con 0x7f2294108d90 2024-04-26T22:55:05.951 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:05.952 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T22:55:05.952 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 9 2024-04-26T22:55:05.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.954+0000 7f229a800640 1 -- 172.21.15.27:0/954894470 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f227c0a3cd0 msgr2=0x7f227c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.954+0000 7f229a800640 1 --2- 172.21.15.27:0/954894470 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f227c0a3cd0 0x7f227c0a6190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2280000c00 tx=0x7f22800314c0 comp rx=0 tx=0).stop 2024-04-26T22:55:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.954+0000 7f229a800640 1 -- 172.21.15.27:0/954894470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2294108d90 msgr2=0x7f229419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.954+0000 7f229a800640 1 --2- 172.21.15.27:0/954894470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2294108d90 0x7f229419ef20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f228800e9c0 tx=0x7f228800ee80 comp rx=0 tx=0).stop 2024-04-26T22:55:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.954+0000 7f229a800640 1 -- 172.21.15.27:0/954894470 shutdown_connections 2024-04-26T22:55:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.954+0000 7f229a800640 1 --2- 172.21.15.27:0/954894470 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f227c0a3cd0 0x7f227c0a6190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.954+0000 7f229a800640 1 --2- 172.21.15.27:0/954894470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2294109f80 0x7f229419f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.954+0000 7f229a800640 1 --2- 172.21.15.27:0/954894470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f229410a940 0x7f22941a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.954+0000 7f229a800640 1 --2- 172.21.15.27:0/954894470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2294108d90 0x7f229419ef20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.954+0000 7f229a800640 1 -- 172.21.15.27:0/954894470 >> 172.21.15.27:0/954894470 conn(0x7f2294077d30 msgr2=0x7f229410d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.954+0000 7f229a800640 1 -- 172.21.15.27:0/954894470 shutdown_connections 2024-04-26T22:55:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:05.954+0000 7f229a800640 1 -- 172.21.15.27:0/954894470 wait complete. 2024-04-26T22:55:06.251 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 22:55:06 smithi094 systemd[1]: Starting Ceph mds.i for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T22:55:06.518 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:06 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/954894470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:06.518 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 22:55:06 smithi094 podman[63808]: 2024-04-26 22:55:06.227255548 +0000 UTC m=+0.018295663 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:55:06.518 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 22:55:06 smithi094 podman[63808]: 2024-04-26 22:55:06.372504602 +0000 UTC m=+0.163544711 container create d19201f63fdd1ad6ba3d24e8bb94eec8c0318fde9527ed5cb0044f96b5b895b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-26T22:55:06.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:06 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/954894470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:06.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:06 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/954894470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:06.770 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 22:55:06 smithi094 podman[63808]: 2024-04-26 22:55:06.662514252 +0000 UTC m=+0.453554365 container init d19201f63fdd1ad6ba3d24e8bb94eec8c0318fde9527ed5cb0044f96b5b895b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-26T22:55:06.770 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 22:55:06 smithi094 podman[63808]: 2024-04-26 22:55:06.665123242 +0000 UTC m=+0.456163354 container start d19201f63fdd1ad6ba3d24e8bb94eec8c0318fde9527ed5cb0044f96b5b895b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:55:06.770 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 22:55:06 smithi094 ceph-mds[63828]: -- 172.21.15.94:0/3121834248 <== mon.1 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dacaa5da40 con 0x55dacaa31000 2024-04-26T22:55:06.770 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 22:55:06 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-i[63824]: starting mds.i at 2024-04-26T22:55:06.770 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 22:55:06 smithi094 bash[63808]: d19201f63fdd1ad6ba3d24e8bb94eec8c0318fde9527ed5cb0044f96b5b895b7 2024-04-26T22:55:07.034 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 22:55:06 smithi094 systemd[1]: Started Ceph mds.i for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:55:07.974 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:55:08.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:07 smithi027 ceph-mon[28034]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:08.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:08.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:08.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:08.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:55:08.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:55:08.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:08.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:07 smithi027 ceph-mon[28034]: Deploying daemon mds.g on smithi027 2024-04-26T22:55:08.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:07 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.94:6838/1620677319,v1:172.21.15.94:6839/1620677319] up:boot 2024-04-26T22:55:08.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:07 smithi027 ceph-mon[28034]: fsmap 9 up:standby 2024-04-26T22:55:08.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:07 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T22:55:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.197+0000 7f6daa800640 1 -- 172.21.15.27:0/2962447589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da4105b90 msgr2=0x7f6da4106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.197+0000 7f6daa800640 1 --2- 172.21.15.27:0/2962447589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da4105b90 0x7f6da4106010 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009900 tx=0x7f6d9002a410 comp rx=0 tx=0).stop 2024-04-26T22:55:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.197+0000 7f6daa800640 1 -- 172.21.15.27:0/2962447589 shutdown_connections 2024-04-26T22:55:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.197+0000 7f6daa800640 1 --2- 172.21.15.27:0/2962447589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6da4106550 0x7f6da410cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.197+0000 7f6daa800640 1 --2- 172.21.15.27:0/2962447589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da4105b90 0x7f6da4106010 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:08.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.197+0000 7f6daa800640 1 --2- 172.21.15.27:0/2962447589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da41049a0 0x7f6da4104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:08.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.197+0000 7f6daa800640 1 -- 172.21.15.27:0/2962447589 >> 172.21.15.27:0/2962447589 conn(0x7f6da40fff10 msgr2=0x7f6da4102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:08.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.198+0000 7f6daa800640 1 -- 172.21.15.27:0/2962447589 shutdown_connections 2024-04-26T22:55:08.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.198+0000 7f6daa800640 1 -- 172.21.15.27:0/2962447589 wait complete. 2024-04-26T22:55:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.199+0000 7f6daa800640 1 Processor -- start 2024-04-26T22:55:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.199+0000 7f6daa800640 1 -- start start 2024-04-26T22:55:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.199+0000 7f6daa800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da41049a0 0x7f6da419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.199+0000 7f6daa800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6da4105b90 0x7f6da419afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.199+0000 7f6daa800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da4106550 0x7f6da41a2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.199+0000 7f6daa800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6da4110520 con 0x7f6da4106550 2024-04-26T22:55:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.199+0000 7f6daa800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6da41103a0 con 0x7f6da41049a0 2024-04-26T22:55:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.199+0000 7f6daa800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6da41106a0 con 0x7f6da4105b90 2024-04-26T22:55:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.199+0000 7f6da9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da4106550 0x7f6da41a2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.199+0000 7f6da9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da4106550 0x7f6da41a2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41764/0 (socket says 172.21.15.27:41764) 2024-04-26T22:55:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.199+0000 7f6da9e00640 1 -- 172.21.15.27:0/1370447921 learned_addr learned my addr 172.21.15.27:0/1370447921 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.200+0000 7f6da9400640 1 --2- 172.21.15.27:0/1370447921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da41049a0 0x7f6da419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.200+0000 7f6da8a00640 1 --2- 172.21.15.27:0/1370447921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6da4105b90 0x7f6da419afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.200+0000 7f6da9e00640 1 -- 172.21.15.27:0/1370447921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6da4105b90 msgr2=0x7f6da419afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.200+0000 7f6da9e00640 1 --2- 172.21.15.27:0/1370447921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6da4105b90 0x7f6da419afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.200+0000 7f6da9e00640 1 -- 172.21.15.27:0/1370447921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da41049a0 msgr2=0x7f6da419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.200+0000 7f6da9e00640 1 --2- 172.21.15.27:0/1370447921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da41049a0 0x7f6da419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.200+0000 7f6da9e00640 1 -- 172.21.15.27:0/1370447921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da41a26c0 con 0x7f6da4106550 2024-04-26T22:55:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.200+0000 7f6da9400640 1 --2- 172.21.15.27:0/1370447921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da41049a0 0x7f6da419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.200+0000 7f6da9e00640 1 --2- 172.21.15.27:0/1370447921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da4106550 0x7f6da41a2050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00eb10 tx=0x7f6d9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.200+0000 7f6da2000640 1 -- 172.21.15.27:0/1370447921 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c004b20 con 0x7f6da4106550 2024-04-26T22:55:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.200+0000 7f6da2000640 1 -- 172.21.15.27:0/1370447921 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d9c004cc0 con 0x7f6da4106550 2024-04-26T22:55:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.201+0000 7f6da2000640 1 -- 172.21.15.27:0/1370447921 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c010670 con 0x7f6da4106550 2024-04-26T22:55:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.201+0000 7f6daa800640 1 -- 172.21.15.27:0/1370447921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da41a29b0 con 0x7f6da4106550 2024-04-26T22:55:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.201+0000 7f6daa800640 1 -- 172.21.15.27:0/1370447921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da41a2f20 con 0x7f6da4106550 2024-04-26T22:55:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.202+0000 7f6daa800640 1 -- 172.21.15.27:0/1370447921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da410bbf0 con 0x7f6da4106550 2024-04-26T22:55:08.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.204+0000 7f6da2000640 1 -- 172.21.15.27:0/1370447921 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6d9c005020 con 0x7f6da4106550 2024-04-26T22:55:08.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.205+0000 7f6da2000640 1 --2- 172.21.15.27:0/1370447921 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6d8c0a3cd0 0x7f6d8c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:08.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.205+0000 7f6da2000640 1 -- 172.21.15.27:0/1370447921 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6d9c0d04a0 con 0x7f6da4106550 2024-04-26T22:55:08.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.206+0000 7f6da9400640 1 --2- 172.21.15.27:0/1370447921 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6d8c0a3cd0 0x7f6d8c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:08.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.206+0000 7f6da9400640 1 --2- 172.21.15.27:0/1370447921 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6d8c0a3cd0 0x7f6d8c0a6190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6da41059f0 tx=0x7f6d9800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:08.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.211+0000 7f6da2000640 1 -- 172.21.15.27:0/1370447921 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6d9c09d280 con 0x7f6da4106550 2024-04-26T22:55:08.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:07 smithi080 ceph-mon[32324]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:08.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:07 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:08.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:07 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:08.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:07 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:08.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:07 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:55:08.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:07 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:55:08.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:07 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:08.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:07 smithi080 ceph-mon[32324]: Deploying daemon mds.g on smithi027 2024-04-26T22:55:08.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:07 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.94:6838/1620677319,v1:172.21.15.94:6839/1620677319] up:boot 2024-04-26T22:55:08.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:07 smithi080 ceph-mon[32324]: fsmap 9 up:standby 2024-04-26T22:55:08.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:07 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T22:55:08.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:07 smithi094 ceph-mon[32889]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:08.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:08.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:08.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:08.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:55:08.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:55:08.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:08.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:07 smithi094 ceph-mon[32889]: Deploying daemon mds.g on smithi027 2024-04-26T22:55:08.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:07 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.94:6838/1620677319,v1:172.21.15.94:6839/1620677319] up:boot 2024-04-26T22:55:08.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:07 smithi094 ceph-mon[32889]: fsmap 9 up:standby 2024-04-26T22:55:08.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:07 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T22:55:08.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.467+0000 7f6daa800640 1 -- 172.21.15.27:0/1370447921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6da4063530 con 0x7f6da4106550 2024-04-26T22:55:08.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.469+0000 7f6da2000640 1 -- 172.21.15.27:0/1370447921 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7989 (secure 0 0 0) 0x7f6d9c09cb80 con 0x7f6da4106550 2024-04-26T22:55:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:08.469 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/1620677319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1620677319},{"type":"v1","addr":"172.21.15.94:6839","nonce":1620677319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T22:55:08.469 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 10 2024-04-26T22:55:08.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.471+0000 7f6daa800640 1 -- 172.21.15.27:0/1370447921 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6d8c0a3cd0 msgr2=0x7f6d8c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:08.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.471+0000 7f6daa800640 1 --2- 172.21.15.27:0/1370447921 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6d8c0a3cd0 0x7f6d8c0a6190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6da41059f0 tx=0x7f6d9800b390 comp rx=0 tx=0).stop 2024-04-26T22:55:08.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.471+0000 7f6daa800640 1 -- 172.21.15.27:0/1370447921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da4106550 msgr2=0x7f6da41a2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:08.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.471+0000 7f6daa800640 1 --2- 172.21.15.27:0/1370447921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da4106550 0x7f6da41a2050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00eb10 tx=0x7f6d9c00c2d0 comp rx=0 tx=0).stop 2024-04-26T22:55:08.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.471+0000 7f6daa800640 1 -- 172.21.15.27:0/1370447921 shutdown_connections 2024-04-26T22:55:08.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.471+0000 7f6daa800640 1 --2- 172.21.15.27:0/1370447921 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6d8c0a3cd0 0x7f6d8c0a6190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:08.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.471+0000 7f6daa800640 1 --2- 172.21.15.27:0/1370447921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da4106550 0x7f6da41a2050 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:08.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.471+0000 7f6daa800640 1 --2- 172.21.15.27:0/1370447921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6da4105b90 0x7f6da419afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:08.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.471+0000 7f6daa800640 1 --2- 172.21.15.27:0/1370447921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da41049a0 0x7f6da419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:08.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.471+0000 7f6daa800640 1 -- 172.21.15.27:0/1370447921 >> 172.21.15.27:0/1370447921 conn(0x7f6da40fff10 msgr2=0x7f6da4108dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:08.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.472+0000 7f6daa800640 1 -- 172.21.15.27:0/1370447921 shutdown_connections 2024-04-26T22:55:08.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:08.472+0000 7f6daa800640 1 -- 172.21.15.27:0/1370447921 wait complete. 2024-04-26T22:55:09.086 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:08 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1370447921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:09.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:08 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1370447921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:09.121 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:08 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1370447921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:09.907 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:09 smithi027 ceph-mon[28034]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:10.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:09 smithi080 ceph-mon[32324]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:10.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:09 smithi094 ceph-mon[32889]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:10.499 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:55:10.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.696+0000 7f20efa00640 1 -- 172.21.15.27:0/3034635163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20e8106720 msgr2=0x7f20e810cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:10.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.696+0000 7f20efa00640 1 --2- 172.21.15.27:0/3034635163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20e8106720 0x7f20e810cfb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009ca0 tx=0x7f20e002a8c0 comp rx=0 tx=0).stop 2024-04-26T22:55:10.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.697+0000 7f20efa00640 1 -- 172.21.15.27:0/3034635163 shutdown_connections 2024-04-26T22:55:10.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.697+0000 7f20efa00640 1 --2- 172.21.15.27:0/3034635163 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20e8105d60 0x7f20e81061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:10.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.697+0000 7f20efa00640 1 --2- 172.21.15.27:0/3034635163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20e8106720 0x7f20e810cfb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:10.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.697+0000 7f20efa00640 1 --2- 172.21.15.27:0/3034635163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20e8104b70 0x7f20e8104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:10.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.697+0000 7f20efa00640 1 -- 172.21.15.27:0/3034635163 >> 172.21.15.27:0/3034635163 conn(0x7f20e81000e0 msgr2=0x7f20e8102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:10.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.697+0000 7f20efa00640 1 -- 172.21.15.27:0/3034635163 shutdown_connections 2024-04-26T22:55:10.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.697+0000 7f20efa00640 1 -- 172.21.15.27:0/3034635163 wait complete. 2024-04-26T22:55:10.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.698+0000 7f20efa00640 1 Processor -- start 2024-04-26T22:55:10.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.698+0000 7f20efa00640 1 -- start start 2024-04-26T22:55:10.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.698+0000 7f20efa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20e8104b70 0x7f20e819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:10.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.698+0000 7f20efa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20e8105d60 0x7f20e819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:10.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.698+0000 7f20efa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20e8106720 0x7f20e81a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:10.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.698+0000 7f20efa00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81106d0 con 0x7f20e8104b70 2024-04-26T22:55:10.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.698+0000 7f20efa00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8110550 con 0x7f20e8105d60 2024-04-26T22:55:10.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.698+0000 7f20efa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8110850 con 0x7f20e8106720 2024-04-26T22:55:10.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.698+0000 7f20ed000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20e8104b70 0x7f20e819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:10.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.698+0000 7f20ed000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20e8104b70 0x7f20e819ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41776/0 (socket says 172.21.15.27:41776) 2024-04-26T22:55:10.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.698+0000 7f20ed000640 1 -- 172.21.15.27:0/1083443443 learned_addr learned my addr 172.21.15.27:0/1083443443 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:10.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.698+0000 7f20eda00640 1 --2- 172.21.15.27:0/1083443443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20e8106720 0x7f20e81a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:10.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.698+0000 7f20e7e00640 1 --2- 172.21.15.27:0/1083443443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20e8105d60 0x7f20e819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:10.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.699+0000 7f20ed000640 1 -- 172.21.15.27:0/1083443443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20e8106720 msgr2=0x7f20e81a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:10.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.699+0000 7f20ed000640 1 --2- 172.21.15.27:0/1083443443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20e8106720 0x7f20e81a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:10.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.699+0000 7f20ed000640 1 -- 172.21.15.27:0/1083443443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20e8105d60 msgr2=0x7f20e819b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:10.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.699+0000 7f20ed000640 1 --2- 172.21.15.27:0/1083443443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20e8105d60 0x7f20e819b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:10.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.699+0000 7f20ed000640 1 -- 172.21.15.27:0/1083443443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e81a2940 con 0x7f20e8104b70 2024-04-26T22:55:10.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.699+0000 7f20ed000640 1 --2- 172.21.15.27:0/1083443443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20e8104b70 0x7f20e819ad30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00e9c0 tx=0x7f20dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:10.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.699+0000 7f20e5600640 1 -- 172.21.15.27:0/1083443443 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc00cd80 con 0x7f20e8104b70 2024-04-26T22:55:10.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.699+0000 7f20efa00640 1 -- 172.21.15.27:0/1083443443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e81a2c30 con 0x7f20e8104b70 2024-04-26T22:55:10.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.699+0000 7f20e5600640 1 -- 172.21.15.27:0/1083443443 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20dc0035f0 con 0x7f20e8104b70 2024-04-26T22:55:10.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.699+0000 7f20efa00640 1 -- 172.21.15.27:0/1083443443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e81a31d0 con 0x7f20e8104b70 2024-04-26T22:55:10.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.699+0000 7f20e5600640 1 -- 172.21.15.27:0/1083443443 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc018630 con 0x7f20e8104b70 2024-04-26T22:55:10.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.701+0000 7f20efa00640 1 -- 172.21.15.27:0/1083443443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e810be90 con 0x7f20e8104b70 2024-04-26T22:55:10.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.703+0000 7f20e5600640 1 -- 172.21.15.27:0/1083443443 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f20dc0037a0 con 0x7f20e8104b70 2024-04-26T22:55:10.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.704+0000 7f20e5600640 1 --2- 172.21.15.27:0/1083443443 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f20d00a3bb0 0x7f20d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:10.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.704+0000 7f20e5600640 1 -- 172.21.15.27:0/1083443443 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f20dc0d03f0 con 0x7f20e8104b70 2024-04-26T22:55:10.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.705+0000 7f20e7e00640 1 --2- 172.21.15.27:0/1083443443 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f20d00a3bb0 0x7f20d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:10.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.705+0000 7f20e7e00640 1 --2- 172.21.15.27:0/1083443443 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f20d00a3bb0 0x7f20d00a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20d40095f0 tx=0x7f20d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:10.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.708+0000 7f20e5600640 1 -- 172.21.15.27:0/1083443443 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f20dc09d1d0 con 0x7f20e8104b70 2024-04-26T22:55:10.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.952+0000 7f20efa00640 1 -- 172.21.15.27:0/1083443443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20e802dad0 con 0x7f20e8104b70 2024-04-26T22:55:10.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.954+0000 7f20e5600640 1 -- 172.21.15.27:0/1083443443 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7989 (secure 0 0 0) 0x7f20dc09cad0 con 0x7f20e8104b70 2024-04-26T22:55:10.954 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:10.955 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/1620677319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1620677319},{"type":"v1","addr":"172.21.15.94:6839","nonce":1620677319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T22:55:10.955 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 10 2024-04-26T22:55:10.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.956+0000 7f20efa00640 1 -- 172.21.15.27:0/1083443443 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f20d00a3bb0 msgr2=0x7f20d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:10.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.956+0000 7f20efa00640 1 --2- 172.21.15.27:0/1083443443 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f20d00a3bb0 0x7f20d00a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20d40095f0 tx=0x7f20d4005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:10.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.956+0000 7f20efa00640 1 -- 172.21.15.27:0/1083443443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20e8104b70 msgr2=0x7f20e819ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:10.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.956+0000 7f20efa00640 1 --2- 172.21.15.27:0/1083443443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20e8104b70 0x7f20e819ad30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00e9c0 tx=0x7f20dc00ee80 comp rx=0 tx=0).stop 2024-04-26T22:55:10.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.957+0000 7f20efa00640 1 -- 172.21.15.27:0/1083443443 shutdown_connections 2024-04-26T22:55:10.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.957+0000 7f20efa00640 1 --2- 172.21.15.27:0/1083443443 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f20d00a3bb0 0x7f20d00a6070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:10.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.957+0000 7f20efa00640 1 --2- 172.21.15.27:0/1083443443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20e8105d60 0x7f20e819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:10.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.957+0000 7f20efa00640 1 --2- 172.21.15.27:0/1083443443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20e8106720 0x7f20e81a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:10.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.957+0000 7f20efa00640 1 --2- 172.21.15.27:0/1083443443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20e8104b70 0x7f20e819ad30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:10.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.957+0000 7f20efa00640 1 -- 172.21.15.27:0/1083443443 >> 172.21.15.27:0/1083443443 conn(0x7f20e81000e0 msgr2=0x7f20e8109060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:10.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.957+0000 7f20efa00640 1 -- 172.21.15.27:0/1083443443 shutdown_connections 2024-04-26T22:55:10.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:10.957+0000 7f20efa00640 1 -- 172.21.15.27:0/1083443443 wait complete. 2024-04-26T22:55:11.614 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 22:55:11 smithi027 systemd[1]: Started Ceph mds.g for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:55:12.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:11 smithi027 ceph-mon[28034]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:12.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:11 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1083443443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:12.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:11 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:12.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:11 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:12.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:11 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:12.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:11 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:55:12.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:11 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:55:12.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:11 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:12.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:11 smithi080 ceph-mon[32324]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:12.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:11 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1083443443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:12.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:12.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:12.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:12.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:55:12.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:55:12.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:11 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:12.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:11 smithi094 ceph-mon[32889]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:12.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:11 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1083443443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:12.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:11 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:12.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:11 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:12.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:11 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:12.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:11 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:55:12.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:11 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:55:12.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:11 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:12.976 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:55:12.989 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:12 smithi027 ceph-mon[28034]: Deploying daemon mds.a on smithi027 2024-04-26T22:55:12.989 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:12 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] up:boot 2024-04-26T22:55:12.989 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:12 smithi027 ceph-mon[28034]: fsmap 10 up:standby 2024-04-26T22:55:12.989 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:12 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T22:55:13.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.186+0000 7fe677000640 1 -- 172.21.15.27:0/1237297897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe67010a940 msgr2=0x7fe6701111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:13.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.186+0000 7fe677000640 1 --2- 172.21.15.27:0/1237297897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe67010a940 0x7fe6701111d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe668009ca0 tx=0x7fe66802a8c0 comp rx=0 tx=0).stop 2024-04-26T22:55:13.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.186+0000 7fe677000640 1 -- 172.21.15.27:0/1237297897 shutdown_connections 2024-04-26T22:55:13.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.186+0000 7fe677000640 1 --2- 172.21.15.27:0/1237297897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe67010a940 0x7fe6701111d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:13.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.186+0000 7fe677000640 1 --2- 172.21.15.27:0/1237297897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe670109f80 0x7fe67010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:13.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.186+0000 7fe677000640 1 --2- 172.21.15.27:0/1237297897 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe670108d90 0x7fe6701091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:13.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.186+0000 7fe677000640 1 -- 172.21.15.27:0/1237297897 >> 172.21.15.27:0/1237297897 conn(0x7fe670077d30 msgr2=0x7fe67007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:13.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.186+0000 7fe677000640 1 -- 172.21.15.27:0/1237297897 shutdown_connections 2024-04-26T22:55:13.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.186+0000 7fe677000640 1 -- 172.21.15.27:0/1237297897 wait complete. 2024-04-26T22:55:13.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.187+0000 7fe677000640 1 Processor -- start 2024-04-26T22:55:13.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.187+0000 7fe677000640 1 -- start start 2024-04-26T22:55:13.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.187+0000 7fe677000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe670108d90 0x7fe67019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:13.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.187+0000 7fe677000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe670109f80 0x7fe67019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:13.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.187+0000 7fe677000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe67010a940 0x7fe6701a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:13.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.187+0000 7fe677000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe670114910 con 0x7fe670109f80 2024-04-26T22:55:13.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.187+0000 7fe677000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe670114790 con 0x7fe67010a940 2024-04-26T22:55:13.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.187+0000 7fe677000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe670114a90 con 0x7fe670108d90 2024-04-26T22:55:13.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.188+0000 7fe675c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe670108d90 0x7fe67019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.188+0000 7fe675c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe670108d90 0x7fe67019ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:47866/0 (socket says 172.21.15.27:47866) 2024-04-26T22:55:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.188+0000 7fe675c00640 1 -- 172.21.15.27:0/3393647160 learned_addr learned my addr 172.21.15.27:0/3393647160 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.188+0000 7fe675200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe670109f80 0x7fe67019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.188+0000 7fe676600640 1 --2- 172.21.15.27:0/3393647160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe67010a940 0x7fe6701a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.188+0000 7fe675c00640 1 -- 172.21.15.27:0/3393647160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe67010a940 msgr2=0x7fe6701a6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.188+0000 7fe675c00640 1 --2- 172.21.15.27:0/3393647160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe67010a940 0x7fe6701a6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.188+0000 7fe675c00640 1 -- 172.21.15.27:0/3393647160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe670109f80 msgr2=0x7fe67019f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.188+0000 7fe675c00640 1 --2- 172.21.15.27:0/3393647160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe670109f80 0x7fe67019f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.188+0000 7fe675c00640 1 -- 172.21.15.27:0/3393647160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6701a6a80 con 0x7fe670108d90 2024-04-26T22:55:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.188+0000 7fe675200640 1 --2- 172.21.15.27:0/3393647160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe670109f80 0x7fe67019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.188+0000 7fe675c00640 1 --2- 172.21.15.27:0/3393647160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe670108d90 0x7fe67019ee60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe66400e9c0 tx=0x7fe66400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.189+0000 7fe66e000640 1 -- 172.21.15.27:0/3393647160 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66400cd80 con 0x7fe670108d90 2024-04-26T22:55:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.189+0000 7fe66e000640 1 -- 172.21.15.27:0/3393647160 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6640035f0 con 0x7fe670108d90 2024-04-26T22:55:13.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.189+0000 7fe66e000640 1 -- 172.21.15.27:0/3393647160 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664010690 con 0x7fe670108d90 2024-04-26T22:55:13.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.189+0000 7fe677000640 1 -- 172.21.15.27:0/3393647160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6701a6d70 con 0x7fe670108d90 2024-04-26T22:55:13.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.189+0000 7fe677000640 1 -- 172.21.15.27:0/3393647160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6701a7200 con 0x7fe670108d90 2024-04-26T22:55:13.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.190+0000 7fe677000640 1 -- 172.21.15.27:0/3393647160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe67010ffe0 con 0x7fe670108d90 2024-04-26T22:55:13.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.192+0000 7fe66e000640 1 -- 172.21.15.27:0/3393647160 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe664004000 con 0x7fe670108d90 2024-04-26T22:55:13.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.194+0000 7fe66e000640 1 --2- 172.21.15.27:0/3393647160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6580a3cd0 0x7fe6580a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:13.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.194+0000 7fe66e000640 1 -- 172.21.15.27:0/3393647160 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe6640d0530 con 0x7fe670108d90 2024-04-26T22:55:13.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.194+0000 7fe675200640 1 --2- 172.21.15.27:0/3393647160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6580a3cd0 0x7fe6580a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:13.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.195+0000 7fe675200640 1 --2- 172.21.15.27:0/3393647160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6580a3cd0 0x7fe6580a6190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe6701a0370 tx=0x7fe65c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:13.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.196+0000 7fe66e000640 1 -- 172.21.15.27:0/3393647160 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe66409d310 con 0x7fe670108d90 2024-04-26T22:55:13.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:12 smithi080 ceph-mon[32324]: Deploying daemon mds.a on smithi027 2024-04-26T22:55:13.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:12 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] up:boot 2024-04-26T22:55:13.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:12 smithi080 ceph-mon[32324]: fsmap 10 up:standby 2024-04-26T22:55:13.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:12 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T22:55:13.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:12 smithi094 ceph-mon[32889]: Deploying daemon mds.a on smithi027 2024-04-26T22:55:13.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:12 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] up:boot 2024-04-26T22:55:13.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:12 smithi094 ceph-mon[32889]: fsmap 10 up:standby 2024-04-26T22:55:13.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:12 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T22:55:13.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.445+0000 7fe677000640 1 -- 172.21.15.27:0/3393647160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe67002dad0 con 0x7fe670108d90 2024-04-26T22:55:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.446+0000 7fe66e000640 1 -- 172.21.15.27:0/3393647160 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8815 (secure 0 0 0) 0x7fe66409cc10 con 0x7fe670108d90 2024-04-26T22:55:13.447 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:13.447 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/1620677319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1620677319},{"type":"v1","addr":"172.21.15.94:6839","nonce":1620677319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3193530521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3193530521},{"type":"v1","addr":"172.21.15.27:6839","nonce":3193530521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-26T22:55:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 11 2024-04-26T22:55:13.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.449+0000 7fe677000640 1 -- 172.21.15.27:0/3393647160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6580a3cd0 msgr2=0x7fe6580a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:13.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.449+0000 7fe677000640 1 --2- 172.21.15.27:0/3393647160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6580a3cd0 0x7fe6580a6190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe6701a0370 tx=0x7fe65c005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:13.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.449+0000 7fe677000640 1 -- 172.21.15.27:0/3393647160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe670108d90 msgr2=0x7fe67019ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:13.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.449+0000 7fe677000640 1 --2- 172.21.15.27:0/3393647160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe670108d90 0x7fe67019ee60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe66400e9c0 tx=0x7fe66400ee80 comp rx=0 tx=0).stop 2024-04-26T22:55:13.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.449+0000 7fe677000640 1 -- 172.21.15.27:0/3393647160 shutdown_connections 2024-04-26T22:55:13.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.449+0000 7fe677000640 1 --2- 172.21.15.27:0/3393647160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6580a3cd0 0x7fe6580a6190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:13.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.449+0000 7fe677000640 1 --2- 172.21.15.27:0/3393647160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe67010a940 0x7fe6701a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:13.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.449+0000 7fe677000640 1 --2- 172.21.15.27:0/3393647160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe670109f80 0x7fe67019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:13.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.449+0000 7fe677000640 1 --2- 172.21.15.27:0/3393647160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe670108d90 0x7fe67019ee60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:13.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.449+0000 7fe677000640 1 -- 172.21.15.27:0/3393647160 >> 172.21.15.27:0/3393647160 conn(0x7fe670077d30 msgr2=0x7fe67010d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:13.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.450+0000 7fe677000640 1 -- 172.21.15.27:0/3393647160 shutdown_connections 2024-04-26T22:55:13.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:13.450+0000 7fe677000640 1 -- 172.21.15.27:0/3393647160 wait complete. 2024-04-26T22:55:14.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:13 smithi027 ceph-mon[28034]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:14.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:13 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3393647160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:14.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:13 smithi080 ceph-mon[32324]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:14.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:13 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3393647160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:14.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:13 smithi094 ceph-mon[32889]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:14.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:13 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3393647160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:15.264 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 22:55:14 smithi027 systemd[1]: Starting Ceph mds.a for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T22:55:15.470 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:55:15.543 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 22:55:15 smithi027 podman[69261]: 2024-04-26 22:55:15.168586054 +0000 UTC m=+0.036188056 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:55:15.543 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 22:55:15 smithi027 podman[69261]: 2024-04-26 22:55:15.287912766 +0000 UTC m=+0.155514813 container create 3666b592592601eb68fa40700dfd31e75910f064767c479d8ee0aa014c996b97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:55:15.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.687+0000 7f2c94400640 1 -- 172.21.15.27:0/385674783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c8c103b60 msgr2=0x7f2c8c103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:15.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.687+0000 7f2c94400640 1 --2- 172.21.15.27:0/385674783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c8c103b60 0x7f2c8c103fe0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009860 tx=0x7f2c7802a340 comp rx=0 tx=0).stop 2024-04-26T22:55:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.687+0000 7f2c94400640 1 -- 172.21.15.27:0/385674783 shutdown_connections 2024-04-26T22:55:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.687+0000 7f2c94400640 1 --2- 172.21.15.27:0/385674783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c8c104520 0x7f2c8c10adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.687+0000 7f2c94400640 1 --2- 172.21.15.27:0/385674783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c8c103b60 0x7f2c8c103fe0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.687+0000 7f2c94400640 1 --2- 172.21.15.27:0/385674783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c8c102970 0x7f2c8c102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.687+0000 7f2c94400640 1 -- 172.21.15.27:0/385674783 >> 172.21.15.27:0/385674783 conn(0x7f2c8c0fdf00 msgr2=0x7f2c8c100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.687+0000 7f2c94400640 1 -- 172.21.15.27:0/385674783 shutdown_connections 2024-04-26T22:55:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.687+0000 7f2c94400640 1 -- 172.21.15.27:0/385674783 wait complete. 2024-04-26T22:55:15.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.688+0000 7f2c94400640 1 Processor -- start 2024-04-26T22:55:15.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.688+0000 7f2c94400640 1 -- start start 2024-04-26T22:55:15.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.688+0000 7f2c94400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c8c102970 0x7f2c8c0766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:15.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.688+0000 7f2c94400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c8c103b60 0x7f2c8c074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:15.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.688+0000 7f2c94400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c8c104520 0x7f2c8c075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:15.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.688+0000 7f2c94400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c10e4a0 con 0x7f2c8c102970 2024-04-26T22:55:15.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.688+0000 7f2c94400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c10e320 con 0x7f2c8c103b60 2024-04-26T22:55:15.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.689+0000 7f2c94400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c10e620 con 0x7f2c8c104520 2024-04-26T22:55:15.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.689+0000 7f2c91a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c8c102970 0x7f2c8c0766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:15.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.689+0000 7f2c91a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c8c102970 0x7f2c8c0766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41874/0 (socket says 172.21.15.27:41874) 2024-04-26T22:55:15.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.689+0000 7f2c91a00640 1 -- 172.21.15.27:0/3969291396 learned_addr learned my addr 172.21.15.27:0/3969291396 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:15.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.689+0000 7f2c91000640 1 --2- 172.21.15.27:0/3969291396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c8c103b60 0x7f2c8c074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:15.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.689+0000 7f2c92400640 1 --2- 172.21.15.27:0/3969291396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c8c104520 0x7f2c8c075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.689+0000 7f2c91a00640 1 -- 172.21.15.27:0/3969291396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c8c104520 msgr2=0x7f2c8c075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.689+0000 7f2c91a00640 1 --2- 172.21.15.27:0/3969291396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c8c104520 0x7f2c8c075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.689+0000 7f2c91a00640 1 -- 172.21.15.27:0/3969291396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c8c103b60 msgr2=0x7f2c8c074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.689+0000 7f2c91a00640 1 --2- 172.21.15.27:0/3969291396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c8c103b60 0x7f2c8c074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.689+0000 7f2c91a00640 1 -- 172.21.15.27:0/3969291396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8c075af0 con 0x7f2c8c102970 2024-04-26T22:55:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.689+0000 7f2c91a00640 1 --2- 172.21.15.27:0/3969291396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c8c102970 0x7f2c8c0766e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000e9c0 tx=0x7f2c8000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.690+0000 7f2c8a000640 1 -- 172.21.15.27:0/3969291396 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8000cd80 con 0x7f2c8c102970 2024-04-26T22:55:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.690+0000 7f2c8a000640 1 -- 172.21.15.27:0/3969291396 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c800035f0 con 0x7f2c8c102970 2024-04-26T22:55:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.690+0000 7f2c8a000640 1 -- 172.21.15.27:0/3969291396 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80010690 con 0x7f2c8c102970 2024-04-26T22:55:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.690+0000 7f2c94400640 1 -- 172.21.15.27:0/3969291396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8c1a2cf0 con 0x7f2c8c102970 2024-04-26T22:55:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.691+0000 7f2c94400640 1 -- 172.21.15.27:0/3969291396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c8c1a31c0 con 0x7f2c8c102970 2024-04-26T22:55:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.692+0000 7f2c94400640 1 -- 172.21.15.27:0/3969291396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c8c069c30 con 0x7f2c8c102970 2024-04-26T22:55:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.694+0000 7f2c8a000640 1 -- 172.21.15.27:0/3969291396 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2c80004000 con 0x7f2c8c102970 2024-04-26T22:55:15.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.695+0000 7f2c8a000640 1 --2- 172.21.15.27:0/3969291396 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2c740a3bb0 0x7f2c740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:15.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.696+0000 7f2c8a000640 1 -- 172.21.15.27:0/3969291396 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2c800d0740 con 0x7f2c8c102970 2024-04-26T22:55:15.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.696+0000 7f2c91000640 1 --2- 172.21.15.27:0/3969291396 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2c740a3bb0 0x7f2c740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.697+0000 7f2c91000640 1 --2- 172.21.15.27:0/3969291396 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2c740a3bb0 0x7f2c740a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2c7802a930 tx=0x7f2c78030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.700+0000 7f2c8a000640 1 -- 172.21.15.27:0/3969291396 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2c8001d060 con 0x7f2c8c102970 2024-04-26T22:55:15.865 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 22:55:15 smithi027 podman[69261]: 2024-04-26 22:55:15.619641949 +0000 UTC m=+0.487243983 container init 3666b592592601eb68fa40700dfd31e75910f064767c479d8ee0aa014c996b97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:55:15.866 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 22:55:15 smithi027 podman[69261]: 2024-04-26 22:55:15.62939253 +0000 UTC m=+0.496994540 container start 3666b592592601eb68fa40700dfd31e75910f064767c479d8ee0aa014c996b97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-26T22:55:15.866 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 22:55:15 smithi027 ceph-mds[69301]: -- 172.21.15.27:0/2917174606 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555ddef01a40 con 0x555ddeed5000 2024-04-26T22:55:15.866 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 22:55:15 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-a[69277]: starting mds.a at 2024-04-26T22:55:15.866 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 22:55:15 smithi027 bash[69261]: 3666b592592601eb68fa40700dfd31e75910f064767c479d8ee0aa014c996b97 2024-04-26T22:55:15.866 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 22:55:15 smithi027 systemd[1]: Started Ceph mds.a for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:55:15.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.959+0000 7f2c94400640 1 -- 172.21.15.27:0/3969291396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c8c063530 con 0x7f2c8c102970 2024-04-26T22:55:15.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.961+0000 7f2c8a000640 1 -- 172.21.15.27:0/3969291396 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9641 (secure 0 0 0) 0x7f2c8009d520 con 0x7f2c8c102970 2024-04-26T22:55:15.961 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:15.961 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/1620677319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1620677319},{"type":"v1","addr":"172.21.15.94:6839","nonce":1620677319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1571355196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1571355196},{"type":"v1","addr":"172.21.15.27:6841","nonce":1571355196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3193530521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3193530521},{"type":"v1","addr":"172.21.15.27:6839","nonce":3193530521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-26T22:55:15.962 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 12 2024-04-26T22:55:15.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.964+0000 7f2c94400640 1 -- 172.21.15.27:0/3969291396 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2c740a3bb0 msgr2=0x7f2c740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:15.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.964+0000 7f2c94400640 1 --2- 172.21.15.27:0/3969291396 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2c740a3bb0 0x7f2c740a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2c7802a930 tx=0x7f2c78030450 comp rx=0 tx=0).stop 2024-04-26T22:55:15.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.964+0000 7f2c94400640 1 -- 172.21.15.27:0/3969291396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c8c102970 msgr2=0x7f2c8c0766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.964+0000 7f2c94400640 1 --2- 172.21.15.27:0/3969291396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c8c102970 0x7f2c8c0766e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000e9c0 tx=0x7f2c8000ee80 comp rx=0 tx=0).stop 2024-04-26T22:55:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.964+0000 7f2c94400640 1 -- 172.21.15.27:0/3969291396 shutdown_connections 2024-04-26T22:55:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.964+0000 7f2c94400640 1 --2- 172.21.15.27:0/3969291396 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2c740a3bb0 0x7f2c740a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.964+0000 7f2c94400640 1 --2- 172.21.15.27:0/3969291396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c8c103b60 0x7f2c8c074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.964+0000 7f2c94400640 1 --2- 172.21.15.27:0/3969291396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c8c104520 0x7f2c8c075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.964+0000 7f2c94400640 1 --2- 172.21.15.27:0/3969291396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c8c102970 0x7f2c8c0766e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.964+0000 7f2c94400640 1 -- 172.21.15.27:0/3969291396 >> 172.21.15.27:0/3969291396 conn(0x7f2c8c0fdf00 msgr2=0x7f2c8c106e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.964+0000 7f2c94400640 1 -- 172.21.15.27:0/3969291396 shutdown_connections 2024-04-26T22:55:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:15.964+0000 7f2c94400640 1 -- 172.21.15.27:0/3969291396 wait complete. 2024-04-26T22:55:16.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:15 smithi080 ceph-mon[32324]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:16.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:15 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:16.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:15 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:16.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:15 smithi094 ceph-mon[32889]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:16.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:15 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:16.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:15 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:16.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:15 smithi027 ceph-mon[28034]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:16.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:15 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:16.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:15 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:17.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:16 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] up:boot 2024-04-26T22:55:17.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:16 smithi080 ceph-mon[32324]: fsmap 11 up:standby 2024-04-26T22:55:17.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:16 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T22:55:17.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:16 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:17.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:16 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:55:17.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:16 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:55:17.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:16 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:17.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:16 smithi080 ceph-mon[32324]: Deploying daemon mds.b on smithi080 2024-04-26T22:55:17.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:16 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3969291396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:17.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:16 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] up:boot 2024-04-26T22:55:17.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:16 smithi094 ceph-mon[32889]: fsmap 11 up:standby 2024-04-26T22:55:17.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T22:55:17.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:17.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:55:17.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:55:17.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:16 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:17.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:16 smithi094 ceph-mon[32889]: Deploying daemon mds.b on smithi080 2024-04-26T22:55:17.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:16 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3969291396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:17.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:16 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] up:boot 2024-04-26T22:55:17.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:16 smithi027 ceph-mon[28034]: fsmap 11 up:standby 2024-04-26T22:55:17.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T22:55:17.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:17.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T22:55:17.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T22:55:17.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:16 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:17.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:16 smithi027 ceph-mon[28034]: Deploying daemon mds.b on smithi080 2024-04-26T22:55:17.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:16 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3969291396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:17.986 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:55:18.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.213+0000 7fc84cc00640 1 -- 172.21.15.27:0/2810085853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848103220 msgr2=0x7fc848109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:18.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.213+0000 7fc84cc00640 1 --2- 172.21.15.27:0/2810085853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848103220 0x7fc848109ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc840009be0 tx=0x7fc84002a6c0 comp rx=0 tx=0).stop 2024-04-26T22:55:18.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.213+0000 7fc84cc00640 1 -- 172.21.15.27:0/2810085853 shutdown_connections 2024-04-26T22:55:18.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.213+0000 7fc84cc00640 1 --2- 172.21.15.27:0/2810085853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848103220 0x7fc848109ab0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:18.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.213+0000 7fc84cc00640 1 --2- 172.21.15.27:0/2810085853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc848102860 0x7fc848102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:18.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.213+0000 7fc84cc00640 1 --2- 172.21.15.27:0/2810085853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc848101670 0x7fc848101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:18.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.213+0000 7fc84cc00640 1 -- 172.21.15.27:0/2810085853 >> 172.21.15.27:0/2810085853 conn(0x7fc8480fcc00 msgr2=0x7fc8480ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:18.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.213+0000 7fc84cc00640 1 -- 172.21.15.27:0/2810085853 shutdown_connections 2024-04-26T22:55:18.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.213+0000 7fc84cc00640 1 -- 172.21.15.27:0/2810085853 wait complete. 2024-04-26T22:55:18.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.214+0000 7fc84cc00640 1 Processor -- start 2024-04-26T22:55:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.215+0000 7fc84cc00640 1 -- start start 2024-04-26T22:55:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.215+0000 7fc84cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc848101670 0x7fc848198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.215+0000 7fc84cc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848102860 0x7fc848198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.215+0000 7fc84cc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc848103220 0x7fc84819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.215+0000 7fc84cc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc84810d1a0 con 0x7fc848103220 2024-04-26T22:55:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.215+0000 7fc84cc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc84810d020 con 0x7fc848102860 2024-04-26T22:55:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.215+0000 7fc84cc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc84810d320 con 0x7fc848101670 2024-04-26T22:55:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.215+0000 7fc847e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc848103220 0x7fc84819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.215+0000 7fc846a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848102860 0x7fc848198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.215+0000 7fc847e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc848103220 0x7fc84819fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37540/0 (socket says 172.21.15.27:37540) 2024-04-26T22:55:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.215+0000 7fc846a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848102860 0x7fc848198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:52026/0 (socket says 172.21.15.27:52026) 2024-04-26T22:55:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.215+0000 7fc847e00640 1 -- 172.21.15.27:0/3570750155 learned_addr learned my addr 172.21.15.27:0/3570750155 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.215+0000 7fc847400640 1 --2- 172.21.15.27:0/3570750155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc848101670 0x7fc848198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.216+0000 7fc847e00640 1 -- 172.21.15.27:0/3570750155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc848101670 msgr2=0x7fc848198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.216+0000 7fc847e00640 1 --2- 172.21.15.27:0/3570750155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc848101670 0x7fc848198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.216+0000 7fc847e00640 1 -- 172.21.15.27:0/3570750155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848102860 msgr2=0x7fc848198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.216+0000 7fc847e00640 1 --2- 172.21.15.27:0/3570750155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848102860 0x7fc848198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.216+0000 7fc847e00640 1 -- 172.21.15.27:0/3570750155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8481a0470 con 0x7fc848103220 2024-04-26T22:55:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.216+0000 7fc846a00640 1 --2- 172.21.15.27:0/3570750155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848102860 0x7fc848198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.216+0000 7fc847400640 1 --2- 172.21.15.27:0/3570750155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc848101670 0x7fc848198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.216+0000 7fc847e00640 1 --2- 172.21.15.27:0/3570750155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc848103220 0x7fc84819fe00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc840004af0 tx=0x7fc840030d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.216+0000 7fc83fe00640 1 -- 172.21.15.27:0/3570750155 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840041020 con 0x7fc848103220 2024-04-26T22:55:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.216+0000 7fc84cc00640 1 -- 172.21.15.27:0/3570750155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8481a0700 con 0x7fc848103220 2024-04-26T22:55:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.216+0000 7fc83fe00640 1 -- 172.21.15.27:0/3570750155 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc840030ec0 con 0x7fc848103220 2024-04-26T22:55:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.216+0000 7fc83fe00640 1 -- 172.21.15.27:0/3570750155 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840036040 con 0x7fc848103220 2024-04-26T22:55:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.216+0000 7fc84cc00640 1 -- 172.21.15.27:0/3570750155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8481a0c40 con 0x7fc848103220 2024-04-26T22:55:18.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.217+0000 7fc84cc00640 1 -- 172.21.15.27:0/3570750155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8481a0f60 con 0x7fc848103220 2024-04-26T22:55:18.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.220+0000 7fc83fe00640 1 -- 172.21.15.27:0/3570750155 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc840044020 con 0x7fc848103220 2024-04-26T22:55:18.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.221+0000 7fc83fe00640 1 --2- 172.21.15.27:0/3570750155 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc82c0a3c50 0x7fc82c0a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:18.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.221+0000 7fc83fe00640 1 -- 172.21.15.27:0/3570750155 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc8400f0920 con 0x7fc848103220 2024-04-26T22:55:18.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.221+0000 7fc847400640 1 --2- 172.21.15.27:0/3570750155 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc82c0a3c50 0x7fc82c0a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:18.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.222+0000 7fc847400640 1 --2- 172.21.15.27:0/3570750155 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc82c0a3c50 0x7fc82c0a6110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc8481026c0 tx=0x7fc838009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.226+0000 7fc83fe00640 1 -- 172.21.15.27:0/3570750155 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc84004b030 con 0x7fc848103220 2024-04-26T22:55:18.234 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:17 smithi080 ceph-mon[32324]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:18.234 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 22:55:18 smithi080 systemd[1]: Starting Ceph mds.b for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T22:55:18.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:17 smithi094 ceph-mon[32889]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:18.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:17 smithi027 ceph-mon[28034]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:18.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.470+0000 7fc84cc00640 1 -- 172.21.15.27:0/3570750155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8481a1130 con 0x7fc848103220 2024-04-26T22:55:18.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.471+0000 7fc83fe00640 1 -- 172.21.15.27:0/3570750155 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9641 (secure 0 0 0) 0x7fc8400bd700 con 0x7fc848103220 2024-04-26T22:55:18.472 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:18.472 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/1620677319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1620677319},{"type":"v1","addr":"172.21.15.94:6839","nonce":1620677319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1571355196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1571355196},{"type":"v1","addr":"172.21.15.27:6841","nonce":1571355196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3193530521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3193530521},{"type":"v1","addr":"172.21.15.27:6839","nonce":3193530521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-26T22:55:18.472 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 12 2024-04-26T22:55:18.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.474+0000 7fc84cc00640 1 -- 172.21.15.27:0/3570750155 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc82c0a3c50 msgr2=0x7fc82c0a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:18.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.474+0000 7fc84cc00640 1 --2- 172.21.15.27:0/3570750155 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc82c0a3c50 0x7fc82c0a6110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc8481026c0 tx=0x7fc838009260 comp rx=0 tx=0).stop 2024-04-26T22:55:18.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.474+0000 7fc84cc00640 1 -- 172.21.15.27:0/3570750155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc848103220 msgr2=0x7fc84819fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:18.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.474+0000 7fc84cc00640 1 --2- 172.21.15.27:0/3570750155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc848103220 0x7fc84819fe00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc840004af0 tx=0x7fc840030d50 comp rx=0 tx=0).stop 2024-04-26T22:55:18.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.474+0000 7fc84cc00640 1 -- 172.21.15.27:0/3570750155 shutdown_connections 2024-04-26T22:55:18.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.474+0000 7fc84cc00640 1 --2- 172.21.15.27:0/3570750155 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc82c0a3c50 0x7fc82c0a6110 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:18.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.474+0000 7fc84cc00640 1 --2- 172.21.15.27:0/3570750155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848102860 0x7fc848198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:18.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.474+0000 7fc84cc00640 1 --2- 172.21.15.27:0/3570750155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc848103220 0x7fc84819fe00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:18.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.474+0000 7fc84cc00640 1 --2- 172.21.15.27:0/3570750155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc848101670 0x7fc848198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:18.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.474+0000 7fc84cc00640 1 -- 172.21.15.27:0/3570750155 >> 172.21.15.27:0/3570750155 conn(0x7fc8480fcc00 msgr2=0x7fc848069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:18.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.474+0000 7fc84cc00640 1 -- 172.21.15.27:0/3570750155 shutdown_connections 2024-04-26T22:55:18.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:18.474+0000 7fc84cc00640 1 -- 172.21.15.27:0/3570750155 wait complete. 2024-04-26T22:55:18.495 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 22:55:18 smithi080 podman[63006]: 2024-04-26 22:55:18.137526107 +0000 UTC m=+0.018075185 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:55:18.495 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 22:55:18 smithi080 podman[63006]: 2024-04-26 22:55:18.260242037 +0000 UTC m=+0.140791101 container create 4f14a8823937eb46ddc14daae622a3492431a000e1d61ff5fab8e6ef37155d16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T22:55:18.495 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 22:55:18 smithi080 podman[63006]: 2024-04-26 22:55:18.496413875 +0000 UTC m=+0.376962939 container init 4f14a8823937eb46ddc14daae622a3492431a000e1d61ff5fab8e6ef37155d16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-26T22:55:18.771 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 22:55:18 smithi080 podman[63006]: 2024-04-26 22:55:18.498971858 +0000 UTC m=+0.379520933 container start 4f14a8823937eb46ddc14daae622a3492431a000e1d61ff5fab8e6ef37155d16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-26T22:55:18.771 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 22:55:18 smithi080 ceph-mds[63025]: -- 172.21.15.80:0/3053268928 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5612e63eba40 con 0x5612e63be800 2024-04-26T22:55:18.771 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 22:55:18 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-b[63021]: starting mds.b at 2024-04-26T22:55:18.771 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 22:55:18 smithi080 bash[63006]: 4f14a8823937eb46ddc14daae622a3492431a000e1d61ff5fab8e6ef37155d16 2024-04-26T22:55:18.771 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 22:55:18 smithi080 systemd[1]: Started Ceph mds.b for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T22:55:19.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:18 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3570750155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:19.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:19.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:19.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:19.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:19.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:18 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:19.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:18 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3570750155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:19.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:18 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:19.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:18 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:19.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:18 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:19.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:18 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:19.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:18 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:19.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:18 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3570750155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:19.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:18 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:19.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:18 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:19.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:18 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:19.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:18 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:19.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:18 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:20.099 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:19 smithi080 ceph-mon[32324]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:20.099 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:19 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] up:boot 2024-04-26T22:55:20.099 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:19 smithi080 ceph-mon[32324]: fsmap 12 up:standby 2024-04-26T22:55:20.100 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:19 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T22:55:20.100 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:19 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:55:20.100 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:19 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:20.112 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:19 smithi094 ceph-mon[32889]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:20.112 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:19 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] up:boot 2024-04-26T22:55:20.112 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:19 smithi094 ceph-mon[32889]: fsmap 12 up:standby 2024-04-26T22:55:20.112 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:19 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T22:55:20.112 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:19 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:55:20.113 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:19 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:20.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:19 smithi027 ceph-mon[28034]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:20.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:19 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] up:boot 2024-04-26T22:55:20.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:19 smithi027 ceph-mon[28034]: fsmap 12 up:standby 2024-04-26T22:55:20.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:19 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T22:55:20.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:19 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:55:20.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:19 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:20.494 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:55:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.703+0000 7f1132000640 1 -- 172.21.15.27:0/1272442922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f112c1049a0 msgr2=0x7f112c104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.703+0000 7f1132000640 1 --2- 172.21.15.27:0/1272442922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f112c1049a0 0x7f112c104dc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1120009900 tx=0x7f112002a410 comp rx=0 tx=0).stop 2024-04-26T22:55:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.703+0000 7f1132000640 1 -- 172.21.15.27:0/1272442922 shutdown_connections 2024-04-26T22:55:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.703+0000 7f1132000640 1 --2- 172.21.15.27:0/1272442922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f112c106550 0x7f112c10cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.703+0000 7f1132000640 1 --2- 172.21.15.27:0/1272442922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f112c105b90 0x7f112c106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.703+0000 7f1132000640 1 --2- 172.21.15.27:0/1272442922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f112c1049a0 0x7f112c104dc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.703+0000 7f1132000640 1 -- 172.21.15.27:0/1272442922 >> 172.21.15.27:0/1272442922 conn(0x7f112c0fff10 msgr2=0x7f112c102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.703+0000 7f1132000640 1 -- 172.21.15.27:0/1272442922 shutdown_connections 2024-04-26T22:55:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.703+0000 7f1132000640 1 -- 172.21.15.27:0/1272442922 wait complete. 2024-04-26T22:55:20.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.704+0000 7f1132000640 1 Processor -- start 2024-04-26T22:55:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.704+0000 7f1132000640 1 -- start start 2024-04-26T22:55:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f1132000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f112c1049a0 0x7f112c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f1132000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f112c105b90 0x7f112c19aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f1132000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f112c106550 0x7f112c1a2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f1132000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f112c1104f0 con 0x7f112c106550 2024-04-26T22:55:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f1132000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f112c110370 con 0x7f112c105b90 2024-04-26T22:55:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f1132000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f112c110670 con 0x7f112c1049a0 2024-04-26T22:55:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f1130c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f112c1049a0 0x7f112c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f1131600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f112c106550 0x7f112c1a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f112be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f112c105b90 0x7f112c19aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f1130c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f112c1049a0 0x7f112c19aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:41674/0 (socket says 172.21.15.27:41674) 2024-04-26T22:55:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f1130c00640 1 -- 172.21.15.27:0/3786791662 learned_addr learned my addr 172.21.15.27:0/3786791662 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f112be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f112c105b90 0x7f112c19aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:52044/0 (socket says 172.21.15.27:52044) 2024-04-26T22:55:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f1130c00640 1 -- 172.21.15.27:0/3786791662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f112c105b90 msgr2=0x7f112c19aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f1130c00640 1 --2- 172.21.15.27:0/3786791662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f112c105b90 0x7f112c19aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f1130c00640 1 -- 172.21.15.27:0/3786791662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f112c106550 msgr2=0x7f112c1a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f1130c00640 1 --2- 172.21.15.27:0/3786791662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f112c106550 0x7f112c1a2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.705+0000 7f1130c00640 1 -- 172.21.15.27:0/3786791662 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112c1a26d0 con 0x7f112c1049a0 2024-04-26T22:55:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.706+0000 7f1130c00640 1 --2- 172.21.15.27:0/3786791662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f112c1049a0 0x7f112c19aab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1120031500 tx=0x7f1120003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.706+0000 7f1129600640 1 -- 172.21.15.27:0/3786791662 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112003f070 con 0x7f112c1049a0 2024-04-26T22:55:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.706+0000 7f1129600640 1 -- 172.21.15.27:0/3786791662 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1120031c50 con 0x7f112c1049a0 2024-04-26T22:55:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.706+0000 7f1132000640 1 -- 172.21.15.27:0/3786791662 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112c1a2960 con 0x7f112c1049a0 2024-04-26T22:55:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.706+0000 7f1129600640 1 -- 172.21.15.27:0/3786791662 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120035030 con 0x7f112c1049a0 2024-04-26T22:55:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.706+0000 7f1132000640 1 -- 172.21.15.27:0/3786791662 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f112c1a2e50 con 0x7f112c1049a0 2024-04-26T22:55:20.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.707+0000 7f1132000640 1 -- 172.21.15.27:0/3786791662 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f112c10bd40 con 0x7f112c1049a0 2024-04-26T22:55:20.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.709+0000 7f1129600640 1 -- 172.21.15.27:0/3786791662 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1120042070 con 0x7f112c1049a0 2024-04-26T22:55:20.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.710+0000 7f1129600640 1 --2- 172.21.15.27:0/3786791662 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f11140a3c50 0x7f11140a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:20.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.711+0000 7f1129600640 1 -- 172.21.15.27:0/3786791662 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f11200f06d0 con 0x7f112c1049a0 2024-04-26T22:55:20.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.711+0000 7f112be00640 1 --2- 172.21.15.27:0/3786791662 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f11140a3c50 0x7f11140a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:20.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.711+0000 7f112be00640 1 --2- 172.21.15.27:0/3786791662 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f11140a3c50 0x7f11140a6110 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f11180095f0 tx=0x7f1118005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:20.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.715+0000 7f1129600640 1 -- 172.21.15.27:0/3786791662 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f11200bd4b0 con 0x7f112c1049a0 2024-04-26T22:55:20.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.967+0000 7f1132000640 1 -- 172.21.15.27:0/3786791662 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f112c02dad0 con 0x7f112c1049a0 2024-04-26T22:55:20.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.968+0000 7f1129600640 1 -- 172.21.15.27:0/3786791662 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10461 (secure 0 0 0) 0x7f11200bcdb0 con 0x7f112c1049a0 2024-04-26T22:55:20.969 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:20.969 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/1620677319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1620677319},{"type":"v1","addr":"172.21.15.94:6839","nonce":1620677319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1571355196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1571355196},{"type":"v1","addr":"172.21.15.27:6841","nonce":1571355196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3193530521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3193530521},{"type":"v1","addr":"172.21.15.27:6839","nonce":3193530521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24578,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/95952000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":95952000},{"type":"v1","addr":"172.21.15.80:6839","nonce":95952000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-26T22:55:20.969 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 13 2024-04-26T22:55:20.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.971+0000 7f1132000640 1 -- 172.21.15.27:0/3786791662 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f11140a3c50 msgr2=0x7f11140a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:20.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.971+0000 7f1132000640 1 --2- 172.21.15.27:0/3786791662 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f11140a3c50 0x7f11140a6110 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f11180095f0 tx=0x7f1118005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:20.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.971+0000 7f1132000640 1 -- 172.21.15.27:0/3786791662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f112c1049a0 msgr2=0x7f112c19aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.971+0000 7f1132000640 1 --2- 172.21.15.27:0/3786791662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f112c1049a0 0x7f112c19aab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1120031500 tx=0x7f1120003c10 comp rx=0 tx=0).stop 2024-04-26T22:55:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.971+0000 7f1132000640 1 -- 172.21.15.27:0/3786791662 shutdown_connections 2024-04-26T22:55:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.971+0000 7f1132000640 1 --2- 172.21.15.27:0/3786791662 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f11140a3c50 0x7f11140a6110 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.971+0000 7f1132000640 1 --2- 172.21.15.27:0/3786791662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f112c105b90 0x7f112c19aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.971+0000 7f1132000640 1 --2- 172.21.15.27:0/3786791662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f112c106550 0x7f112c1a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.971+0000 7f1132000640 1 --2- 172.21.15.27:0/3786791662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f112c1049a0 0x7f112c19aab0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.971+0000 7f1132000640 1 -- 172.21.15.27:0/3786791662 >> 172.21.15.27:0/3786791662 conn(0x7f112c0fff10 msgr2=0x7f112c108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.971+0000 7f1132000640 1 -- 172.21.15.27:0/3786791662 shutdown_connections 2024-04-26T22:55:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:20.971+0000 7f1132000640 1 -- 172.21.15.27:0/3786791662 wait complete. 2024-04-26T22:55:20.992 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:55:21.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.208+0000 7f8bd0400640 1 -- 172.21.15.27:0/553779156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc8102980 msgr2=0x7f8bc8102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:21.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.208+0000 7f8bd0400640 1 --2- 172.21.15.27:0/553779156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc8102980 0x7f8bc8102da0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009900 tx=0x7f8bbc02a410 comp rx=0 tx=0).stop 2024-04-26T22:55:21.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.208+0000 7f8bd0400640 1 -- 172.21.15.27:0/553779156 shutdown_connections 2024-04-26T22:55:21.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.208+0000 7f8bd0400640 1 --2- 172.21.15.27:0/553779156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bc8104530 0x7f8bc810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.208+0000 7f8bd0400640 1 --2- 172.21.15.27:0/553779156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc8103b70 0x7f8bc8103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.208+0000 7f8bd0400640 1 --2- 172.21.15.27:0/553779156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc8102980 0x7f8bc8102da0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.208+0000 7f8bd0400640 1 -- 172.21.15.27:0/553779156 >> 172.21.15.27:0/553779156 conn(0x7f8bc80fdf50 msgr2=0x7f8bc8100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:21.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.209+0000 7f8bd0400640 1 -- 172.21.15.27:0/553779156 shutdown_connections 2024-04-26T22:55:21.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.209+0000 7f8bd0400640 1 -- 172.21.15.27:0/553779156 wait complete. 2024-04-26T22:55:21.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.210+0000 7f8bd0400640 1 Processor -- start 2024-04-26T22:55:21.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.210+0000 7f8bd0400640 1 -- start start 2024-04-26T22:55:21.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.210+0000 7f8bd0400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc8102980 0x7f8bc819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:21.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.210+0000 7f8bd0400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bc8103b70 0x7f8bc819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:21.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.210+0000 7f8bd0400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc8104530 0x7f8bc81a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:21.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.210+0000 7f8bd0400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc810e500 con 0x7f8bc8104530 2024-04-26T22:55:21.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.210+0000 7f8bd0400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc810e380 con 0x7f8bc8103b70 2024-04-26T22:55:21.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.210+0000 7f8bd0400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc810e680 con 0x7f8bc8102980 2024-04-26T22:55:21.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.211+0000 7f8bcda00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc8102980 0x7f8bc819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:21.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.211+0000 7f8bcda00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc8102980 0x7f8bc819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:41698/0 (socket says 172.21.15.27:41698) 2024-04-26T22:55:21.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.211+0000 7f8bcda00640 1 -- 172.21.15.27:0/3948510001 learned_addr learned my addr 172.21.15.27:0/3948510001 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:21.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.211+0000 7f8bce400640 1 --2- 172.21.15.27:0/3948510001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc8104530 0x7f8bc81a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:21.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.211+0000 7f8bcd000640 1 --2- 172.21.15.27:0/3948510001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bc8103b70 0x7f8bc819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:21.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.211+0000 7f8bcda00640 1 -- 172.21.15.27:0/3948510001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bc8103b70 msgr2=0x7f8bc819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:21.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.211+0000 7f8bcda00640 1 --2- 172.21.15.27:0/3948510001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bc8103b70 0x7f8bc819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.211+0000 7f8bcda00640 1 -- 172.21.15.27:0/3948510001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc8104530 msgr2=0x7f8bc81a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:21.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.211+0000 7f8bcda00640 1 --2- 172.21.15.27:0/3948510001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc8104530 0x7f8bc81a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.211+0000 7f8bcda00640 1 -- 172.21.15.27:0/3948510001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc81a29b0 con 0x7f8bc8102980 2024-04-26T22:55:21.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.211+0000 7f8bce400640 1 --2- 172.21.15.27:0/3948510001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc8104530 0x7f8bc81a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:55:21.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.211+0000 7f8bcda00640 1 --2- 172.21.15.27:0/3948510001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc8102980 0x7f8bc819ada0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc031500 tx=0x7f8bbc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:21.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.212+0000 7f8bc6000640 1 -- 172.21.15.27:0/3948510001 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc03f070 con 0x7f8bc8102980 2024-04-26T22:55:21.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.212+0000 7f8bc6000640 1 -- 172.21.15.27:0/3948510001 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bbc031c50 con 0x7f8bc8102980 2024-04-26T22:55:21.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.212+0000 7f8bc6000640 1 -- 172.21.15.27:0/3948510001 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc035030 con 0x7f8bc8102980 2024-04-26T22:55:21.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.212+0000 7f8bd0400640 1 -- 172.21.15.27:0/3948510001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc81a2c40 con 0x7f8bc8102980 2024-04-26T22:55:21.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.212+0000 7f8bd0400640 1 -- 172.21.15.27:0/3948510001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc81a30e0 con 0x7f8bc8102980 2024-04-26T22:55:21.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.213+0000 7f8bd0400640 1 -- 172.21.15.27:0/3948510001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc81a3310 con 0x7f8bc8102980 2024-04-26T22:55:21.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.215+0000 7f8bc6000640 1 -- 172.21.15.27:0/3948510001 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8bbc042070 con 0x7f8bc8102980 2024-04-26T22:55:21.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.216+0000 7f8bc6000640 1 --2- 172.21.15.27:0/3948510001 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8bb00a3bb0 0x7f8bb00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:21.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.217+0000 7f8bc6000640 1 -- 172.21.15.27:0/3948510001 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8bbc0f08a0 con 0x7f8bc8102980 2024-04-26T22:55:21.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.217+0000 7f8bcd000640 1 --2- 172.21.15.27:0/3948510001 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8bb00a3bb0 0x7f8bb00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:21.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.217+0000 7f8bcd000640 1 --2- 172.21.15.27:0/3948510001 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8bb00a3bb0 0x7f8bb00a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4009bd0 tx=0x7f8bb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:21.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.220+0000 7f8bc6000640 1 -- 172.21.15.27:0/3948510001 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8bbc0bd680 con 0x7f8bc8102980 2024-04-26T22:55:21.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.492+0000 7f8bd0400640 1 -- 172.21.15.27:0/3948510001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8bc81a3b70 con 0x7f8bc8102980 2024-04-26T22:55:21.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.493+0000 7f8bc6000640 1 -- 172.21.15.27:0/3948510001 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18337 (secure 0 0 0) 0x7f8bbc0bcf80 con 0x7f8bc8102980 2024-04-26T22:55:21.494 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:21.494 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":81,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:54:23.783142+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:55:21.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.496+0000 7f8bd0400640 1 -- 172.21.15.27:0/3948510001 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8bb00a3bb0 msgr2=0x7f8bb00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:21.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.496+0000 7f8bd0400640 1 --2- 172.21.15.27:0/3948510001 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8bb00a3bb0 0x7f8bb00a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4009bd0 tx=0x7f8bb4005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:21.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.496+0000 7f8bd0400640 1 -- 172.21.15.27:0/3948510001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc8102980 msgr2=0x7f8bc819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:21.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.496+0000 7f8bd0400640 1 --2- 172.21.15.27:0/3948510001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc8102980 0x7f8bc819ada0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc031500 tx=0x7f8bbc003c10 comp rx=0 tx=0).stop 2024-04-26T22:55:21.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.496+0000 7f8bd0400640 1 -- 172.21.15.27:0/3948510001 shutdown_connections 2024-04-26T22:55:21.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.496+0000 7f8bd0400640 1 --2- 172.21.15.27:0/3948510001 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8bb00a3bb0 0x7f8bb00a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.496+0000 7f8bd0400640 1 --2- 172.21.15.27:0/3948510001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bc8103b70 0x7f8bc819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.496+0000 7f8bd0400640 1 --2- 172.21.15.27:0/3948510001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc8104530 0x7f8bc81a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.496+0000 7f8bd0400640 1 --2- 172.21.15.27:0/3948510001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc8102980 0x7f8bc819ada0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.496+0000 7f8bd0400640 1 -- 172.21.15.27:0/3948510001 >> 172.21.15.27:0/3948510001 conn(0x7f8bc80fdf50 msgr2=0x7f8bc8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:21.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.496+0000 7f8bd0400640 1 -- 172.21.15.27:0/3948510001 shutdown_connections 2024-04-26T22:55:21.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.496+0000 7f8bd0400640 1 -- 172.21.15.27:0/3948510001 wait complete. 2024-04-26T22:55:21.515 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:49:54.482679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:55:21.515 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:55:21.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.710+0000 7efd2f800640 1 -- 172.21.15.27:0/650932278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd28103b70 msgr2=0x7efd28103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:21.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.710+0000 7efd2f800640 1 --2- 172.21.15.27:0/650932278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd28103b70 0x7efd28103ff0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efd14009900 tx=0x7efd1402a3f0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.710+0000 7efd2f800640 1 -- 172.21.15.27:0/650932278 shutdown_connections 2024-04-26T22:55:21.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.710+0000 7efd2f800640 1 --2- 172.21.15.27:0/650932278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd28104530 0x7efd2810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.710+0000 7efd2f800640 1 --2- 172.21.15.27:0/650932278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd28103b70 0x7efd28103ff0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.710+0000 7efd2f800640 1 --2- 172.21.15.27:0/650932278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd28102980 0x7efd28102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.710+0000 7efd2f800640 1 -- 172.21.15.27:0/650932278 >> 172.21.15.27:0/650932278 conn(0x7efd280fdef0 msgr2=0x7efd28100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:21.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.710+0000 7efd2f800640 1 -- 172.21.15.27:0/650932278 shutdown_connections 2024-04-26T22:55:21.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.710+0000 7efd2f800640 1 -- 172.21.15.27:0/650932278 wait complete. 2024-04-26T22:55:21.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.711+0000 7efd2f800640 1 Processor -- start 2024-04-26T22:55:21.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.712+0000 7efd2f800640 1 -- start start 2024-04-26T22:55:21.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.712+0000 7efd2f800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd28102980 0x7efd2819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:21.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.712+0000 7efd2f800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd28103b70 0x7efd2819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:21.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.712+0000 7efd2f800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd28104530 0x7efd281a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:21.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.712+0000 7efd2f800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efd2810e500 con 0x7efd28102980 2024-04-26T22:55:21.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.712+0000 7efd2f800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efd2810e380 con 0x7efd28103b70 2024-04-26T22:55:21.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.712+0000 7efd2f800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd2810e680 con 0x7efd28104530 2024-04-26T22:55:21.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.712+0000 7efd2ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd28102980 0x7efd2819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:21.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.712+0000 7efd27e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd28103b70 0x7efd2819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:21.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.712+0000 7efd2ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd28102980 0x7efd2819adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37580/0 (socket says 172.21.15.27:37580) 2024-04-26T22:55:21.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.712+0000 7efd2ce00640 1 -- 172.21.15.27:0/2960351259 learned_addr learned my addr 172.21.15.27:0/2960351259 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:21.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.712+0000 7efd2d800640 1 --2- 172.21.15.27:0/2960351259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd28104530 0x7efd281a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:21.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.713+0000 7efd27e00640 1 -- 172.21.15.27:0/2960351259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd28104530 msgr2=0x7efd281a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:21.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.713+0000 7efd27e00640 1 --2- 172.21.15.27:0/2960351259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd28104530 0x7efd281a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.713+0000 7efd27e00640 1 -- 172.21.15.27:0/2960351259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd28102980 msgr2=0x7efd2819adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:21.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.713+0000 7efd27e00640 1 --2- 172.21.15.27:0/2960351259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd28102980 0x7efd2819adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.713+0000 7efd27e00640 1 -- 172.21.15.27:0/2960351259 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd281a29e0 con 0x7efd28103b70 2024-04-26T22:55:21.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.713+0000 7efd27e00640 1 --2- 172.21.15.27:0/2960351259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd28103b70 0x7efd2819b300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efd1402a9e0 tx=0x7efd14003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:21.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.714+0000 7efd25600640 1 -- 172.21.15.27:0/2960351259 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1403f070 con 0x7efd28103b70 2024-04-26T22:55:21.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.714+0000 7efd25600640 1 -- 172.21.15.27:0/2960351259 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd14031c50 con 0x7efd28103b70 2024-04-26T22:55:21.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.714+0000 7efd25600640 1 -- 172.21.15.27:0/2960351259 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14035030 con 0x7efd28103b70 2024-04-26T22:55:21.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.714+0000 7efd2f800640 1 -- 172.21.15.27:0/2960351259 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd281a2c70 con 0x7efd28103b70 2024-04-26T22:55:21.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.714+0000 7efd2f800640 1 -- 172.21.15.27:0/2960351259 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd281a3130 con 0x7efd28103b70 2024-04-26T22:55:21.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.715+0000 7efd2f800640 1 -- 172.21.15.27:0/2960351259 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd281a3300 con 0x7efd28103b70 2024-04-26T22:55:21.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.717+0000 7efd25600640 1 -- 172.21.15.27:0/2960351259 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efd14042070 con 0x7efd28103b70 2024-04-26T22:55:21.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.718+0000 7efd25600640 1 --2- 172.21.15.27:0/2960351259 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7efd100a3bb0 0x7efd100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:21.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.719+0000 7efd25600640 1 -- 172.21.15.27:0/2960351259 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7efd140f0cd0 con 0x7efd28103b70 2024-04-26T22:55:21.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.719+0000 7efd2ce00640 1 --2- 172.21.15.27:0/2960351259 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7efd100a3bb0 0x7efd100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:21.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.720+0000 7efd2ce00640 1 --2- 172.21.15.27:0/2960351259 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7efd100a3bb0 0x7efd100a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0095f0 tx=0x7efd1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:21.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.721+0000 7efd25600640 1 -- 172.21.15.27:0/2960351259 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efd140bd460 con 0x7efd28103b70 2024-04-26T22:55:21.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:21 smithi027 ceph-mon[28034]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:21.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:21.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:21.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:21.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:21.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:21 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3786791662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:21.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:21 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3948510001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:21.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.967+0000 7efd2f800640 1 -- 172.21.15.27:0/2960351259 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd281a3690 con 0x7efd28103b70 2024-04-26T22:55:21.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.968+0000 7efd25600640 1 -- 172.21.15.27:0/2960351259 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7efd140c1760 con 0x7efd28103b70 2024-04-26T22:55:21.968 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T22:55:21.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.971+0000 7efd2f800640 1 -- 172.21.15.27:0/2960351259 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7efd100a3bb0 msgr2=0x7efd100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:21.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.971+0000 7efd2f800640 1 --2- 172.21.15.27:0/2960351259 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7efd100a3bb0 0x7efd100a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0095f0 tx=0x7efd1c005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:21.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.971+0000 7efd2f800640 1 -- 172.21.15.27:0/2960351259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd28103b70 msgr2=0x7efd2819b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:21.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.971+0000 7efd2f800640 1 --2- 172.21.15.27:0/2960351259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd28103b70 0x7efd2819b300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efd1402a9e0 tx=0x7efd14003c10 comp rx=0 tx=0).stop 2024-04-26T22:55:21.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.971+0000 7efd2f800640 1 -- 172.21.15.27:0/2960351259 shutdown_connections 2024-04-26T22:55:21.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.971+0000 7efd2f800640 1 --2- 172.21.15.27:0/2960351259 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7efd100a3bb0 0x7efd100a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.971+0000 7efd2f800640 1 --2- 172.21.15.27:0/2960351259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd28103b70 0x7efd2819b300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.971+0000 7efd2f800640 1 --2- 172.21.15.27:0/2960351259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd28104530 0x7efd281a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.971+0000 7efd2f800640 1 --2- 172.21.15.27:0/2960351259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd28102980 0x7efd2819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:21.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.971+0000 7efd2f800640 1 -- 172.21.15.27:0/2960351259 >> 172.21.15.27:0/2960351259 conn(0x7efd280fdef0 msgr2=0x7efd28109310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:21.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.972+0000 7efd2f800640 1 -- 172.21.15.27:0/2960351259 shutdown_connections 2024-04-26T22:55:21.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:21.972+0000 7efd2f800640 1 -- 172.21.15.27:0/2960351259 wait complete. 2024-04-26T22:55:21.994 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T22:55:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:21 smithi080 ceph-mon[32324]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:21 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3786791662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:22.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:21 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3948510001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:22.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:21 smithi094 ceph-mon[32889]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:22.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:22.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:22.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:22.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:22.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:21 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3786791662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:22.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:21 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3948510001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:22.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.204+0000 7f240a000640 1 -- 172.21.15.27:0/2266443498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2404076970 msgr2=0x7f2404074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:22.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.204+0000 7f240a000640 1 --2- 172.21.15.27:0/2266443498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2404076970 0x7f2404074dc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009900 tx=0x7f23f802a400 comp rx=0 tx=0).stop 2024-04-26T22:55:22.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.204+0000 7f240a000640 1 -- 172.21.15.27:0/2266443498 shutdown_connections 2024-04-26T22:55:22.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.204+0000 7f240a000640 1 --2- 172.21.15.27:0/2266443498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f240410ee50 0x7f2404111240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:22.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.204+0000 7f240a000640 1 --2- 172.21.15.27:0/2266443498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2404075390 0x7f2404075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:22.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.204+0000 7f240a000640 1 --2- 172.21.15.27:0/2266443498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2404076970 0x7f2404074dc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:22.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.204+0000 7f240a000640 1 -- 172.21.15.27:0/2266443498 >> 172.21.15.27:0/2266443498 conn(0x7f24040fddf0 msgr2=0x7f2404100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:22.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.204+0000 7f240a000640 1 -- 172.21.15.27:0/2266443498 shutdown_connections 2024-04-26T22:55:22.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.205+0000 7f240a000640 1 -- 172.21.15.27:0/2266443498 wait complete. 2024-04-26T22:55:22.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.206+0000 7f240a000640 1 Processor -- start 2024-04-26T22:55:22.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.206+0000 7f240a000640 1 -- start start 2024-04-26T22:55:22.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.206+0000 7f240a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2404075390 0x7f240419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:22.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.206+0000 7f240a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2404076970 0x7f240419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:22.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.206+0000 7f240a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f240410ee50 0x7f24041a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:22.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.206+0000 7f240a000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24041149d0 con 0x7f2404075390 2024-04-26T22:55:22.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.206+0000 7f240a000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2404114850 con 0x7f240410ee50 2024-04-26T22:55:22.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.206+0000 7f240a000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2404114b50 con 0x7f2404076970 2024-04-26T22:55:22.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.207+0000 7f2403e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2404076970 0x7f240419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:22.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.207+0000 7f2403e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2404076970 0x7f240419f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:41728/0 (socket says 172.21.15.27:41728) 2024-04-26T22:55:22.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.207+0000 7f2408c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2404075390 0x7f240419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:22.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.207+0000 7f2403e00640 1 -- 172.21.15.27:0/497077190 learned_addr learned my addr 172.21.15.27:0/497077190 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:22.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.207+0000 7f2409600640 1 --2- 172.21.15.27:0/497077190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f240410ee50 0x7f24041a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:22.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.207+0000 7f2403e00640 1 -- 172.21.15.27:0/497077190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f240410ee50 msgr2=0x7f24041a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:22.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.207+0000 7f2403e00640 1 --2- 172.21.15.27:0/497077190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f240410ee50 0x7f24041a6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:22.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.207+0000 7f2403e00640 1 -- 172.21.15.27:0/497077190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2404075390 msgr2=0x7f240419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:22.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.207+0000 7f2403e00640 1 --2- 172.21.15.27:0/497077190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2404075390 0x7f240419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:22.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.207+0000 7f2403e00640 1 -- 172.21.15.27:0/497077190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24041a6cf0 con 0x7f2404076970 2024-04-26T22:55:22.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.207+0000 7f2403e00640 1 --2- 172.21.15.27:0/497077190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2404076970 0x7f240419f610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f23f000ea60 tx=0x7f23f000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:22.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.208+0000 7f2401600640 1 -- 172.21.15.27:0/497077190 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f000cde0 con 0x7f2404076970 2024-04-26T22:55:22.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.208+0000 7f2401600640 1 -- 172.21.15.27:0/497077190 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f00035f0 con 0x7f2404076970 2024-04-26T22:55:22.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.208+0000 7f2401600640 1 -- 172.21.15.27:0/497077190 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0010690 con 0x7f2404076970 2024-04-26T22:55:22.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.208+0000 7f240a000640 1 -- 172.21.15.27:0/497077190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24041a6fe0 con 0x7f2404076970 2024-04-26T22:55:22.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.208+0000 7f240a000640 1 -- 172.21.15.27:0/497077190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24041a74f0 con 0x7f2404076970 2024-04-26T22:55:22.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.210+0000 7f240a000640 1 -- 172.21.15.27:0/497077190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2404050730 con 0x7f2404076970 2024-04-26T22:55:22.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.211+0000 7f2401600640 1 -- 172.21.15.27:0/497077190 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f23f00037a0 con 0x7f2404076970 2024-04-26T22:55:22.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.213+0000 7f2401600640 1 --2- 172.21.15.27:0/497077190 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f23ec0a3c20 0x7f23ec0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:22.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.213+0000 7f2401600640 1 -- 172.21.15.27:0/497077190 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f23f00d02e0 con 0x7f2404076970 2024-04-26T22:55:22.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.214+0000 7f2408c00640 1 --2- 172.21.15.27:0/497077190 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f23ec0a3c20 0x7f23ec0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:22.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.214+0000 7f2408c00640 1 --2- 172.21.15.27:0/497077190 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f23ec0a3c20 0x7f23ec0a60e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f23f8005ab0 tx=0x7f23f8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:22.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.216+0000 7f2401600640 1 -- 172.21.15.27:0/497077190 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f23f009d0c0 con 0x7f2404076970 2024-04-26T22:55:22.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.432+0000 7f240a000640 1 -- 172.21.15.27:0/497077190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2404063530 con 0x7f2404076970 2024-04-26T22:55:22.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.432+0000 7f2401600640 1 -- 172.21.15.27:0/497077190 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f23f009c9c0 con 0x7f2404076970 2024-04-26T22:55:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:22.433 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-04-26T22:55:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T22:55:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-04-26T22:55:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-04-26T22:55:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T22:55:22.433 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-04-26T22:55:22.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.435+0000 7f240a000640 1 -- 172.21.15.27:0/497077190 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f23ec0a3c20 msgr2=0x7f23ec0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:22.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.435+0000 7f240a000640 1 --2- 172.21.15.27:0/497077190 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f23ec0a3c20 0x7f23ec0a60e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f23f8005ab0 tx=0x7f23f8005a20 comp rx=0 tx=0).stop 2024-04-26T22:55:22.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.435+0000 7f240a000640 1 -- 172.21.15.27:0/497077190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2404076970 msgr2=0x7f240419f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:22.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.435+0000 7f240a000640 1 --2- 172.21.15.27:0/497077190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2404076970 0x7f240419f610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f23f000ea60 tx=0x7f23f000ef20 comp rx=0 tx=0).stop 2024-04-26T22:55:22.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.435+0000 7f240a000640 1 -- 172.21.15.27:0/497077190 shutdown_connections 2024-04-26T22:55:22.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.435+0000 7f240a000640 1 --2- 172.21.15.27:0/497077190 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f23ec0a3c20 0x7f23ec0a60e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:22.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.435+0000 7f240a000640 1 --2- 172.21.15.27:0/497077190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f240410ee50 0x7f24041a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.435+0000 7f240a000640 1 --2- 172.21.15.27:0/497077190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2404076970 0x7f240419f610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.435+0000 7f240a000640 1 --2- 172.21.15.27:0/497077190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2404075390 0x7f240419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.435+0000 7f240a000640 1 -- 172.21.15.27:0/497077190 >> 172.21.15.27:0/497077190 conn(0x7f24040fddf0 msgr2=0x7f24041021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.436+0000 7f240a000640 1 -- 172.21.15.27:0/497077190 shutdown_connections 2024-04-26T22:55:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.436+0000 7f240a000640 1 -- 172.21.15.27:0/497077190 wait complete. 2024-04-26T22:55:22.454 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T22:55:22.454 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T22:55:22.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.646+0000 7ff4f8a00640 1 -- 172.21.15.27:0/3969001519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4108670 msgr2=0x7ff4f410aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:22.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.646+0000 7ff4f8a00640 1 --2- 172.21.15.27:0/3969001519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4108670 0x7ff4f410aac0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009ca0 tx=0x7ff4ec02a8c0 comp rx=0 tx=0).stop 2024-04-26T22:55:22.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.646+0000 7ff4f8a00640 1 -- 172.21.15.27:0/3969001519 shutdown_connections 2024-04-26T22:55:22.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.646+0000 7ff4f8a00640 1 --2- 172.21.15.27:0/3969001519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f4105d40 0x7ff4f4108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:22.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.646+0000 7ff4f8a00640 1 --2- 172.21.15.27:0/3969001519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4108670 0x7ff4f410aac0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:22.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.646+0000 7ff4f8a00640 1 --2- 172.21.15.27:0/3969001519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4103410 0x7ff4f4105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:22.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.646+0000 7ff4f8a00640 1 -- 172.21.15.27:0/3969001519 >> 172.21.15.27:0/3969001519 conn(0x7ff4f40fce00 msgr2=0x7ff4f40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:22.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.646+0000 7ff4f8a00640 1 -- 172.21.15.27:0/3969001519 shutdown_connections 2024-04-26T22:55:22.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.647+0000 7ff4f8a00640 1 -- 172.21.15.27:0/3969001519 wait complete. 2024-04-26T22:55:22.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.647+0000 7ff4f8a00640 1 Processor -- start 2024-04-26T22:55:22.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.648+0000 7ff4f8a00640 1 -- start start 2024-04-26T22:55:22.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.648+0000 7ff4f8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4103410 0x7ff4f419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:22.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.648+0000 7ff4f8a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f4105d40 0x7ff4f419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:22.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.648+0000 7ff4f8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4108670 0x7ff4f41a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:22.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.648+0000 7ff4f8a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f410e530 con 0x7ff4f4108670 2024-04-26T22:55:22.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.648+0000 7ff4f8a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f410e3b0 con 0x7ff4f4105d40 2024-04-26T22:55:22.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.648+0000 7ff4f8a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f410e6b0 con 0x7ff4f4103410 2024-04-26T22:55:22.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.648+0000 7ff4f1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4103410 0x7ff4f419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:22.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.648+0000 7ff4f1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4103410 0x7ff4f419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:41754/0 (socket says 172.21.15.27:41754) 2024-04-26T22:55:22.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.648+0000 7ff4f1e00640 1 -- 172.21.15.27:0/2673429780 learned_addr learned my addr 172.21.15.27:0/2673429780 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:22.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.648+0000 7ff4f1400640 1 --2- 172.21.15.27:0/2673429780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f4105d40 0x7ff4f419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:22.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.648+0000 7ff4f2800640 1 --2- 172.21.15.27:0/2673429780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4108670 0x7ff4f41a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:22.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.649+0000 7ff4f1e00640 1 -- 172.21.15.27:0/2673429780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f4105d40 msgr2=0x7ff4f419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:22.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.649+0000 7ff4f1e00640 1 --2- 172.21.15.27:0/2673429780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f4105d40 0x7ff4f419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:22.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.649+0000 7ff4f1e00640 1 -- 172.21.15.27:0/2673429780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4108670 msgr2=0x7ff4f41a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:22.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.649+0000 7ff4f1e00640 1 --2- 172.21.15.27:0/2673429780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4108670 0x7ff4f41a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:22.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.649+0000 7ff4f1e00640 1 -- 172.21.15.27:0/2673429780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f41a29d0 con 0x7ff4f4103410 2024-04-26T22:55:22.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.649+0000 7ff4f1e00640 1 --2- 172.21.15.27:0/2673429780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4103410 0x7ff4f419adc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400ea60 tx=0x7ff4e400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:22.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.649+0000 7ff4eaa00640 1 -- 172.21.15.27:0/2673429780 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e400cde0 con 0x7ff4f4103410 2024-04-26T22:55:22.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.649+0000 7ff4f8a00640 1 -- 172.21.15.27:0/2673429780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f41a2cc0 con 0x7ff4f4103410 2024-04-26T22:55:22.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.649+0000 7ff4eaa00640 1 -- 172.21.15.27:0/2673429780 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e40035f0 con 0x7ff4f4103410 2024-04-26T22:55:22.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.649+0000 7ff4eaa00640 1 -- 172.21.15.27:0/2673429780 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4010690 con 0x7ff4f4103410 2024-04-26T22:55:22.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.649+0000 7ff4f8a00640 1 -- 172.21.15.27:0/2673429780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f41a31c0 con 0x7ff4f4103410 2024-04-26T22:55:22.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.650+0000 7ff4f8a00640 1 -- 172.21.15.27:0/2673429780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f410a160 con 0x7ff4f4103410 2024-04-26T22:55:22.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.652+0000 7ff4eaa00640 1 -- 172.21.15.27:0/2673429780 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff4e40037a0 con 0x7ff4f4103410 2024-04-26T22:55:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.654+0000 7ff4eaa00640 1 --2- 172.21.15.27:0/2673429780 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4d80a3bb0 0x7ff4d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.654+0000 7ff4eaa00640 1 -- 172.21.15.27:0/2673429780 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff4e40d0300 con 0x7ff4f4103410 2024-04-26T22:55:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.654+0000 7ff4f1400640 1 --2- 172.21.15.27:0/2673429780 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4d80a3bb0 0x7ff4d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:22.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.654+0000 7ff4f1400640 1 --2- 172.21.15.27:0/2673429780 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4d80a3bb0 0x7ff4d80a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff4f406b540 tx=0x7ff4e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.662+0000 7ff4eaa00640 1 -- 172.21.15.27:0/2673429780 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff4e409d0e0 con 0x7ff4f4103410 2024-04-26T22:55:22.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:22.881+0000 7ff4f8a00640 1 -- 172.21.15.27:0/2673429780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff4f4063530 con 0x7ff4f4103410 2024-04-26T22:55:23.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:22 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2960351259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:55:23.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:23.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:23.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:23.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:55:23.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:23.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:22 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/497077190' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T22:55:23.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:55:23.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:22 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2960351259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:55:23.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:23.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:23.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:23.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:55:23.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:23.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:22 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/497077190' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T22:55:23.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:55:23.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:22 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2960351259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:55:23.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:23.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:23.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:23.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:55:23.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:23.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:22 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/497077190' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T22:55:23.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:55:23.731 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:23 smithi080 ceph-mon[32324]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:23.731 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:23 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2673429780' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T22:55:23.731 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:23 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T22:55:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.745+0000 7ff4eaa00640 1 -- 172.21.15.27:0/2673429780 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7ff4e409c9e0 con 0x7ff4f4103410 2024-04-26T22:55:23.746 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-04-26T22:55:23.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.752+0000 7ff4f8a00640 1 -- 172.21.15.27:0/2673429780 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4d80a3bb0 msgr2=0x7ff4d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:23.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.752+0000 7ff4f8a00640 1 --2- 172.21.15.27:0/2673429780 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4d80a3bb0 0x7ff4d80a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff4f406b540 tx=0x7ff4e0005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:23.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.752+0000 7ff4f8a00640 1 -- 172.21.15.27:0/2673429780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4103410 msgr2=0x7ff4f419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:23.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.752+0000 7ff4f8a00640 1 --2- 172.21.15.27:0/2673429780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4103410 0x7ff4f419adc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400ea60 tx=0x7ff4e400ef20 comp rx=0 tx=0).stop 2024-04-26T22:55:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.753+0000 7ff4f8a00640 1 -- 172.21.15.27:0/2673429780 shutdown_connections 2024-04-26T22:55:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.753+0000 7ff4f8a00640 1 --2- 172.21.15.27:0/2673429780 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4d80a3bb0 0x7ff4d80a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.753+0000 7ff4f8a00640 1 --2- 172.21.15.27:0/2673429780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f4105d40 0x7ff4f419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.753+0000 7ff4f8a00640 1 --2- 172.21.15.27:0/2673429780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4108670 0x7ff4f41a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.753+0000 7ff4f8a00640 1 --2- 172.21.15.27:0/2673429780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4103410 0x7ff4f419adc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.753+0000 7ff4f8a00640 1 -- 172.21.15.27:0/2673429780 >> 172.21.15.27:0/2673429780 conn(0x7ff4f40fce00 msgr2=0x7ff4f4102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.753+0000 7ff4f8a00640 1 -- 172.21.15.27:0/2673429780 shutdown_connections 2024-04-26T22:55:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.754+0000 7ff4f8a00640 1 -- 172.21.15.27:0/2673429780 wait complete. 2024-04-26T22:55:23.782 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-26T22:55:23.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.973+0000 7f81c5a00640 1 -- 172.21.15.27:0/610264721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81c0105d70 msgr2=0x7f81c01061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:23.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.973+0000 7f81c5a00640 1 --2- 172.21.15.27:0/610264721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81c0105d70 0x7f81c01061f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009900 tx=0x7f81ac02a410 comp rx=0 tx=0).stop 2024-04-26T22:55:23.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.973+0000 7f81c5a00640 1 -- 172.21.15.27:0/610264721 shutdown_connections 2024-04-26T22:55:23.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.973+0000 7f81c5a00640 1 --2- 172.21.15.27:0/610264721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81c0105d70 0x7f81c01061f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:23.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.973+0000 7f81c5a00640 1 --2- 172.21.15.27:0/610264721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81c0106730 0x7f81c010cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:23.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.973+0000 7f81c5a00640 1 --2- 172.21.15.27:0/610264721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c0104b80 0x7f81c0104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:23.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.973+0000 7f81c5a00640 1 -- 172.21.15.27:0/610264721 >> 172.21.15.27:0/610264721 conn(0x7f81c01000f0 msgr2=0x7f81c0102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:23.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.974+0000 7f81c5a00640 1 -- 172.21.15.27:0/610264721 shutdown_connections 2024-04-26T22:55:23.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.974+0000 7f81c5a00640 1 -- 172.21.15.27:0/610264721 wait complete. 2024-04-26T22:55:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.975+0000 7f81c5a00640 1 Processor -- start 2024-04-26T22:55:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.975+0000 7f81c5a00640 1 -- start start 2024-04-26T22:55:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.975+0000 7f81c5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81c0104b80 0x7f81c019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.975+0000 7f81c5a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c0105d70 0x7f81c019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.975+0000 7f81c5a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81c0106730 0x7f81c01a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.975+0000 7f81c5a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f81c0110700 con 0x7f81c0104b80 2024-04-26T22:55:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.975+0000 7f81c5a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f81c0110580 con 0x7f81c0106730 2024-04-26T22:55:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.975+0000 7f81c5a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f81c0110880 con 0x7f81c0105d70 2024-04-26T22:55:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.975+0000 7f81be000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c0105d70 0x7f81c019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.975+0000 7f81bea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81c0104b80 0x7f81c019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:23.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.975+0000 7f81be000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c0105d70 0x7f81c019b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:41774/0 (socket says 172.21.15.27:41774) 2024-04-26T22:55:23.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.975+0000 7f81bf400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81c0106730 0x7f81c01a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:23.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.975+0000 7f81be000640 1 -- 172.21.15.27:0/1196132396 learned_addr learned my addr 172.21.15.27:0/1196132396 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:23.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.976+0000 7f81be000640 1 -- 172.21.15.27:0/1196132396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81c0106730 msgr2=0x7f81c01a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:23.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.976+0000 7f81be000640 1 --2- 172.21.15.27:0/1196132396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81c0106730 0x7f81c01a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:23.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.976+0000 7f81be000640 1 -- 172.21.15.27:0/1196132396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81c0104b80 msgr2=0x7f81c019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:23.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.976+0000 7f81be000640 1 --2- 172.21.15.27:0/1196132396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81c0104b80 0x7f81c019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:23.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.976+0000 7f81be000640 1 -- 172.21.15.27:0/1196132396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c01a29f0 con 0x7f81c0105d70 2024-04-26T22:55:23.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.976+0000 7f81be000640 1 --2- 172.21.15.27:0/1196132396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c0105d70 0x7f81c019b310 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81ac005ab0 tx=0x7f81ac003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:23.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.976+0000 7f81b7400640 1 -- 172.21.15.27:0/1196132396 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac03f070 con 0x7f81c0105d70 2024-04-26T22:55:23.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.976+0000 7f81b7400640 1 -- 172.21.15.27:0/1196132396 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81ac031c50 con 0x7f81c0105d70 2024-04-26T22:55:23.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.976+0000 7f81b7400640 1 -- 172.21.15.27:0/1196132396 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac035030 con 0x7f81c0105d70 2024-04-26T22:55:23.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.976+0000 7f81c5a00640 1 -- 172.21.15.27:0/1196132396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c01a2c80 con 0x7f81c0105d70 2024-04-26T22:55:23.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.976+0000 7f81c5a00640 1 -- 172.21.15.27:0/1196132396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c01a3170 con 0x7f81c0105d70 2024-04-26T22:55:23.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.978+0000 7f81c5a00640 1 -- 172.21.15.27:0/1196132396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c010bea0 con 0x7f81c0105d70 2024-04-26T22:55:23.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.979+0000 7f81b7400640 1 -- 172.21.15.27:0/1196132396 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f81ac042070 con 0x7f81c0105d70 2024-04-26T22:55:23.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.981+0000 7f81b7400640 1 --2- 172.21.15.27:0/1196132396 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f81a40a3bb0 0x7f81a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:23.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.981+0000 7f81b7400640 1 -- 172.21.15.27:0/1196132396 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f81ac0f0850 con 0x7f81c0105d70 2024-04-26T22:55:23.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.981+0000 7f81bea00640 1 --2- 172.21.15.27:0/1196132396 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f81a40a3bb0 0x7f81a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:23.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.982+0000 7f81bea00640 1 --2- 172.21.15.27:0/1196132396 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f81a40a3bb0 0x7f81a40a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f81c0105bd0 tx=0x7f81b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:23.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:23.985+0000 7f81b7400640 1 -- 172.21.15.27:0/1196132396 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f81ac0bce70 con 0x7f81c0105d70 2024-04-26T22:55:24.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:23 smithi094 ceph-mon[32889]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:24.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:23 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2673429780' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T22:55:24.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:23 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T22:55:24.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:23 smithi027 ceph-mon[28034]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:24.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:23 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2673429780' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T22:55:24.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:23 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T22:55:24.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.198+0000 7f81c5a00640 1 -- 172.21.15.27:0/1196132396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f81c0063530 con 0x7f81c0105d70 2024-04-26T22:55:24.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.749+0000 7f81b7400640 1 -- 172.21.15.27:0/1196132396 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7f81ac0c1170 con 0x7f81c0105d70 2024-04-26T22:55:24.750 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-04-26T22:55:24.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.758+0000 7f81c5a00640 1 -- 172.21.15.27:0/1196132396 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f81a40a3bb0 msgr2=0x7f81a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:24.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.758+0000 7f81c5a00640 1 --2- 172.21.15.27:0/1196132396 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f81a40a3bb0 0x7f81a40a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f81c0105bd0 tx=0x7f81b0005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:24.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.758+0000 7f81c5a00640 1 -- 172.21.15.27:0/1196132396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c0105d70 msgr2=0x7f81c019b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:24.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.758+0000 7f81c5a00640 1 --2- 172.21.15.27:0/1196132396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c0105d70 0x7f81c019b310 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81ac005ab0 tx=0x7f81ac003b90 comp rx=0 tx=0).stop 2024-04-26T22:55:24.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.759+0000 7f81c5a00640 1 -- 172.21.15.27:0/1196132396 shutdown_connections 2024-04-26T22:55:24.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.759+0000 7f81c5a00640 1 --2- 172.21.15.27:0/1196132396 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f81a40a3bb0 0x7f81a40a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:24.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.759+0000 7f81c5a00640 1 --2- 172.21.15.27:0/1196132396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81c0106730 0x7f81c01a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:24.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.759+0000 7f81c5a00640 1 --2- 172.21.15.27:0/1196132396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c0105d70 0x7f81c019b310 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:24.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.759+0000 7f81c5a00640 1 --2- 172.21.15.27:0/1196132396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81c0104b80 0x7f81c019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:24.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.759+0000 7f81c5a00640 1 -- 172.21.15.27:0/1196132396 >> 172.21.15.27:0/1196132396 conn(0x7f81c01000f0 msgr2=0x7f81c0109070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:24.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.760+0000 7f81c5a00640 1 -- 172.21.15.27:0/1196132396 shutdown_connections 2024-04-26T22:55:24.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.762+0000 7f81c5a00640 1 -- 172.21.15.27:0/1196132396 wait complete. 2024-04-26T22:55:24.799 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T22:55:24.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.989+0000 7fc1b4800640 1 -- 172.21.15.27:0/2418766057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1ac105d80 msgr2=0x7fc1ac106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:24.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.989+0000 7fc1b4800640 1 --2- 172.21.15.27:0/2418766057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1ac105d80 0x7fc1ac106200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009900 tx=0x7fc19c02a3f0 comp rx=0 tx=0).stop 2024-04-26T22:55:24.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.989+0000 7fc1b4800640 1 -- 172.21.15.27:0/2418766057 shutdown_connections 2024-04-26T22:55:24.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.989+0000 7fc1b4800640 1 --2- 172.21.15.27:0/2418766057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1ac106740 0x7fc1ac10cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:24.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.989+0000 7fc1b4800640 1 --2- 172.21.15.27:0/2418766057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1ac105d80 0x7fc1ac106200 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:24.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.989+0000 7fc1b4800640 1 --2- 172.21.15.27:0/2418766057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1ac104b90 0x7fc1ac104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:24.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.989+0000 7fc1b4800640 1 -- 172.21.15.27:0/2418766057 >> 172.21.15.27:0/2418766057 conn(0x7fc1ac100120 msgr2=0x7fc1ac102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:24.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.989+0000 7fc1b4800640 1 -- 172.21.15.27:0/2418766057 shutdown_connections 2024-04-26T22:55:24.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.989+0000 7fc1b4800640 1 -- 172.21.15.27:0/2418766057 wait complete. 2024-04-26T22:55:24.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.990+0000 7fc1b4800640 1 Processor -- start 2024-04-26T22:55:24.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.991+0000 7fc1b4800640 1 -- start start 2024-04-26T22:55:24.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.991+0000 7fc1b4800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1ac104b90 0x7fc1ac19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:24.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.991+0000 7fc1b4800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1ac105d80 0x7fc1ac19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:24.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.991+0000 7fc1b4800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1ac106740 0x7fc1ac1a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:24.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.991+0000 7fc1b4800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac1106f0 con 0x7fc1ac106740 2024-04-26T22:55:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.991+0000 7fc1b4800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac110570 con 0x7fc1ac105d80 2024-04-26T22:55:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.991+0000 7fc1b4800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac110870 con 0x7fc1ac104b90 2024-04-26T22:55:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.991+0000 7fc1b1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1ac104b90 0x7fc1ac19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.991+0000 7fc1b1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1ac104b90 0x7fc1ac19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:41804/0 (socket says 172.21.15.27:41804) 2024-04-26T22:55:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.991+0000 7fc1b2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1ac106740 0x7fc1ac1a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.991+0000 7fc1b1e00640 1 -- 172.21.15.27:0/2749344226 learned_addr learned my addr 172.21.15.27:0/2749344226 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.991+0000 7fc1b1400640 1 --2- 172.21.15.27:0/2749344226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1ac105d80 0x7fc1ac19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.992+0000 7fc1b1e00640 1 -- 172.21.15.27:0/2749344226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1ac105d80 msgr2=0x7fc1ac19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.992+0000 7fc1b1e00640 1 --2- 172.21.15.27:0/2749344226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1ac105d80 0x7fc1ac19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.992+0000 7fc1b1e00640 1 -- 172.21.15.27:0/2749344226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1ac106740 msgr2=0x7fc1ac1a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.992+0000 7fc1b1e00640 1 --2- 172.21.15.27:0/2749344226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1ac106740 0x7fc1ac1a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.992+0000 7fc1b1e00640 1 -- 172.21.15.27:0/2749344226 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac1a2a10 con 0x7fc1ac104b90 2024-04-26T22:55:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.992+0000 7fc1b1400640 1 --2- 172.21.15.27:0/2749344226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1ac105d80 0x7fc1ac19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:55:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.992+0000 7fc1b1e00640 1 --2- 172.21.15.27:0/2749344226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1ac104b90 0x7fc1ac19adf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000e9c0 tx=0x7fc1a000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:24.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.992+0000 7fc1a6a00640 1 -- 172.21.15.27:0/2749344226 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a000cd80 con 0x7fc1ac104b90 2024-04-26T22:55:24.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.992+0000 7fc1a6a00640 1 -- 172.21.15.27:0/2749344226 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a00035f0 con 0x7fc1ac104b90 2024-04-26T22:55:24.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.992+0000 7fc1b4800640 1 -- 172.21.15.27:0/2749344226 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ac1a2d00 con 0x7fc1ac104b90 2024-04-26T22:55:24.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.992+0000 7fc1a6a00640 1 -- 172.21.15.27:0/2749344226 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0010690 con 0x7fc1ac104b90 2024-04-26T22:55:24.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.992+0000 7fc1b4800640 1 -- 172.21.15.27:0/2749344226 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ac1a31c0 con 0x7fc1ac104b90 2024-04-26T22:55:24.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.994+0000 7fc1b4800640 1 -- 172.21.15.27:0/2749344226 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ac10bf30 con 0x7fc1ac104b90 2024-04-26T22:55:24.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.995+0000 7fc1a6a00640 1 -- 172.21.15.27:0/2749344226 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc1a00037a0 con 0x7fc1ac104b90 2024-04-26T22:55:24.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.997+0000 7fc1a6a00640 1 --2- 172.21.15.27:0/2749344226 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc1940a3c30 0x7fc1940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:24.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.997+0000 7fc1a6a00640 1 -- 172.21.15.27:0/2749344226 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7fc1a00d06a0 con 0x7fc1ac104b90 2024-04-26T22:55:24.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.997+0000 7fc1b1400640 1 --2- 172.21.15.27:0/2749344226 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc1940a3c30 0x7fc1940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:24.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:24.998+0000 7fc1b1400640 1 --2- 172.21.15.27:0/2749344226 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc1940a3c30 0x7fc1940a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc19c000c00 tx=0x7fc19c0314c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:25.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:25.001+0000 7fc1a6a00640 1 -- 172.21.15.27:0/2749344226 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc1a009d190 con 0x7fc1ac104b90 2024-04-26T22:55:25.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:24 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T22:55:25.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:24 smithi080 ceph-mon[32324]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T22:55:25.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:24 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:25.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:24 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:25.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:24 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1196132396' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T22:55:25.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:24 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:25.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:24 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T22:55:25.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:24 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:25.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:24 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T22:55:25.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:24 smithi094 ceph-mon[32889]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T22:55:25.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:24 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:25.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:24 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:25.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:24 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1196132396' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T22:55:25.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:24 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:25.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:24 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T22:55:25.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:24 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:25.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:24 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T22:55:25.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:24 smithi027 ceph-mon[28034]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T22:55:25.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:24 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:25.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:24 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:25.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:24 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1196132396' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T22:55:25.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:24 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:25.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:24 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T22:55:25.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:24 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:25.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:25.263+0000 7fc1b4800640 1 -- 172.21.15.27:0/2749344226 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc1ac02dad0 con 0x7fc1ac104b90 2024-04-26T22:55:26.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:25 smithi080 ceph-mon[32324]: pgmap v322: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:26.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:25 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-26T22:55:26.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:25 smithi080 ceph-mon[32324]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T22:55:26.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:25 smithi080 ceph-mon[32324]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T22:55:26.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:25 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2749344226' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T22:55:26.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:25 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T22:55:26.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:25 smithi080 ceph-mon[32324]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T22:55:26.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:26.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:25 smithi080 ceph-mon[32324]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T22:55:26.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:26.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:26.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:55:26.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:26.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:25 smithi094 ceph-mon[32889]: pgmap v322: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:26.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:25 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-26T22:55:26.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:25 smithi094 ceph-mon[32889]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T22:55:26.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:25 smithi094 ceph-mon[32889]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T22:55:26.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:25 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2749344226' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T22:55:26.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:25 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T22:55:26.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:25 smithi094 ceph-mon[32889]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T22:55:26.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:26.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:25 smithi094 ceph-mon[32889]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T22:55:26.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:26.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:26.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:55:26.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:26.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:25 smithi027 ceph-mon[28034]: pgmap v322: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:26.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:25 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-26T22:55:26.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:25 smithi027 ceph-mon[28034]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T22:55:26.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:25 smithi027 ceph-mon[28034]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T22:55:26.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:25 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2749344226' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T22:55:26.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:25 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T22:55:26.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:25 smithi027 ceph-mon[28034]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T22:55:26.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:26.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:25 smithi027 ceph-mon[28034]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T22:55:26.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:26.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:26.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:55:26.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:55:26.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.247+0000 7fc1a6a00640 1 -- 172.21.15.27:0/2749344226 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fc1ac02dad0 con 0x7fc1ac104b90 2024-04-26T22:55:26.247 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T22:55:26.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.253+0000 7fc1b4800640 1 -- 172.21.15.27:0/2749344226 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc1940a3c30 msgr2=0x7fc1940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:26.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.253+0000 7fc1b4800640 1 --2- 172.21.15.27:0/2749344226 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc1940a3c30 0x7fc1940a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc19c000c00 tx=0x7fc19c0314c0 comp rx=0 tx=0).stop 2024-04-26T22:55:26.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.253+0000 7fc1b4800640 1 -- 172.21.15.27:0/2749344226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1ac104b90 msgr2=0x7fc1ac19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:26.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.253+0000 7fc1b4800640 1 --2- 172.21.15.27:0/2749344226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1ac104b90 0x7fc1ac19adf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000e9c0 tx=0x7fc1a000ee80 comp rx=0 tx=0).stop 2024-04-26T22:55:26.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.253+0000 7fc1b4800640 1 -- 172.21.15.27:0/2749344226 shutdown_connections 2024-04-26T22:55:26.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.253+0000 7fc1b4800640 1 --2- 172.21.15.27:0/2749344226 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fc1940a3c30 0x7fc1940a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:26.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.253+0000 7fc1b4800640 1 --2- 172.21.15.27:0/2749344226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1ac105d80 0x7fc1ac19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:26.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.253+0000 7fc1b4800640 1 --2- 172.21.15.27:0/2749344226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1ac106740 0x7fc1ac1a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:26.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.253+0000 7fc1b4800640 1 --2- 172.21.15.27:0/2749344226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1ac104b90 0x7fc1ac19adf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:26.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.253+0000 7fc1b4800640 1 -- 172.21.15.27:0/2749344226 >> 172.21.15.27:0/2749344226 conn(0x7fc1ac100120 msgr2=0x7fc1ac109100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:26.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.255+0000 7fc1b4800640 1 -- 172.21.15.27:0/2749344226 shutdown_connections 2024-04-26T22:55:26.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.255+0000 7fc1b4800640 1 -- 172.21.15.27:0/2749344226 wait complete. 2024-04-26T22:55:26.285 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-26T22:55:26.286 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-26T22:55:26.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.496+0000 7fa23be00640 1 -- 172.21.15.27:0/2338725979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa23c105b80 msgr2=0x7fa23c106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:26.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.496+0000 7fa23be00640 1 --2- 172.21.15.27:0/2338725979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa23c105b80 0x7fa23c106000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa228009900 tx=0x7fa22802a3f0 comp rx=0 tx=0).stop 2024-04-26T22:55:26.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.497+0000 7fa23be00640 1 -- 172.21.15.27:0/2338725979 shutdown_connections 2024-04-26T22:55:26.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.497+0000 7fa23be00640 1 --2- 172.21.15.27:0/2338725979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa23c106540 0x7fa23c10cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:26.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.497+0000 7fa23be00640 1 --2- 172.21.15.27:0/2338725979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa23c105b80 0x7fa23c106000 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:26.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.497+0000 7fa23be00640 1 --2- 172.21.15.27:0/2338725979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa23c104990 0x7fa23c104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:26.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.497+0000 7fa23be00640 1 -- 172.21.15.27:0/2338725979 >> 172.21.15.27:0/2338725979 conn(0x7fa23c0fff00 msgr2=0x7fa23c102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:26.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.497+0000 7fa23be00640 1 -- 172.21.15.27:0/2338725979 shutdown_connections 2024-04-26T22:55:26.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.497+0000 7fa23be00640 1 -- 172.21.15.27:0/2338725979 wait complete. 2024-04-26T22:55:26.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.498+0000 7fa23be00640 1 Processor -- start 2024-04-26T22:55:26.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.498+0000 7fa23be00640 1 -- start start 2024-04-26T22:55:26.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.499+0000 7fa23be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa23c104990 0x7fa23c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:26.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.499+0000 7fa23be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa23c105b80 0x7fa23c19b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:26.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.499+0000 7fa23be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa23c106540 0x7fa23c1a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:26.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.499+0000 7fa23be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c1104f0 con 0x7fa23c106540 2024-04-26T22:55:26.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.499+0000 7fa23be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c110370 con 0x7fa23c105b80 2024-04-26T22:55:26.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.499+0000 7fa23be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c110670 con 0x7fa23c104990 2024-04-26T22:55:26.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.499+0000 7fa23b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa23c106540 0x7fa23c1a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.499+0000 7fa23aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa23c104990 0x7fa23c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.499+0000 7fa23b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa23c106540 0x7fa23c1a2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37654/0 (socket says 172.21.15.27:37654) 2024-04-26T22:55:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.499+0000 7fa23b400640 1 -- 172.21.15.27:0/3481873185 learned_addr learned my addr 172.21.15.27:0/3481873185 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.499+0000 7fa23a000640 1 --2- 172.21.15.27:0/3481873185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa23c105b80 0x7fa23c19b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.499+0000 7fa23b400640 1 -- 172.21.15.27:0/3481873185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa23c104990 msgr2=0x7fa23c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.499+0000 7fa23b400640 1 --2- 172.21.15.27:0/3481873185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa23c104990 0x7fa23c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.499+0000 7fa23b400640 1 -- 172.21.15.27:0/3481873185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa23c105b80 msgr2=0x7fa23c19b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.499+0000 7fa23b400640 1 --2- 172.21.15.27:0/3481873185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa23c105b80 0x7fa23c19b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.499+0000 7fa23b400640 1 -- 172.21.15.27:0/3481873185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23c1a27e0 con 0x7fa23c106540 2024-04-26T22:55:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.500+0000 7fa23b400640 1 --2- 172.21.15.27:0/3481873185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa23c106540 0x7fa23c1a2170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa23400eb10 tx=0x7fa23400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.500+0000 7fa233400640 1 -- 172.21.15.27:0/3481873185 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234004b20 con 0x7fa23c106540 2024-04-26T22:55:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.500+0000 7fa23be00640 1 -- 172.21.15.27:0/3481873185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa23c1a2ad0 con 0x7fa23c106540 2024-04-26T22:55:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.500+0000 7fa233400640 1 -- 172.21.15.27:0/3481873185 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa234004cc0 con 0x7fa23c106540 2024-04-26T22:55:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.500+0000 7fa233400640 1 -- 172.21.15.27:0/3481873185 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234010670 con 0x7fa23c106540 2024-04-26T22:55:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.500+0000 7fa23be00640 1 -- 172.21.15.27:0/3481873185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa23c1a2ff0 con 0x7fa23c106540 2024-04-26T22:55:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.501+0000 7fa23be00640 1 -- 172.21.15.27:0/3481873185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa23c10bd00 con 0x7fa23c106540 2024-04-26T22:55:26.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.504+0000 7fa233400640 1 -- 172.21.15.27:0/3481873185 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa234005000 con 0x7fa23c106540 2024-04-26T22:55:26.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.506+0000 7fa233400640 1 --2- 172.21.15.27:0/3481873185 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2200a3c30 0x7fa2200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:26.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.507+0000 7fa233400640 1 -- 172.21.15.27:0/3481873185 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa2340d0a80 con 0x7fa23c106540 2024-04-26T22:55:26.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.507+0000 7fa23aa00640 1 --2- 172.21.15.27:0/3481873185 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2200a3c30 0x7fa2200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:26.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.507+0000 7fa23aa00640 1 --2- 172.21.15.27:0/3481873185 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2200a3c30 0x7fa2200a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0095f0 tx=0x7fa22c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:26.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.510+0000 7fa233400640 1 -- 172.21.15.27:0/3481873185 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa234054060 con 0x7fa23c106540 2024-04-26T22:55:26.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:26.727+0000 7fa23be00640 1 -- 172.21.15.27:0/3481873185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fa23c063530 con 0x7fa23c106540 2024-04-26T22:55:27.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.366+0000 7fa233400640 1 -- 172.21.15.27:0/3481873185 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7fa23409d510 con 0x7fa23c106540 2024-04-26T22:55:27.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.369+0000 7fa23be00640 1 -- 172.21.15.27:0/3481873185 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2200a3c30 msgr2=0x7fa2200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:27.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.369+0000 7fa23be00640 1 --2- 172.21.15.27:0/3481873185 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2200a3c30 0x7fa2200a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0095f0 tx=0x7fa22c005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:27.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.369+0000 7fa23be00640 1 -- 172.21.15.27:0/3481873185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa23c106540 msgr2=0x7fa23c1a2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:27.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.369+0000 7fa23be00640 1 --2- 172.21.15.27:0/3481873185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa23c106540 0x7fa23c1a2170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa23400eb10 tx=0x7fa23400c2d0 comp rx=0 tx=0).stop 2024-04-26T22:55:27.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.369+0000 7fa23be00640 1 -- 172.21.15.27:0/3481873185 shutdown_connections 2024-04-26T22:55:27.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.369+0000 7fa23be00640 1 --2- 172.21.15.27:0/3481873185 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2200a3c30 0x7fa2200a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:27.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.369+0000 7fa23be00640 1 --2- 172.21.15.27:0/3481873185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa23c105b80 0x7fa23c19b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:27.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.369+0000 7fa23be00640 1 --2- 172.21.15.27:0/3481873185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa23c106540 0x7fa23c1a2170 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:27.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.369+0000 7fa23be00640 1 --2- 172.21.15.27:0/3481873185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa23c104990 0x7fa23c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:27.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.369+0000 7fa23be00640 1 -- 172.21.15.27:0/3481873185 >> 172.21.15.27:0/3481873185 conn(0x7fa23c0fff00 msgr2=0x7fa23c108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:27.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.370+0000 7fa23be00640 1 -- 172.21.15.27:0/3481873185 shutdown_connections 2024-04-26T22:55:27.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.370+0000 7fa23be00640 1 -- 172.21.15.27:0/3481873185 wait complete. 2024-04-26T22:55:27.392 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T22:55:27.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:27 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T22:55:27.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:27 smithi080 ceph-mon[32324]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T22:55:27.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:27 smithi080 ceph-mon[32324]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T22:55:27.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:27 smithi080 ceph-mon[32324]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T22:55:27.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:27 smithi080 ceph-mon[32324]: pgmap v326: 97 pgs: 37 active+clean, 20 creating+peering, 40 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:27.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:27 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3481873185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T22:55:27.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:27 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T22:55:27.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:27 smithi094 ceph-mon[32889]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T22:55:27.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:27 smithi094 ceph-mon[32889]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T22:55:27.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:27 smithi094 ceph-mon[32889]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T22:55:27.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:27 smithi094 ceph-mon[32889]: pgmap v326: 97 pgs: 37 active+clean, 20 creating+peering, 40 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:27.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:27 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3481873185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T22:55:27.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:27 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T22:55:27.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:27 smithi027 ceph-mon[28034]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T22:55:27.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:27 smithi027 ceph-mon[28034]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T22:55:27.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:27 smithi027 ceph-mon[28034]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T22:55:27.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:27 smithi027 ceph-mon[28034]: pgmap v326: 97 pgs: 37 active+clean, 20 creating+peering, 40 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:27.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:27 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3481873185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T22:55:27.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.615+0000 7f029fe00640 1 -- 172.21.15.27:0/897687554 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a0109fd0 msgr2=0x7f02a010a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:27.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.615+0000 7f029fe00640 1 --2- 172.21.15.27:0/897687554 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a0109fd0 0x7f02a010a450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0288009860 tx=0x7f028802a340 comp rx=0 tx=0).stop 2024-04-26T22:55:27.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.616+0000 7f029fe00640 1 -- 172.21.15.27:0/897687554 shutdown_connections 2024-04-26T22:55:27.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.616+0000 7f029fe00640 1 --2- 172.21.15.27:0/897687554 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a0109fd0 0x7f02a010a450 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:27.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.616+0000 7f029fe00640 1 --2- 172.21.15.27:0/897687554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02a010a990 0x7f02a01112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:27.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.616+0000 7f029fe00640 1 --2- 172.21.15.27:0/897687554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a0108ed0 0x7f02a01092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:27.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.616+0000 7f029fe00640 1 -- 172.21.15.27:0/897687554 >> 172.21.15.27:0/897687554 conn(0x7f02a0077e20 msgr2=0x7f02a007a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:27.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.616+0000 7f029fe00640 1 -- 172.21.15.27:0/897687554 shutdown_connections 2024-04-26T22:55:27.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.616+0000 7f029fe00640 1 -- 172.21.15.27:0/897687554 wait complete. 2024-04-26T22:55:27.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.617+0000 7f029fe00640 1 Processor -- start 2024-04-26T22:55:27.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.617+0000 7f029fe00640 1 -- start start 2024-04-26T22:55:27.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.618+0000 7f029fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a0108ed0 0x7f02a019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:27.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.618+0000 7f029fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02a0109fd0 0x7f02a019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:27.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.618+0000 7f029fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a010a990 0x7f02a01a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:27.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.618+0000 7f029fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f02a01149e0 con 0x7f02a0109fd0 2024-04-26T22:55:27.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.618+0000 7f029fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f02a0114860 con 0x7f02a010a990 2024-04-26T22:55:27.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.618+0000 7f029fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f02a0114b60 con 0x7f02a0108ed0 2024-04-26T22:55:27.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.618+0000 7f029f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a010a990 0x7f02a01a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:27.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.618+0000 7f029ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a0108ed0 0x7f02a019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:27.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.618+0000 7f029f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a010a990 0x7f02a01a6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:52174/0 (socket says 172.21.15.27:52174) 2024-04-26T22:55:27.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.618+0000 7f0297e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02a0109fd0 0x7f02a019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:27.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.618+0000 7f029f400640 1 -- 172.21.15.27:0/1487378697 learned_addr learned my addr 172.21.15.27:0/1487378697 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:27.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.619+0000 7f029f400640 1 -- 172.21.15.27:0/1487378697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a0108ed0 msgr2=0x7f02a019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:27.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.619+0000 7f029f400640 1 --2- 172.21.15.27:0/1487378697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a0108ed0 0x7f02a019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:27.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.619+0000 7f029f400640 1 -- 172.21.15.27:0/1487378697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02a0109fd0 msgr2=0x7f02a019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:27.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.619+0000 7f029f400640 1 --2- 172.21.15.27:0/1487378697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02a0109fd0 0x7f02a019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:27.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.619+0000 7f029f400640 1 -- 172.21.15.27:0/1487378697 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a01a6d00 con 0x7f02a010a990 2024-04-26T22:55:27.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.619+0000 7f029ea00640 1 --2- 172.21.15.27:0/1487378697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a0108ed0 0x7f02a019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:27.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.619+0000 7f029f400640 1 --2- 172.21.15.27:0/1487378697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a010a990 0x7f02a01a6690 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f029800efc0 tx=0x7f029800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:27.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.619+0000 7f02a4a00640 1 -- 172.21.15.27:0/1487378697 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0298003970 con 0x7f02a010a990 2024-04-26T22:55:27.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.619+0000 7f02a4a00640 1 -- 172.21.15.27:0/1487378697 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0298004020 con 0x7f02a010a990 2024-04-26T22:55:27.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.619+0000 7f029fe00640 1 -- 172.21.15.27:0/1487378697 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a01a6ff0 con 0x7f02a010a990 2024-04-26T22:55:27.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.619+0000 7f02a4a00640 1 -- 172.21.15.27:0/1487378697 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029800b940 con 0x7f02a010a990 2024-04-26T22:55:27.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.619+0000 7f029fe00640 1 -- 172.21.15.27:0/1487378697 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a01a7460 con 0x7f02a010a990 2024-04-26T22:55:27.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.621+0000 7f029fe00640 1 -- 172.21.15.27:0/1487378697 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a0110230 con 0x7f02a010a990 2024-04-26T22:55:27.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.623+0000 7f02a4a00640 1 -- 172.21.15.27:0/1487378697 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f029801d020 con 0x7f02a010a990 2024-04-26T22:55:27.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.624+0000 7f02a4a00640 1 --2- 172.21.15.27:0/1487378697 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f02840a3ba0 0x7f02840a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:27.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.624+0000 7f02a4a00640 1 -- 172.21.15.27:0/1487378697 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7f02980d05a0 con 0x7f02a010a990 2024-04-26T22:55:27.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.625+0000 7f029ea00640 1 --2- 172.21.15.27:0/1487378697 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f02840a3ba0 0x7f02840a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:27.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.626+0000 7f029ea00640 1 --2- 172.21.15.27:0/1487378697 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f02840a3ba0 0x7f02840a6060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0290009fc0 tx=0x7f0290005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:27.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.627+0000 7f02a4a00640 1 -- 172.21.15.27:0/1487378697 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f029809cef0 con 0x7f02a010a990 2024-04-26T22:55:27.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:27.834+0000 7f029fe00640 1 -- 172.21.15.27:0/1487378697 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f02a002dad0 con 0x7f02a010a990 2024-04-26T22:55:28.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:28 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] up:active 2024-04-26T22:55:28.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:28 smithi080 ceph-mon[32324]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:55:28.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:28 smithi080 ceph-mon[32324]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:55:28.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:28 smithi080 ceph-mon[32324]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T22:55:28.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:28 smithi080 ceph-mon[32324]: Cluster is now healthy 2024-04-26T22:55:28.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:28 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3481873185' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-26T22:55:28.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:28 smithi080 ceph-mon[32324]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T22:55:28.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:28 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1487378697' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T22:55:28.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:28 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T22:55:28.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:28 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] up:active 2024-04-26T22:55:28.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:28 smithi094 ceph-mon[32889]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:55:28.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:28 smithi094 ceph-mon[32889]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:55:28.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:28 smithi094 ceph-mon[32889]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T22:55:28.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:28 smithi094 ceph-mon[32889]: Cluster is now healthy 2024-04-26T22:55:28.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:28 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3481873185' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-26T22:55:28.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:28 smithi094 ceph-mon[32889]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T22:55:28.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:28 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1487378697' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T22:55:28.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:28 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T22:55:28.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:28 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] up:active 2024-04-26T22:55:28.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:28 smithi027 ceph-mon[28034]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:55:28.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:28 smithi027 ceph-mon[28034]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:55:28.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:28 smithi027 ceph-mon[28034]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T22:55:28.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:28 smithi027 ceph-mon[28034]: Cluster is now healthy 2024-04-26T22:55:28.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:28 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3481873185' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-26T22:55:28.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:28 smithi027 ceph-mon[28034]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T22:55:28.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:28 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1487378697' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T22:55:28.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:28 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T22:55:29.271 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 22:55:28 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-b[63021]: 2024-04-26T22:55:28.919+0000 7f6a2c29f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:55:29.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.394+0000 7f02a4a00640 1 -- 172.21.15.27:0/1487378697 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7f029809c7f0 con 0x7f02a010a990 2024-04-26T22:55:29.395 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data_ec' created 2024-04-26T22:55:29.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.401+0000 7f029fe00640 1 -- 172.21.15.27:0/1487378697 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f02840a3ba0 msgr2=0x7f02840a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:29.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.401+0000 7f029fe00640 1 --2- 172.21.15.27:0/1487378697 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f02840a3ba0 0x7f02840a6060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0290009fc0 tx=0x7f0290005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:29.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.401+0000 7f029fe00640 1 -- 172.21.15.27:0/1487378697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a010a990 msgr2=0x7f02a01a6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:29.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.401+0000 7f029fe00640 1 --2- 172.21.15.27:0/1487378697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a010a990 0x7f02a01a6690 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f029800efc0 tx=0x7f029800c800 comp rx=0 tx=0).stop 2024-04-26T22:55:29.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.402+0000 7f029fe00640 1 -- 172.21.15.27:0/1487378697 shutdown_connections 2024-04-26T22:55:29.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.402+0000 7f029fe00640 1 --2- 172.21.15.27:0/1487378697 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f02840a3ba0 0x7f02840a6060 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:29.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.402+0000 7f029fe00640 1 --2- 172.21.15.27:0/1487378697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a010a990 0x7f02a01a6690 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:29.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.402+0000 7f029fe00640 1 --2- 172.21.15.27:0/1487378697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02a0109fd0 0x7f02a019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:29.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.402+0000 7f029fe00640 1 --2- 172.21.15.27:0/1487378697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a0108ed0 0x7f02a019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:29.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.402+0000 7f029fe00640 1 -- 172.21.15.27:0/1487378697 >> 172.21.15.27:0/1487378697 conn(0x7f02a0077e20 msgr2=0x7f02a010d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:29.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.404+0000 7f029fe00640 1 -- 172.21.15.27:0/1487378697 shutdown_connections 2024-04-26T22:55:29.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.404+0000 7f029fe00640 1 -- 172.21.15.27:0/1487378697 wait complete. 2024-04-26T22:55:29.431 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-26T22:55:29.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.622+0000 7f7577200640 1 -- 172.21.15.27:0/373902130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7570100a70 msgr2=0x7f7570100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:29.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.622+0000 7f7577200640 1 --2- 172.21.15.27:0/373902130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7570100a70 0x7f7570100e90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7564009930 tx=0x7f756402a420 comp rx=0 tx=0).stop 2024-04-26T22:55:29.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.622+0000 7f7577200640 1 -- 172.21.15.27:0/373902130 shutdown_connections 2024-04-26T22:55:29.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.622+0000 7f7577200640 1 --2- 172.21.15.27:0/373902130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7570103a40 0x7f7570103ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:29.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.622+0000 7f7577200640 1 --2- 172.21.15.27:0/373902130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7570104400 0x7f757010ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:29.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.622+0000 7f7577200640 1 --2- 172.21.15.27:0/373902130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7570100a70 0x7f7570100e90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:29.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.622+0000 7f7577200640 1 -- 172.21.15.27:0/373902130 >> 172.21.15.27:0/373902130 conn(0x7f75700fc3c0 msgr2=0x7f75700fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:29.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.622+0000 7f7577200640 1 -- 172.21.15.27:0/373902130 shutdown_connections 2024-04-26T22:55:29.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.622+0000 7f7577200640 1 -- 172.21.15.27:0/373902130 wait complete. 2024-04-26T22:55:29.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.623+0000 7f7577200640 1 Processor -- start 2024-04-26T22:55:29.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.624+0000 7f7577200640 1 -- start start 2024-04-26T22:55:29.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.624+0000 7f7577200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7570100a70 0x7f757019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:29.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.624+0000 7f7577200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7570103a40 0x7f757019b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:29.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.624+0000 7f7577200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7570104400 0x7f75701a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:29.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.624+0000 7f7577200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f757010e320 con 0x7f7570103a40 2024-04-26T22:55:29.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.624+0000 7f7577200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f757010e1a0 con 0x7f7570100a70 2024-04-26T22:55:29.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.624+0000 7f7577200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f757010e4a0 con 0x7f7570104400 2024-04-26T22:55:29.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.624+0000 7f7575400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7570103a40 0x7f757019b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.624+0000 7f7575e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7570100a70 0x7f757019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.624+0000 7f7575400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7570103a40 0x7f757019b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60792/0 (socket says 172.21.15.27:60792) 2024-04-26T22:55:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.624+0000 7f7575e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7570100a70 0x7f757019ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:54138/0 (socket says 172.21.15.27:54138) 2024-04-26T22:55:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.624+0000 7f7575400640 1 -- 172.21.15.27:0/1352984690 learned_addr learned my addr 172.21.15.27:0/1352984690 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.625+0000 7f7576800640 1 --2- 172.21.15.27:0/1352984690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7570104400 0x7f75701a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.625+0000 7f7575400640 1 -- 172.21.15.27:0/1352984690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7570104400 msgr2=0x7f75701a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.625+0000 7f7575400640 1 --2- 172.21.15.27:0/1352984690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7570104400 0x7f75701a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.625+0000 7f7575400640 1 -- 172.21.15.27:0/1352984690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7570100a70 msgr2=0x7f757019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.625+0000 7f7575400640 1 --2- 172.21.15.27:0/1352984690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7570100a70 0x7f757019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.625+0000 7f7575400640 1 -- 172.21.15.27:0/1352984690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75701a2890 con 0x7f7570103a40 2024-04-26T22:55:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.625+0000 7f7575e00640 1 --2- 172.21.15.27:0/1352984690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7570100a70 0x7f757019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.625+0000 7f7575400640 1 --2- 172.21.15.27:0/1352984690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7570103a40 0x7f757019b1c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f756000e9f0 tx=0x7f756000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.626+0000 7f756aa00640 1 -- 172.21.15.27:0/1352984690 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756000ccc0 con 0x7f7570103a40 2024-04-26T22:55:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.626+0000 7f756aa00640 1 -- 172.21.15.27:0/1352984690 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75600108e0 con 0x7f7570103a40 2024-04-26T22:55:29.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.626+0000 7f7577200640 1 -- 172.21.15.27:0/1352984690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75701a2b80 con 0x7f7570103a40 2024-04-26T22:55:29.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.626+0000 7f756aa00640 1 -- 172.21.15.27:0/1352984690 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560018880 con 0x7f7570103a40 2024-04-26T22:55:29.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.626+0000 7f7577200640 1 -- 172.21.15.27:0/1352984690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75701a30f0 con 0x7f7570103a40 2024-04-26T22:55:29.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.627+0000 7f7577200640 1 -- 172.21.15.27:0/1352984690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75701a32c0 con 0x7f7570103a40 2024-04-26T22:55:29.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.631+0000 7f756aa00640 1 -- 172.21.15.27:0/1352984690 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7560010430 con 0x7f7570103a40 2024-04-26T22:55:29.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.632+0000 7f756aa00640 1 --2- 172.21.15.27:0/1352984690 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f75580a3c30 0x7f75580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:29.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.633+0000 7f756aa00640 1 -- 172.21.15.27:0/1352984690 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f75600d0ef0 con 0x7f7570103a40 2024-04-26T22:55:29.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.633+0000 7f7575e00640 1 --2- 172.21.15.27:0/1352984690 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f75580a3c30 0x7f75580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:29.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.634+0000 7f7575e00640 1 --2- 172.21.15.27:0/1352984690 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f75580a3c30 0x7f75580a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7564009900 tx=0x7f7564005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:29.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.634+0000 7f756aa00640 1 -- 172.21.15.27:0/1352984690 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f756009d650 con 0x7f7570103a40 2024-04-26T22:55:29.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:29 smithi080 ceph-mon[32324]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T22:55:29.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:29 smithi080 ceph-mon[32324]: pgmap v329: 97 pgs: 59 active+clean, 20 creating+peering, 18 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T22:55:29.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:29 smithi094 ceph-mon[32889]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T22:55:29.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:29 smithi094 ceph-mon[32889]: pgmap v329: 97 pgs: 59 active+clean, 20 creating+peering, 18 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T22:55:29.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:29.845+0000 7f7577200640 1 -- 172.21.15.27:0/1352984690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f75701a3720 con 0x7f7570103a40 2024-04-26T22:55:29.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:29 smithi027 ceph-mon[28034]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T22:55:29.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:29 smithi027 ceph-mon[28034]: pgmap v329: 97 pgs: 59 active+clean, 20 creating+peering, 18 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T22:55:30.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.390+0000 7f756aa00640 1 -- 172.21.15.27:0/1352984690 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7f756009cf50 con 0x7f7570103a40 2024-04-26T22:55:30.391 INFO:teuthology.orchestra.run.smithi027.stderr:set pool 4 allow_ec_overwrites to true 2024-04-26T22:55:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.395+0000 7f7577200640 1 -- 172.21.15.27:0/1352984690 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f75580a3c30 msgr2=0x7f75580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.395+0000 7f7577200640 1 --2- 172.21.15.27:0/1352984690 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f75580a3c30 0x7f75580a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7564009900 tx=0x7f7564005a20 comp rx=0 tx=0).stop 2024-04-26T22:55:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.395+0000 7f7577200640 1 -- 172.21.15.27:0/1352984690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7570103a40 msgr2=0x7f757019b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.395+0000 7f7577200640 1 --2- 172.21.15.27:0/1352984690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7570103a40 0x7f757019b1c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f756000e9f0 tx=0x7f756000edf0 comp rx=0 tx=0).stop 2024-04-26T22:55:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.395+0000 7f7577200640 1 -- 172.21.15.27:0/1352984690 shutdown_connections 2024-04-26T22:55:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.395+0000 7f7577200640 1 --2- 172.21.15.27:0/1352984690 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f75580a3c30 0x7f75580a60f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.395+0000 7f7577200640 1 --2- 172.21.15.27:0/1352984690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7570104400 0x7f75701a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.395+0000 7f7577200640 1 --2- 172.21.15.27:0/1352984690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7570103a40 0x7f757019b1c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.395+0000 7f7577200640 1 --2- 172.21.15.27:0/1352984690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7570100a70 0x7f757019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.396+0000 7f7577200640 1 -- 172.21.15.27:0/1352984690 >> 172.21.15.27:0/1352984690 conn(0x7f75700fc3c0 msgr2=0x7f7570101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.396+0000 7f7577200640 1 -- 172.21.15.27:0/1352984690 shutdown_connections 2024-04-26T22:55:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.397+0000 7f7577200640 1 -- 172.21.15.27:0/1352984690 wait complete. 2024-04-26T22:55:30.438 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-26T22:55:30.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.622+0000 7fbbe3e00640 1 -- 172.21.15.27:0/3407021609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbdc104b90 msgr2=0x7fbbdc104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:30.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.622+0000 7fbbe3e00640 1 --2- 172.21.15.27:0/3407021609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbdc104b90 0x7fbbdc104fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009860 tx=0x7fbbd002a340 comp rx=0 tx=0).stop 2024-04-26T22:55:30.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.622+0000 7fbbe3e00640 1 -- 172.21.15.27:0/3407021609 shutdown_connections 2024-04-26T22:55:30.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.622+0000 7fbbe3e00640 1 --2- 172.21.15.27:0/3407021609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbdc106740 0x7fbbdc10cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:30.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.622+0000 7fbbe3e00640 1 --2- 172.21.15.27:0/3407021609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbdc105d80 0x7fbbdc106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:30.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.622+0000 7fbbe3e00640 1 --2- 172.21.15.27:0/3407021609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbdc104b90 0x7fbbdc104fb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:30.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.622+0000 7fbbe3e00640 1 -- 172.21.15.27:0/3407021609 >> 172.21.15.27:0/3407021609 conn(0x7fbbdc100120 msgr2=0x7fbbdc102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:30.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.622+0000 7fbbe3e00640 1 -- 172.21.15.27:0/3407021609 shutdown_connections 2024-04-26T22:55:30.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.622+0000 7fbbe3e00640 1 -- 172.21.15.27:0/3407021609 wait complete. 2024-04-26T22:55:30.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.623+0000 7fbbe3e00640 1 Processor -- start 2024-04-26T22:55:30.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.624+0000 7fbbe3e00640 1 -- start start 2024-04-26T22:55:30.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.624+0000 7fbbe3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbdc104b90 0x7fbbdc19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:30.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.624+0000 7fbbe3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbdc105d80 0x7fbbdc19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:30.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.624+0000 7fbbe3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbdc106740 0x7fbbdc1a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:30.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.624+0000 7fbbe3e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc110690 con 0x7fbbdc104b90 2024-04-26T22:55:30.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.624+0000 7fbbe3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc110510 con 0x7fbbdc106740 2024-04-26T22:55:30.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.624+0000 7fbbe3e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc110810 con 0x7fbbdc105d80 2024-04-26T22:55:30.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.624+0000 7fbbe1400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbdc104b90 0x7fbbdc19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:30.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.624+0000 7fbbe1400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbdc104b90 0x7fbbdc19ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60812/0 (socket says 172.21.15.27:60812) 2024-04-26T22:55:30.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.624+0000 7fbbe1400640 1 -- 172.21.15.27:0/606077817 learned_addr learned my addr 172.21.15.27:0/606077817 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:30.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.624+0000 7fbbe0a00640 1 --2- 172.21.15.27:0/606077817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbdc105d80 0x7fbbdc19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:30.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.624+0000 7fbbe1e00640 1 --2- 172.21.15.27:0/606077817 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbdc106740 0x7fbbdc1a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:30.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.625+0000 7fbbe0a00640 1 -- 172.21.15.27:0/606077817 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbdc106740 msgr2=0x7fbbdc1a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:30.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.625+0000 7fbbe0a00640 1 --2- 172.21.15.27:0/606077817 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbdc106740 0x7fbbdc1a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:30.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.625+0000 7fbbe0a00640 1 -- 172.21.15.27:0/606077817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbdc104b90 msgr2=0x7fbbdc19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:30.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.625+0000 7fbbe0a00640 1 --2- 172.21.15.27:0/606077817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbdc104b90 0x7fbbdc19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:30.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.625+0000 7fbbe0a00640 1 -- 172.21.15.27:0/606077817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbdc1a2a30 con 0x7fbbdc105d80 2024-04-26T22:55:30.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.625+0000 7fbbe0a00640 1 --2- 172.21.15.27:0/606077817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbdc105d80 0x7fbbdc19b350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800e9f0 tx=0x7fbbc800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:30.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.625+0000 7fbbda000640 1 -- 172.21.15.27:0/606077817 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc800cd80 con 0x7fbbdc105d80 2024-04-26T22:55:30.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.626+0000 7fbbe3e00640 1 -- 172.21.15.27:0/606077817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbdc1a2d20 con 0x7fbbdc105d80 2024-04-26T22:55:30.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.626+0000 7fbbda000640 1 -- 172.21.15.27:0/606077817 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc80035f0 con 0x7fbbdc105d80 2024-04-26T22:55:30.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.626+0000 7fbbda000640 1 -- 172.21.15.27:0/606077817 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8010690 con 0x7fbbdc105d80 2024-04-26T22:55:30.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.626+0000 7fbbe3e00640 1 -- 172.21.15.27:0/606077817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbdc1a3210 con 0x7fbbdc105d80 2024-04-26T22:55:30.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.627+0000 7fbbe3e00640 1 -- 172.21.15.27:0/606077817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbdc10bf80 con 0x7fbbdc105d80 2024-04-26T22:55:30.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.629+0000 7fbbda000640 1 -- 172.21.15.27:0/606077817 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbbc80037a0 con 0x7fbbdc105d80 2024-04-26T22:55:30.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.631+0000 7fbbda000640 1 --2- 172.21.15.27:0/606077817 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbbc40a3bb0 0x7fbbc40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:30.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.631+0000 7fbbda000640 1 -- 172.21.15.27:0/606077817 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7fbbc80d08b0 con 0x7fbbdc105d80 2024-04-26T22:55:30.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.631+0000 7fbbe1400640 1 --2- 172.21.15.27:0/606077817 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbbc40a3bb0 0x7fbbc40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:30.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.632+0000 7fbbe1400640 1 --2- 172.21.15.27:0/606077817 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbbc40a3bb0 0x7fbbc40a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00095f0 tx=0x7fbbd0030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:30.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.635+0000 7fbbda000640 1 -- 172.21.15.27:0/606077817 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbbc809d010 con 0x7fbbdc105d80 2024-04-26T22:55:30.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:30 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T22:55:30.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:30 smithi080 ceph-mon[32324]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T22:55:30.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:30 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1352984690' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-26T22:55:30.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:30 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T22:55:30.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:30 smithi094 ceph-mon[32889]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T22:55:30.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:30 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1352984690' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-26T22:55:30.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:30 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T22:55:30.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:30 smithi027 ceph-mon[28034]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T22:55:30.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:30 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1352984690' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-26T22:55:30.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:30.883+0000 7fbbe3e00640 1 -- 172.21.15.27:0/606077817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fbbdc0635d0 con 0x7fbbdc105d80 2024-04-26T22:55:31.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.422+0000 7fbbda000640 1 -- 172.21.15.27:0/606077817 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fbbc809c910 con 0x7fbbdc105d80 2024-04-26T22:55:31.423 INFO:teuthology.orchestra.run.smithi027.stderr:added data pool 4 to fsmap 2024-04-26T22:55:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.427+0000 7fbbe3e00640 1 -- 172.21.15.27:0/606077817 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbbc40a3bb0 msgr2=0x7fbbc40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.427+0000 7fbbe3e00640 1 --2- 172.21.15.27:0/606077817 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbbc40a3bb0 0x7fbbc40a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00095f0 tx=0x7fbbd0030450 comp rx=0 tx=0).stop 2024-04-26T22:55:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.427+0000 7fbbe3e00640 1 -- 172.21.15.27:0/606077817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbdc105d80 msgr2=0x7fbbdc19b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.427+0000 7fbbe3e00640 1 --2- 172.21.15.27:0/606077817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbdc105d80 0x7fbbdc19b350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800e9f0 tx=0x7fbbc800eeb0 comp rx=0 tx=0).stop 2024-04-26T22:55:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.427+0000 7fbbe3e00640 1 -- 172.21.15.27:0/606077817 shutdown_connections 2024-04-26T22:55:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.427+0000 7fbbe3e00640 1 --2- 172.21.15.27:0/606077817 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbbc40a3bb0 0x7fbbc40a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.427+0000 7fbbe3e00640 1 --2- 172.21.15.27:0/606077817 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbdc106740 0x7fbbdc1a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.427+0000 7fbbe3e00640 1 --2- 172.21.15.27:0/606077817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbdc105d80 0x7fbbdc19b350 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.427+0000 7fbbe3e00640 1 --2- 172.21.15.27:0/606077817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbdc104b90 0x7fbbdc19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.427+0000 7fbbe3e00640 1 -- 172.21.15.27:0/606077817 >> 172.21.15.27:0/606077817 conn(0x7fbbdc100120 msgr2=0x7fbbdc109150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.427+0000 7fbbe3e00640 1 -- 172.21.15.27:0/606077817 shutdown_connections 2024-04-26T22:55:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.427+0000 7fbbe3e00640 1 -- 172.21.15.27:0/606077817 wait complete. 2024-04-26T22:55:31.450 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:55:31.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.670+0000 7fbc4f200640 1 -- 172.21.15.27:0/2523745600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48104990 msgr2=0x7fbc48104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:31.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.670+0000 7fbc4f200640 1 --2- 172.21.15.27:0/2523745600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48104990 0x7fbc48104db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009860 tx=0x7fbc3c02a340 comp rx=0 tx=0).stop 2024-04-26T22:55:31.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.670+0000 7fbc4f200640 1 -- 172.21.15.27:0/2523745600 shutdown_connections 2024-04-26T22:55:31.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.670+0000 7fbc4f200640 1 --2- 172.21.15.27:0/2523745600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc48106540 0x7fbc4810cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:31.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.670+0000 7fbc4f200640 1 --2- 172.21.15.27:0/2523745600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc48105b80 0x7fbc48106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:31.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.670+0000 7fbc4f200640 1 --2- 172.21.15.27:0/2523745600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48104990 0x7fbc48104db0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:31.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.670+0000 7fbc4f200640 1 -- 172.21.15.27:0/2523745600 >> 172.21.15.27:0/2523745600 conn(0x7fbc480fff00 msgr2=0x7fbc48102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:31.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.670+0000 7fbc4f200640 1 -- 172.21.15.27:0/2523745600 shutdown_connections 2024-04-26T22:55:31.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.670+0000 7fbc4f200640 1 -- 172.21.15.27:0/2523745600 wait complete. 2024-04-26T22:55:31.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.671+0000 7fbc4f200640 1 Processor -- start 2024-04-26T22:55:31.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.672+0000 7fbc4f200640 1 -- start start 2024-04-26T22:55:31.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.672+0000 7fbc4f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc48104990 0x7fbc4819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:31.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.672+0000 7fbc4f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48105b80 0x7fbc4819b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:31.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.672+0000 7fbc4f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc48106540 0x7fbc481a2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:31.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.672+0000 7fbc4f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481104c0 con 0x7fbc48106540 2024-04-26T22:55:31.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.672+0000 7fbc4f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbc48110340 con 0x7fbc48104990 2024-04-26T22:55:31.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.672+0000 7fbc4f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc48110640 con 0x7fbc48105b80 2024-04-26T22:55:31.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.672+0000 7fbc4de00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc48104990 0x7fbc4819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:31.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.673+0000 7fbc4e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc48106540 0x7fbc481a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:31.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.673+0000 7fbc4d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48105b80 0x7fbc4819b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:31.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.673+0000 7fbc4d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48105b80 0x7fbc4819b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:39758/0 (socket says 172.21.15.27:39758) 2024-04-26T22:55:31.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.673+0000 7fbc4d400640 1 -- 172.21.15.27:0/4159805101 learned_addr learned my addr 172.21.15.27:0/4159805101 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:31.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.673+0000 7fbc4d400640 1 -- 172.21.15.27:0/4159805101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc48104990 msgr2=0x7fbc4819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:31.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.673+0000 7fbc4d400640 1 --2- 172.21.15.27:0/4159805101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc48104990 0x7fbc4819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:31.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.673+0000 7fbc4d400640 1 -- 172.21.15.27:0/4159805101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc48106540 msgr2=0x7fbc481a2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:31.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.673+0000 7fbc4d400640 1 --2- 172.21.15.27:0/4159805101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc48106540 0x7fbc481a2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:31.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.673+0000 7fbc4d400640 1 -- 172.21.15.27:0/4159805101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc481a26f0 con 0x7fbc48105b80 2024-04-26T22:55:31.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.673+0000 7fbc4d400640 1 --2- 172.21.15.27:0/4159805101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48105b80 0x7fbc4819b010 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800e9c0 tx=0x7fbc3800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:31.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.673+0000 7fbc42a00640 1 -- 172.21.15.27:0/4159805101 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3800ccc0 con 0x7fbc48105b80 2024-04-26T22:55:31.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.674+0000 7fbc42a00640 1 -- 172.21.15.27:0/4159805101 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc380054d0 con 0x7fbc48105b80 2024-04-26T22:55:31.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.674+0000 7fbc4f200640 1 -- 172.21.15.27:0/4159805101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc481a29e0 con 0x7fbc48105b80 2024-04-26T22:55:31.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.674+0000 7fbc42a00640 1 -- 172.21.15.27:0/4159805101 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38010430 con 0x7fbc48105b80 2024-04-26T22:55:31.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.674+0000 7fbc4f200640 1 -- 172.21.15.27:0/4159805101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc481a2f80 con 0x7fbc48105b80 2024-04-26T22:55:31.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.675+0000 7fbc4f200640 1 -- 172.21.15.27:0/4159805101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc4810bcb0 con 0x7fbc48105b80 2024-04-26T22:55:31.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.677+0000 7fbc42a00640 1 -- 172.21.15.27:0/4159805101 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbc38004f80 con 0x7fbc48105b80 2024-04-26T22:55:31.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.678+0000 7fbc42a00640 1 --2- 172.21.15.27:0/4159805101 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbc300a3cd0 0x7fbc300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:31.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.678+0000 7fbc42a00640 1 -- 172.21.15.27:0/4159805101 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbc380d09d0 con 0x7fbc48105b80 2024-04-26T22:55:31.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.679+0000 7fbc4de00640 1 --2- 172.21.15.27:0/4159805101 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbc300a3cd0 0x7fbc300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:31.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.679+0000 7fbc4de00640 1 --2- 172.21.15.27:0/4159805101 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbc300a3cd0 0x7fbc300a6190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c02a930 tx=0x7fbc3c030470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:31.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.683+0000 7fbc42a00640 1 -- 172.21.15.27:0/4159805101 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbc3809d110 con 0x7fbc48105b80 2024-04-26T22:55:31.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:31 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1352984690' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-26T22:55:31.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:31 smithi080 ceph-mon[32324]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T22:55:31.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:31 smithi080 ceph-mon[32324]: pgmap v332: 129 pgs: 9 creating+peering, 97 active+clean, 23 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 3 op/s 2024-04-26T22:55:31.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:31 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/606077817' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T22:55:31.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:31 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T22:55:31.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:31 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1352984690' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-26T22:55:31.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:31 smithi094 ceph-mon[32889]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T22:55:31.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:31 smithi094 ceph-mon[32889]: pgmap v332: 129 pgs: 9 creating+peering, 97 active+clean, 23 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 3 op/s 2024-04-26T22:55:31.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:31 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/606077817' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T22:55:31.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:31 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T22:55:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:31 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1352984690' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-26T22:55:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:31 smithi027 ceph-mon[28034]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T22:55:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:31 smithi027 ceph-mon[28034]: pgmap v332: 129 pgs: 9 creating+peering, 97 active+clean, 23 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 3 op/s 2024-04-26T22:55:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:31 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/606077817' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T22:55:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:31 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T22:55:31.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.930+0000 7fbc4f200640 1 -- 172.21.15.27:0/4159805101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc4802dad0 con 0x7fbc48105b80 2024-04-26T22:55:31.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.931+0000 7fbc42a00640 1 -- 172.21.15.27:0/4159805101 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11779 (secure 0 0 0) 0x7fbc3809ca10 con 0x7fbc48105b80 2024-04-26T22:55:31.956 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:31.956 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/1620677319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1620677319},{"type":"v1","addr":"172.21.15.94:6839","nonce":1620677319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1571355196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1571355196},{"type":"v1","addr":"172.21.15.27:6841","nonce":1571355196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3193530521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3193530521},{"type":"v1","addr":"172.21.15.27:6839","nonce":3193530521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:55:25.265331+0000","modified":"2024-04-26T22:55:30.887610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/95952000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":95952000},{"type":"v1","addr":"172.21.15.80:6839","nonce":95952000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24578,"qdb_cluster":[24578]},"id":1}]} 2024-04-26T22:55:31.958 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-04-26T22:55:31.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.934+0000 7fbc4f200640 1 -- 172.21.15.27:0/4159805101 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbc300a3cd0 msgr2=0x7fbc300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:31.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.934+0000 7fbc4f200640 1 --2- 172.21.15.27:0/4159805101 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbc300a3cd0 0x7fbc300a6190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c02a930 tx=0x7fbc3c030470 comp rx=0 tx=0).stop 2024-04-26T22:55:31.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.934+0000 7fbc4f200640 1 -- 172.21.15.27:0/4159805101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48105b80 msgr2=0x7fbc4819b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:31.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.934+0000 7fbc4f200640 1 --2- 172.21.15.27:0/4159805101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48105b80 0x7fbc4819b010 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800e9c0 tx=0x7fbc3800ee80 comp rx=0 tx=0).stop 2024-04-26T22:55:31.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.934+0000 7fbc4f200640 1 -- 172.21.15.27:0/4159805101 shutdown_connections 2024-04-26T22:55:31.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.934+0000 7fbc4f200640 1 --2- 172.21.15.27:0/4159805101 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbc300a3cd0 0x7fbc300a6190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:31.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.934+0000 7fbc4f200640 1 --2- 172.21.15.27:0/4159805101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc48106540 0x7fbc481a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:31.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.934+0000 7fbc4f200640 1 --2- 172.21.15.27:0/4159805101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48105b80 0x7fbc4819b010 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:31.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.934+0000 7fbc4f200640 1 --2- 172.21.15.27:0/4159805101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc48104990 0x7fbc4819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:31.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.934+0000 7fbc4f200640 1 -- 172.21.15.27:0/4159805101 >> 172.21.15.27:0/4159805101 conn(0x7fbc480fff00 msgr2=0x7fbc48108e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:31.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.934+0000 7fbc4f200640 1 -- 172.21.15.27:0/4159805101 shutdown_connections 2024-04-26T22:55:31.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:31.934+0000 7fbc4f200640 1 -- 172.21.15.27:0/4159805101 wait complete. 2024-04-26T22:55:31.959 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:55:32.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.177+0000 7f122d400640 1 -- 172.21.15.27:0/2582921761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1228106540 msgr2=0x7f122810cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:32.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.177+0000 7f122d400640 1 --2- 172.21.15.27:0/2582921761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1228106540 0x7f122810cdd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1220009ca0 tx=0x7f122002a8c0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.178+0000 7f122d400640 1 -- 172.21.15.27:0/2582921761 shutdown_connections 2024-04-26T22:55:32.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.178+0000 7f122d400640 1 --2- 172.21.15.27:0/2582921761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1228106540 0x7f122810cdd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.178+0000 7f122d400640 1 --2- 172.21.15.27:0/2582921761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1228105b80 0x7f1228106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.178+0000 7f122d400640 1 --2- 172.21.15.27:0/2582921761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1228104990 0x7f1228104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.178+0000 7f122d400640 1 -- 172.21.15.27:0/2582921761 >> 172.21.15.27:0/2582921761 conn(0x7f12280fff00 msgr2=0x7f1228102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:32.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.178+0000 7f122d400640 1 -- 172.21.15.27:0/2582921761 shutdown_connections 2024-04-26T22:55:32.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.178+0000 7f122d400640 1 -- 172.21.15.27:0/2582921761 wait complete. 2024-04-26T22:55:32.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.179+0000 7f122d400640 1 Processor -- start 2024-04-26T22:55:32.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.179+0000 7f122d400640 1 -- start start 2024-04-26T22:55:32.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.179+0000 7f122d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1228104990 0x7f122819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:32.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.179+0000 7f122d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1228105b80 0x7f122819af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:32.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.179+0000 7f122d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1228106540 0x7f12281a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:32.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.179+0000 7f122d400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f12281104f0 con 0x7f1228105b80 2024-04-26T22:55:32.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.179+0000 7f122d400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1228110370 con 0x7f1228106540 2024-04-26T22:55:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.179+0000 7f122d400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1228110670 con 0x7f1228104990 2024-04-26T22:55:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.179+0000 7f1227400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1228105b80 0x7f122819af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.179+0000 7f1227400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1228105b80 0x7f122819af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60838/0 (socket says 172.21.15.27:60838) 2024-04-26T22:55:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.179+0000 7f1227400640 1 -- 172.21.15.27:0/1865979446 learned_addr learned my addr 172.21.15.27:0/1865979446 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.180+0000 7f1227e00640 1 --2- 172.21.15.27:0/1865979446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1228104990 0x7f122819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.180+0000 7f122ca00640 1 --2- 172.21.15.27:0/1865979446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1228106540 0x7f12281a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.180+0000 7f1227400640 1 -- 172.21.15.27:0/1865979446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1228104990 msgr2=0x7f122819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.180+0000 7f1227400640 1 --2- 172.21.15.27:0/1865979446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1228104990 0x7f122819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.180+0000 7f1227400640 1 -- 172.21.15.27:0/1865979446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1228106540 msgr2=0x7f12281a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.180+0000 7f1227400640 1 --2- 172.21.15.27:0/1865979446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1228106540 0x7f12281a1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.180+0000 7f1227400640 1 -- 172.21.15.27:0/1865979446 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12281a2640 con 0x7f1228105b80 2024-04-26T22:55:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.180+0000 7f1227400640 1 --2- 172.21.15.27:0/1865979446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1228105b80 0x7f122819af60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f121400ea60 tx=0x7f121400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.180+0000 7f1224c00640 1 -- 172.21.15.27:0/1865979446 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121400cde0 con 0x7f1228105b80 2024-04-26T22:55:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.181+0000 7f1224c00640 1 -- 172.21.15.27:0/1865979446 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12140035f0 con 0x7f1228105b80 2024-04-26T22:55:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.181+0000 7f122d400640 1 -- 172.21.15.27:0/1865979446 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12281a2930 con 0x7f1228105b80 2024-04-26T22:55:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.181+0000 7f1224c00640 1 -- 172.21.15.27:0/1865979446 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214010690 con 0x7f1228105b80 2024-04-26T22:55:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.181+0000 7f122d400640 1 -- 172.21.15.27:0/1865979446 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12281a2e80 con 0x7f1228105b80 2024-04-26T22:55:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.182+0000 7f122d400640 1 -- 172.21.15.27:0/1865979446 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122810bc50 con 0x7f1228105b80 2024-04-26T22:55:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.185+0000 7f1224c00640 1 -- 172.21.15.27:0/1865979446 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f12140037a0 con 0x7f1228105b80 2024-04-26T22:55:32.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.186+0000 7f1224c00640 1 --2- 172.21.15.27:0/1865979446 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f120c0a3c50 0x7f120c0a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:32.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.186+0000 7f1224c00640 1 -- 172.21.15.27:0/1865979446 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f12140d0930 con 0x7f1228105b80 2024-04-26T22:55:32.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.186+0000 7f1227e00640 1 --2- 172.21.15.27:0/1865979446 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f120c0a3c50 0x7f120c0a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:32.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.187+0000 7f1227e00640 1 --2- 172.21.15.27:0/1865979446 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f120c0a3c50 0x7f120c0a6110 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12180095f0 tx=0x7f1218005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.191+0000 7f1224c00640 1 -- 172.21.15.27:0/1865979446 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f121409d070 con 0x7f1228105b80 2024-04-26T22:55:32.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.397+0000 7f122d400640 1 -- 172.21.15.27:0/1865979446 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1228063530 con 0x7f1228105b80 2024-04-26T22:55:32.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.400+0000 7f1224c00640 1 -- 172.21.15.27:0/1865979446 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24009 (secure 0 0 0) 0x7f121409c970 con 0x7f1228105b80 2024-04-26T22:55:32.401 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:32.402 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:55:30.886974+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:55:32.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.403+0000 7f122d400640 1 -- 172.21.15.27:0/1865979446 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f120c0a3c50 msgr2=0x7f120c0a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:32.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.403+0000 7f122d400640 1 --2- 172.21.15.27:0/1865979446 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f120c0a3c50 0x7f120c0a6110 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12180095f0 tx=0x7f1218005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:32.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.403+0000 7f122d400640 1 -- 172.21.15.27:0/1865979446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1228105b80 msgr2=0x7f122819af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:32.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.403+0000 7f122d400640 1 --2- 172.21.15.27:0/1865979446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1228105b80 0x7f122819af60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f121400ea60 tx=0x7f121400ef20 comp rx=0 tx=0).stop 2024-04-26T22:55:32.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.404+0000 7f122d400640 1 -- 172.21.15.27:0/1865979446 shutdown_connections 2024-04-26T22:55:32.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.404+0000 7f122d400640 1 --2- 172.21.15.27:0/1865979446 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f120c0a3c50 0x7f120c0a6110 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.404+0000 7f122d400640 1 --2- 172.21.15.27:0/1865979446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1228106540 0x7f12281a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.404+0000 7f122d400640 1 --2- 172.21.15.27:0/1865979446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1228105b80 0x7f122819af60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.404+0000 7f122d400640 1 --2- 172.21.15.27:0/1865979446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1228104990 0x7f122819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.404+0000 7f122d400640 1 -- 172.21.15.27:0/1865979446 >> 172.21.15.27:0/1865979446 conn(0x7f12280fff00 msgr2=0x7f1228108e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:32.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.404+0000 7f122d400640 1 -- 172.21.15.27:0/1865979446 shutdown_connections 2024-04-26T22:55:32.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.404+0000 7f122d400640 1 -- 172.21.15.27:0/1865979446 wait complete. 2024-04-26T22:55:32.425 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:55:32.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.636+0000 7ff343e00640 1 -- 172.21.15.27:0/2692504807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff344108450 msgr2=0x7ff34410a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:32.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.636+0000 7ff343e00640 1 --2- 172.21.15.27:0/2692504807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff344108450 0x7ff34410a8a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009860 tx=0x7ff33c02a340 comp rx=0 tx=0).stop 2024-04-26T22:55:32.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.636+0000 7ff343e00640 1 -- 172.21.15.27:0/2692504807 shutdown_connections 2024-04-26T22:55:32.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.636+0000 7ff343e00640 1 --2- 172.21.15.27:0/2692504807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff344108450 0x7ff34410a8a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.636+0000 7ff343e00640 1 --2- 172.21.15.27:0/2692504807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff344105b20 0x7ff344107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.636+0000 7ff343e00640 1 --2- 172.21.15.27:0/2692504807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3441031f0 0x7ff3441055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.636+0000 7ff343e00640 1 -- 172.21.15.27:0/2692504807 >> 172.21.15.27:0/2692504807 conn(0x7ff3440fcc00 msgr2=0x7ff3440ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:32.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.636+0000 7ff343e00640 1 -- 172.21.15.27:0/2692504807 shutdown_connections 2024-04-26T22:55:32.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.636+0000 7ff343e00640 1 -- 172.21.15.27:0/2692504807 wait complete. 2024-04-26T22:55:32.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.637+0000 7ff343e00640 1 Processor -- start 2024-04-26T22:55:32.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.637+0000 7ff343e00640 1 -- start start 2024-04-26T22:55:32.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.638+0000 7ff343e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3441031f0 0x7ff34419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:32.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.638+0000 7ff343e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff344105b20 0x7ff34419af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:32.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.638+0000 7ff343e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff344108450 0x7ff3441a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:32.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.638+0000 7ff343e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff34410e2a0 con 0x7ff344105b20 2024-04-26T22:55:32.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.638+0000 7ff343e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff34410e120 con 0x7ff3441031f0 2024-04-26T22:55:32.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.638+0000 7ff343e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff34410e420 con 0x7ff344108450 2024-04-26T22:55:32.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.638+0000 7ff342000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff344105b20 0x7ff34419af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:32.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.638+0000 7ff342a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3441031f0 0x7ff34419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:32.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.638+0000 7ff342000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff344105b20 0x7ff34419af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60854/0 (socket says 172.21.15.27:60854) 2024-04-26T22:55:32.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.638+0000 7ff342000640 1 -- 172.21.15.27:0/2014543234 learned_addr learned my addr 172.21.15.27:0/2014543234 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:32.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.638+0000 7ff343400640 1 --2- 172.21.15.27:0/2014543234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff344108450 0x7ff3441a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:32.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.638+0000 7ff342000640 1 -- 172.21.15.27:0/2014543234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff344108450 msgr2=0x7ff3441a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:32.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.638+0000 7ff342000640 1 --2- 172.21.15.27:0/2014543234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff344108450 0x7ff3441a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.638+0000 7ff342000640 1 -- 172.21.15.27:0/2014543234 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3441031f0 msgr2=0x7ff34419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:32.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.638+0000 7ff342000640 1 --2- 172.21.15.27:0/2014543234 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3441031f0 0x7ff34419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.638+0000 7ff342000640 1 -- 172.21.15.27:0/2014543234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3441a2670 con 0x7ff344105b20 2024-04-26T22:55:32.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.639+0000 7ff342000640 1 --2- 172.21.15.27:0/2014543234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff344105b20 0x7ff34419af90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00e9c0 tx=0x7ff32c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:32.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.639+0000 7ff33b400640 1 -- 172.21.15.27:0/2014543234 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c00cd80 con 0x7ff344105b20 2024-04-26T22:55:32.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.639+0000 7ff33b400640 1 -- 172.21.15.27:0/2014543234 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff32c0054d0 con 0x7ff344105b20 2024-04-26T22:55:32.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.639+0000 7ff33b400640 1 -- 172.21.15.27:0/2014543234 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c010430 con 0x7ff344105b20 2024-04-26T22:55:32.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.639+0000 7ff343e00640 1 -- 172.21.15.27:0/2014543234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3441a2960 con 0x7ff344105b20 2024-04-26T22:55:32.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.639+0000 7ff343e00640 1 -- 172.21.15.27:0/2014543234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3441a2e60 con 0x7ff344105b20 2024-04-26T22:55:32.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.640+0000 7ff343e00640 1 -- 172.21.15.27:0/2014543234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff344109f40 con 0x7ff344105b20 2024-04-26T22:55:32.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.642+0000 7ff33b400640 1 -- 172.21.15.27:0/2014543234 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff32c004fd0 con 0x7ff344105b20 2024-04-26T22:55:32.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.644+0000 7ff33b400640 1 --2- 172.21.15.27:0/2014543234 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff3280a3cd0 0x7ff3280a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:32.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.644+0000 7ff33b400640 1 -- 172.21.15.27:0/2014543234 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff32c0d09e0 con 0x7ff344105b20 2024-04-26T22:55:32.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.644+0000 7ff342a00640 1 --2- 172.21.15.27:0/2014543234 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff3280a3cd0 0x7ff3280a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:32.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.645+0000 7ff342a00640 1 --2- 172.21.15.27:0/2014543234 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff3280a3cd0 0x7ff3280a6190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff334005fd0 tx=0x7ff334010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:32.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.648+0000 7ff33b400640 1 -- 172.21.15.27:0/2014543234 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff32c09d120 con 0x7ff344105b20 2024-04-26T22:55:32.740 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:32 smithi094 ceph-mon[32889]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T22:55:32.740 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:32 smithi094 ceph-mon[32889]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T22:55:32.740 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:32 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-26T22:55:32.740 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:32 smithi094 ceph-mon[32889]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:55:32.740 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:32 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/4159805101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:32.740 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:32 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1865979446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:32.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:32 smithi080 ceph-mon[32324]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T22:55:32.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:32 smithi080 ceph-mon[32324]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T22:55:32.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:32 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-26T22:55:32.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:32 smithi080 ceph-mon[32324]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:55:32.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:32 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/4159805101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:32.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:32 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1865979446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.863+0000 7ff343e00640 1 -- 172.21.15.27:0/2014543234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff344063560 con 0x7ff344105b20 2024-04-26T22:55:32.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:32 smithi027 ceph-mon[28034]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T22:55:32.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:32 smithi027 ceph-mon[28034]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T22:55:32.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:32 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-26T22:55:32.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:32 smithi027 ceph-mon[28034]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:55:32.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:32 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/4159805101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:32.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:32 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1865979446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.868+0000 7ff33b400640 1 -- 172.21.15.27:0/2014543234 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24009 (secure 0 0 0) 0x7ff32c09ca20 con 0x7ff344105b20 2024-04-26T22:55:32.869 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:32.869 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:55:30.886974+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:55:32.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.870+0000 7ff343e00640 1 -- 172.21.15.27:0/2014543234 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff3280a3cd0 msgr2=0x7ff3280a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:32.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.870+0000 7ff343e00640 1 --2- 172.21.15.27:0/2014543234 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff3280a3cd0 0x7ff3280a6190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff334005fd0 tx=0x7ff334010040 comp rx=0 tx=0).stop 2024-04-26T22:55:32.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.870+0000 7ff343e00640 1 -- 172.21.15.27:0/2014543234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff344105b20 msgr2=0x7ff34419af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:32.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.870+0000 7ff343e00640 1 --2- 172.21.15.27:0/2014543234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff344105b20 0x7ff34419af90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00e9c0 tx=0x7ff32c00ee80 comp rx=0 tx=0).stop 2024-04-26T22:55:32.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.871+0000 7ff343e00640 1 -- 172.21.15.27:0/2014543234 shutdown_connections 2024-04-26T22:55:32.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.871+0000 7ff343e00640 1 --2- 172.21.15.27:0/2014543234 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff3280a3cd0 0x7ff3280a6190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.871+0000 7ff343e00640 1 --2- 172.21.15.27:0/2014543234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff344108450 0x7ff3441a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.871+0000 7ff343e00640 1 --2- 172.21.15.27:0/2014543234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff344105b20 0x7ff34419af90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.871+0000 7ff343e00640 1 --2- 172.21.15.27:0/2014543234 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3441031f0 0x7ff34419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.871+0000 7ff343e00640 1 -- 172.21.15.27:0/2014543234 >> 172.21.15.27:0/2014543234 conn(0x7ff3440fcc00 msgr2=0x7ff344102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.871+0000 7ff343e00640 1 -- 172.21.15.27:0/2014543234 shutdown_connections 2024-04-26T22:55:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:32.871+0000 7ff343e00640 1 -- 172.21.15.27:0/2014543234 wait complete. 2024-04-26T22:55:32.905 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:55:33.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.119+0000 7f8fa7a00640 1 -- 172.21.15.27:0/929921030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa0104b70 msgr2=0x7f8fa0104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:33.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.119+0000 7f8fa7a00640 1 --2- 172.21.15.27:0/929921030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa0104b70 0x7f8fa0104f90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009860 tx=0x7f8f9002a340 comp rx=0 tx=0).stop 2024-04-26T22:55:33.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.119+0000 7f8fa7a00640 1 -- 172.21.15.27:0/929921030 shutdown_connections 2024-04-26T22:55:33.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.119+0000 7f8fa7a00640 1 --2- 172.21.15.27:0/929921030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fa0106720 0x7f8fa010cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.119+0000 7f8fa7a00640 1 --2- 172.21.15.27:0/929921030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fa0105d60 0x7f8fa01061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.119+0000 7f8fa7a00640 1 --2- 172.21.15.27:0/929921030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa0104b70 0x7f8fa0104f90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.119+0000 7f8fa7a00640 1 -- 172.21.15.27:0/929921030 >> 172.21.15.27:0/929921030 conn(0x7f8fa01000e0 msgr2=0x7f8fa0102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:33.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.119+0000 7f8fa7a00640 1 -- 172.21.15.27:0/929921030 shutdown_connections 2024-04-26T22:55:33.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.119+0000 7f8fa7a00640 1 -- 172.21.15.27:0/929921030 wait complete. 2024-04-26T22:55:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.121+0000 7f8fa7a00640 1 Processor -- start 2024-04-26T22:55:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.121+0000 7f8fa7a00640 1 -- start start 2024-04-26T22:55:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.121+0000 7f8fa7a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fa0104b70 0x7f8fa019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.121+0000 7f8fa7a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa0105d60 0x7f8fa019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.121+0000 7f8fa7a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fa0106720 0x7f8fa01a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.121+0000 7f8fa7a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa0110670 con 0x7f8fa0104b70 2024-04-26T22:55:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.121+0000 7f8fa7a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa01104f0 con 0x7f8fa0106720 2024-04-26T22:55:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.121+0000 7f8fa7a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa01107f0 con 0x7f8fa0105d60 2024-04-26T22:55:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.122+0000 7f8fa5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fa0104b70 0x7f8fa019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.122+0000 7f8f97e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa0105d60 0x7f8fa019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.122+0000 7f8f97e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa0105d60 0x7f8fa019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:39808/0 (socket says 172.21.15.27:39808) 2024-04-26T22:55:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.122+0000 7f8f97e00640 1 -- 172.21.15.27:0/2099973835 learned_addr learned my addr 172.21.15.27:0/2099973835 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.122+0000 7f8fa5a00640 1 --2- 172.21.15.27:0/2099973835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fa0106720 0x7f8fa01a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.122+0000 7f8f97e00640 1 -- 172.21.15.27:0/2099973835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fa0106720 msgr2=0x7f8fa01a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.122+0000 7f8f97e00640 1 --2- 172.21.15.27:0/2099973835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fa0106720 0x7f8fa01a2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.122+0000 7f8f97e00640 1 -- 172.21.15.27:0/2099973835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fa0104b70 msgr2=0x7f8fa019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.122+0000 7f8f97e00640 1 --2- 172.21.15.27:0/2099973835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fa0104b70 0x7f8fa019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.122+0000 7f8f97e00640 1 -- 172.21.15.27:0/2099973835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa01a29d0 con 0x7f8fa0105d60 2024-04-26T22:55:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.122+0000 7f8f97e00640 1 --2- 172.21.15.27:0/2099973835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa0105d60 0x7f8fa019b2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800e9c0 tx=0x7f8f8800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.123+0000 7f8f9e000640 1 -- 172.21.15.27:0/2099973835 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8800cde0 con 0x7f8fa0105d60 2024-04-26T22:55:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.123+0000 7f8f9e000640 1 -- 172.21.15.27:0/2099973835 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f880035f0 con 0x7f8fa0105d60 2024-04-26T22:55:33.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.123+0000 7f8f9e000640 1 -- 172.21.15.27:0/2099973835 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88010690 con 0x7f8fa0105d60 2024-04-26T22:55:33.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.123+0000 7f8fa7a00640 1 -- 172.21.15.27:0/2099973835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa01a2cc0 con 0x7f8fa0105d60 2024-04-26T22:55:33.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.123+0000 7f8fa7a00640 1 -- 172.21.15.27:0/2099973835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa01a31c0 con 0x7f8fa0105d60 2024-04-26T22:55:33.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.124+0000 7f8fa7a00640 1 -- 172.21.15.27:0/2099973835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa010bfb0 con 0x7f8fa0105d60 2024-04-26T22:55:33.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.126+0000 7f8f9e000640 1 -- 172.21.15.27:0/2099973835 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8f88004000 con 0x7f8fa0105d60 2024-04-26T22:55:33.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.128+0000 7f8f9e000640 1 --2- 172.21.15.27:0/2099973835 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8f840a3bb0 0x7f8f840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:33.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.128+0000 7f8f9e000640 1 -- 172.21.15.27:0/2099973835 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8f880d0990 con 0x7f8fa0105d60 2024-04-26T22:55:33.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.128+0000 7f8fa5000640 1 --2- 172.21.15.27:0/2099973835 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8f840a3bb0 0x7f8f840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:33.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.129+0000 7f8fa5000640 1 --2- 172.21.15.27:0/2099973835 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8f840a3bb0 0x7f8f840a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009830 tx=0x7f8f90030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:33.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.132+0000 7f8f9e000640 1 -- 172.21.15.27:0/2099973835 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8f88009510 con 0x7f8fa0105d60 2024-04-26T22:55:33.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.381+0000 7f8fa7a00640 1 -- 172.21.15.27:0/2099973835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fa002dad0 con 0x7f8fa0105d60 2024-04-26T22:55:33.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.382+0000 7f8f9e000640 1 -- 172.21.15.27:0/2099973835 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11779 (secure 0 0 0) 0x7f8f8809d0d0 con 0x7f8fa0105d60 2024-04-26T22:55:33.383 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:33.383 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/1620677319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1620677319},{"type":"v1","addr":"172.21.15.94:6839","nonce":1620677319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1571355196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1571355196},{"type":"v1","addr":"172.21.15.27:6841","nonce":1571355196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3193530521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3193530521},{"type":"v1","addr":"172.21.15.27:6839","nonce":3193530521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:55:25.265331+0000","modified":"2024-04-26T22:55:30.887610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/95952000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":95952000},{"type":"v1","addr":"172.21.15.80:6839","nonce":95952000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24578,"qdb_cluster":[24578]},"id":1}]} 2024-04-26T22:55:33.383 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-04-26T22:55:33.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.385+0000 7f8fa7a00640 1 -- 172.21.15.27:0/2099973835 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8f840a3bb0 msgr2=0x7f8f840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:33.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.385+0000 7f8fa7a00640 1 --2- 172.21.15.27:0/2099973835 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8f840a3bb0 0x7f8f840a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009830 tx=0x7f8f90030450 comp rx=0 tx=0).stop 2024-04-26T22:55:33.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.385+0000 7f8fa7a00640 1 -- 172.21.15.27:0/2099973835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa0105d60 msgr2=0x7f8fa019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:33.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.385+0000 7f8fa7a00640 1 --2- 172.21.15.27:0/2099973835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa0105d60 0x7f8fa019b2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800e9c0 tx=0x7f8f8800ee80 comp rx=0 tx=0).stop 2024-04-26T22:55:33.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.385+0000 7f8fa7a00640 1 -- 172.21.15.27:0/2099973835 shutdown_connections 2024-04-26T22:55:33.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.385+0000 7f8fa7a00640 1 --2- 172.21.15.27:0/2099973835 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8f840a3bb0 0x7f8f840a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.385+0000 7f8fa7a00640 1 --2- 172.21.15.27:0/2099973835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fa0106720 0x7f8fa01a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.385+0000 7f8fa7a00640 1 --2- 172.21.15.27:0/2099973835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa0105d60 0x7f8fa019b2f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.385+0000 7f8fa7a00640 1 --2- 172.21.15.27:0/2099973835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fa0104b70 0x7f8fa019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.385+0000 7f8fa7a00640 1 -- 172.21.15.27:0/2099973835 >> 172.21.15.27:0/2099973835 conn(0x7f8fa01000e0 msgr2=0x7f8fa0109180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:33.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.386+0000 7f8fa7a00640 1 -- 172.21.15.27:0/2099973835 shutdown_connections 2024-04-26T22:55:33.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.386+0000 7f8fa7a00640 1 -- 172.21.15.27:0/2099973835 wait complete. 2024-04-26T22:55:33.408 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:55:33.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.605+0000 7f7d03200640 1 -- 172.21.15.27:0/4110539611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d0410a940 msgr2=0x7f7d041111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:33.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.605+0000 7f7d03200640 1 --2- 172.21.15.27:0/4110539611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d0410a940 0x7f7d041111d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009ca0 tx=0x7f7cfc02a8c0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.605+0000 7f7d03200640 1 -- 172.21.15.27:0/4110539611 shutdown_connections 2024-04-26T22:55:33.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.605+0000 7f7d03200640 1 --2- 172.21.15.27:0/4110539611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d04109f80 0x7f7d0410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.605+0000 7f7d03200640 1 --2- 172.21.15.27:0/4110539611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d0410a940 0x7f7d041111d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.605+0000 7f7d03200640 1 --2- 172.21.15.27:0/4110539611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d04108d90 0x7f7d041091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.605+0000 7f7d03200640 1 -- 172.21.15.27:0/4110539611 >> 172.21.15.27:0/4110539611 conn(0x7f7d04077d30 msgr2=0x7f7d0407a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:33.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.605+0000 7f7d03200640 1 -- 172.21.15.27:0/4110539611 shutdown_connections 2024-04-26T22:55:33.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.605+0000 7f7d03200640 1 -- 172.21.15.27:0/4110539611 wait complete. 2024-04-26T22:55:33.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.606+0000 7f7d03200640 1 Processor -- start 2024-04-26T22:55:33.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.606+0000 7f7d03200640 1 -- start start 2024-04-26T22:55:33.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.606+0000 7f7d03200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d04108d90 0x7f7d0419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.606+0000 7f7d03200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d04109f80 0x7f7d0419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.606+0000 7f7d03200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d0410a940 0x7f7d041a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.606+0000 7f7d03200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041148f0 con 0x7f7d0410a940 2024-04-26T22:55:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.606+0000 7f7d03200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04114770 con 0x7f7d04109f80 2024-04-26T22:55:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.606+0000 7f7d03200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04114a70 con 0x7f7d04108d90 2024-04-26T22:55:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.606+0000 7f7d01e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d04108d90 0x7f7d0419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.606+0000 7f7d01e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d04108d90 0x7f7d0419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:39822/0 (socket says 172.21.15.27:39822) 2024-04-26T22:55:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.607+0000 7f7d01e00640 1 -- 172.21.15.27:0/4028387904 learned_addr learned my addr 172.21.15.27:0/4028387904 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.607+0000 7f7d02800640 1 --2- 172.21.15.27:0/4028387904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d0410a940 0x7f7d041a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.607+0000 7f7cfbe00640 1 --2- 172.21.15.27:0/4028387904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d04109f80 0x7f7d0419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.607+0000 7f7d01e00640 1 -- 172.21.15.27:0/4028387904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d04109f80 msgr2=0x7f7d0419f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.607+0000 7f7d01e00640 1 --2- 172.21.15.27:0/4028387904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d04109f80 0x7f7d0419f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.607+0000 7f7d01e00640 1 -- 172.21.15.27:0/4028387904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d0410a940 msgr2=0x7f7d041a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.607+0000 7f7d01e00640 1 --2- 172.21.15.27:0/4028387904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d0410a940 0x7f7d041a64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.607+0000 7f7d01e00640 1 -- 172.21.15.27:0/4028387904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d041a6b40 con 0x7f7d04108d90 2024-04-26T22:55:33.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.607+0000 7f7d01e00640 1 --2- 172.21.15.27:0/4028387904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d04108d90 0x7f7d0419ef20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400e9c0 tx=0x7f7cf400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:33.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.608+0000 7f7cfaa00640 1 -- 172.21.15.27:0/4028387904 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf400cd80 con 0x7f7d04108d90 2024-04-26T22:55:33.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.608+0000 7f7cfaa00640 1 -- 172.21.15.27:0/4028387904 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cf40035f0 con 0x7f7d04108d90 2024-04-26T22:55:33.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.608+0000 7f7cfaa00640 1 -- 172.21.15.27:0/4028387904 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4010690 con 0x7f7d04108d90 2024-04-26T22:55:33.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.608+0000 7f7d03200640 1 -- 172.21.15.27:0/4028387904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d041a6e30 con 0x7f7d04108d90 2024-04-26T22:55:33.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.608+0000 7f7d03200640 1 -- 172.21.15.27:0/4028387904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d041a71e0 con 0x7f7d04108d90 2024-04-26T22:55:33.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.609+0000 7f7d03200640 1 -- 172.21.15.27:0/4028387904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0410ffe0 con 0x7f7d04108d90 2024-04-26T22:55:33.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.611+0000 7f7cfaa00640 1 -- 172.21.15.27:0/4028387904 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7cf40037a0 con 0x7f7d04108d90 2024-04-26T22:55:33.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.612+0000 7f7cfaa00640 1 --2- 172.21.15.27:0/4028387904 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7ce80a3cd0 0x7f7ce80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:33.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.612+0000 7f7cfaa00640 1 -- 172.21.15.27:0/4028387904 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7cf40d09a0 con 0x7f7d04108d90 2024-04-26T22:55:33.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.613+0000 7f7cfbe00640 1 --2- 172.21.15.27:0/4028387904 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7ce80a3cd0 0x7f7ce80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:33.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.613+0000 7f7cfbe00640 1 --2- 172.21.15.27:0/4028387904 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7ce80a3cd0 0x7f7ce80a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7d041a0430 tx=0x7f7cec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:33.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.617+0000 7f7cfaa00640 1 -- 172.21.15.27:0/4028387904 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7cf409d0e0 con 0x7f7d04108d90 2024-04-26T22:55:33.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:33 smithi080 ceph-mon[32324]: pgmap v334: 129 pgs: 9 creating+peering, 109 active+clean, 11 unknown; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s wr, 10 op/s 2024-04-26T22:55:33.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:33 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2014543234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:33.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:33 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2099973835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:33.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:33 smithi094 ceph-mon[32889]: pgmap v334: 129 pgs: 9 creating+peering, 109 active+clean, 11 unknown; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s wr, 10 op/s 2024-04-26T22:55:33.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:33 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2014543234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:33.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:33 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2099973835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:33.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.824+0000 7f7d03200640 1 -- 172.21.15.27:0/4028387904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d0402dd80 con 0x7f7d04108d90 2024-04-26T22:55:33.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.826+0000 7f7cfaa00640 1 -- 172.21.15.27:0/4028387904 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24009 (secure 0 0 0) 0x7f7cf409c9e0 con 0x7f7d04108d90 2024-04-26T22:55:33.828 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:33.828 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:55:30.886974+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:55:33.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.830+0000 7f7d03200640 1 -- 172.21.15.27:0/4028387904 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7ce80a3cd0 msgr2=0x7f7ce80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:33.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.830+0000 7f7d03200640 1 --2- 172.21.15.27:0/4028387904 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7ce80a3cd0 0x7f7ce80a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7d041a0430 tx=0x7f7cec005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:33.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.830+0000 7f7d03200640 1 -- 172.21.15.27:0/4028387904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d04108d90 msgr2=0x7f7d0419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:33.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.830+0000 7f7d03200640 1 --2- 172.21.15.27:0/4028387904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d04108d90 0x7f7d0419ef20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400e9c0 tx=0x7f7cf400ee80 comp rx=0 tx=0).stop 2024-04-26T22:55:33.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.830+0000 7f7d03200640 1 -- 172.21.15.27:0/4028387904 shutdown_connections 2024-04-26T22:55:33.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.830+0000 7f7d03200640 1 --2- 172.21.15.27:0/4028387904 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7ce80a3cd0 0x7f7ce80a6190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.830+0000 7f7d03200640 1 --2- 172.21.15.27:0/4028387904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d04109f80 0x7f7d0419f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.830+0000 7f7d03200640 1 --2- 172.21.15.27:0/4028387904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d0410a940 0x7f7d041a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.830+0000 7f7d03200640 1 --2- 172.21.15.27:0/4028387904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d04108d90 0x7f7d0419ef20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:33.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.830+0000 7f7d03200640 1 -- 172.21.15.27:0/4028387904 >> 172.21.15.27:0/4028387904 conn(0x7f7d04077d30 msgr2=0x7f7d0410d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:33.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.830+0000 7f7d03200640 1 -- 172.21.15.27:0/4028387904 shutdown_connections 2024-04-26T22:55:33.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:33.830+0000 7f7d03200640 1 -- 172.21.15.27:0/4028387904 wait complete. 2024-04-26T22:55:33.848 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:33 smithi027 ceph-mon[28034]: pgmap v334: 129 pgs: 9 creating+peering, 109 active+clean, 11 unknown; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s wr, 10 op/s 2024-04-26T22:55:33.848 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:33 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2014543234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:33.849 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:33 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2099973835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:33.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T22:55:25.265331+0000', 'modified': '2024-04-26T22:55:30.887610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24578}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6839/95952000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6838', 'nonce': 95952000}, {'type': 'v1', 'addr': '172.21.15.80:6839', 'nonce': 95952000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24578, 'qdb_cluster': [24578]} 2024-04-26T22:55:33.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T22:55:33.852 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T22:55:34.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.063+0000 7f2e3d200640 1 -- 172.21.15.27:0/1910347537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e38076910 msgr2=0x7f2e38074d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:34.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.063+0000 7f2e3d200640 1 --2- 172.21.15.27:0/1910347537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e38076910 0x7f2e38074d60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009900 tx=0x7f2e2802a480 comp rx=0 tx=0).stop 2024-04-26T22:55:34.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.063+0000 7f2e3d200640 1 -- 172.21.15.27:0/1910347537 shutdown_connections 2024-04-26T22:55:34.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.063+0000 7f2e3d200640 1 --2- 172.21.15.27:0/1910347537 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3810a650 0x7f2e3810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.063+0000 7f2e3d200640 1 --2- 172.21.15.27:0/1910347537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e38075330 0x7f2e38075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.063+0000 7f2e3d200640 1 --2- 172.21.15.27:0/1910347537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e38076910 0x7f2e38074d60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.063+0000 7f2e3d200640 1 -- 172.21.15.27:0/1910347537 >> 172.21.15.27:0/1910347537 conn(0x7f2e380fdd30 msgr2=0x7f2e38100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:34.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.063+0000 7f2e3d200640 1 -- 172.21.15.27:0/1910347537 shutdown_connections 2024-04-26T22:55:34.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.063+0000 7f2e3d200640 1 -- 172.21.15.27:0/1910347537 wait complete. 2024-04-26T22:55:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.064+0000 7f2e3d200640 1 Processor -- start 2024-04-26T22:55:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.064+0000 7f2e3d200640 1 -- start start 2024-04-26T22:55:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.064+0000 7f2e3d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e38075330 0x7f2e3819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.064+0000 7f2e3d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e38076910 0x7f2e3819b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.064+0000 7f2e3d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3810a650 0x7f2e381a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.064+0000 7f2e3d200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e381104c0 con 0x7f2e38075330 2024-04-26T22:55:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.065+0000 7f2e3d200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2e38110340 con 0x7f2e3810a650 2024-04-26T22:55:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.065+0000 7f2e3d200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e38110640 con 0x7f2e38076910 2024-04-26T22:55:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.065+0000 7f2e37400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e38075330 0x7f2e3819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.065+0000 7f2e37400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e38075330 0x7f2e3819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60922/0 (socket says 172.21.15.27:60922) 2024-04-26T22:55:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.065+0000 7f2e37400640 1 -- 172.21.15.27:0/1429668402 learned_addr learned my addr 172.21.15.27:0/1429668402 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.065+0000 7f2e36a00640 1 --2- 172.21.15.27:0/1429668402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e38076910 0x7f2e3819b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.065+0000 7f2e37e00640 1 --2- 172.21.15.27:0/1429668402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3810a650 0x7f2e381a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.065+0000 7f2e37400640 1 -- 172.21.15.27:0/1429668402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e38076910 msgr2=0x7f2e3819b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:34.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.065+0000 7f2e37400640 1 --2- 172.21.15.27:0/1429668402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e38076910 0x7f2e3819b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.065+0000 7f2e37400640 1 -- 172.21.15.27:0/1429668402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3810a650 msgr2=0x7f2e381a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:34.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.065+0000 7f2e37400640 1 --2- 172.21.15.27:0/1429668402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3810a650 0x7f2e381a2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.065+0000 7f2e37400640 1 -- 172.21.15.27:0/1429668402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e381a27b0 con 0x7f2e38075330 2024-04-26T22:55:34.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.065+0000 7f2e37400640 1 --2- 172.21.15.27:0/1429668402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e38075330 0x7f2e3819ab90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2e28005940 tx=0x7f2e280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:34.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.066+0000 7f2e2fe00640 1 -- 172.21.15.27:0/1429668402 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2803f030 con 0x7f2e38075330 2024-04-26T22:55:34.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.066+0000 7f2e2fe00640 1 -- 172.21.15.27:0/1429668402 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e28003cb0 con 0x7f2e38075330 2024-04-26T22:55:34.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.066+0000 7f2e3d200640 1 -- 172.21.15.27:0/1429668402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e381a2a40 con 0x7f2e38075330 2024-04-26T22:55:34.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.066+0000 7f2e2fe00640 1 -- 172.21.15.27:0/1429668402 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28035070 con 0x7f2e38075330 2024-04-26T22:55:34.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.066+0000 7f2e3d200640 1 -- 172.21.15.27:0/1429668402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e381a2f30 con 0x7f2e38075330 2024-04-26T22:55:34.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.069+0000 7f2e415da740 1 Processor -- start 2024-04-26T22:55:34.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.069+0000 7f2e2fe00640 1 -- 172.21.15.27:0/1429668402 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2e28042030 con 0x7f2e38075330 2024-04-26T22:55:34.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.070+0000 7f2e415da740 1 -- start start 2024-04-26T22:55:34.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.070+0000 7f2e415da740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e5612c0 0x55a02e5616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.070+0000 7f2e415da740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e561cb0 0x55a02e5667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.070+0000 7f2e415da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e566e20 0x55a02e5692b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.070+0000 7f2e415da740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a02e4c5600 con 0x55a02e561cb0 2024-04-26T22:55:34.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.070+0000 7f2e415da740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55a02e462100 con 0x55a02e5612c0 2024-04-26T22:55:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.070+0000 7f2e415da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a02e4c15c0 con 0x55a02e566e20 2024-04-26T22:55:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.070+0000 7f2e37400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e561cb0 0x55a02e5667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.070+0000 7f2e36a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e566e20 0x55a02e5692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.070+0000 7f2e36a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e566e20 0x55a02e5692b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:39856/0 (socket says 172.21.15.27:39856) 2024-04-26T22:55:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.070+0000 7f2e36a00640 1 -- 172.21.15.27:0/3725784488 learned_addr learned my addr 172.21.15.27:0/3725784488 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.070+0000 7f2e37e00640 1 --2- 172.21.15.27:0/3725784488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e5612c0 0x55a02e5616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.071+0000 7f2e36a00640 1 -- 172.21.15.27:0/3725784488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e5612c0 msgr2=0x55a02e5616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:55:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.071+0000 7f2e36a00640 1 --2- 172.21.15.27:0/3725784488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e5612c0 0x55a02e5616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.071+0000 7f2e36a00640 1 -- 172.21.15.27:0/3725784488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e561cb0 msgr2=0x55a02e5667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:55:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.071+0000 7f2e36a00640 1 --2- 172.21.15.27:0/3725784488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e561cb0 0x55a02e5667b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.071+0000 7f2e36a00640 1 -- 172.21.15.27:0/3725784488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a02e1d72d0 con 0x55a02e566e20 2024-04-26T22:55:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.071+0000 7f2e36a00640 1 --2- 172.21.15.27:0/3725784488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e566e20 0x55a02e5692b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009c90 tx=0x7f2e24011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f191abd75f0ebcb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:34.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.071+0000 7f2e2fe00640 1 --2- 172.21.15.27:0/1429668402 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2e1c0a3ba0 0x7f2e1c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.071+0000 7f2e2cc00640 1 -- 172.21.15.27:0/3725784488 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2401e070 con 0x55a02e566e20 2024-04-26T22:55:34.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.071+0000 7f2e2cc00640 1 -- 172.21.15.27:0/3725784488 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e24019ae0 con 0x55a02e566e20 2024-04-26T22:55:34.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.071+0000 7f2e2cc00640 1 -- 172.21.15.27:0/3725784488 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2400ea40 con 0x55a02e566e20 2024-04-26T22:55:34.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.072+0000 7f2e2fe00640 1 -- 172.21.15.27:0/1429668402 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2e280f09f0 con 0x7f2e38075330 2024-04-26T22:55:34.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.072+0000 7f2e37e00640 1 --2- 172.21.15.27:0/1429668402 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2e1c0a3ba0 0x7f2e1c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.072+0000 7f2e415da740 1 -- 172.21.15.27:0/3725784488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e566e20 msgr2=0x55a02e5692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:34.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.072+0000 7f2e415da740 1 --2- 172.21.15.27:0/3725784488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e566e20 0x55a02e5692b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009c90 tx=0x7f2e24011810 comp rx=0 tx=0).stop 2024-04-26T22:55:34.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.072+0000 7f2e37e00640 1 --2- 172.21.15.27:0/1429668402 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2e1c0a3ba0 0x7f2e1c0a6060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55a02e562510 tx=0x7f2e30009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:34.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.072+0000 7f2e415da740 1 -- 172.21.15.27:0/3725784488 shutdown_connections 2024-04-26T22:55:34.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.072+0000 7f2e415da740 1 --2- 172.21.15.27:0/3725784488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e566e20 0x55a02e5692b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.072+0000 7f2e415da740 1 --2- 172.21.15.27:0/3725784488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e561cb0 0x55a02e5667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.072+0000 7f2e415da740 1 --2- 172.21.15.27:0/3725784488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e5612c0 0x55a02e5616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.072+0000 7f2e415da740 1 -- 172.21.15.27:0/3725784488 >> 172.21.15.27:0/3725784488 conn(0x55a02e4f5d10 msgr2=0x55a02e55cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:34.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.073+0000 7f2e415da740 1 -- 172.21.15.27:0/3725784488 shutdown_connections 2024-04-26T22:55:34.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.073+0000 7f2e415da740 1 -- 172.21.15.27:0/3725784488 wait complete. 2024-04-26T22:55:34.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.073+0000 7f2e415da740 1 -- start start 2024-04-26T22:55:34.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.073+0000 7f2e415da740 1 Processor -- start 2024-04-26T22:55:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e415da740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e5612c0 0x55a02e56b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e415da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e561cb0 0x55a02e56bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e415da740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e566e20 0x55a02e572c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e415da740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a02e569dd0 con 0x55a02e566e20 2024-04-26T22:55:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e415da740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55a02e44cf70 con 0x55a02e5612c0 2024-04-26T22:55:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e415da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a02e569f50 con 0x55a02e561cb0 2024-04-26T22:55:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e415da740 10 client.0 _ll_register_callbacks cb 0x55a02e4bb480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:55:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e36a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e561cb0 0x55a02e56bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e36a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e561cb0 0x55a02e56bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:39862/0 (socket says 172.21.15.27:39862) 2024-04-26T22:55:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e36a00640 1 -- 172.21.15.27:0/2566800570 learned_addr learned my addr 172.21.15.27:0/2566800570 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e37400640 1 --2- 172.21.15.27:0/2566800570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e5612c0 0x55a02e56b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e37e00640 1 --2- 172.21.15.27:0/2566800570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e566e20 0x55a02e572c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e36a00640 1 -- 172.21.15.27:0/2566800570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e5612c0 msgr2=0x55a02e56b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:55:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e36a00640 1 --2- 172.21.15.27:0/2566800570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e5612c0 0x55a02e56b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e36a00640 1 -- 172.21.15.27:0/2566800570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e566e20 msgr2=0x55a02e572c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:55:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e36a00640 1 --2- 172.21.15.27:0/2566800570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e566e20 0x55a02e572c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e36a00640 1 -- 172.21.15.27:0/2566800570 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a02e2a16b0 con 0x55a02e561cb0 2024-04-26T22:55:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.074+0000 7f2e37400640 1 --2- 172.21.15.27:0/2566800570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e5612c0 0x55a02e56b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.075+0000 7f2e36a00640 1 --2- 172.21.15.27:0/2566800570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e561cb0 0x55a02e56bc20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2e24019660 tx=0x7f2e24019690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee3b43f330eac3be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.075+0000 7f2e21600640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T22:55:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.075+0000 7f2e21600640 1 -- 172.21.15.27:0/2566800570 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24029030 con 0x55a02e561cb0 2024-04-26T22:55:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.075+0000 7f2e21600640 1 -- 172.21.15.27:0/2566800570 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2e240198b0 con 0x55a02e561cb0 2024-04-26T22:55:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.075+0000 7f2e415da740 1 -- 172.21.15.27:0/2566800570 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e24019ae0 con 0x55a02e561cb0 2024-04-26T22:55:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.075+0000 7f2e21600640 1 -- 172.21.15.27:0/2566800570 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f2e24033050 con 0x55a02e561cb0 2024-04-26T22:55:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.075+0000 7f2e415da740 10 client.24715 fetch_fsmap learned FSMap version 17 2024-04-26T22:55:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.075+0000 7f2e415da740 1 -- 172.21.15.27:0/2566800570 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55a02e575c40 con 0x55a02e561cb0 2024-04-26T22:55:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.076+0000 7f2e21600640 1 -- 172.21.15.27:0/2566800570 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e24019ae0 con 0x55a02e561cb0 2024-04-26T22:55:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.076+0000 7f2e21600640 1 -- 172.21.15.27:0/2566800570 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f2e2400ea40 con 0x55a02e561cb0 2024-04-26T22:55:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.076+0000 7f2e415da740 10 client.24715 fetch_fsmap finished waiting for FSMap version 17 2024-04-26T22:55:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.076+0000 7f2e415da740 10 client.24715 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T22:55:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.076+0000 7f2e415da740 20 client.24715 populate_metadata read hostname 'smithi027' 2024-04-26T22:55:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.076+0000 7f2e415da740 1 --2- 172.21.15.27:0/2566800570 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55a02e578020 0x55a02e579490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.076+0000 7f2e415da740 4 client.24715 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:55:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.076+0000 7f2e415da740 1 -- 172.21.15.27:0/2566800570 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a02e4d3210 con 0x55a02e578020 2024-04-26T22:55:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.077+0000 7f2e37400640 1 --2- 172.21.15.27:0/2566800570 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55a02e578020 0x55a02e579490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.077+0000 7f2e37400640 1 --2- 172.21.15.27:0/2566800570 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55a02e578020 0x55a02e579490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff6dbbcefcf594 server_cookie=b98b80c9853d2242 in_seq=0 out_seq=0 2024-04-26T22:55:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.077+0000 7f2e21600640 10 client.24715 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:55:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.078+0000 7f2e21600640 1 -- 172.21.15.27:0/2566800570 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a02e4d3210 con 0x55a02e578020 2024-04-26T22:55:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.078+0000 7f2e21600640 10 client.24715 handle_command_reply: tid=0 2024-04-26T22:55:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.078+0000 7f2e415da740 1 client.24715 shutdown 2024-04-26T22:55:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.078+0000 7f2e415da740 1 -- 172.21.15.27:0/2566800570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e561cb0 msgr2=0x55a02e56bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.078+0000 7f2e415da740 1 --2- 172.21.15.27:0/2566800570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e561cb0 0x55a02e56bc20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2e24019660 tx=0x7f2e24019690 comp rx=0 tx=0).stop 2024-04-26T22:55:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.078+0000 7f2e415da740 1 -- 172.21.15.27:0/2566800570 shutdown_connections 2024-04-26T22:55:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.078+0000 7f2e415da740 1 --2- 172.21.15.27:0/2566800570 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55a02e578020 0x55a02e579490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.078+0000 7f2e415da740 1 --2- 172.21.15.27:0/2566800570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e566e20 0x55a02e572c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.078+0000 7f2e415da740 1 --2- 172.21.15.27:0/2566800570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e561cb0 0x55a02e56bc20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.079+0000 7f2e415da740 1 --2- 172.21.15.27:0/2566800570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e5612c0 0x55a02e56b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.079+0000 7f2e21600640 0 client.24715 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:55:34.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.079+0000 7f2e415da740 1 -- 172.21.15.27:0/2566800570 >> 172.21.15.27:0/2566800570 conn(0x55a02e4f5d10 msgr2=0x55a02e55bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:34.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.079+0000 7f2e415da740 1 -- 172.21.15.27:0/2566800570 shutdown_connections 2024-04-26T22:55:34.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.079+0000 7f2e415da740 1 -- 172.21.15.27:0/2566800570 wait complete. 2024-04-26T22:55:34.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.079+0000 7f2e415da740 20 client.24715 trim_cache size 0 max 16384 2024-04-26T22:55:34.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.099+0000 7f2e415da740 1 Processor -- start 2024-04-26T22:55:34.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.099+0000 7f2e415da740 1 -- start start 2024-04-26T22:55:34.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e415da740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e5612c0 0x55a02e569210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e415da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e561cb0 0x55a02e572c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e415da740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e566e20 0x55a02e56b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e415da740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a02e5777d0 con 0x55a02e5612c0 2024-04-26T22:55:34.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e415da740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55a02e577ea0 con 0x55a02e566e20 2024-04-26T22:55:34.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e415da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a02e578570 con 0x55a02e561cb0 2024-04-26T22:55:34.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e37400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e5612c0 0x55a02e569210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e37e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e566e20 0x55a02e56b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e37400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e5612c0 0x55a02e569210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60944/0 (socket says 172.21.15.27:60944) 2024-04-26T22:55:34.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e37400640 1 -- 172.21.15.27:0/845220347 learned_addr learned my addr 172.21.15.27:0/845220347 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:34.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e36a00640 1 --2- 172.21.15.27:0/845220347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e561cb0 0x55a02e572c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e37400640 1 -- 172.21.15.27:0/845220347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e561cb0 msgr2=0x55a02e572c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:55:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e37400640 1 --2- 172.21.15.27:0/845220347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e561cb0 0x55a02e572c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e37400640 1 -- 172.21.15.27:0/845220347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e566e20 msgr2=0x55a02e56b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:55:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e37400640 1 --2- 172.21.15.27:0/845220347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e566e20 0x55a02e56b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e37400640 1 -- 172.21.15.27:0/845220347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a02e562090 con 0x55a02e5612c0 2024-04-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.100+0000 7f2e36a00640 1 --2- 172.21.15.27:0/845220347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e561cb0 0x55a02e572c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.101+0000 7f2e37400640 1 --2- 172.21.15.27:0/845220347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e5612c0 0x55a02e569210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2e28006010 tx=0x7f2e280ee180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af602e35e975beb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.101+0000 7f2e23e00640 1 -- 172.21.15.27:0/845220347 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28049040 con 0x55a02e5612c0 2024-04-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.101+0000 7f2e23e00640 1 -- 172.21.15.27:0/845220347 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e280dd0e0 con 0x55a02e5612c0 2024-04-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.101+0000 7f2e23e00640 1 -- 172.21.15.27:0/845220347 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e280c0e40 con 0x55a02e5612c0 2024-04-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.102+0000 7f2e415da740 1 -- 172.21.15.27:0/845220347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e5612c0 msgr2=0x55a02e569210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.102+0000 7f2e415da740 1 --2- 172.21.15.27:0/845220347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e5612c0 0x55a02e569210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2e28006010 tx=0x7f2e280ee180 comp rx=0 tx=0).stop 2024-04-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.102+0000 7f2e415da740 1 -- 172.21.15.27:0/845220347 shutdown_connections 2024-04-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.102+0000 7f2e415da740 1 --2- 172.21.15.27:0/845220347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e566e20 0x55a02e56b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.102+0000 7f2e415da740 1 --2- 172.21.15.27:0/845220347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e561cb0 0x55a02e572c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.102+0000 7f2e415da740 1 --2- 172.21.15.27:0/845220347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e5612c0 0x55a02e569210 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.102+0000 7f2e415da740 1 -- 172.21.15.27:0/845220347 >> 172.21.15.27:0/845220347 conn(0x55a02e4f5d10 msgr2=0x55a02e557df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.102+0000 7f2e415da740 1 -- 172.21.15.27:0/845220347 shutdown_connections 2024-04-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.102+0000 7f2e415da740 1 -- 172.21.15.27:0/845220347 wait complete. 2024-04-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.102+0000 7f2e415da740 1 -- start start 2024-04-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.103+0000 7f2e415da740 1 Processor -- start 2024-04-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.103+0000 7f2e415da740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e5612c0 0x55a02e56e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.103+0000 7f2e415da740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e561cb0 0x55a02e56e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.103+0000 7f2e415da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e566e20 0x55a02e571960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.103+0000 7f2e415da740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a02e579310 con 0x55a02e5612c0 2024-04-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.103+0000 7f2e415da740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55a02e578c40 con 0x55a02e561cb0 2024-04-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.103+0000 7f2e415da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a02e5799e0 con 0x55a02e566e20 2024-04-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.103+0000 7f2e415da740 10 client.0 _ll_register_callbacks cb 0x7f2e2401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.103+0000 7f2e37400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e5612c0 0x55a02e56e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.103+0000 7f2e36a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e561cb0 0x55a02e56e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.103+0000 7f2e37400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e5612c0 0x55a02e56e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60958/0 (socket says 172.21.15.27:60958) 2024-04-26T22:55:34.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.103+0000 7f2e36a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e561cb0 0x55a02e56e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:54292/0 (socket says 172.21.15.27:54292) 2024-04-26T22:55:34.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.103+0000 7f2e37400640 1 -- 172.21.15.27:0/1857847092 learned_addr learned my addr 172.21.15.27:0/1857847092 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:34.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.103+0000 7f2e37e00640 1 --2- 172.21.15.27:0/1857847092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e566e20 0x55a02e571960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.104+0000 7f2e36a00640 1 -- 172.21.15.27:0/1857847092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e566e20 msgr2=0x55a02e571960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:55:34.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.104+0000 7f2e36a00640 1 --2- 172.21.15.27:0/1857847092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e566e20 0x55a02e571960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.104+0000 7f2e36a00640 1 -- 172.21.15.27:0/1857847092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e5612c0 msgr2=0x55a02e56e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:55:34.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.104+0000 7f2e36a00640 1 --2- 172.21.15.27:0/1857847092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e5612c0 0x55a02e56e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.104+0000 7f2e36a00640 1 -- 172.21.15.27:0/1857847092 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a02e571f50 con 0x55a02e561cb0 2024-04-26T22:55:34.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.104+0000 7f2e36a00640 1 --2- 172.21.15.27:0/1857847092 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e561cb0 0x55a02e56e940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009c90 tx=0x7f2e24004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cb357aa11b60311 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:34.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.104+0000 7f2e21600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T22:55:34.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.104+0000 7f2e21600640 1 -- 172.21.15.27:0/1857847092 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2403c060 con 0x55a02e561cb0 2024-04-26T22:55:34.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.104+0000 7f2e415da740 1 -- 172.21.15.27:0/1857847092 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e280dd0e0 con 0x55a02e561cb0 2024-04-26T22:55:34.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.104+0000 7f2e21600640 1 -- 172.21.15.27:0/1857847092 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2e240035f0 con 0x55a02e561cb0 2024-04-26T22:55:34.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.105+0000 7f2e21600640 1 -- 172.21.15.27:0/1857847092 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f2e24043050 con 0x55a02e561cb0 2024-04-26T22:55:34.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.105+0000 7f2e415da740 10 client.24692 fetch_fsmap learned FSMap version 17 2024-04-26T22:55:34.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.105+0000 7f2e415da740 1 -- 172.21.15.27:0/1857847092 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55a02e5de030 con 0x55a02e561cb0 2024-04-26T22:55:34.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.105+0000 7f2e21600640 1 -- 172.21.15.27:0/1857847092 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e280dd0e0 con 0x55a02e561cb0 2024-04-26T22:55:34.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.105+0000 7f2e21600640 1 -- 172.21.15.27:0/1857847092 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f2e24036070 con 0x55a02e561cb0 2024-04-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.105+0000 7f2e415da740 10 client.24692 fetch_fsmap finished waiting for FSMap version 17 2024-04-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.105+0000 7f2e415da740 10 client.24692 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.105+0000 7f2e415da740 20 client.24692 populate_metadata read hostname 'smithi027' 2024-04-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.105+0000 7f2e415da740 1 --2- 172.21.15.27:0/1857847092 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55a02e5e2bc0 0x55a02e5e18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.105+0000 7f2e415da740 4 client.24692 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.105+0000 7f2e415da740 1 -- 172.21.15.27:0/1857847092 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "status"}) -- 0x55a02e4cd080 con 0x55a02e5e2bc0 2024-04-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.106+0000 7f2e37400640 1 --2- 172.21.15.27:0/1857847092 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55a02e5e2bc0 0x55a02e5e18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.106+0000 7f2e37400640 1 --2- 172.21.15.27:0/1857847092 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55a02e5e2bc0 0x55a02e5e18d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=affe565a05b7c5bb server_cookie=b0e3a0c79d1552cd in_seq=0 out_seq=0 2024-04-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.106+0000 7f2e21600640 10 client.24692 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.106+0000 7f2e21600640 1 -- 172.21.15.27:0/1857847092 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55a02e4cd080 con 0x55a02e5e2bc0 2024-04-26T22:55:34.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.106+0000 7f2e21600640 10 client.24692 handle_command_reply: tid=0 2024-04-26T22:55:34.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.106+0000 7f2e415da740 1 client.24692 shutdown 2024-04-26T22:55:34.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.107+0000 7f2e415da740 1 -- 172.21.15.27:0/1857847092 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e561cb0 msgr2=0x55a02e56e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:34.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.107+0000 7f2e415da740 1 --2- 172.21.15.27:0/1857847092 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e561cb0 0x55a02e56e940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009c90 tx=0x7f2e24004270 comp rx=0 tx=0).stop 2024-04-26T22:55:34.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.107+0000 7f2e415da740 1 -- 172.21.15.27:0/1857847092 shutdown_connections 2024-04-26T22:55:34.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.107+0000 7f2e415da740 1 --2- 172.21.15.27:0/1857847092 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55a02e5e2bc0 0x55a02e5e18d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.107+0000 7f2e415da740 1 --2- 172.21.15.27:0/1857847092 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a02e561cb0 0x55a02e56e940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.107+0000 7f2e415da740 1 --2- 172.21.15.27:0/1857847092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a02e566e20 0x55a02e571960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.107+0000 7f2e415da740 1 --2- 172.21.15.27:0/1857847092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a02e5612c0 0x55a02e56e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.107+0000 7f2e415da740 1 -- 172.21.15.27:0/1857847092 >> 172.21.15.27:0/1857847092 conn(0x55a02e4f5d10 msgr2=0x55a02e56b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:34.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.107+0000 7f2e21600640 0 client.24692 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:55:34.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.107+0000 7f2e415da740 1 -- 172.21.15.27:0/1857847092 shutdown_connections 2024-04-26T22:55:34.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.107+0000 7f2e415da740 1 -- 172.21.15.27:0/1857847092 wait complete. 2024-04-26T22:55:34.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.107+0000 7f2e415da740 20 client.24692 trim_cache size 0 max 16384 2024-04-26T22:55:34.113 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T22:55:34.113 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "95ebdda6-041e-11ef-bc93-c7b262605968", 2024-04-26T22:55:34.113 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-04-26T22:55:34.113 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24578, 2024-04-26T22:55:34.113 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T22:55:34.113 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T22:55:34.113 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T22:55:34.113 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 7.8627794580000003, 2024-04-26T22:55:34.113 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 17, 2024-04-26T22:55:34.113 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-04-26T22:55:34.114 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T22:55:34.114 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 15.583187638 2024-04-26T22:55:34.114 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T22:55:34.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.108+0000 7f2e3d200640 1 -- 172.21.15.27:0/1429668402 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2e1c0a3ba0 msgr2=0x7f2e1c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:34.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.108+0000 7f2e3d200640 1 --2- 172.21.15.27:0/1429668402 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2e1c0a3ba0 0x7f2e1c0a6060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55a02e562510 tx=0x7f2e30009500 comp rx=0 tx=0).stop 2024-04-26T22:55:34.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.108+0000 7f2e3d200640 1 -- 172.21.15.27:0/1429668402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e38075330 msgr2=0x7f2e3819ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:34.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.108+0000 7f2e3d200640 1 --2- 172.21.15.27:0/1429668402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e38075330 0x7f2e3819ab90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2e28005940 tx=0x7f2e280042f0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.108+0000 7f2e3d200640 1 -- 172.21.15.27:0/1429668402 shutdown_connections 2024-04-26T22:55:34.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.108+0000 7f2e3d200640 1 --2- 172.21.15.27:0/1429668402 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2e1c0a3ba0 0x7f2e1c0a6060 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.108+0000 7f2e3d200640 1 --2- 172.21.15.27:0/1429668402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3810a650 0x7f2e381a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.108+0000 7f2e3d200640 1 --2- 172.21.15.27:0/1429668402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e38076910 0x7f2e3819b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.108+0000 7f2e3d200640 1 --2- 172.21.15.27:0/1429668402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e38075330 0x7f2e3819ab90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:34.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.108+0000 7f2e3d200640 1 -- 172.21.15.27:0/1429668402 >> 172.21.15.27:0/1429668402 conn(0x7f2e380fdd30 msgr2=0x7f2e381065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:34.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.108+0000 7f2e3d200640 1 -- 172.21.15.27:0/1429668402 shutdown_connections 2024-04-26T22:55:34.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:34.108+0000 7f2e3d200640 1 -- 172.21.15.27:0/1429668402 wait complete. 2024-04-26T22:55:34.127 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T22:55:34.128 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T22:55:34.128 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T22:55:34.620 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:55:34.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:34 smithi080 ceph-mon[32324]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T22:55:34.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:34 smithi080 ceph-mon[32324]: Cluster is now healthy 2024-04-26T22:55:34.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:34 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/4028387904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:34.777 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:34 smithi027 ceph-mon[28034]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T22:55:34.777 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:34 smithi027 ceph-mon[28034]: Cluster is now healthy 2024-04-26T22:55:34.777 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:34 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/4028387904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:34.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:34 smithi094 ceph-mon[32889]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T22:55:34.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:34 smithi094 ceph-mon[32889]: Cluster is now healthy 2024-04-26T22:55:34.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:34 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/4028387904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:35.702 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:35 smithi027 ceph-mon[28034]: pgmap v335: 129 pgs: 9 creating+peering, 120 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-04-26T22:55:35.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:35 smithi080 ceph-mon[32324]: pgmap v335: 129 pgs: 9 creating+peering, 120 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-04-26T22:55:35.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:35 smithi094 ceph-mon[32889]: pgmap v335: 129 pgs: 9 creating+peering, 120 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-04-26T22:55:37.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.223+0000 7fcac7200700 1 -- 172.21.15.27:0/2837251772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac810fcb0 msgr2=0x7fcac81120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:37.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.223+0000 7fcac7200700 1 --2- 172.21.15.27:0/2837251772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac810fcb0 0x7fcac81120a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcac000b0a0 tx=0x7fcac000b3b0 comp rx=0 tx=0).stop 2024-04-26T22:55:37.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.223+0000 7fcac7200700 1 -- 172.21.15.27:0/2837251772 shutdown_connections 2024-04-26T22:55:37.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.223+0000 7fcac7200700 1 --2- 172.21.15.27:0/2837251772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac810fcb0 0x7fcac81120a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:37.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.223+0000 7fcac7200700 1 --2- 172.21.15.27:0/2837251772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcac8102100 0x7fcac81024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:37.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.223+0000 7fcac7200700 1 --2- 172.21.15.27:0/2837251772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcac8102ab0 0x7fcac810f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:37.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.223+0000 7fcac7200700 1 -- 172.21.15.27:0/2837251772 >> 172.21.15.27:0/2837251772 conn(0x7fcac80fdd40 msgr2=0x7fcac8100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:37.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.223+0000 7fcac7200700 1 -- 172.21.15.27:0/2837251772 shutdown_connections 2024-04-26T22:55:37.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.223+0000 7fcac7200700 1 -- 172.21.15.27:0/2837251772 wait complete. 2024-04-26T22:55:37.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.224+0000 7fcac7200700 1 Processor -- start 2024-04-26T22:55:37.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.225+0000 7fcac7200700 1 -- start start 2024-04-26T22:55:37.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.225+0000 7fcac7200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcac8102100 0x7fcac81a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:37.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.225+0000 7fcac7200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcac8102ab0 0x7fcac81a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:37.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.225+0000 7fcac7200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac810fcb0 0x7fcac819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:37.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.225+0000 7fcac7200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81157a0 con 0x7fcac8102100 2024-04-26T22:55:37.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.225+0000 7fcac7200700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8115620 con 0x7fcac8102ab0 2024-04-26T22:55:37.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.225+0000 7fcac7200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8115920 con 0x7fcac810fcb0 2024-04-26T22:55:37.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.226+0000 7fcabd400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcac8102ab0 0x7fcac81a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:37.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.226+0000 7fcac5e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcac8102100 0x7fcac81a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:37.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.226+0000 7fcabd400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcac8102ab0 0x7fcac81a1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:54316/0 (socket says 172.21.15.27:54316) 2024-04-26T22:55:37.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.226+0000 7fcabd400700 1 -- 172.21.15.27:0/3045312632 learned_addr learned my addr 172.21.15.27:0/3045312632 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:37.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.226+0000 7fcac6800700 1 --2- 172.21.15.27:0/3045312632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac810fcb0 0x7fcac819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:37.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.226+0000 7fcabd400700 1 -- 172.21.15.27:0/3045312632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac810fcb0 msgr2=0x7fcac819ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:37.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.226+0000 7fcabd400700 1 --2- 172.21.15.27:0/3045312632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac810fcb0 0x7fcac819ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:37.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.226+0000 7fcabd400700 1 -- 172.21.15.27:0/3045312632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcac8102100 msgr2=0x7fcac81a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:37.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.226+0000 7fcabd400700 1 --2- 172.21.15.27:0/3045312632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcac8102100 0x7fcac81a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:37.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.226+0000 7fcabd400700 1 -- 172.21.15.27:0/3045312632 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac819c1c0 con 0x7fcac8102ab0 2024-04-26T22:55:37.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.226+0000 7fcac6800700 1 --2- 172.21.15.27:0/3045312632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac810fcb0 0x7fcac819ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:37.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.226+0000 7fcabd400700 1 --2- 172.21.15.27:0/3045312632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcac8102ab0 0x7fcac81a1de0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcab000e9c0 tx=0x7fcab000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:37.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.227+0000 7fcabf400700 1 -- 172.21.15.27:0/3045312632 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab000cbd0 con 0x7fcac8102ab0 2024-04-26T22:55:37.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.227+0000 7fcabf400700 1 -- 172.21.15.27:0/3045312632 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcab000ee60 con 0x7fcac8102ab0 2024-04-26T22:55:37.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.227+0000 7fcac7200700 1 -- 172.21.15.27:0/3045312632 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac819c4b0 con 0x7fcac8102ab0 2024-04-26T22:55:37.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.227+0000 7fcabf400700 1 -- 172.21.15.27:0/3045312632 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab00055c0 con 0x7fcac8102ab0 2024-04-26T22:55:37.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.227+0000 7fcac7200700 1 -- 172.21.15.27:0/3045312632 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac81a8770 con 0x7fcac8102ab0 2024-04-26T22:55:37.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.228+0000 7fcac7200700 1 -- 172.21.15.27:0/3045312632 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac806a110 con 0x7fcac8102ab0 2024-04-26T22:55:37.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.230+0000 7fcabf400700 1 -- 172.21.15.27:0/3045312632 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcab0004750 con 0x7fcac8102ab0 2024-04-26T22:55:37.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.232+0000 7fcabf400700 1 --2- 172.21.15.27:0/3045312632 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcaac0a3cc0 0x7fcaac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:37.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.232+0000 7fcabf400700 1 -- 172.21.15.27:0/3045312632 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fcab00d07f0 con 0x7fcac8102ab0 2024-04-26T22:55:37.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.232+0000 7fcac5e00700 1 --2- 172.21.15.27:0/3045312632 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcaac0a3cc0 0x7fcaac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:37.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.233+0000 7fcac5e00700 1 --2- 172.21.15.27:0/3045312632 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcaac0a3cc0 0x7fcaac0a6180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcac819cba0 tx=0x7fcab8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:37.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.236+0000 7fcabf400700 1 -- 172.21.15.27:0/3045312632 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcab009cf30 con 0x7fcac8102ab0 2024-04-26T22:55:37.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.521+0000 7fcac7200700 1 -- 172.21.15.27:0/3045312632 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcac819cf90 con 0x7fcac8102ab0 2024-04-26T22:55:37.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.521+0000 7fcabf400700 1 -- 172.21.15.27:0/3045312632 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fcab009c830 con 0x7fcac8102ab0 2024-04-26T22:55:37.522 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T22:55:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.524+0000 7fcac7200700 1 -- 172.21.15.27:0/3045312632 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcaac0a3cc0 msgr2=0x7fcaac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.524+0000 7fcac7200700 1 --2- 172.21.15.27:0/3045312632 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcaac0a3cc0 0x7fcaac0a6180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcac819cba0 tx=0x7fcab8009260 comp rx=0 tx=0).stop 2024-04-26T22:55:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.525+0000 7fcac7200700 1 -- 172.21.15.27:0/3045312632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcac8102ab0 msgr2=0x7fcac81a1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.525+0000 7fcac7200700 1 --2- 172.21.15.27:0/3045312632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcac8102ab0 0x7fcac81a1de0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcab000e9c0 tx=0x7fcab000ecd0 comp rx=0 tx=0).stop 2024-04-26T22:55:37.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.525+0000 7fcac7200700 1 -- 172.21.15.27:0/3045312632 shutdown_connections 2024-04-26T22:55:37.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.525+0000 7fcac7200700 1 --2- 172.21.15.27:0/3045312632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac810fcb0 0x7fcac819ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:37.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.525+0000 7fcac7200700 1 --2- 172.21.15.27:0/3045312632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcac8102100 0x7fcac81a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:37.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.525+0000 7fcac7200700 1 --2- 172.21.15.27:0/3045312632 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcaac0a3cc0 0x7fcaac0a6180 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:37.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.525+0000 7fcac7200700 1 --2- 172.21.15.27:0/3045312632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcac8102ab0 0x7fcac81a1de0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:37.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.525+0000 7fcac7200700 1 -- 172.21.15.27:0/3045312632 >> 172.21.15.27:0/3045312632 conn(0x7fcac80fdd40 msgr2=0x7fcac8100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:37.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.525+0000 7fcac7200700 1 -- 172.21.15.27:0/3045312632 shutdown_connections 2024-04-26T22:55:37.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:37.525+0000 7fcac7200700 1 -- 172.21.15.27:0/3045312632 wait complete. 2024-04-26T22:55:37.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:37 smithi080 ceph-mon[32324]: pgmap v336: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 B/s wr, 5 op/s 2024-04-26T22:55:37.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:37 smithi094 ceph-mon[32889]: pgmap v336: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 B/s wr, 5 op/s 2024-04-26T22:55:37.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:37 smithi027 ceph-mon[28034]: pgmap v336: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 B/s wr, 5 op/s 2024-04-26T22:55:38.443 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T22:55:38.444 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T22:55:38.444 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-04-26T22:55:38.444 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-26T22:55:38.444 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T22:55:38.444 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T22:55:38.444 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T22:55:38.444 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T22:55:38.444 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T22:55:38.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:55:38.444 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-04-26T22:55:38.505 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:55:38.506 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:55:38.506 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:55:38.506 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:55:38.506 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-04-26T22:55:38.507 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:55:38.507 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:55:38.507 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:55:38.507 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-04-26T22:55:38.507 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:55:38.507 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:55:38.508 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-04-26T22:55:38.508 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T22:55:38.508 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-04-26T22:55:38.508 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-04-26T22:55:38.508 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T22:55:38.508 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42044sec preferred_lft 42044sec 2024-04-26T22:55:38.509 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2024-04-26T22:55:38.509 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:55:38.509 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:55:38.509 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-04-26T22:55:38.509 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-04-26T22:55:38.512 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T22:55:38.512 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:55:38.512 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-26T22:55:38.512 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2024-04-26T22:55:38.512 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2024-04-26T22:55:38.513 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2024-04-26T22:55:38.513 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T22:55:38.513 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-26T22:55:38.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:38 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3045312632' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T22:55:38.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:55:38.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:38 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3045312632' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T22:55:38.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:38 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3045312632' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T22:55:39.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:55:39.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:55:39.039 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T22:55:39.109 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-04-26T22:55:39.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:55:39.113 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-04-26T22:55:39.171 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-04-26T22:55:39.171 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T22:55:39.171 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T22:55:39.171 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T22:55:39.171 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T22:55:39.171 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:55:39.171 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-26T22:55:39.172 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T22:55:39.172 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T22:55:39.172 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T22:55:39.172 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-26T22:55:39.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:55:39.590 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:39 smithi027 ceph-mon[28034]: pgmap v337: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 762 B/s wr, 3 op/s 2024-04-26T22:55:39.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:39 smithi080 ceph-mon[32324]: pgmap v337: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 762 B/s wr, 3 op/s 2024-04-26T22:55:39.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:39 smithi094 ceph-mon[32889]: pgmap v337: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 762 B/s wr, 3 op/s 2024-04-26T22:55:39.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:55:39.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:55:39.985 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-04-26T22:55:40.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:55:40.036 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2024-04-26T22:55:40.075 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:55:40.076 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-26T22:55:40.076 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T22:55:40.076 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-26T22:55:40.076 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-26T22:55:40.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:55:40.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:55:40.287 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-26T22:55:40.287 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:55:40.287 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-26T22:55:40.287 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-26T22:55:40.287 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T22:55:40.287 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-26T22:55:40.287 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-26T22:55:40.287 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-26T22:55:40.287 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-26T22:55:40.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:55:40.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:55:40.563 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:55:40.563 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-26T22:55:40.563 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-04-26T22:55:40.563 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T22:55:40.563 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-26T22:55:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:55:40.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:55:40.758 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-26T22:55:40.758 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:55:40.758 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-26T22:55:40.805 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T22:55:40.805 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:55:40.805 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-26T22:55:40.898 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-04-26T22:55:40.976 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T22:55:41.029 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-04-26T22:55:41.029 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-04-26T22:55:41.029 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-04-26T22:55:41.030 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-04-26T22:55:41.030 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-04-26T22:55:41.030 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-04-26T22:55:41.030 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-04-26T22:55:41.030 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-04-26T22:55:41.030 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-04-26T22:55:41.030 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-04-26T22:55:41.030 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-04-26T22:55:41.030 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-04-26T22:55:41.030 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T22:55:41.030 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-04-26T22:55:41.030 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-04-26T22:55:41.030 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-04-26T22:55:41.031 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-04-26T22:55:41.031 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-04-26T22:55:41.031 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-04-26T22:55:41.031 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T22:55:41.031 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T22:55:41.031 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T22:55:41.031 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T22:55:41.031 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-04-26T22:55:41.031 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-04-26T22:55:41.031 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-04-26T22:55:41.031 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/7f5bab4b027dddd3c39cd19cb622e30c5ae7cb90dc9dacafba1792cc38d36284/merged 2024-04-26T22:55:41.031 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/ae11748494a13abbc9dcdec7e588d25f68d45861f5c4aaa8e89d9fdb241653e1/merged 2024-04-26T22:55:41.031 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-04-26T22:55:41.031 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T22:55:41.032 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/06cdfcc577ee7df5e5ca5a916ac1dda2906b405bcb305d61887bc56cbaa3b30b/merged 2024-04-26T22:55:41.032 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/39f6c5c7e461e6627c7fc13101fd86ad8f05a6a83b9f83bbab0278f9355c054a/merged 2024-04-26T22:55:41.032 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/1e8d13d25538689a1d906d512cfd7bafb9857b7481b43675a8e8aff3e84f5626/merged 2024-04-26T22:55:41.032 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/f4854361a65643301aacc8bb7405f594ca9f993ea46c1b900d8bfe929d4e10bd/merged 2024-04-26T22:55:41.032 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/5b1a16aac8906fb41d919a527cde2c8cf3f532754335edc9dd634fef2bb71a30/merged 2024-04-26T22:55:41.032 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/bd63769c207bad0f1c68d351defdf537320fc94302b84694a899be8de4f136c4/merged 2024-04-26T22:55:41.032 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/abb82c6eab0d5408dfdfe77c6a4cee6c115b0f366f37015708198afc6470a9b8/merged 2024-04-26T22:55:41.032 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/7f73ec5b8f0477626f45a10c55ae7e3673239de9c02c22152650989c7dcbb037/merged 2024-04-26T22:55:41.032 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-04-26T22:55:41.032 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T22:55:41.032 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T22:55:41.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:55:41.033 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-04-26T22:55:41.084 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T22:55:41.084 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-26T22:55:41.089 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-04-26T22:55:41.160 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T22:55:41.204 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-04-26T22:55:41.204 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-04-26T22:55:41.204 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-04-26T22:55:41.204 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-04-26T22:55:41.204 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-04-26T22:55:41.204 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-04-26T22:55:41.205 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-04-26T22:55:41.205 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-04-26T22:55:41.205 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-04-26T22:55:41.205 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-04-26T22:55:41.205 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-04-26T22:55:41.205 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-04-26T22:55:41.205 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T22:55:41.205 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-04-26T22:55:41.205 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-04-26T22:55:41.205 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-04-26T22:55:41.205 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-04-26T22:55:41.205 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-04-26T22:55:41.205 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-04-26T22:55:41.205 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T22:55:41.206 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T22:55:41.206 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T22:55:41.206 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T22:55:41.206 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-04-26T22:55:41.206 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-04-26T22:55:41.206 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-04-26T22:55:41.206 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/7f5bab4b027dddd3c39cd19cb622e30c5ae7cb90dc9dacafba1792cc38d36284/merged 2024-04-26T22:55:41.206 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/ae11748494a13abbc9dcdec7e588d25f68d45861f5c4aaa8e89d9fdb241653e1/merged 2024-04-26T22:55:41.206 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-04-26T22:55:41.206 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T22:55:41.206 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/06cdfcc577ee7df5e5ca5a916ac1dda2906b405bcb305d61887bc56cbaa3b30b/merged 2024-04-26T22:55:41.206 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/39f6c5c7e461e6627c7fc13101fd86ad8f05a6a83b9f83bbab0278f9355c054a/merged 2024-04-26T22:55:41.206 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/1e8d13d25538689a1d906d512cfd7bafb9857b7481b43675a8e8aff3e84f5626/merged 2024-04-26T22:55:41.206 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/f4854361a65643301aacc8bb7405f594ca9f993ea46c1b900d8bfe929d4e10bd/merged 2024-04-26T22:55:41.207 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/5b1a16aac8906fb41d919a527cde2c8cf3f532754335edc9dd634fef2bb71a30/merged 2024-04-26T22:55:41.207 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/bd63769c207bad0f1c68d351defdf537320fc94302b84694a899be8de4f136c4/merged 2024-04-26T22:55:41.207 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/abb82c6eab0d5408dfdfe77c6a4cee6c115b0f366f37015708198afc6470a9b8/merged 2024-04-26T22:55:41.207 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/7f73ec5b8f0477626f45a10c55ae7e3673239de9c02c22152650989c7dcbb037/merged 2024-04-26T22:55:41.207 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-04-26T22:55:41.207 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T22:55:41.207 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T22:55:41.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:55:41.208 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-04-26T22:55:41.302 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:2024-04-26T22:55:41.302+0000 7f42280f1480 -1 init, newargv = 0x55d6726018a0 newargc=15 2024-04-26T22:55:41.303 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[71447]: starting ceph client 2024-04-26T22:55:41.313 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[71447]: starting fuse 2024-04-26T22:55:41.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:41 smithi094 ceph-mon[32889]: pgmap v338: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s wr, 3 op/s 2024-04-26T22:55:41.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:41 smithi027 ceph-mon[28034]: pgmap v338: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s wr, 3 op/s 2024-04-26T22:55:42.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:41 smithi080 ceph-mon[32324]: pgmap v338: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s wr, 3 op/s 2024-04-26T22:55:42.398 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-04-26T22:55:42.495 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T22:55:42.553 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-04-26T22:55:42.553 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-04-26T22:55:42.553 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-04-26T22:55:42.553 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-04-26T22:55:42.553 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-04-26T22:55:42.553 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-04-26T22:55:42.553 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-04-26T22:55:42.553 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-04-26T22:55:42.553 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-04-26T22:55:42.554 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-04-26T22:55:42.554 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-04-26T22:55:42.554 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-04-26T22:55:42.554 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T22:55:42.554 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-04-26T22:55:42.554 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-04-26T22:55:42.554 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-04-26T22:55:42.554 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-04-26T22:55:42.554 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-04-26T22:55:42.554 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-04-26T22:55:42.554 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T22:55:42.554 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T22:55:42.555 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T22:55:42.555 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T22:55:42.555 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-04-26T22:55:42.555 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-04-26T22:55:42.555 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-04-26T22:55:42.555 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/7f5bab4b027dddd3c39cd19cb622e30c5ae7cb90dc9dacafba1792cc38d36284/merged 2024-04-26T22:55:42.555 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/ae11748494a13abbc9dcdec7e588d25f68d45861f5c4aaa8e89d9fdb241653e1/merged 2024-04-26T22:55:42.555 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-04-26T22:55:42.555 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T22:55:42.555 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/06cdfcc577ee7df5e5ca5a916ac1dda2906b405bcb305d61887bc56cbaa3b30b/merged 2024-04-26T22:55:42.555 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/39f6c5c7e461e6627c7fc13101fd86ad8f05a6a83b9f83bbab0278f9355c054a/merged 2024-04-26T22:55:42.555 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/1e8d13d25538689a1d906d512cfd7bafb9857b7481b43675a8e8aff3e84f5626/merged 2024-04-26T22:55:42.555 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/f4854361a65643301aacc8bb7405f594ca9f993ea46c1b900d8bfe929d4e10bd/merged 2024-04-26T22:55:42.555 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/5b1a16aac8906fb41d919a527cde2c8cf3f532754335edc9dd634fef2bb71a30/merged 2024-04-26T22:55:42.555 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/bd63769c207bad0f1c68d351defdf537320fc94302b84694a899be8de4f136c4/merged 2024-04-26T22:55:42.556 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/abb82c6eab0d5408dfdfe77c6a4cee6c115b0f366f37015708198afc6470a9b8/merged 2024-04-26T22:55:42.556 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/7f73ec5b8f0477626f45a10c55ae7e3673239de9c02c22152650989c7dcbb037/merged 2024-04-26T22:55:42.556 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-04-26T22:55:42.556 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T22:55:42.556 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T22:55:42.556 INFO:teuthology.orchestra.run.smithi027.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-26T22:55:42.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:55:42.557 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-04-26T22:55:42.613 INFO:teuthology.orchestra.run.smithi027.stdout:114 2024-04-26T22:55:42.613 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-04-26T22:55:42.613 DEBUG:teuthology.orchestra.run.smithi027:> sudo stdin-killer -- python3 -c ' 2024-04-26T22:55:42.614 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2024-04-26T22:55:42.614 DEBUG:teuthology.orchestra.run.smithi027:> import re 2024-04-26T22:55:42.614 DEBUG:teuthology.orchestra.run.smithi027:> import os 2024-04-26T22:55:42.614 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2024-04-26T22:55:42.614 DEBUG:teuthology.orchestra.run.smithi027:> 2024-04-26T22:55:42.614 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2024-04-26T22:55:42.614 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-26T22:55:42.615 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2024-04-26T22:55:42.615 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-26T22:55:42.615 DEBUG:teuthology.orchestra.run.smithi027:> 2024-04-26T22:55:42.615 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2024-04-26T22:55:42.615 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2024-04-26T22:55:42.615 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2024-04-26T22:55:42.615 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2024-04-26T22:55:42.615 DEBUG:teuthology.orchestra.run.smithi027:> 2024-04-26T22:55:42.615 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2024-04-26T22:55:42.616 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T22:55:42.616 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T22:55:42.616 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T22:55:42.616 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2024-04-26T22:55:42.616 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2024-04-26T22:55:42.616 DEBUG:teuthology.orchestra.run.smithi027:> return f 2024-04-26T22:55:42.616 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T22:55:42.616 DEBUG:teuthology.orchestra.run.smithi027:> 2024-04-26T22:55:42.616 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.admin")) 2024-04-26T22:55:42.617 DEBUG:teuthology.orchestra.run.smithi027:> ' 2024-04-26T22:55:42.797 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.admin.71447.asok 2024-04-26T22:55:42.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:55:42.814 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.71447.asok 2024-04-26T22:55:42.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:55:42.814 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.71447.asok status 2024-04-26T22:55:43.044 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T22:55:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2024-04-26T22:55:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T22:55:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T22:55:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "admin", 2024-04-26T22:55:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2024-04-26T22:55:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-26T22:55:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "71447", 2024-04-26T22:55:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2024-04-26T22:55:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T22:55:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2024-04-26T22:55:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2024-04-26T22:55:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24704, 2024-04-26T22:55:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2024-04-26T22:55:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2024-04-26T22:55:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2024-04-26T22:55:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 24704 2024-04-26T22:55:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T22:55:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2024-04-26T22:55:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-04-26T22:55:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-04-26T22:55:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1664146308 2024-04-26T22:55:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T22:55:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T22:55:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2024-04-26T22:55:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-04-26T22:55:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-04-26T22:55:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1664146308 2024-04-26T22:55:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T22:55:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.24704 172.21.15.27:0/1664146308", 2024-04-26T22:55:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "172.21.15.27:0/1664146308", 2024-04-26T22:55:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2024-04-26T22:55:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 17, 2024-04-26T22:55:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 90, 2024-04-26T22:55:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2024-04-26T22:55:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2024-04-26T22:55:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2024-04-26T22:55:43.048 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T22:55:43.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:55:43.060 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2024-04-26T22:55:43.060 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-26T22:55:43.119 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2024-04-26T22:55:43.120 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-26T22:55:43.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:55:43.120 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-26T22:55:43.216 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-26T22:55:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:55:43.378 INFO:teuthology.orchestra.run.smithi027.stdout:# file: . 2024-04-26T22:55:43.379 INFO:teuthology.orchestra.run.smithi027.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-26T22:55:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:43.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:55:43.390 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:55:43.390 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:55:43.443 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:55:43.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.651+0000 7f3eea000640 1 -- 172.21.15.27:0/1501998545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee4103960 msgr2=0x7f3ee4103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:43.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.651+0000 7f3eea000640 1 --2- 172.21.15.27:0/1501998545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee4103960 0x7f3ee4103de0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009900 tx=0x7f3ed002a3f0 comp rx=0 tx=0).stop 2024-04-26T22:55:43.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.651+0000 7f3eea000640 1 -- 172.21.15.27:0/1501998545 shutdown_connections 2024-04-26T22:55:43.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.651+0000 7f3eea000640 1 --2- 172.21.15.27:0/1501998545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ee4104320 0x7f3ee410abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:43.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.651+0000 7f3eea000640 1 --2- 172.21.15.27:0/1501998545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee4103960 0x7f3ee4103de0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:43.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.651+0000 7f3eea000640 1 --2- 172.21.15.27:0/1501998545 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ee4102770 0x7f3ee4102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:43.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.651+0000 7f3eea000640 1 -- 172.21.15.27:0/1501998545 >> 172.21.15.27:0/1501998545 conn(0x7f3ee40fdce0 msgr2=0x7f3ee4100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:43.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.651+0000 7f3eea000640 1 -- 172.21.15.27:0/1501998545 shutdown_connections 2024-04-26T22:55:43.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.651+0000 7f3eea000640 1 -- 172.21.15.27:0/1501998545 wait complete. 2024-04-26T22:55:43.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.652+0000 7f3eea000640 1 Processor -- start 2024-04-26T22:55:43.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.653+0000 7f3eea000640 1 -- start start 2024-04-26T22:55:43.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.653+0000 7f3eea000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ee4102770 0x7f3ee419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:43.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.653+0000 7f3eea000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee4103960 0x7f3ee419b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:43.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.653+0000 7f3eea000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ee4104320 0x7f3ee41a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:43.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.653+0000 7f3eea000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee410e2d0 con 0x7f3ee4104320 2024-04-26T22:55:43.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.653+0000 7f3eea000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee410e150 con 0x7f3ee4102770 2024-04-26T22:55:43.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.653+0000 7f3eea000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee410e450 con 0x7f3ee4103960 2024-04-26T22:55:43.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.653+0000 7f3ee9600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ee4104320 0x7f3ee41a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:43.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.653+0000 7f3ee3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee4103960 0x7f3ee419b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:43.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.654+0000 7f3ee8c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ee4102770 0x7f3ee419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:43.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.654+0000 7f3ee3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee4103960 0x7f3ee419b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:40860/0 (socket says 172.21.15.27:40860) 2024-04-26T22:55:43.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.654+0000 7f3ee3e00640 1 -- 172.21.15.27:0/3689506910 learned_addr learned my addr 172.21.15.27:0/3689506910 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:43.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.654+0000 7f3ee3e00640 1 -- 172.21.15.27:0/3689506910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ee4102770 msgr2=0x7f3ee419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:43.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.654+0000 7f3ee3e00640 1 --2- 172.21.15.27:0/3689506910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ee4102770 0x7f3ee419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:43.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.654+0000 7f3ee3e00640 1 -- 172.21.15.27:0/3689506910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ee4104320 msgr2=0x7f3ee41a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:43.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.654+0000 7f3ee3e00640 1 --2- 172.21.15.27:0/3689506910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ee4104320 0x7f3ee41a2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:43.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.654+0000 7f3ee3e00640 1 -- 172.21.15.27:0/3689506910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee41a2770 con 0x7f3ee4103960 2024-04-26T22:55:43.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.654+0000 7f3ee3e00640 1 --2- 172.21.15.27:0/3689506910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee4103960 0x7f3ee419b090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0031440 tx=0x7f3ed0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:43.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.655+0000 7f3ee1600640 1 -- 172.21.15.27:0/3689506910 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed003f070 con 0x7f3ee4103960 2024-04-26T22:55:43.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.655+0000 7f3ee1600640 1 -- 172.21.15.27:0/3689506910 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ed0031c50 con 0x7f3ee4103960 2024-04-26T22:55:43.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.655+0000 7f3ee1600640 1 -- 172.21.15.27:0/3689506910 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0035030 con 0x7f3ee4103960 2024-04-26T22:55:43.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.655+0000 7f3eea000640 1 -- 172.21.15.27:0/3689506910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee41a2a00 con 0x7f3ee4103960 2024-04-26T22:55:43.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.655+0000 7f3eea000640 1 -- 172.21.15.27:0/3689506910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee41a2ea0 con 0x7f3ee4103960 2024-04-26T22:55:43.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.657+0000 7f3eea000640 1 -- 172.21.15.27:0/3689506910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee41a30d0 con 0x7f3ee4103960 2024-04-26T22:55:43.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.658+0000 7f3ee1600640 1 -- 172.21.15.27:0/3689506910 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3ed0042070 con 0x7f3ee4103960 2024-04-26T22:55:43.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.660+0000 7f3ee1600640 1 --2- 172.21.15.27:0/3689506910 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3ecc0a3c50 0x7f3ecc0a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:43.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.660+0000 7f3ee1600640 1 -- 172.21.15.27:0/3689506910 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3ed00f0e00 con 0x7f3ee4103960 2024-04-26T22:55:43.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.660+0000 7f3ee8c00640 1 --2- 172.21.15.27:0/3689506910 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3ecc0a3c50 0x7f3ecc0a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:43.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.661+0000 7f3ee8c00640 1 --2- 172.21.15.27:0/3689506910 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3ecc0a3c50 0x7f3ecc0a6110 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3ee41037c0 tx=0x7f3ed8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:43.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.664+0000 7f3ee1600640 1 -- 172.21.15.27:0/3689506910 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3ed00bcef0 con 0x7f3ee4103960 2024-04-26T22:55:43.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:43 smithi094 ceph-mon[32889]: pgmap v339: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:43.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:43 smithi027 ceph-mon[28034]: pgmap v339: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:43.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.877+0000 7f3eea000640 1 -- 172.21.15.27:0/3689506910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ee41a33c0 con 0x7f3ee4103960 2024-04-26T22:55:43.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.879+0000 7f3ee1600640 1 -- 172.21.15.27:0/3689506910 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24009 (secure 0 0 0) 0x7f3ed00c11f0 con 0x7f3ee4103960 2024-04-26T22:55:43.880 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:43.880 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:55:30.886974+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:55:43.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.882+0000 7f3eea000640 1 -- 172.21.15.27:0/3689506910 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3ecc0a3c50 msgr2=0x7f3ecc0a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:43.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.882+0000 7f3eea000640 1 --2- 172.21.15.27:0/3689506910 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3ecc0a3c50 0x7f3ecc0a6110 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3ee41037c0 tx=0x7f3ed8005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:43.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.882+0000 7f3eea000640 1 -- 172.21.15.27:0/3689506910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee4103960 msgr2=0x7f3ee419b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:43.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.882+0000 7f3eea000640 1 --2- 172.21.15.27:0/3689506910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee4103960 0x7f3ee419b090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0031440 tx=0x7f3ed0003c10 comp rx=0 tx=0).stop 2024-04-26T22:55:43.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.882+0000 7f3eea000640 1 -- 172.21.15.27:0/3689506910 shutdown_connections 2024-04-26T22:55:43.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.882+0000 7f3eea000640 1 --2- 172.21.15.27:0/3689506910 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3ecc0a3c50 0x7f3ecc0a6110 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:43.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.882+0000 7f3eea000640 1 --2- 172.21.15.27:0/3689506910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ee4104320 0x7f3ee41a2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:43.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.882+0000 7f3eea000640 1 --2- 172.21.15.27:0/3689506910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee4103960 0x7f3ee419b090 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:43.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.882+0000 7f3eea000640 1 --2- 172.21.15.27:0/3689506910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ee4102770 0x7f3ee419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:43.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.882+0000 7f3eea000640 1 -- 172.21.15.27:0/3689506910 >> 172.21.15.27:0/3689506910 conn(0x7f3ee40fdce0 msgr2=0x7f3ee4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:43.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.882+0000 7f3eea000640 1 -- 172.21.15.27:0/3689506910 shutdown_connections 2024-04-26T22:55:43.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:43.883+0000 7f3eea000640 1 -- 172.21.15.27:0/3689506910 wait complete. 2024-04-26T22:55:43.902 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:49:54.482679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:55:22.899174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:55:24.218295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T22:55:28.394547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T22:55:43.903 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:55:44.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:43 smithi080 ceph-mon[32324]: pgmap v339: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:44.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.118+0000 7f650fe00640 1 -- 172.21.15.27:0/184561790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65101027a0 msgr2=0x7f6510102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:44.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.119+0000 7f650fe00640 1 --2- 172.21.15.27:0/184561790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65101027a0 0x7f6510102bc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6500009860 tx=0x7f650002a340 comp rx=0 tx=0).stop 2024-04-26T22:55:44.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.119+0000 7f650fe00640 1 -- 172.21.15.27:0/184561790 shutdown_connections 2024-04-26T22:55:44.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.119+0000 7f650fe00640 1 --2- 172.21.15.27:0/184561790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6510103990 0x7f6510103e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.119+0000 7f650fe00640 1 --2- 172.21.15.27:0/184561790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6510104350 0x7f651010abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.119+0000 7f650fe00640 1 --2- 172.21.15.27:0/184561790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65101027a0 0x7f6510102bc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.119+0000 7f650fe00640 1 -- 172.21.15.27:0/184561790 >> 172.21.15.27:0/184561790 conn(0x7f65100fdd10 msgr2=0x7f6510100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:44.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.119+0000 7f650fe00640 1 -- 172.21.15.27:0/184561790 shutdown_connections 2024-04-26T22:55:44.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.119+0000 7f650fe00640 1 -- 172.21.15.27:0/184561790 wait complete. 2024-04-26T22:55:44.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.120+0000 7f650fe00640 1 Processor -- start 2024-04-26T22:55:44.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.120+0000 7f650fe00640 1 -- start start 2024-04-26T22:55:44.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.120+0000 7f650fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65101027a0 0x7f651019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:44.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.120+0000 7f650fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6510103990 0x7f651019b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:44.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.120+0000 7f650fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6510104350 0x7f65101a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:44.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.120+0000 7f650fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f651010e2a0 con 0x7f65101027a0 2024-04-26T22:55:44.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.120+0000 7f650fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f651010e120 con 0x7f6510103990 2024-04-26T22:55:44.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.120+0000 7f650fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f651010e420 con 0x7f6510104350 2024-04-26T22:55:44.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.121+0000 7f650ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65101027a0 0x7f651019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:44.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.121+0000 7f650f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6510104350 0x7f65101a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:44.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.121+0000 7f650ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65101027a0 0x7f651019ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50030/0 (socket says 172.21.15.27:50030) 2024-04-26T22:55:44.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.121+0000 7f650ea00640 1 -- 172.21.15.27:0/2763019931 learned_addr learned my addr 172.21.15.27:0/2763019931 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:44.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.121+0000 7f650e000640 1 --2- 172.21.15.27:0/2763019931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6510103990 0x7f651019b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:44.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.121+0000 7f650f400640 1 -- 172.21.15.27:0/2763019931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6510103990 msgr2=0x7f651019b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:44.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.121+0000 7f650f400640 1 --2- 172.21.15.27:0/2763019931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6510103990 0x7f651019b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.121+0000 7f650f400640 1 -- 172.21.15.27:0/2763019931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65101027a0 msgr2=0x7f651019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:44.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.121+0000 7f650f400640 1 --2- 172.21.15.27:0/2763019931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65101027a0 0x7f651019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.121+0000 7f650f400640 1 -- 172.21.15.27:0/2763019931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65101a2870 con 0x7f6510104350 2024-04-26T22:55:44.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.121+0000 7f650ea00640 1 --2- 172.21.15.27:0/2763019931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65101027a0 0x7f651019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:44.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.121+0000 7f650f400640 1 --2- 172.21.15.27:0/2763019931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6510104350 0x7f65101a21d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f650800ec50 tx=0x7f650800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:44.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.122+0000 7f6507400640 1 -- 172.21.15.27:0/2763019931 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6508003b70 con 0x7f6510104350 2024-04-26T22:55:44.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.122+0000 7f6507400640 1 -- 172.21.15.27:0/2763019931 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6508003d10 con 0x7f6510104350 2024-04-26T22:55:44.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.122+0000 7f650fe00640 1 -- 172.21.15.27:0/2763019931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65101a2b60 con 0x7f6510104350 2024-04-26T22:55:44.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.122+0000 7f6507400640 1 -- 172.21.15.27:0/2763019931 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6508010830 con 0x7f6510104350 2024-04-26T22:55:44.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.122+0000 7f650fe00640 1 -- 172.21.15.27:0/2763019931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65101a30b0 con 0x7f6510104350 2024-04-26T22:55:44.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.123+0000 7f650fe00640 1 -- 172.21.15.27:0/2763019931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65101a3280 con 0x7f6510104350 2024-04-26T22:55:44.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.125+0000 7f6507400640 1 -- 172.21.15.27:0/2763019931 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6508003ec0 con 0x7f6510104350 2024-04-26T22:55:44.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.126+0000 7f6507400640 1 --2- 172.21.15.27:0/2763019931 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64f40a3c30 0x7f64f40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:44.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.127+0000 7f6507400640 1 -- 172.21.15.27:0/2763019931 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f65080d0be0 con 0x7f6510104350 2024-04-26T22:55:44.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.127+0000 7f650ea00640 1 --2- 172.21.15.27:0/2763019931 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64f40a3c30 0x7f64f40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:44.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.128+0000 7f650ea00640 1 --2- 172.21.15.27:0/2763019931 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64f40a3c30 0x7f64f40a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6500000c00 tx=0x7f6500030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:44.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.131+0000 7f6507400640 1 -- 172.21.15.27:0/2763019931 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f650809d320 con 0x7f6510104350 2024-04-26T22:55:44.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.340+0000 7f650fe00640 1 -- 172.21.15.27:0/2763019931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f65101a3450 con 0x7f6510104350 2024-04-26T22:55:44.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.341+0000 7f6507400640 1 -- 172.21.15.27:0/2763019931 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f650809cc20 con 0x7f6510104350 2024-04-26T22:55:44.342 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T22:55:44.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.344+0000 7f650fe00640 1 -- 172.21.15.27:0/2763019931 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64f40a3c30 msgr2=0x7f64f40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:44.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.344+0000 7f650fe00640 1 --2- 172.21.15.27:0/2763019931 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64f40a3c30 0x7f64f40a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6500000c00 tx=0x7f6500030450 comp rx=0 tx=0).stop 2024-04-26T22:55:44.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.344+0000 7f650fe00640 1 -- 172.21.15.27:0/2763019931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6510104350 msgr2=0x7f65101a21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:44.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.344+0000 7f650fe00640 1 --2- 172.21.15.27:0/2763019931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6510104350 0x7f65101a21d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f650800ec50 tx=0x7f650800c6a0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.344+0000 7f650fe00640 1 -- 172.21.15.27:0/2763019931 shutdown_connections 2024-04-26T22:55:44.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.344+0000 7f650fe00640 1 --2- 172.21.15.27:0/2763019931 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64f40a3c30 0x7f64f40a60f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.344+0000 7f650fe00640 1 --2- 172.21.15.27:0/2763019931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6510103990 0x7f651019b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.344+0000 7f650fe00640 1 --2- 172.21.15.27:0/2763019931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6510104350 0x7f65101a21d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.344+0000 7f650fe00640 1 --2- 172.21.15.27:0/2763019931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65101027a0 0x7f651019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.344+0000 7f650fe00640 1 -- 172.21.15.27:0/2763019931 >> 172.21.15.27:0/2763019931 conn(0x7f65100fdd10 msgr2=0x7f6510074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:44.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.345+0000 7f650fe00640 1 -- 172.21.15.27:0/2763019931 shutdown_connections 2024-04-26T22:55:44.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.345+0000 7f650fe00640 1 -- 172.21.15.27:0/2763019931 wait complete. 2024-04-26T22:55:44.367 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:55:44.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.567+0000 7f18e2000640 1 -- 172.21.15.27:0/3470153015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc104b80 msgr2=0x7f18dc104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:44.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.567+0000 7f18e2000640 1 --2- 172.21.15.27:0/3470153015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc104b80 0x7f18dc104fa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009900 tx=0x7f18cc02a3f0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.567+0000 7f18e2000640 1 -- 172.21.15.27:0/3470153015 shutdown_connections 2024-04-26T22:55:44.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.567+0000 7f18e2000640 1 --2- 172.21.15.27:0/3470153015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18dc106730 0x7f18dc10cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.567+0000 7f18e2000640 1 --2- 172.21.15.27:0/3470153015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18dc105d70 0x7f18dc1061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.567+0000 7f18e2000640 1 --2- 172.21.15.27:0/3470153015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc104b80 0x7f18dc104fa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.567+0000 7f18e2000640 1 -- 172.21.15.27:0/3470153015 >> 172.21.15.27:0/3470153015 conn(0x7f18dc1000f0 msgr2=0x7f18dc102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:44.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.567+0000 7f18e2000640 1 -- 172.21.15.27:0/3470153015 shutdown_connections 2024-04-26T22:55:44.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.567+0000 7f18e2000640 1 -- 172.21.15.27:0/3470153015 wait complete. 2024-04-26T22:55:44.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.568+0000 7f18e2000640 1 Processor -- start 2024-04-26T22:55:44.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.568+0000 7f18e2000640 1 -- start start 2024-04-26T22:55:44.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.568+0000 7f18e2000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18dc104b80 0x7f18dc19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:44.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.568+0000 7f18e2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18dc105d70 0x7f18dc19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:44.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.568+0000 7f18e2000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc106730 0x7f18dc1a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:44.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.568+0000 7f18e2000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1106d0 con 0x7f18dc105d70 2024-04-26T22:55:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.568+0000 7f18e2000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc110550 con 0x7f18dc106730 2024-04-26T22:55:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.568+0000 7f18e2000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc110850 con 0x7f18dc104b80 2024-04-26T22:55:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.569+0000 7f18db400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18dc104b80 0x7f18dc19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.569+0000 7f18d3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18dc105d70 0x7f18dc19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.569+0000 7f18dbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc106730 0x7f18dc1a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.569+0000 7f18db400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18dc104b80 0x7f18dc19add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:40896/0 (socket says 172.21.15.27:40896) 2024-04-26T22:55:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.569+0000 7f18db400640 1 -- 172.21.15.27:0/3502677130 learned_addr learned my addr 172.21.15.27:0/3502677130 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.569+0000 7f18dbe00640 1 -- 172.21.15.27:0/3502677130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18dc104b80 msgr2=0x7f18dc19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.569+0000 7f18dbe00640 1 --2- 172.21.15.27:0/3502677130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18dc104b80 0x7f18dc19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.569+0000 7f18dbe00640 1 -- 172.21.15.27:0/3502677130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18dc105d70 msgr2=0x7f18dc19b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.569+0000 7f18dbe00640 1 --2- 172.21.15.27:0/3502677130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18dc105d70 0x7f18dc19b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.569+0000 7f18dbe00640 1 -- 172.21.15.27:0/3502677130 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18dc1a29f0 con 0x7f18dc106730 2024-04-26T22:55:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.569+0000 7f18db400640 1 --2- 172.21.15.27:0/3502677130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18dc104b80 0x7f18dc19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.569+0000 7f18dbe00640 1 --2- 172.21.15.27:0/3502677130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc106730 0x7f18dc1a2380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009fd0 tx=0x7f18d400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.570+0000 7f18d8c00640 1 -- 172.21.15.27:0/3502677130 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d400ce20 con 0x7f18dc106730 2024-04-26T22:55:44.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.570+0000 7f18d8c00640 1 -- 172.21.15.27:0/3502677130 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18d4005020 con 0x7f18dc106730 2024-04-26T22:55:44.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.570+0000 7f18d8c00640 1 -- 172.21.15.27:0/3502677130 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4010690 con 0x7f18dc106730 2024-04-26T22:55:44.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.570+0000 7f18e2000640 1 -- 172.21.15.27:0/3502677130 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18dc1a2ce0 con 0x7f18dc106730 2024-04-26T22:55:44.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.570+0000 7f18e2000640 1 -- 172.21.15.27:0/3502677130 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18dc1a3180 con 0x7f18dc106730 2024-04-26T22:55:44.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.571+0000 7f18e2000640 1 -- 172.21.15.27:0/3502677130 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18dc10bf70 con 0x7f18dc106730 2024-04-26T22:55:44.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.573+0000 7f18d8c00640 1 -- 172.21.15.27:0/3502677130 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f18d40054d0 con 0x7f18dc106730 2024-04-26T22:55:44.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.574+0000 7f18d8c00640 1 --2- 172.21.15.27:0/3502677130 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f18c00a3bb0 0x7f18c00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:44.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.574+0000 7f18d8c00640 1 -- 172.21.15.27:0/3502677130 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f18d40d0d60 con 0x7f18dc106730 2024-04-26T22:55:44.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.575+0000 7f18db400640 1 --2- 172.21.15.27:0/3502677130 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f18c00a3bb0 0x7f18c00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:44.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.575+0000 7f18db400640 1 --2- 172.21.15.27:0/3502677130 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f18c00a3bb0 0x7f18c00a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f18cc000c00 tx=0x7f18cc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.579+0000 7f18d8c00640 1 -- 172.21.15.27:0/3502677130 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f18d409d4a0 con 0x7f18dc106730 2024-04-26T22:55:44.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:44 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3689506910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:44.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:44 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2763019931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:55:44.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.799+0000 7f18e2000640 1 -- 172.21.15.27:0/3502677130 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f18dc0635d0 con 0x7f18dc106730 2024-04-26T22:55:44.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.800+0000 7f18d8c00640 1 -- 172.21.15.27:0/3502677130 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f18d409cda0 con 0x7f18dc106730 2024-04-26T22:55:44.801 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T22:55:44.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.803+0000 7f18e2000640 1 -- 172.21.15.27:0/3502677130 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f18c00a3bb0 msgr2=0x7f18c00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:44.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.803+0000 7f18e2000640 1 --2- 172.21.15.27:0/3502677130 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f18c00a3bb0 0x7f18c00a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f18cc000c00 tx=0x7f18cc005a20 comp rx=0 tx=0).stop 2024-04-26T22:55:44.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.803+0000 7f18e2000640 1 -- 172.21.15.27:0/3502677130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc106730 msgr2=0x7f18dc1a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:44.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.803+0000 7f18e2000640 1 --2- 172.21.15.27:0/3502677130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc106730 0x7f18dc1a2380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009fd0 tx=0x7f18d400ef10 comp rx=0 tx=0).stop 2024-04-26T22:55:44.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.803+0000 7f18e2000640 1 -- 172.21.15.27:0/3502677130 shutdown_connections 2024-04-26T22:55:44.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.803+0000 7f18e2000640 1 --2- 172.21.15.27:0/3502677130 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f18c00a3bb0 0x7f18c00a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.803+0000 7f18e2000640 1 --2- 172.21.15.27:0/3502677130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc106730 0x7f18dc1a2380 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.803+0000 7f18e2000640 1 --2- 172.21.15.27:0/3502677130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18dc105d70 0x7f18dc19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.803+0000 7f18e2000640 1 --2- 172.21.15.27:0/3502677130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18dc104b80 0x7f18dc19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:44.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.803+0000 7f18e2000640 1 -- 172.21.15.27:0/3502677130 >> 172.21.15.27:0/3502677130 conn(0x7f18dc1000f0 msgr2=0x7f18dc109140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:44.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.804+0000 7f18e2000640 1 -- 172.21.15.27:0/3502677130 shutdown_connections 2024-04-26T22:55:44.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:44.804+0000 7f18e2000640 1 -- 172.21.15.27:0/3502677130 wait complete. 2024-04-26T22:55:44.823 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:55:45.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:44 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3689506910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:45.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:44 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2763019931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:55:45.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:44 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3689506910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:45.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:44 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2763019931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:55:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.044+0000 7f5c29400640 1 -- 172.21.15.27:0/1751439850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c24101060 msgr2=0x7f5c24101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.044+0000 7f5c29400640 1 --2- 172.21.15.27:0/1751439850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c24101060 0x7f5c24101480 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009900 tx=0x7f5c1402a480 comp rx=0 tx=0).stop 2024-04-26T22:55:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.044+0000 7f5c29400640 1 -- 172.21.15.27:0/1751439850 shutdown_connections 2024-04-26T22:55:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.044+0000 7f5c29400640 1 --2- 172.21.15.27:0/1751439850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c24102360 0x7f5c241027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.044+0000 7f5c29400640 1 --2- 172.21.15.27:0/1751439850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c241019c0 0x7f5c24101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.044+0000 7f5c29400640 1 --2- 172.21.15.27:0/1751439850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c24101060 0x7f5c24101480 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.044+0000 7f5c29400640 1 -- 172.21.15.27:0/1751439850 >> 172.21.15.27:0/1751439850 conn(0x7f5c240fcc00 msgr2=0x7f5c240ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.044+0000 7f5c29400640 1 -- 172.21.15.27:0/1751439850 shutdown_connections 2024-04-26T22:55:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.044+0000 7f5c29400640 1 -- 172.21.15.27:0/1751439850 wait complete. 2024-04-26T22:55:45.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.046+0000 7f5c29400640 1 Processor -- start 2024-04-26T22:55:45.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.046+0000 7f5c29400640 1 -- start start 2024-04-26T22:55:45.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.046+0000 7f5c29400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c24101060 0x7f5c2419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:45.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.046+0000 7f5c29400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c241019c0 0x7f5c2419b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:45.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.046+0000 7f5c29400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c24102360 0x7f5c241a2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:45.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.046+0000 7f5c29400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5c24110490 con 0x7f5c24101060 2024-04-26T22:55:45.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.046+0000 7f5c29400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5c24110310 con 0x7f5c241019c0 2024-04-26T22:55:45.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.046+0000 7f5c29400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5c24110610 con 0x7f5c24102360 2024-04-26T22:55:45.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.047+0000 7f5c23e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c24101060 0x7f5c2419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:45.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.047+0000 7f5c23e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c24101060 0x7f5c2419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50070/0 (socket says 172.21.15.27:50070) 2024-04-26T22:55:45.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.047+0000 7f5c23e00640 1 -- 172.21.15.27:0/1831400089 learned_addr learned my addr 172.21.15.27:0/1831400089 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:45.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.047+0000 7f5c28a00640 1 --2- 172.21.15.27:0/1831400089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c24102360 0x7f5c241a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:45.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.047+0000 7f5c23400640 1 --2- 172.21.15.27:0/1831400089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c241019c0 0x7f5c2419b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:45.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.047+0000 7f5c23e00640 1 -- 172.21.15.27:0/1831400089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c24102360 msgr2=0x7f5c241a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:45.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.047+0000 7f5c23e00640 1 --2- 172.21.15.27:0/1831400089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c24102360 0x7f5c241a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.047+0000 7f5c23e00640 1 -- 172.21.15.27:0/1831400089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c241019c0 msgr2=0x7f5c2419b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:45.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.047+0000 7f5c23e00640 1 --2- 172.21.15.27:0/1831400089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c241019c0 0x7f5c2419b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.047+0000 7f5c23e00640 1 -- 172.21.15.27:0/1831400089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c241a2780 con 0x7f5c24101060 2024-04-26T22:55:45.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.047+0000 7f5c28a00640 1 --2- 172.21.15.27:0/1831400089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c24102360 0x7f5c241a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:45.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.047+0000 7f5c23400640 1 --2- 172.21.15.27:0/1831400089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c241019c0 0x7f5c2419b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:45.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.047+0000 7f5c23e00640 1 --2- 172.21.15.27:0/1831400089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c24101060 0x7f5c2419ab60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5c14005940 tx=0x7f5c140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:45.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.048+0000 7f5c20c00640 1 -- 172.21.15.27:0/1831400089 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1403f030 con 0x7f5c24101060 2024-04-26T22:55:45.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.048+0000 7f5c29400640 1 -- 172.21.15.27:0/1831400089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c241a2a10 con 0x7f5c24101060 2024-04-26T22:55:45.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.048+0000 7f5c20c00640 1 -- 172.21.15.27:0/1831400089 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c14003cb0 con 0x7f5c24101060 2024-04-26T22:55:45.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.048+0000 7f5c20c00640 1 -- 172.21.15.27:0/1831400089 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14035070 con 0x7f5c24101060 2024-04-26T22:55:45.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.048+0000 7f5c29400640 1 -- 172.21.15.27:0/1831400089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c241a2f00 con 0x7f5c24101060 2024-04-26T22:55:45.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.049+0000 7f5c29400640 1 -- 172.21.15.27:0/1831400089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c2410bdb0 con 0x7f5c24101060 2024-04-26T22:55:45.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.051+0000 7f5c20c00640 1 -- 172.21.15.27:0/1831400089 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5c14042030 con 0x7f5c24101060 2024-04-26T22:55:45.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.053+0000 7f5c20c00640 1 --2- 172.21.15.27:0/1831400089 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5c080a3ba0 0x7f5c080a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:45.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.053+0000 7f5c20c00640 1 -- 172.21.15.27:0/1831400089 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5c140f0a50 con 0x7f5c24101060 2024-04-26T22:55:45.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.053+0000 7f5c23400640 1 --2- 172.21.15.27:0/1831400089 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5c080a3ba0 0x7f5c080a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.054+0000 7f5c23400640 1 --2- 172.21.15.27:0/1831400089 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5c080a3ba0 0x7f5c080a6060 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5c2419c070 tx=0x7f5c10009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:45.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.058+0000 7f5c20c00640 1 -- 172.21.15.27:0/1831400089 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5c14003e60 con 0x7f5c24101060 2024-04-26T22:55:45.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.271+0000 7f5c29400640 1 -- 172.21.15.27:0/1831400089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5c24063530 con 0x7f5c24101060 2024-04-26T22:55:45.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.272+0000 7f5c20c00640 1 -- 172.21.15.27:0/1831400089 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f5c140bcb40 con 0x7f5c24101060 2024-04-26T22:55:45.273 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-26T22:55:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.275+0000 7f5c29400640 1 -- 172.21.15.27:0/1831400089 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5c080a3ba0 msgr2=0x7f5c080a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.275+0000 7f5c29400640 1 --2- 172.21.15.27:0/1831400089 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5c080a3ba0 0x7f5c080a6060 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5c2419c070 tx=0x7f5c10009500 comp rx=0 tx=0).stop 2024-04-26T22:55:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.276+0000 7f5c29400640 1 -- 172.21.15.27:0/1831400089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c24101060 msgr2=0x7f5c2419ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.276+0000 7f5c29400640 1 --2- 172.21.15.27:0/1831400089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c24101060 0x7f5c2419ab60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5c14005940 tx=0x7f5c140042f0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.276+0000 7f5c29400640 1 -- 172.21.15.27:0/1831400089 shutdown_connections 2024-04-26T22:55:45.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.276+0000 7f5c29400640 1 --2- 172.21.15.27:0/1831400089 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5c080a3ba0 0x7f5c080a6060 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.276+0000 7f5c29400640 1 --2- 172.21.15.27:0/1831400089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c241019c0 0x7f5c2419b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.276+0000 7f5c29400640 1 --2- 172.21.15.27:0/1831400089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c24102360 0x7f5c241a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.276+0000 7f5c29400640 1 --2- 172.21.15.27:0/1831400089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c24101060 0x7f5c2419ab60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.276+0000 7f5c29400640 1 -- 172.21.15.27:0/1831400089 >> 172.21.15.27:0/1831400089 conn(0x7f5c240fcc00 msgr2=0x7f5c240fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:45.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.276+0000 7f5c29400640 1 -- 172.21.15.27:0/1831400089 shutdown_connections 2024-04-26T22:55:45.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.276+0000 7f5c29400640 1 -- 172.21.15.27:0/1831400089 wait complete. 2024-04-26T22:55:45.294 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T22:55:45.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.507+0000 7feee5200640 1 -- 172.21.15.27:0/2178531086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee010a180 msgr2=0x7feee010a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:45.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.507+0000 7feee5200640 1 --2- 172.21.15.27:0/2178531086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee010a180 0x7feee010a600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feecc009900 tx=0x7feecc02a400 comp rx=0 tx=0).stop 2024-04-26T22:55:45.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.507+0000 7feee5200640 1 -- 172.21.15.27:0/2178531086 shutdown_connections 2024-04-26T22:55:45.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.507+0000 7feee5200640 1 --2- 172.21.15.27:0/2178531086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee010a180 0x7feee010a600 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.507+0000 7feee5200640 1 --2- 172.21.15.27:0/2178531086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feee010ab40 0x7feee01113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.507+0000 7feee5200640 1 --2- 172.21.15.27:0/2178531086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee0108f90 0x7feee01093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.507+0000 7feee5200640 1 -- 172.21.15.27:0/2178531086 >> 172.21.15.27:0/2178531086 conn(0x7feee0077f40 msgr2=0x7feee007a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:45.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.507+0000 7feee5200640 1 -- 172.21.15.27:0/2178531086 shutdown_connections 2024-04-26T22:55:45.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.507+0000 7feee5200640 1 -- 172.21.15.27:0/2178531086 wait complete. 2024-04-26T22:55:45.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.509+0000 7feee5200640 1 Processor -- start 2024-04-26T22:55:45.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.509+0000 7feee5200640 1 -- start start 2024-04-26T22:55:45.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.509+0000 7feee5200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee0108f90 0x7feee019f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:45.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.509+0000 7feee5200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee010a180 0x7feee019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:45.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.509+0000 7feee5200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feee010ab40 0x7feee01a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:45.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.509+0000 7feee5200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feee0114ad0 con 0x7feee010ab40 2024-04-26T22:55:45.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.509+0000 7feee5200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7feee0114950 con 0x7feee010a180 2024-04-26T22:55:45.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.509+0000 7feee5200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feee0114c50 con 0x7feee0108f90 2024-04-26T22:55:45.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.510+0000 7feedde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee0108f90 0x7feee019f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:45.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.510+0000 7feedde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee0108f90 0x7feee019f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:40936/0 (socket says 172.21.15.27:40936) 2024-04-26T22:55:45.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.510+0000 7feedde00640 1 -- 172.21.15.27:0/1321333187 learned_addr learned my addr 172.21.15.27:0/1321333187 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:45.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.510+0000 7feede800640 1 --2- 172.21.15.27:0/1321333187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feee010ab40 0x7feee01a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:45.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.510+0000 7feedd400640 1 --2- 172.21.15.27:0/1321333187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee010a180 0x7feee019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:45.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.510+0000 7feedde00640 1 -- 172.21.15.27:0/1321333187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee010a180 msgr2=0x7feee019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:45.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.510+0000 7feedde00640 1 --2- 172.21.15.27:0/1321333187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee010a180 0x7feee019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.510+0000 7feedde00640 1 -- 172.21.15.27:0/1321333187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feee010ab40 msgr2=0x7feee01a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:45.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.510+0000 7feedde00640 1 --2- 172.21.15.27:0/1321333187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feee010ab40 0x7feee01a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.510+0000 7feedde00640 1 -- 172.21.15.27:0/1321333187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee01a6e20 con 0x7feee0108f90 2024-04-26T22:55:45.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.510+0000 7feedd400640 1 --2- 172.21.15.27:0/1321333187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee010a180 0x7feee019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:55:45.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.510+0000 7feede800640 1 --2- 172.21.15.27:0/1321333187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feee010ab40 0x7feee01a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:55:45.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.510+0000 7feedde00640 1 --2- 172.21.15.27:0/1321333187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee0108f90 0x7feee019f200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feed000eb70 tx=0x7feed000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:45.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.510+0000 7feed6a00640 1 -- 172.21.15.27:0/1321333187 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0003b70 con 0x7feee0108f90 2024-04-26T22:55:45.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.511+0000 7feed6a00640 1 -- 172.21.15.27:0/1321333187 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed000ee70 con 0x7feee0108f90 2024-04-26T22:55:45.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.511+0000 7feee5200640 1 -- 172.21.15.27:0/1321333187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee01a7110 con 0x7feee0108f90 2024-04-26T22:55:45.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.511+0000 7feed6a00640 1 -- 172.21.15.27:0/1321333187 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0010650 con 0x7feee0108f90 2024-04-26T22:55:45.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.511+0000 7feee5200640 1 -- 172.21.15.27:0/1321333187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee01a7660 con 0x7feee0108f90 2024-04-26T22:55:45.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.512+0000 7feee5200640 1 -- 172.21.15.27:0/1321333187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee0110350 con 0x7feee0108f90 2024-04-26T22:55:45.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.514+0000 7feed6a00640 1 -- 172.21.15.27:0/1321333187 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feed0004020 con 0x7feee0108f90 2024-04-26T22:55:45.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.515+0000 7feed6a00640 1 --2- 172.21.15.27:0/1321333187 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7feec40a3c30 0x7feec40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:45.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.516+0000 7feed6a00640 1 -- 172.21.15.27:0/1321333187 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7feed00d0de0 con 0x7feee0108f90 2024-04-26T22:55:45.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.516+0000 7feedd400640 1 --2- 172.21.15.27:0/1321333187 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7feec40a3c30 0x7feec40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:45.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.516+0000 7feedd400640 1 --2- 172.21.15.27:0/1321333187 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7feec40a3c30 0x7feec40a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feee01a0710 tx=0x7feecc0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:45.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.520+0000 7feed6a00640 1 -- 172.21.15.27:0/1321333187 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7feed009d520 con 0x7feee0108f90 2024-04-26T22:55:45.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.730+0000 7feee5200640 1 -- 172.21.15.27:0/1321333187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7feee0063530 con 0x7feee0108f90 2024-04-26T22:55:45.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.731+0000 7feed6a00640 1 -- 172.21.15.27:0/1321333187 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7feed009ce20 con 0x7feee0108f90 2024-04-26T22:55:45.732 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T22:55:45.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.734+0000 7feee5200640 1 -- 172.21.15.27:0/1321333187 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7feec40a3c30 msgr2=0x7feec40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:45.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.734+0000 7feee5200640 1 --2- 172.21.15.27:0/1321333187 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7feec40a3c30 0x7feec40a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feee01a0710 tx=0x7feecc0314a0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.734+0000 7feee5200640 1 -- 172.21.15.27:0/1321333187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee0108f90 msgr2=0x7feee019f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:45.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.734+0000 7feee5200640 1 --2- 172.21.15.27:0/1321333187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee0108f90 0x7feee019f200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feed000eb70 tx=0x7feed000c6a0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.735+0000 7feee5200640 1 -- 172.21.15.27:0/1321333187 shutdown_connections 2024-04-26T22:55:45.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.735+0000 7feee5200640 1 --2- 172.21.15.27:0/1321333187 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7feec40a3c30 0x7feec40a60f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.735+0000 7feee5200640 1 --2- 172.21.15.27:0/1321333187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee010a180 0x7feee019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.735+0000 7feee5200640 1 --2- 172.21.15.27:0/1321333187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feee010ab40 0x7feee01a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.735+0000 7feee5200640 1 --2- 172.21.15.27:0/1321333187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee0108f90 0x7feee019f200 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.735+0000 7feee5200640 1 -- 172.21.15.27:0/1321333187 >> 172.21.15.27:0/1321333187 conn(0x7feee0077f40 msgr2=0x7feee010d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:45.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.735+0000 7feee5200640 1 -- 172.21.15.27:0/1321333187 shutdown_connections 2024-04-26T22:55:45.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.735+0000 7feee5200640 1 -- 172.21.15.27:0/1321333187 wait complete. 2024-04-26T22:55:45.755 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T22:55:45.826 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:45 smithi027 ceph-mon[28034]: pgmap v340: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:45.826 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:45 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3502677130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:55:45.826 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:45 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1831400089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:55:45.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.972+0000 7fdec6a00640 1 -- 172.21.15.27:0/3414598580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec0109fc0 msgr2=0x7fdec010a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:45.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.972+0000 7fdec6a00640 1 --2- 172.21.15.27:0/3414598580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec0109fc0 0x7fdec010a440 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009900 tx=0x7fdeac02a3f0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.972+0000 7fdec6a00640 1 -- 172.21.15.27:0/3414598580 shutdown_connections 2024-04-26T22:55:45.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.972+0000 7fdec6a00640 1 --2- 172.21.15.27:0/3414598580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdec010a980 0x7fdec0111210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.972+0000 7fdec6a00640 1 --2- 172.21.15.27:0/3414598580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec0109fc0 0x7fdec010a440 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.972+0000 7fdec6a00640 1 --2- 172.21.15.27:0/3414598580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec0108dd0 0x7fdec01091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.972+0000 7fdec6a00640 1 -- 172.21.15.27:0/3414598580 >> 172.21.15.27:0/3414598580 conn(0x7fdec0077d70 msgr2=0x7fdec007a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:45.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.972+0000 7fdec6a00640 1 -- 172.21.15.27:0/3414598580 shutdown_connections 2024-04-26T22:55:45.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.972+0000 7fdec6a00640 1 -- 172.21.15.27:0/3414598580 wait complete. 2024-04-26T22:55:45.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.974+0000 7fdec6a00640 1 Processor -- start 2024-04-26T22:55:45.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.974+0000 7fdec6a00640 1 -- start start 2024-04-26T22:55:45.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.974+0000 7fdec6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec0108dd0 0x7fdec019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:45.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.974+0000 7fdec6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec0109fc0 0x7fdec019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:45.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.975+0000 7fdec6a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdec010a980 0x7fdec01a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:45.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.975+0000 7fdec6a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdec0114900 con 0x7fdec0109fc0 2024-04-26T22:55:45.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.975+0000 7fdec6a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdec0114780 con 0x7fdec010a980 2024-04-26T22:55:45.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.975+0000 7fdec6a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdec0114a80 con 0x7fdec0108dd0 2024-04-26T22:55:45.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.975+0000 7fdec4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec0109fc0 0x7fdec019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:45.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.975+0000 7fdec5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec0108dd0 0x7fdec019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:45.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.975+0000 7fdec4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec0109fc0 0x7fdec019f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50116/0 (socket says 172.21.15.27:50116) 2024-04-26T22:55:45.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.975+0000 7fdec4c00640 1 -- 172.21.15.27:0/3546676029 learned_addr learned my addr 172.21.15.27:0/3546676029 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:45.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.975+0000 7fdec6000640 1 --2- 172.21.15.27:0/3546676029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdec010a980 0x7fdec01a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:45.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.975+0000 7fdec5600640 1 -- 172.21.15.27:0/3546676029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdec010a980 msgr2=0x7fdec01a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:45.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.975+0000 7fdec5600640 1 --2- 172.21.15.27:0/3546676029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdec010a980 0x7fdec01a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.975+0000 7fdec5600640 1 -- 172.21.15.27:0/3546676029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec0109fc0 msgr2=0x7fdec019f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:45.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.975+0000 7fdec5600640 1 --2- 172.21.15.27:0/3546676029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec0109fc0 0x7fdec019f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:45.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.975+0000 7fdec5600640 1 -- 172.21.15.27:0/3546676029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec01a6bd0 con 0x7fdec0108dd0 2024-04-26T22:55:45.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.975+0000 7fdec4c00640 1 --2- 172.21.15.27:0/3546676029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec0109fc0 0x7fdec019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:45.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.976+0000 7fdec5600640 1 --2- 172.21.15.27:0/3546676029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec0108dd0 0x7fdec019efb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400e9c0 tx=0x7fdeb400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:45.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.976+0000 7fdebe000640 1 -- 172.21.15.27:0/3546676029 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb400ccc0 con 0x7fdec0108dd0 2024-04-26T22:55:45.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.976+0000 7fdec6a00640 1 -- 172.21.15.27:0/3546676029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec01a6ec0 con 0x7fdec0108dd0 2024-04-26T22:55:45.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.976+0000 7fdebe000640 1 -- 172.21.15.27:0/3546676029 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeb40035f0 con 0x7fdec0108dd0 2024-04-26T22:55:45.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.976+0000 7fdebe000640 1 -- 172.21.15.27:0/3546676029 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb400ce60 con 0x7fdec0108dd0 2024-04-26T22:55:45.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.976+0000 7fdec6a00640 1 -- 172.21.15.27:0/3546676029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec01a7330 con 0x7fdec0108dd0 2024-04-26T22:55:45.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.978+0000 7fdec6a00640 1 -- 172.21.15.27:0/3546676029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec01100f0 con 0x7fdec0108dd0 2024-04-26T22:55:45.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.980+0000 7fdebe000640 1 -- 172.21.15.27:0/3546676029 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdeb4004000 con 0x7fdec0108dd0 2024-04-26T22:55:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.981+0000 7fdebe000640 1 --2- 172.21.15.27:0/3546676029 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdea80a3c20 0x7fdea80a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.982+0000 7fdebe000640 1 -- 172.21.15.27:0/3546676029 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fdeb40d0c00 con 0x7fdec0108dd0 2024-04-26T22:55:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.982+0000 7fdec4c00640 1 --2- 172.21.15.27:0/3546676029 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdea80a3c20 0x7fdea80a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:45.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.983+0000 7fdec4c00640 1 --2- 172.21.15.27:0/3546676029 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdea80a3c20 0x7fdea80a60e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009230 tx=0x7fdeac0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:45.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:45.983+0000 7fdebe000640 1 -- 172.21.15.27:0/3546676029 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdeb409d340 con 0x7fdec0108dd0 2024-04-26T22:55:46.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:45 smithi080 ceph-mon[32324]: pgmap v340: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:46.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:45 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3502677130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:55:46.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:45 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1831400089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:55:46.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:45 smithi094 ceph-mon[32889]: pgmap v340: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:55:46.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:45 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3502677130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:55:46.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:45 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1831400089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:55:46.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.213+0000 7fdec6a00640 1 -- 172.21.15.27:0/3546676029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdec002dd30 con 0x7fdec0108dd0 2024-04-26T22:55:46.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.214+0000 7fdebe000640 1 -- 172.21.15.27:0/3546676029 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+591 (secure 0 0 0) 0x7fdeb409cc40 con 0x7fdec0108dd0 2024-04-26T22:55:46.214 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/1661629783 2024-04-27T22:47:04.292223+0000 2024-04-26T22:55:46.214 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3136245138 2024-04-27T22:47:04.292223+0000 2024-04-26T22:55:46.214 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/1661629783 2024-04-27T22:47:04.292223+0000 2024-04-26T22:55:46.215 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/1521074971 2024-04-27T22:46:28.107236+0000 2024-04-26T22:55:46.215 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/1521074971 2024-04-27T22:46:28.107236+0000 2024-04-26T22:55:46.215 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/542026788 2024-04-27T22:46:28.107236+0000 2024-04-26T22:55:46.215 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1524274936 2024-04-27T22:46:28.107236+0000 2024-04-26T22:55:46.215 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3814309846 2024-04-27T22:47:04.292223+0000 2024-04-26T22:55:46.215 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2767985348 2024-04-27T22:46:28.107236+0000 2024-04-26T22:55:46.215 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3451701535 2024-04-27T22:47:04.292223+0000 2024-04-26T22:55:46.215 INFO:teuthology.orchestra.run.smithi027.stderr:listed 10 entries 2024-04-26T22:55:46.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.216+0000 7fdec6a00640 1 -- 172.21.15.27:0/3546676029 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdea80a3c20 msgr2=0x7fdea80a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:46.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.217+0000 7fdec6a00640 1 --2- 172.21.15.27:0/3546676029 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdea80a3c20 0x7fdea80a60e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009230 tx=0x7fdeac0058d0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.217+0000 7fdec6a00640 1 -- 172.21.15.27:0/3546676029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec0108dd0 msgr2=0x7fdec019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.217+0000 7fdec6a00640 1 --2- 172.21.15.27:0/3546676029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec0108dd0 0x7fdec019efb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400e9c0 tx=0x7fdeb400ee80 comp rx=0 tx=0).stop 2024-04-26T22:55:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.217+0000 7fdec6a00640 1 -- 172.21.15.27:0/3546676029 shutdown_connections 2024-04-26T22:55:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.217+0000 7fdec6a00640 1 --2- 172.21.15.27:0/3546676029 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdea80a3c20 0x7fdea80a60e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.217+0000 7fdec6a00640 1 --2- 172.21.15.27:0/3546676029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdec010a980 0x7fdec01a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.217+0000 7fdec6a00640 1 --2- 172.21.15.27:0/3546676029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec0109fc0 0x7fdec019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.217+0000 7fdec6a00640 1 --2- 172.21.15.27:0/3546676029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec0108dd0 0x7fdec019efb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.217+0000 7fdec6a00640 1 -- 172.21.15.27:0/3546676029 >> 172.21.15.27:0/3546676029 conn(0x7fdec0077d70 msgr2=0x7fdec010d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.217+0000 7fdec6a00640 1 -- 172.21.15.27:0/3546676029 shutdown_connections 2024-04-26T22:55:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.217+0000 7fdec6a00640 1 -- 172.21.15.27:0/3546676029 wait complete. 2024-04-26T22:55:46.237 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:55:46.237 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:55:46.293 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:55:46.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.516+0000 7f6c3e400640 1 -- 172.21.15.27:0/3744755050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c3810a620 msgr2=0x7f6c3810ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:46.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.516+0000 7f6c3e400640 1 --2- 172.21.15.27:0/3744755050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c3810a620 0x7f6c3810ca90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009ca0 tx=0x7f6c3002a8c0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.516+0000 7f6c3e400640 1 -- 172.21.15.27:0/3744755050 shutdown_connections 2024-04-26T22:55:46.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.516+0000 7f6c3e400640 1 --2- 172.21.15.27:0/3744755050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c3810a620 0x7f6c3810ca90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.516+0000 7f6c3e400640 1 --2- 172.21.15.27:0/3744755050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c38075350 0x7f6c380757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.516+0000 7f6c3e400640 1 --2- 172.21.15.27:0/3744755050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c38076930 0x7f6c38074d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.516+0000 7f6c3e400640 1 -- 172.21.15.27:0/3744755050 >> 172.21.15.27:0/3744755050 conn(0x7f6c380fdce0 msgr2=0x7f6c38100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:46.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.516+0000 7f6c3e400640 1 -- 172.21.15.27:0/3744755050 shutdown_connections 2024-04-26T22:55:46.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.517+0000 7f6c3e400640 1 -- 172.21.15.27:0/3744755050 wait complete. 2024-04-26T22:55:46.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.517+0000 7f6c3e400640 1 Processor -- start 2024-04-26T22:55:46.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.518+0000 7f6c3e400640 1 -- start start 2024-04-26T22:55:46.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.518+0000 7f6c3e400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c38075350 0x7f6c3819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:46.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.518+0000 7f6c3e400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c38076930 0x7f6c3819af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:46.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.518+0000 7f6c3e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c3810a620 0x7f6c381a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:46.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.518+0000 7f6c3e400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6c38110480 con 0x7f6c38075350 2024-04-26T22:55:46.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.518+0000 7f6c3e400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6c38110300 con 0x7f6c38076930 2024-04-26T22:55:46.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.518+0000 7f6c3e400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c38110600 con 0x7f6c3810a620 2024-04-26T22:55:46.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.518+0000 7f6c3d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c38075350 0x7f6c3819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.518+0000 7f6c3d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c38075350 0x7f6c3819aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50142/0 (socket says 172.21.15.27:50142) 2024-04-26T22:55:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.518+0000 7f6c3d000640 1 -- 172.21.15.27:0/1685923314 learned_addr learned my addr 172.21.15.27:0/1685923314 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.518+0000 7f6c37e00640 1 --2- 172.21.15.27:0/1685923314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c38076930 0x7f6c3819af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.518+0000 7f6c3da00640 1 --2- 172.21.15.27:0/1685923314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c3810a620 0x7f6c381a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.519+0000 7f6c3d000640 1 -- 172.21.15.27:0/1685923314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c3810a620 msgr2=0x7f6c381a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.519+0000 7f6c3d000640 1 --2- 172.21.15.27:0/1685923314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c3810a620 0x7f6c381a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.519+0000 7f6c3d000640 1 -- 172.21.15.27:0/1685923314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c38076930 msgr2=0x7f6c3819af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.519+0000 7f6c3d000640 1 --2- 172.21.15.27:0/1685923314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c38076930 0x7f6c3819af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.519+0000 7f6c3d000640 1 -- 172.21.15.27:0/1685923314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c381a2650 con 0x7f6c38075350 2024-04-26T22:55:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.519+0000 7f6c3d000640 1 --2- 172.21.15.27:0/1685923314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c38075350 0x7f6c3819aa30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00e9f0 tx=0x7f6c2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.519+0000 7f6c35600640 1 -- 172.21.15.27:0/1685923314 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c00cd80 con 0x7f6c38075350 2024-04-26T22:55:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.519+0000 7f6c35600640 1 -- 172.21.15.27:0/1685923314 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c2c0054d0 con 0x7f6c38075350 2024-04-26T22:55:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.519+0000 7f6c3e400640 1 -- 172.21.15.27:0/1685923314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c381a2940 con 0x7f6c38075350 2024-04-26T22:55:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.519+0000 7f6c35600640 1 -- 172.21.15.27:0/1685923314 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c010690 con 0x7f6c38075350 2024-04-26T22:55:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.520+0000 7f6c3e400640 1 -- 172.21.15.27:0/1685923314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c381a2e40 con 0x7f6c38075350 2024-04-26T22:55:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.521+0000 7f6c3e400640 1 -- 172.21.15.27:0/1685923314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c380506c0 con 0x7f6c38075350 2024-04-26T22:55:46.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.523+0000 7f6c35600640 1 -- 172.21.15.27:0/1685923314 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6c2c004e90 con 0x7f6c38075350 2024-04-26T22:55:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.524+0000 7f6c35600640 1 --2- 172.21.15.27:0/1685923314 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6c200a3c50 0x7f6c200a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.524+0000 7f6c35600640 1 -- 172.21.15.27:0/1685923314 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f6c2c0d0a90 con 0x7f6c38075350 2024-04-26T22:55:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.525+0000 7f6c37e00640 1 --2- 172.21.15.27:0/1685923314 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6c200a3c50 0x7f6c200a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.525+0000 7f6c37e00640 1 --2- 172.21.15.27:0/1685923314 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6c200a3c50 0x7f6c200a6110 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6c3819bf40 tx=0x7f6c24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:46.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.529+0000 7f6c35600640 1 -- 172.21.15.27:0/1685923314 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6c2c09d1d0 con 0x7f6c38075350 2024-04-26T22:55:46.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.742+0000 7f6c3e400640 1 -- 172.21.15.27:0/1685923314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c38063530 con 0x7f6c38075350 2024-04-26T22:55:46.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.746+0000 7f6c35600640 1 -- 172.21.15.27:0/1685923314 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24009 (secure 0 0 0) 0x7f6c2c09cad0 con 0x7f6c38075350 2024-04-26T22:55:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:46.748 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:55:30.886974+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:55:46.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.749+0000 7f6c3e400640 1 -- 172.21.15.27:0/1685923314 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6c200a3c50 msgr2=0x7f6c200a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:46.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.749+0000 7f6c3e400640 1 --2- 172.21.15.27:0/1685923314 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6c200a3c50 0x7f6c200a6110 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6c3819bf40 tx=0x7f6c24005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:46.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.749+0000 7f6c3e400640 1 -- 172.21.15.27:0/1685923314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c38075350 msgr2=0x7f6c3819aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:46.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.749+0000 7f6c3e400640 1 --2- 172.21.15.27:0/1685923314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c38075350 0x7f6c3819aa30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00e9f0 tx=0x7f6c2c00edf0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.749+0000 7f6c3e400640 1 -- 172.21.15.27:0/1685923314 shutdown_connections 2024-04-26T22:55:46.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.749+0000 7f6c3e400640 1 --2- 172.21.15.27:0/1685923314 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f6c200a3c50 0x7f6c200a6110 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.749+0000 7f6c3e400640 1 --2- 172.21.15.27:0/1685923314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c38076930 0x7f6c3819af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.749+0000 7f6c3e400640 1 --2- 172.21.15.27:0/1685923314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c3810a620 0x7f6c381a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.749+0000 7f6c3e400640 1 --2- 172.21.15.27:0/1685923314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c38075350 0x7f6c3819aa30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.750+0000 7f6c3e400640 1 -- 172.21.15.27:0/1685923314 >> 172.21.15.27:0/1685923314 conn(0x7f6c380fdce0 msgr2=0x7f6c38106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:46.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.750+0000 7f6c3e400640 1 -- 172.21.15.27:0/1685923314 shutdown_connections 2024-04-26T22:55:46.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.750+0000 7f6c3e400640 1 -- 172.21.15.27:0/1685923314 wait complete. 2024-04-26T22:55:46.770 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:49:54.482679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:55:22.899174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:55:24.218295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T22:55:28.394547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T22:55:46.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:55:46.847 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:46 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1321333187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:55:46.848 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:46 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3546676029' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:55:46.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.995+0000 7f0519800640 1 -- 172.21.15.27:0/1391708237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0514105b70 msgr2=0x7f0514107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.995+0000 7f0519800640 1 --2- 172.21.15.27:0/1391708237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0514105b70 0x7f0514107f60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009900 tx=0x7f04fc02a3f0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.995+0000 7f0519800640 1 -- 172.21.15.27:0/1391708237 shutdown_connections 2024-04-26T22:55:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.995+0000 7f0519800640 1 --2- 172.21.15.27:0/1391708237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0514105b70 0x7f0514107f60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.995+0000 7f0519800640 1 --2- 172.21.15.27:0/1391708237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05141084a0 0x7f051410a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.995+0000 7f0519800640 1 --2- 172.21.15.27:0/1391708237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0514103240 0x7f0514105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.995+0000 7f0519800640 1 -- 172.21.15.27:0/1391708237 >> 172.21.15.27:0/1391708237 conn(0x7f05140fcc50 msgr2=0x7f05140ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.995+0000 7f0519800640 1 -- 172.21.15.27:0/1391708237 shutdown_connections 2024-04-26T22:55:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.995+0000 7f0519800640 1 -- 172.21.15.27:0/1391708237 wait complete. 2024-04-26T22:55:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.996+0000 7f0519800640 1 Processor -- start 2024-04-26T22:55:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.996+0000 7f0519800640 1 -- start start 2024-04-26T22:55:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.997+0000 7f0519800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0514103240 0x7f0514198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.997+0000 7f0519800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0514105b70 0x7f0514198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.997+0000 7f0519800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05141084a0 0x7f05141a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.997+0000 7f0519800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f051410e330 con 0x7f05141084a0 2024-04-26T22:55:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.997+0000 7f0519800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f051410e1b0 con 0x7f0514105b70 2024-04-26T22:55:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.997+0000 7f0519800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f051410e4b0 con 0x7f0514103240 2024-04-26T22:55:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.997+0000 7f0513e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0514103240 0x7f0514198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.997+0000 7f0513e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0514103240 0x7f0514198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:40980/0 (socket says 172.21.15.27:40980) 2024-04-26T22:55:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.997+0000 7f0513e00640 1 -- 172.21.15.27:0/566535051 learned_addr learned my addr 172.21.15.27:0/566535051 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.997+0000 7f0518e00640 1 --2- 172.21.15.27:0/566535051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05141084a0 0x7f05141a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.997+0000 7f0513400640 1 --2- 172.21.15.27:0/566535051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0514105b70 0x7f0514198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:46.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.998+0000 7f0513e00640 1 -- 172.21.15.27:0/566535051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0514105b70 msgr2=0x7f0514198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:46.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.998+0000 7f0513e00640 1 --2- 172.21.15.27:0/566535051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0514105b70 0x7f0514198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:46.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.998+0000 7f0513e00640 1 -- 172.21.15.27:0/566535051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05141084a0 msgr2=0x7f05141a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:47.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.998+0000 7f0513e00640 1 --2- 172.21.15.27:0/566535051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05141084a0 0x7f05141a0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.998+0000 7f0513e00640 1 -- 172.21.15.27:0/566535051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05141a0670 con 0x7f0514103240 2024-04-26T22:55:47.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.998+0000 7f0513e00640 1 --2- 172.21.15.27:0/566535051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0514103240 0x7f0514198aa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f050400e9f0 tx=0x7f050400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:47.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.998+0000 7f0510c00640 1 -- 172.21.15.27:0/566535051 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050400cd80 con 0x7f0514103240 2024-04-26T22:55:47.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.998+0000 7f0510c00640 1 -- 172.21.15.27:0/566535051 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05040108e0 con 0x7f0514103240 2024-04-26T22:55:47.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.998+0000 7f0519800640 1 -- 172.21.15.27:0/566535051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05141a0960 con 0x7f0514103240 2024-04-26T22:55:47.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.998+0000 7f0510c00640 1 -- 172.21.15.27:0/566535051 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504018980 con 0x7f0514103240 2024-04-26T22:55:47.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:46.999+0000 7f0519800640 1 -- 172.21.15.27:0/566535051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05141a0e80 con 0x7f0514103240 2024-04-26T22:55:47.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.000+0000 7f0519800640 1 -- 172.21.15.27:0/566535051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05140fe740 con 0x7f0514103240 2024-04-26T22:55:47.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.002+0000 7f0510c00640 1 -- 172.21.15.27:0/566535051 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0504010430 con 0x7f0514103240 2024-04-26T22:55:47.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.003+0000 7f0510c00640 1 --2- 172.21.15.27:0/566535051 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f04f80a3bb0 0x7f04f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:47.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.003+0000 7f0510c00640 1 -- 172.21.15.27:0/566535051 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f05040d0970 con 0x7f0514103240 2024-04-26T22:55:47.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.003+0000 7f0513400640 1 --2- 172.21.15.27:0/566535051 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f04f80a3bb0 0x7f04f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:47.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.004+0000 7f0513400640 1 --2- 172.21.15.27:0/566535051 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f04f80a3bb0 0x7f04f80a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f04fc005ab0 tx=0x7f04fc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:47.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.008+0000 7f0510c00640 1 -- 172.21.15.27:0/566535051 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f050409ca60 con 0x7f0514103240 2024-04-26T22:55:47.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:46 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1321333187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:55:47.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:46 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3546676029' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:55:47.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:46 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1321333187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:55:47.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:46 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3546676029' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:55:47.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.215+0000 7f0519800640 1 -- 172.21.15.27:0/566535051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0514063530 con 0x7f0514103240 2024-04-26T22:55:47.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.216+0000 7f0510c00640 1 -- 172.21.15.27:0/566535051 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f05040a0d60 con 0x7f0514103240 2024-04-26T22:55:47.217 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T22:55:47.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.219+0000 7f0519800640 1 -- 172.21.15.27:0/566535051 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f04f80a3bb0 msgr2=0x7f04f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.219+0000 7f0519800640 1 --2- 172.21.15.27:0/566535051 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f04f80a3bb0 0x7f04f80a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f04fc005ab0 tx=0x7f04fc005a20 comp rx=0 tx=0).stop 2024-04-26T22:55:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.219+0000 7f0519800640 1 -- 172.21.15.27:0/566535051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0514103240 msgr2=0x7f0514198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.219+0000 7f0519800640 1 --2- 172.21.15.27:0/566535051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0514103240 0x7f0514198aa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f050400e9f0 tx=0x7f050400eeb0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.219+0000 7f0519800640 1 -- 172.21.15.27:0/566535051 shutdown_connections 2024-04-26T22:55:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.219+0000 7f0519800640 1 --2- 172.21.15.27:0/566535051 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f04f80a3bb0 0x7f04f80a6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.219+0000 7f0519800640 1 --2- 172.21.15.27:0/566535051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0514105b70 0x7f0514198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.219+0000 7f0519800640 1 --2- 172.21.15.27:0/566535051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05141084a0 0x7f05141a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.219+0000 7f0519800640 1 --2- 172.21.15.27:0/566535051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0514103240 0x7f0514198aa0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.219+0000 7f0519800640 1 -- 172.21.15.27:0/566535051 >> 172.21.15.27:0/566535051 conn(0x7f05140fcc50 msgr2=0x7f0514103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.219+0000 7f0519800640 1 -- 172.21.15.27:0/566535051 shutdown_connections 2024-04-26T22:55:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.219+0000 7f0519800640 1 -- 172.21.15.27:0/566535051 wait complete. 2024-04-26T22:55:47.240 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:55:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.462+0000 7f696ae00640 1 -- 172.21.15.27:0/4184293731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6964103410 msgr2=0x7f6964105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.462+0000 7f696ae00640 1 --2- 172.21.15.27:0/4184293731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6964103410 0x7f6964105800 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6954009900 tx=0x7f695402a3f0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.462+0000 7f696ae00640 1 -- 172.21.15.27:0/4184293731 shutdown_connections 2024-04-26T22:55:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.462+0000 7f696ae00640 1 --2- 172.21.15.27:0/4184293731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6964108670 0x7f696410aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.462+0000 7f696ae00640 1 --2- 172.21.15.27:0/4184293731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6964105d40 0x7f6964108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.462+0000 7f696ae00640 1 --2- 172.21.15.27:0/4184293731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6964103410 0x7f6964105800 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.462+0000 7f696ae00640 1 -- 172.21.15.27:0/4184293731 >> 172.21.15.27:0/4184293731 conn(0x7f69640fce00 msgr2=0x7f69640ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.463+0000 7f696ae00640 1 -- 172.21.15.27:0/4184293731 shutdown_connections 2024-04-26T22:55:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.463+0000 7f696ae00640 1 -- 172.21.15.27:0/4184293731 wait complete. 2024-04-26T22:55:47.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.464+0000 7f696ae00640 1 Processor -- start 2024-04-26T22:55:47.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.464+0000 7f696ae00640 1 -- start start 2024-04-26T22:55:47.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.464+0000 7f696ae00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6964103410 0x7f6964198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:47.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.464+0000 7f696ae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6964105d40 0x7f6964199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:47.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.464+0000 7f696ae00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6964108670 0x7f69641a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:47.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.464+0000 7f696ae00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f696410e4c0 con 0x7f6964108670 2024-04-26T22:55:47.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.464+0000 7f696ae00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f696410e340 con 0x7f6964103410 2024-04-26T22:55:47.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.464+0000 7f696ae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f696410e640 con 0x7f6964105d40 2024-04-26T22:55:47.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.465+0000 7f6963400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6964105d40 0x7f6964199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:47.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.465+0000 7f6963e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6964103410 0x7f6964198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:47.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.465+0000 7f6968e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6964108670 0x7f69641a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:47.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.465+0000 7f6963400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6964105d40 0x7f6964199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:41002/0 (socket says 172.21.15.27:41002) 2024-04-26T22:55:47.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.465+0000 7f6963400640 1 -- 172.21.15.27:0/298585821 learned_addr learned my addr 172.21.15.27:0/298585821 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:47.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.465+0000 7f6963400640 1 -- 172.21.15.27:0/298585821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6964103410 msgr2=0x7f6964198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:47.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.465+0000 7f6963400640 1 --2- 172.21.15.27:0/298585821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6964103410 0x7f6964198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.465+0000 7f6963400640 1 -- 172.21.15.27:0/298585821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6964108670 msgr2=0x7f69641a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:47.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.465+0000 7f6963400640 1 --2- 172.21.15.27:0/298585821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6964108670 0x7f69641a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.465+0000 7f6963400640 1 -- 172.21.15.27:0/298585821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69641a07f0 con 0x7f6964105d40 2024-04-26T22:55:47.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.465+0000 7f6963400640 1 --2- 172.21.15.27:0/298585821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6964105d40 0x7f6964199110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f694c00ea80 tx=0x7f694c00ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:47.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.466+0000 7f6960c00640 1 -- 172.21.15.27:0/298585821 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c00ce40 con 0x7f6964105d40 2024-04-26T22:55:47.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.466+0000 7f6960c00640 1 -- 172.21.15.27:0/298585821 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f694c011920 con 0x7f6964105d40 2024-04-26T22:55:47.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.466+0000 7f6960c00640 1 -- 172.21.15.27:0/298585821 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c01aa20 con 0x7f6964105d40 2024-04-26T22:55:47.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.466+0000 7f696ae00640 1 -- 172.21.15.27:0/298585821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69641a0ae0 con 0x7f6964105d40 2024-04-26T22:55:47.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.466+0000 7f696ae00640 1 -- 172.21.15.27:0/298585821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69641a0fe0 con 0x7f6964105d40 2024-04-26T22:55:47.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.467+0000 7f696ae00640 1 -- 172.21.15.27:0/298585821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69640fe8f0 con 0x7f6964105d40 2024-04-26T22:55:47.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.469+0000 7f6960c00640 1 -- 172.21.15.27:0/298585821 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f694c01abc0 con 0x7f6964105d40 2024-04-26T22:55:47.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.471+0000 7f6960c00640 1 --2- 172.21.15.27:0/298585821 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f69440a3c30 0x7f69440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:47.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.471+0000 7f6960c00640 1 -- 172.21.15.27:0/298585821 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f694c0d0c80 con 0x7f6964105d40 2024-04-26T22:55:47.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.471+0000 7f6963e00640 1 --2- 172.21.15.27:0/298585821 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f69440a3c30 0x7f69440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:47.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.472+0000 7f6963e00640 1 --2- 172.21.15.27:0/298585821 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f69440a3c30 0x7f69440a60f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6954000c00 tx=0x7f69540314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:47.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.473+0000 7f6960c00640 1 -- 172.21.15.27:0/298585821 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f694c016030 con 0x7f6964105d40 2024-04-26T22:55:47.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.681+0000 7f696ae00640 1 -- 172.21.15.27:0/298585821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f696402dd20 con 0x7f6964105d40 2024-04-26T22:55:47.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.682+0000 7f6960c00640 1 -- 172.21.15.27:0/298585821 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f694c09d440 con 0x7f6964105d40 2024-04-26T22:55:47.682 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T22:55:47.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.684+0000 7f696ae00640 1 -- 172.21.15.27:0/298585821 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f69440a3c30 msgr2=0x7f69440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:47.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.684+0000 7f696ae00640 1 --2- 172.21.15.27:0/298585821 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f69440a3c30 0x7f69440a60f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6954000c00 tx=0x7f69540314a0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.684+0000 7f696ae00640 1 -- 172.21.15.27:0/298585821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6964105d40 msgr2=0x7f6964199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:47.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.684+0000 7f696ae00640 1 --2- 172.21.15.27:0/298585821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6964105d40 0x7f6964199110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f694c00ea80 tx=0x7f694c00ef40 comp rx=0 tx=0).stop 2024-04-26T22:55:47.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.684+0000 7f696ae00640 1 -- 172.21.15.27:0/298585821 shutdown_connections 2024-04-26T22:55:47.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.684+0000 7f696ae00640 1 --2- 172.21.15.27:0/298585821 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f69440a3c30 0x7f69440a60f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.684+0000 7f696ae00640 1 --2- 172.21.15.27:0/298585821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6964108670 0x7f69641a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.684+0000 7f696ae00640 1 --2- 172.21.15.27:0/298585821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6964105d40 0x7f6964199110 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.684+0000 7f696ae00640 1 --2- 172.21.15.27:0/298585821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6964103410 0x7f6964198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.684+0000 7f696ae00640 1 -- 172.21.15.27:0/298585821 >> 172.21.15.27:0/298585821 conn(0x7f69640fce00 msgr2=0x7f6964103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:47.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.685+0000 7f696ae00640 1 -- 172.21.15.27:0/298585821 shutdown_connections 2024-04-26T22:55:47.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.685+0000 7f696ae00640 1 -- 172.21.15.27:0/298585821 wait complete. 2024-04-26T22:55:47.703 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:55:47.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:47 smithi027 ceph-mon[28034]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:55:47.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:47 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1685923314' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:47.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:47 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/566535051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:55:47.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.925+0000 7f67c9600640 1 -- 172.21.15.27:0/2776118697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67c4103220 msgr2=0x7f67c4109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:47.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.925+0000 7f67c9600640 1 --2- 172.21.15.27:0/2776118697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67c4103220 0x7f67c4109ab0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009ca0 tx=0x7f67bc02a8c0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.925+0000 7f67c9600640 1 -- 172.21.15.27:0/2776118697 shutdown_connections 2024-04-26T22:55:47.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.925+0000 7f67c9600640 1 --2- 172.21.15.27:0/2776118697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67c4102860 0x7f67c4102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.925+0000 7f67c9600640 1 --2- 172.21.15.27:0/2776118697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67c4103220 0x7f67c4109ab0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.925+0000 7f67c9600640 1 --2- 172.21.15.27:0/2776118697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67c4101670 0x7f67c4101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.925+0000 7f67c9600640 1 -- 172.21.15.27:0/2776118697 >> 172.21.15.27:0/2776118697 conn(0x7f67c40fcc00 msgr2=0x7f67c40ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:47.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.925+0000 7f67c9600640 1 -- 172.21.15.27:0/2776118697 shutdown_connections 2024-04-26T22:55:47.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.925+0000 7f67c9600640 1 -- 172.21.15.27:0/2776118697 wait complete. 2024-04-26T22:55:47.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.926+0000 7f67c9600640 1 Processor -- start 2024-04-26T22:55:47.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.926+0000 7f67c9600640 1 -- start start 2024-04-26T22:55:47.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.927+0000 7f67c9600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67c4101670 0x7f67c419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:47.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.927+0000 7f67c9600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67c4102860 0x7f67c419b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:47.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.927+0000 7f67c9600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67c4103220 0x7f67c41a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:47.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.927+0000 7f67c9600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f67c410d1d0 con 0x7f67c4101670 2024-04-26T22:55:47.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.927+0000 7f67c9600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f67c410d050 con 0x7f67c4103220 2024-04-26T22:55:47.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.927+0000 7f67c9600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f67c410d350 con 0x7f67c4102860 2024-04-26T22:55:47.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.927+0000 7f67c3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67c4101670 0x7f67c419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:47.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.927+0000 7f67c3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67c4101670 0x7f67c419abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50214/0 (socket says 172.21.15.27:50214) 2024-04-26T22:55:47.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.927+0000 7f67c3e00640 1 -- 172.21.15.27:0/3720209297 learned_addr learned my addr 172.21.15.27:0/3720209297 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:47.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.927+0000 7f67c8c00640 1 --2- 172.21.15.27:0/3720209297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67c4103220 0x7f67c41a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:47.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.927+0000 7f67c3400640 1 --2- 172.21.15.27:0/3720209297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67c4102860 0x7f67c419b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:47.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.928+0000 7f67c3e00640 1 -- 172.21.15.27:0/3720209297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67c4102860 msgr2=0x7f67c419b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:47.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.928+0000 7f67c3e00640 1 --2- 172.21.15.27:0/3720209297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67c4102860 0x7f67c419b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.928+0000 7f67c3e00640 1 -- 172.21.15.27:0/3720209297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67c4103220 msgr2=0x7f67c41a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:47.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.928+0000 7f67c3e00640 1 --2- 172.21.15.27:0/3720209297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67c4103220 0x7f67c41a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:47.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.928+0000 7f67c3e00640 1 -- 172.21.15.27:0/3720209297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c41a27e0 con 0x7f67c4101670 2024-04-26T22:55:47.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.928+0000 7f67c3e00640 1 --2- 172.21.15.27:0/3720209297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67c4101670 0x7f67c419abc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f67b400e9c0 tx=0x7f67b400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:47.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.929+0000 7f67c0c00640 1 -- 172.21.15.27:0/3720209297 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b400cd80 con 0x7f67c4101670 2024-04-26T22:55:47.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.929+0000 7f67c0c00640 1 -- 172.21.15.27:0/3720209297 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67b40035f0 con 0x7f67c4101670 2024-04-26T22:55:47.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.929+0000 7f67c0c00640 1 -- 172.21.15.27:0/3720209297 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4010690 con 0x7f67c4101670 2024-04-26T22:55:47.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.929+0000 7f67c9600640 1 -- 172.21.15.27:0/3720209297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c41a2ad0 con 0x7f67c4101670 2024-04-26T22:55:47.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.929+0000 7f67c9600640 1 -- 172.21.15.27:0/3720209297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c41a2fd0 con 0x7f67c4101670 2024-04-26T22:55:47.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.930+0000 7f67c9600640 1 -- 172.21.15.27:0/3720209297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c41a32c0 con 0x7f67c4101670 2024-04-26T22:55:47.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.932+0000 7f67c0c00640 1 -- 172.21.15.27:0/3720209297 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f67b40037a0 con 0x7f67c4101670 2024-04-26T22:55:47.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.934+0000 7f67c0c00640 1 --2- 172.21.15.27:0/3720209297 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f67a80a3cf0 0x7f67a80a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:47.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.934+0000 7f67c0c00640 1 -- 172.21.15.27:0/3720209297 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f67b40d08a0 con 0x7f67c4101670 2024-04-26T22:55:47.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.934+0000 7f67c3400640 1 --2- 172.21.15.27:0/3720209297 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f67a80a3cf0 0x7f67a80a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:47.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.935+0000 7f67c3400640 1 --2- 172.21.15.27:0/3720209297 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f67a80a3cf0 0x7f67a80a61b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f67c419c0d0 tx=0x7f67b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:47.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:47.938+0000 7f67c0c00640 1 -- 172.21.15.27:0/3720209297 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f67b409d040 con 0x7f67c4101670 2024-04-26T22:55:48.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:47 smithi080 ceph-mon[32324]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:55:48.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:47 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1685923314' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:48.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:47 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/566535051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:55:48.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:47 smithi094 ceph-mon[32889]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:55:48.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:47 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1685923314' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:48.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:47 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/566535051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:55:48.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.147+0000 7f67c9600640 1 -- 172.21.15.27:0/3720209297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f67c41a3c00 con 0x7f67c4101670 2024-04-26T22:55:48.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.148+0000 7f67c0c00640 1 -- 172.21.15.27:0/3720209297 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f67b409c940 con 0x7f67c4101670 2024-04-26T22:55:48.149 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-26T22:55:48.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.151+0000 7f67c9600640 1 -- 172.21.15.27:0/3720209297 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f67a80a3cf0 msgr2=0x7f67a80a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:48.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.151+0000 7f67c9600640 1 --2- 172.21.15.27:0/3720209297 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f67a80a3cf0 0x7f67a80a61b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f67c419c0d0 tx=0x7f67b0005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:48.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.151+0000 7f67c9600640 1 -- 172.21.15.27:0/3720209297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67c4101670 msgr2=0x7f67c419abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:48.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.151+0000 7f67c9600640 1 --2- 172.21.15.27:0/3720209297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67c4101670 0x7f67c419abc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f67b400e9c0 tx=0x7f67b400ee80 comp rx=0 tx=0).stop 2024-04-26T22:55:48.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.151+0000 7f67c9600640 1 -- 172.21.15.27:0/3720209297 shutdown_connections 2024-04-26T22:55:48.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.151+0000 7f67c9600640 1 --2- 172.21.15.27:0/3720209297 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f67a80a3cf0 0x7f67a80a61b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.151+0000 7f67c9600640 1 --2- 172.21.15.27:0/3720209297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67c4103220 0x7f67c41a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.151+0000 7f67c9600640 1 --2- 172.21.15.27:0/3720209297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67c4102860 0x7f67c419b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.151+0000 7f67c9600640 1 --2- 172.21.15.27:0/3720209297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67c4101670 0x7f67c419abc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.151+0000 7f67c9600640 1 -- 172.21.15.27:0/3720209297 >> 172.21.15.27:0/3720209297 conn(0x7f67c40fcc00 msgr2=0x7f67c40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:48.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.152+0000 7f67c9600640 1 -- 172.21.15.27:0/3720209297 shutdown_connections 2024-04-26T22:55:48.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.152+0000 7f67c9600640 1 -- 172.21.15.27:0/3720209297 wait complete. 2024-04-26T22:55:48.172 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T22:55:48.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.385+0000 7f0e10e00640 1 -- 172.21.15.27:0/3759936456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e0c1027a0 msgr2=0x7f0e0c102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:48.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.385+0000 7f0e10e00640 1 --2- 172.21.15.27:0/3759936456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e0c1027a0 0x7f0e0c102bc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009900 tx=0x7f0dfc02a3f0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.385+0000 7f0e10e00640 1 -- 172.21.15.27:0/3759936456 shutdown_connections 2024-04-26T22:55:48.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.385+0000 7f0e10e00640 1 --2- 172.21.15.27:0/3759936456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c104350 0x7f0e0c10abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.385+0000 7f0e10e00640 1 --2- 172.21.15.27:0/3759936456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e0c103990 0x7f0e0c103e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.385+0000 7f0e10e00640 1 --2- 172.21.15.27:0/3759936456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e0c1027a0 0x7f0e0c102bc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.385+0000 7f0e10e00640 1 -- 172.21.15.27:0/3759936456 >> 172.21.15.27:0/3759936456 conn(0x7f0e0c0fdd10 msgr2=0x7f0e0c100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:48.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.385+0000 7f0e10e00640 1 -- 172.21.15.27:0/3759936456 shutdown_connections 2024-04-26T22:55:48.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.385+0000 7f0e10e00640 1 -- 172.21.15.27:0/3759936456 wait complete. 2024-04-26T22:55:48.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.386+0000 7f0e10e00640 1 Processor -- start 2024-04-26T22:55:48.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.386+0000 7f0e10e00640 1 -- start start 2024-04-26T22:55:48.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.387+0000 7f0e10e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c1027a0 0x7f0e0c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:48.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.387+0000 7f0e10e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e0c103990 0x7f0e0c198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:48.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.387+0000 7f0e10e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e0c104350 0x7f0e0c19ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:48.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.387+0000 7f0e10e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c10e310 con 0x7f0e0c104350 2024-04-26T22:55:48.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.387+0000 7f0e10e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c10e190 con 0x7f0e0c1027a0 2024-04-26T22:55:48.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.387+0000 7f0e10e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c10e490 con 0x7f0e0c103990 2024-04-26T22:55:48.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.387+0000 7f0e0be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e0c104350 0x7f0e0c19ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:48.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.387+0000 7f0e0b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c1027a0 0x7f0e0c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:48.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.387+0000 7f0e0aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e0c103990 0x7f0e0c198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:48.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.387+0000 7f0e0be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e0c104350 0x7f0e0c19ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50388/0 (socket says 172.21.15.27:50388) 2024-04-26T22:55:48.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.387+0000 7f0e0b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c1027a0 0x7f0e0c198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:35250/0 (socket says 172.21.15.27:35250) 2024-04-26T22:55:48.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.387+0000 7f0e0be00640 1 -- 172.21.15.27:0/4154632547 learned_addr learned my addr 172.21.15.27:0/4154632547 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:48.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.388+0000 7f0e0b400640 1 -- 172.21.15.27:0/4154632547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e0c103990 msgr2=0x7f0e0c198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:48.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.388+0000 7f0e0b400640 1 --2- 172.21.15.27:0/4154632547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e0c103990 0x7f0e0c198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.388+0000 7f0e0b400640 1 -- 172.21.15.27:0/4154632547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e0c104350 msgr2=0x7f0e0c19ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:48.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.388+0000 7f0e0b400640 1 --2- 172.21.15.27:0/4154632547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e0c104350 0x7f0e0c19ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.388+0000 7f0e0b400640 1 -- 172.21.15.27:0/4154632547 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c1a0650 con 0x7f0e0c1027a0 2024-04-26T22:55:48.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.388+0000 7f0e0aa00640 1 --2- 172.21.15.27:0/4154632547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e0c103990 0x7f0e0c198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:48.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.388+0000 7f0e0be00640 1 --2- 172.21.15.27:0/4154632547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e0c104350 0x7f0e0c19ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:48.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.388+0000 7f0e0b400640 1 --2- 172.21.15.27:0/4154632547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c1027a0 0x7f0e0c198a30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc02a910 tx=0x7f0dfc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:48.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.388+0000 7f0e03e00640 1 -- 172.21.15.27:0/4154632547 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc03f070 con 0x7f0e0c1027a0 2024-04-26T22:55:48.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.388+0000 7f0e03e00640 1 -- 172.21.15.27:0/4154632547 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dfc031c50 con 0x7f0e0c1027a0 2024-04-26T22:55:48.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.388+0000 7f0e03e00640 1 -- 172.21.15.27:0/4154632547 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc035030 con 0x7f0e0c1027a0 2024-04-26T22:55:48.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.388+0000 7f0e10e00640 1 -- 172.21.15.27:0/4154632547 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0c1a08e0 con 0x7f0e0c1027a0 2024-04-26T22:55:48.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.388+0000 7f0e10e00640 1 -- 172.21.15.27:0/4154632547 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0c1a0da0 con 0x7f0e0c1027a0 2024-04-26T22:55:48.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.390+0000 7f0e10e00640 1 -- 172.21.15.27:0/4154632547 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e0c109b90 con 0x7f0e0c1027a0 2024-04-26T22:55:48.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.392+0000 7f0e03e00640 1 -- 172.21.15.27:0/4154632547 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0dfc042070 con 0x7f0e0c1027a0 2024-04-26T22:55:48.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.393+0000 7f0e03e00640 1 --2- 172.21.15.27:0/4154632547 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0df00a3bb0 0x7f0df00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:48.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.393+0000 7f0e03e00640 1 -- 172.21.15.27:0/4154632547 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0dfc0f0d30 con 0x7f0e0c1027a0 2024-04-26T22:55:48.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.393+0000 7f0e0aa00640 1 --2- 172.21.15.27:0/4154632547 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0df00a3bb0 0x7f0df00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:48.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.394+0000 7f0e0aa00640 1 --2- 172.21.15.27:0/4154632547 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0df00a3bb0 0x7f0df00a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c199f40 tx=0x7f0df8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:48.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.397+0000 7f0e03e00640 1 -- 172.21.15.27:0/4154632547 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0dfc0bce20 con 0x7f0e0c1027a0 2024-04-26T22:55:48.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.606+0000 7f0e10e00640 1 -- 172.21.15.27:0/4154632547 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0e0c02dcf0 con 0x7f0e0c1027a0 2024-04-26T22:55:48.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.607+0000 7f0e03e00640 1 -- 172.21.15.27:0/4154632547 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f0dfc0c1120 con 0x7f0e0c1027a0 2024-04-26T22:55:48.607 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T22:55:48.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.609+0000 7f0e10e00640 1 -- 172.21.15.27:0/4154632547 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0df00a3bb0 msgr2=0x7f0df00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:48.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.609+0000 7f0e10e00640 1 --2- 172.21.15.27:0/4154632547 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0df00a3bb0 0x7f0df00a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c199f40 tx=0x7f0df8005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:48.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.609+0000 7f0e10e00640 1 -- 172.21.15.27:0/4154632547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c1027a0 msgr2=0x7f0e0c198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:48.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.609+0000 7f0e10e00640 1 --2- 172.21.15.27:0/4154632547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c1027a0 0x7f0e0c198a30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc02a910 tx=0x7f0dfc003c10 comp rx=0 tx=0).stop 2024-04-26T22:55:48.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.610+0000 7f0e10e00640 1 -- 172.21.15.27:0/4154632547 shutdown_connections 2024-04-26T22:55:48.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.610+0000 7f0e10e00640 1 --2- 172.21.15.27:0/4154632547 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0df00a3bb0 0x7f0df00a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.610+0000 7f0e10e00640 1 --2- 172.21.15.27:0/4154632547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e0c104350 0x7f0e0c19ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.610+0000 7f0e10e00640 1 --2- 172.21.15.27:0/4154632547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e0c103990 0x7f0e0c198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.610+0000 7f0e10e00640 1 --2- 172.21.15.27:0/4154632547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c1027a0 0x7f0e0c198a30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.610+0000 7f0e10e00640 1 -- 172.21.15.27:0/4154632547 >> 172.21.15.27:0/4154632547 conn(0x7f0e0c0fdd10 msgr2=0x7f0e0c106d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:48.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.610+0000 7f0e10e00640 1 -- 172.21.15.27:0/4154632547 shutdown_connections 2024-04-26T22:55:48.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.610+0000 7f0e10e00640 1 -- 172.21.15.27:0/4154632547 wait complete. 2024-04-26T22:55:48.631 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T22:55:48.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.854+0000 7f39b4a00640 1 -- 172.21.15.27:0/249235032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39b010ac00 msgr2=0x7f39b0111490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:48.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.854+0000 7f39b4a00640 1 --2- 172.21.15.27:0/249235032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39b010ac00 0x7f39b0111490 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009be0 tx=0x7f39a802a6d0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.854+0000 7f39b4a00640 1 -- 172.21.15.27:0/249235032 shutdown_connections 2024-04-26T22:55:48.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.854+0000 7f39b4a00640 1 --2- 172.21.15.27:0/249235032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39b010ac00 0x7f39b0111490 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.854+0000 7f39b4a00640 1 --2- 172.21.15.27:0/249235032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39b010a240 0x7f39b010a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.854+0000 7f39b4a00640 1 --2- 172.21.15.27:0/249235032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39b0109050 0x7f39b0109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.854+0000 7f39b4a00640 1 -- 172.21.15.27:0/249235032 >> 172.21.15.27:0/249235032 conn(0x7f39b0077fe0 msgr2=0x7f39b007a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:48.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.854+0000 7f39b4a00640 1 -- 172.21.15.27:0/249235032 shutdown_connections 2024-04-26T22:55:48.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.854+0000 7f39b4a00640 1 -- 172.21.15.27:0/249235032 wait complete. 2024-04-26T22:55:48.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.855+0000 7f39b4a00640 1 Processor -- start 2024-04-26T22:55:48.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.855+0000 7f39b4a00640 1 -- start start 2024-04-26T22:55:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.856+0000 7f39b4a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39b0109050 0x7f39b019f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.856+0000 7f39b4a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39b010a240 0x7f39b019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.856+0000 7f39b4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39b010ac00 0x7f39b01a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.856+0000 7f39b4a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f39b0114b90 con 0x7f39b0109050 2024-04-26T22:55:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.856+0000 7f39b4a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f39b0114a10 con 0x7f39b010a240 2024-04-26T22:55:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.856+0000 7f39b4a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f39b0114d10 con 0x7f39b010ac00 2024-04-26T22:55:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.856+0000 7f39ade00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39b0109050 0x7f39b019f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.856+0000 7f39ade00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39b0109050 0x7f39b019f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50420/0 (socket says 172.21.15.27:50420) 2024-04-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.856+0000 7f39ade00640 1 -- 172.21.15.27:0/3986162414 learned_addr learned my addr 172.21.15.27:0/3986162414 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.856+0000 7f39ae800640 1 --2- 172.21.15.27:0/3986162414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39b010ac00 0x7f39b01a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.856+0000 7f39ad400640 1 --2- 172.21.15.27:0/3986162414 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39b010a240 0x7f39b019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.857+0000 7f39ade00640 1 -- 172.21.15.27:0/3986162414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39b010ac00 msgr2=0x7f39b01a6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.857+0000 7f39ade00640 1 --2- 172.21.15.27:0/3986162414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39b010ac00 0x7f39b01a6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.857+0000 7f39ade00640 1 -- 172.21.15.27:0/3986162414 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39b010a240 msgr2=0x7f39b019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.857+0000 7f39ade00640 1 --2- 172.21.15.27:0/3986162414 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39b010a240 0x7f39b019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.857+0000 7f39ade00640 1 -- 172.21.15.27:0/3986162414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b01a6de0 con 0x7f39b0109050 2024-04-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.857+0000 7f39ade00640 1 --2- 172.21.15.27:0/3986162414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39b0109050 0x7f39b019f1c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f39a000e9c0 tx=0x7f39a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.857+0000 7f39a6a00640 1 -- 172.21.15.27:0/3986162414 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a000cd80 con 0x7f39b0109050 2024-04-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.857+0000 7f39b4a00640 1 -- 172.21.15.27:0/3986162414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b01a70d0 con 0x7f39b0109050 2024-04-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.857+0000 7f39a6a00640 1 -- 172.21.15.27:0/3986162414 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a00035f0 con 0x7f39b0109050 2024-04-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.857+0000 7f39a6a00640 1 -- 172.21.15.27:0/3986162414 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0018630 con 0x7f39b0109050 2024-04-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.858+0000 7f39b4a00640 1 -- 172.21.15.27:0/3986162414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b01a7620 con 0x7f39b0109050 2024-04-26T22:55:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.859+0000 7f39b4a00640 1 -- 172.21.15.27:0/3986162414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b0110370 con 0x7f39b0109050 2024-04-26T22:55:48.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.861+0000 7f39a6a00640 1 -- 172.21.15.27:0/3986162414 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f39a00037a0 con 0x7f39b0109050 2024-04-26T22:55:48.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.862+0000 7f39a6a00640 1 --2- 172.21.15.27:0/3986162414 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f39940a3bb0 0x7f39940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:48.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.862+0000 7f39a6a00640 1 -- 172.21.15.27:0/3986162414 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f39a00d09a0 con 0x7f39b0109050 2024-04-26T22:55:48.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.862+0000 7f39ad400640 1 --2- 172.21.15.27:0/3986162414 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f39940a3bb0 0x7f39940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:48.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.863+0000 7f39ad400640 1 --2- 172.21.15.27:0/3986162414 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f39940a3bb0 0x7f39940a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f39b01a06d0 tx=0x7f399c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:48.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:48 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/298585821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:55:48.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:48 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3720209297' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:55:48.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:48.867+0000 7f39a6a00640 1 -- 172.21.15.27:0/3986162414 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f39a009d140 con 0x7f39b0109050 2024-04-26T22:55:49.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:48 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/298585821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:55:49.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:48 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3720209297' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:55:49.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:48 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/298585821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:55:49.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:48 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3720209297' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:55:49.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.088+0000 7f39b4a00640 1 -- 172.21.15.27:0/3986162414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f39b002dad0 con 0x7f39b0109050 2024-04-26T22:55:49.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.089+0000 7f39a6a00640 1 -- 172.21.15.27:0/3986162414 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+591 (secure 0 0 0) 0x7f39a009ca40 con 0x7f39b0109050 2024-04-26T22:55:49.090 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/1661629783 2024-04-27T22:47:04.292223+0000 2024-04-26T22:55:49.090 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3136245138 2024-04-27T22:47:04.292223+0000 2024-04-26T22:55:49.090 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/1661629783 2024-04-27T22:47:04.292223+0000 2024-04-26T22:55:49.090 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/1521074971 2024-04-27T22:46:28.107236+0000 2024-04-26T22:55:49.090 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/1521074971 2024-04-27T22:46:28.107236+0000 2024-04-26T22:55:49.090 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/542026788 2024-04-27T22:46:28.107236+0000 2024-04-26T22:55:49.090 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1524274936 2024-04-27T22:46:28.107236+0000 2024-04-26T22:55:49.090 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3814309846 2024-04-27T22:47:04.292223+0000 2024-04-26T22:55:49.090 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2767985348 2024-04-27T22:46:28.107236+0000 2024-04-26T22:55:49.090 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3451701535 2024-04-27T22:47:04.292223+0000 2024-04-26T22:55:49.091 INFO:teuthology.orchestra.run.smithi027.stderr:listed 10 entries 2024-04-26T22:55:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.092+0000 7f39b4a00640 1 -- 172.21.15.27:0/3986162414 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f39940a3bb0 msgr2=0x7f39940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.092+0000 7f39b4a00640 1 --2- 172.21.15.27:0/3986162414 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f39940a3bb0 0x7f39940a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f39b01a06d0 tx=0x7f399c009260 comp rx=0 tx=0).stop 2024-04-26T22:55:49.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.092+0000 7f39b4a00640 1 -- 172.21.15.27:0/3986162414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39b0109050 msgr2=0x7f39b019f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:49.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.092+0000 7f39b4a00640 1 --2- 172.21.15.27:0/3986162414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39b0109050 0x7f39b019f1c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f39a000e9c0 tx=0x7f39a000ee80 comp rx=0 tx=0).stop 2024-04-26T22:55:49.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.092+0000 7f39b4a00640 1 -- 172.21.15.27:0/3986162414 shutdown_connections 2024-04-26T22:55:49.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.092+0000 7f39b4a00640 1 --2- 172.21.15.27:0/3986162414 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f39940a3bb0 0x7f39940a6070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:49.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.092+0000 7f39b4a00640 1 --2- 172.21.15.27:0/3986162414 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39b010a240 0x7f39b019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:49.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.092+0000 7f39b4a00640 1 --2- 172.21.15.27:0/3986162414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39b010ac00 0x7f39b01a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:49.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.092+0000 7f39b4a00640 1 --2- 172.21.15.27:0/3986162414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39b0109050 0x7f39b019f1c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:49.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.092+0000 7f39b4a00640 1 -- 172.21.15.27:0/3986162414 >> 172.21.15.27:0/3986162414 conn(0x7f39b0077fe0 msgr2=0x7f39b010d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:49.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.092+0000 7f39b4a00640 1 -- 172.21.15.27:0/3986162414 shutdown_connections 2024-04-26T22:55:49.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.092+0000 7f39b4a00640 1 -- 172.21.15.27:0/3986162414 wait complete. 2024-04-26T22:55:49.111 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2024-04-26T22:55:49.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:55:49.111 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-26T22:55:49.189 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[71447]: fuse finished with error 0 and tester_r 0 2024-04-26T22:55:49.195 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:55:49.274 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T22:55:49.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:55:49.275 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-26T22:55:49.341 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:55:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.568+0000 7f4171800640 1 -- 172.21.15.27:0/73059002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f416c104320 msgr2=0x7f416c10abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.568+0000 7f4171800640 1 --2- 172.21.15.27:0/73059002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f416c104320 0x7f416c10abb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4164009ca0 tx=0x7f416402a8b0 comp rx=0 tx=0).stop 2024-04-26T22:55:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.568+0000 7f4171800640 1 -- 172.21.15.27:0/73059002 shutdown_connections 2024-04-26T22:55:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.568+0000 7f4171800640 1 --2- 172.21.15.27:0/73059002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f416c104320 0x7f416c10abb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.568+0000 7f4171800640 1 --2- 172.21.15.27:0/73059002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f416c103960 0x7f416c103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.568+0000 7f4171800640 1 --2- 172.21.15.27:0/73059002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c102770 0x7f416c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.568+0000 7f4171800640 1 -- 172.21.15.27:0/73059002 >> 172.21.15.27:0/73059002 conn(0x7f416c0fdce0 msgr2=0x7f416c100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.568+0000 7f4171800640 1 -- 172.21.15.27:0/73059002 shutdown_connections 2024-04-26T22:55:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.569+0000 7f4171800640 1 -- 172.21.15.27:0/73059002 wait complete. 2024-04-26T22:55:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.570+0000 7f4171800640 1 Processor -- start 2024-04-26T22:55:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.570+0000 7f4171800640 1 -- start start 2024-04-26T22:55:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.570+0000 7f4171800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c102770 0x7f416c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.570+0000 7f4171800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f416c103960 0x7f416c19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.570+0000 7f4171800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f416c104320 0x7f416c1a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.570+0000 7f4171800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f416c10e2d0 con 0x7f416c102770 2024-04-26T22:55:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.570+0000 7f4171800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f416c10e150 con 0x7f416c104320 2024-04-26T22:55:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.570+0000 7f4171800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f416c10e450 con 0x7f416c103960 2024-04-26T22:55:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.571+0000 7f416be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c102770 0x7f416c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.571+0000 7f416b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f416c103960 0x7f416c19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.571+0000 7f416b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f416c103960 0x7f416c19af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:49410/0 (socket says 172.21.15.27:49410) 2024-04-26T22:55:49.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.571+0000 7f416b400640 1 -- 172.21.15.27:0/503041458 learned_addr learned my addr 172.21.15.27:0/503041458 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:49.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.571+0000 7f4170e00640 1 --2- 172.21.15.27:0/503041458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f416c104320 0x7f416c1a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.571+0000 7f416be00640 1 -- 172.21.15.27:0/503041458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f416c103960 msgr2=0x7f416c19af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.571+0000 7f416be00640 1 --2- 172.21.15.27:0/503041458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f416c103960 0x7f416c19af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.571+0000 7f416be00640 1 -- 172.21.15.27:0/503041458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f416c104320 msgr2=0x7f416c1a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.571+0000 7f416be00640 1 --2- 172.21.15.27:0/503041458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f416c104320 0x7f416c1a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.571+0000 7f416be00640 1 -- 172.21.15.27:0/503041458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c1a2650 con 0x7f416c102770 2024-04-26T22:55:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.571+0000 7f416be00640 1 --2- 172.21.15.27:0/503041458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c102770 0x7f416c19aa30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f415c00e9c0 tx=0x7f415c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.572+0000 7f4168c00640 1 -- 172.21.15.27:0/503041458 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c00ccc0 con 0x7f416c102770 2024-04-26T22:55:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.572+0000 7f4168c00640 1 -- 172.21.15.27:0/503041458 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f415c0035f0 con 0x7f416c102770 2024-04-26T22:55:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.572+0000 7f4171800640 1 -- 172.21.15.27:0/503041458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c1a2940 con 0x7f416c102770 2024-04-26T22:55:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.572+0000 7f4168c00640 1 -- 172.21.15.27:0/503041458 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c00ce60 con 0x7f416c102770 2024-04-26T22:55:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.572+0000 7f4171800640 1 -- 172.21.15.27:0/503041458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c1a2e90 con 0x7f416c102770 2024-04-26T22:55:49.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.573+0000 7f4171800640 1 -- 172.21.15.27:0/503041458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c0506c0 con 0x7f416c102770 2024-04-26T22:55:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.575+0000 7f4168c00640 1 -- 172.21.15.27:0/503041458 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f415c004000 con 0x7f416c102770 2024-04-26T22:55:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.577+0000 7f4168c00640 1 --2- 172.21.15.27:0/503041458 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f41500a3c50 0x7f41500a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.577+0000 7f4168c00640 1 -- 172.21.15.27:0/503041458 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f415c0d0ac0 con 0x7f416c102770 2024-04-26T22:55:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.577+0000 7f416b400640 1 --2- 172.21.15.27:0/503041458 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f41500a3c50 0x7f41500a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.578+0000 7f416b400640 1 --2- 172.21.15.27:0/503041458 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f41500a3c50 0x7f41500a6110 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f41580095f0 tx=0x7f4158005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.581+0000 7f4168c00640 1 -- 172.21.15.27:0/503041458 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f415c09d200 con 0x7f416c102770 2024-04-26T22:55:49.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.787+0000 7f4171800640 1 -- 172.21.15.27:0/503041458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f416c1a38d0 con 0x7f416c102770 2024-04-26T22:55:49.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.789+0000 7f4168c00640 1 -- 172.21.15.27:0/503041458 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24009 (secure 0 0 0) 0x7f415c09cb00 con 0x7f416c102770 2024-04-26T22:55:49.790 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:49.790 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:55:30.886974+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:55:49.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.791+0000 7f4171800640 1 -- 172.21.15.27:0/503041458 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f41500a3c50 msgr2=0x7f41500a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.791+0000 7f4171800640 1 --2- 172.21.15.27:0/503041458 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f41500a3c50 0x7f41500a6110 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f41580095f0 tx=0x7f4158005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.791+0000 7f4171800640 1 -- 172.21.15.27:0/503041458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c102770 msgr2=0x7f416c19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.791+0000 7f4171800640 1 --2- 172.21.15.27:0/503041458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c102770 0x7f416c19aa30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f415c00e9c0 tx=0x7f415c00ee80 comp rx=0 tx=0).stop 2024-04-26T22:55:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.791+0000 7f4171800640 1 -- 172.21.15.27:0/503041458 shutdown_connections 2024-04-26T22:55:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.791+0000 7f4171800640 1 --2- 172.21.15.27:0/503041458 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f41500a3c50 0x7f41500a6110 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.791+0000 7f4171800640 1 --2- 172.21.15.27:0/503041458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f416c104320 0x7f416c1a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:49.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.791+0000 7f4171800640 1 --2- 172.21.15.27:0/503041458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f416c103960 0x7f416c19af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:49.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.791+0000 7f4171800640 1 --2- 172.21.15.27:0/503041458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c102770 0x7f416c19aa30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:49.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.792+0000 7f4171800640 1 -- 172.21.15.27:0/503041458 >> 172.21.15.27:0/503041458 conn(0x7f416c0fdce0 msgr2=0x7f416c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:49.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.792+0000 7f4171800640 1 -- 172.21.15.27:0/503041458 shutdown_connections 2024-04-26T22:55:49.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:49.792+0000 7f4171800640 1 -- 172.21.15.27:0/503041458 wait complete. 2024-04-26T22:55:49.809 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:49 smithi027 ceph-mon[28034]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:55:49.809 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:49 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/4154632547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:55:49.809 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:49 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3986162414' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:55:49.814 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:55:50.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:49 smithi080 ceph-mon[32324]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:55:50.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:49 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/4154632547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:55:50.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:49 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3986162414' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:55:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.028+0000 7f933fe00640 1 -- 172.21.15.27:0/4262772428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9340103950 msgr2=0x7f9340105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.028+0000 7f933fe00640 1 --2- 172.21.15.27:0/4262772428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9340103950 0x7f9340105dc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9328009900 tx=0x7f932802a3f0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.029+0000 7f933fe00640 1 -- 172.21.15.27:0/4262772428 shutdown_connections 2024-04-26T22:55:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.029+0000 7f933fe00640 1 --2- 172.21.15.27:0/4262772428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9340106300 0x7f9340106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.029+0000 7f933fe00640 1 --2- 172.21.15.27:0/4262772428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9340103950 0x7f9340105dc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.029+0000 7f933fe00640 1 --2- 172.21.15.27:0/4262772428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9340100fe0 0x7f9340103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.029+0000 7f933fe00640 1 -- 172.21.15.27:0/4262772428 >> 172.21.15.27:0/4262772428 conn(0x7f93400fcc00 msgr2=0x7f93400ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.029+0000 7f933fe00640 1 -- 172.21.15.27:0/4262772428 shutdown_connections 2024-04-26T22:55:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.029+0000 7f933fe00640 1 -- 172.21.15.27:0/4262772428 wait complete. 2024-04-26T22:55:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.030+0000 7f933fe00640 1 Processor -- start 2024-04-26T22:55:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.030+0000 7f933fe00640 1 -- start start 2024-04-26T22:55:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.030+0000 7f933fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9340100fe0 0x7f934019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.030+0000 7f933fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9340103950 0x7f934019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.030+0000 7f933fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9340106300 0x7f93401a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.030+0000 7f933fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9340114890 con 0x7f9340100fe0 2024-04-26T22:55:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.030+0000 7f933fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9340114710 con 0x7f9340106300 2024-04-26T22:55:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.030+0000 7f933fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9340114a10 con 0x7f9340103950 2024-04-26T22:55:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.031+0000 7f933ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9340100fe0 0x7f934019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.031+0000 7f933ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9340100fe0 0x7f934019ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50448/0 (socket says 172.21.15.27:50448) 2024-04-26T22:55:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.031+0000 7f933ea00640 1 -- 172.21.15.27:0/1318140122 learned_addr learned my addr 172.21.15.27:0/1318140122 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:50.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.031+0000 7f933f400640 1 --2- 172.21.15.27:0/1318140122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9340106300 0x7f93401a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:50.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.031+0000 7f933e000640 1 --2- 172.21.15.27:0/1318140122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9340103950 0x7f934019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:50.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.031+0000 7f933ea00640 1 -- 172.21.15.27:0/1318140122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9340103950 msgr2=0x7f934019f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:50.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.031+0000 7f933ea00640 1 --2- 172.21.15.27:0/1318140122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9340103950 0x7f934019f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.031+0000 7f933ea00640 1 -- 172.21.15.27:0/1318140122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9340106300 msgr2=0x7f93401a6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:50.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.031+0000 7f933ea00640 1 --2- 172.21.15.27:0/1318140122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9340106300 0x7f93401a6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.031+0000 7f933ea00640 1 -- 172.21.15.27:0/1318140122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93401a6a90 con 0x7f9340100fe0 2024-04-26T22:55:50.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.031+0000 7f933f400640 1 --2- 172.21.15.27:0/1318140122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9340106300 0x7f93401a6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:50.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.031+0000 7f933ea00640 1 --2- 172.21.15.27:0/1318140122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9340100fe0 0x7f934019ee70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f933000e9f0 tx=0x7f933000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:50.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.032+0000 7f9337400640 1 -- 172.21.15.27:0/1318140122 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933000ccc0 con 0x7f9340100fe0 2024-04-26T22:55:50.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.032+0000 7f9337400640 1 -- 172.21.15.27:0/1318140122 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f933000ce60 con 0x7f9340100fe0 2024-04-26T22:55:50.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.032+0000 7f933fe00640 1 -- 172.21.15.27:0/1318140122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93401a6d80 con 0x7f9340100fe0 2024-04-26T22:55:50.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.032+0000 7f9337400640 1 -- 172.21.15.27:0/1318140122 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330010430 con 0x7f9340100fe0 2024-04-26T22:55:50.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.032+0000 7f933fe00640 1 -- 172.21.15.27:0/1318140122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93401a71e0 con 0x7f9340100fe0 2024-04-26T22:55:50.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.033+0000 7f933fe00640 1 -- 172.21.15.27:0/1318140122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93400fe6f0 con 0x7f9340100fe0 2024-04-26T22:55:50.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:49 smithi094 ceph-mon[32889]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:55:50.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:49 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/4154632547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:55:50.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:49 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3986162414' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:55:50.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.035+0000 7f9337400640 1 -- 172.21.15.27:0/1318140122 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f93300046c0 con 0x7f9340100fe0 2024-04-26T22:55:50.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.037+0000 7f9337400640 1 --2- 172.21.15.27:0/1318140122 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f93240a3cd0 0x7f93240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:50.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.037+0000 7f9337400640 1 -- 172.21.15.27:0/1318140122 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f93300d0a60 con 0x7f9340100fe0 2024-04-26T22:55:50.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.037+0000 7f933e000640 1 --2- 172.21.15.27:0/1318140122 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f93240a3cd0 0x7f93240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:50.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.038+0000 7f933e000640 1 --2- 172.21.15.27:0/1318140122 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f93240a3cd0 0x7f93240a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9328000c00 tx=0x7f93280314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:50.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.041+0000 7f9337400640 1 -- 172.21.15.27:0/1318140122 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f933009d1a0 con 0x7f9340100fe0 2024-04-26T22:55:50.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.251+0000 7f933fe00640 1 -- 172.21.15.27:0/1318140122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f934002dd10 con 0x7f9340100fe0 2024-04-26T22:55:50.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.254+0000 7f9337400640 1 -- 172.21.15.27:0/1318140122 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24009 (secure 0 0 0) 0x7f933009caa0 con 0x7f9340100fe0 2024-04-26T22:55:50.255 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:50.255 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:55:30.886974+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:55:50.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.257+0000 7f933fe00640 1 -- 172.21.15.27:0/1318140122 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f93240a3cd0 msgr2=0x7f93240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:50.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.257+0000 7f933fe00640 1 --2- 172.21.15.27:0/1318140122 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f93240a3cd0 0x7f93240a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9328000c00 tx=0x7f93280314a0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.257+0000 7f933fe00640 1 -- 172.21.15.27:0/1318140122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9340100fe0 msgr2=0x7f934019ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:50.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.257+0000 7f933fe00640 1 --2- 172.21.15.27:0/1318140122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9340100fe0 0x7f934019ee70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f933000e9f0 tx=0x7f933000edf0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.257+0000 7f933fe00640 1 -- 172.21.15.27:0/1318140122 shutdown_connections 2024-04-26T22:55:50.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.257+0000 7f933fe00640 1 --2- 172.21.15.27:0/1318140122 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f93240a3cd0 0x7f93240a6190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.257+0000 7f933fe00640 1 --2- 172.21.15.27:0/1318140122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9340106300 0x7f93401a6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.257+0000 7f933fe00640 1 --2- 172.21.15.27:0/1318140122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9340103950 0x7f934019f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.257+0000 7f933fe00640 1 --2- 172.21.15.27:0/1318140122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9340100fe0 0x7f934019ee70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.257+0000 7f933fe00640 1 -- 172.21.15.27:0/1318140122 >> 172.21.15.27:0/1318140122 conn(0x7f93400fcc00 msgr2=0x7f9340101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:50.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.257+0000 7f933fe00640 1 -- 172.21.15.27:0/1318140122 shutdown_connections 2024-04-26T22:55:50.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.257+0000 7f933fe00640 1 -- 172.21.15.27:0/1318140122 wait complete. 2024-04-26T22:55:50.278 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T22:55:50.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.483+0000 7f8064800640 1 -- 172.21.15.27:0/754119350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f805c108f90 msgr2=0x7f805c1093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.483+0000 7f8064800640 1 --2- 172.21.15.27:0/754119350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f805c108f90 0x7f805c1093b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8050009900 tx=0x7f805002a3f0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.483+0000 7f8064800640 1 -- 172.21.15.27:0/754119350 shutdown_connections 2024-04-26T22:55:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.483+0000 7f8064800640 1 --2- 172.21.15.27:0/754119350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f805c10a180 0x7f805c10a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.483+0000 7f8064800640 1 --2- 172.21.15.27:0/754119350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f805c10ab40 0x7f805c1113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.483+0000 7f8064800640 1 --2- 172.21.15.27:0/754119350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f805c108f90 0x7f805c1093b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.483+0000 7f8064800640 1 -- 172.21.15.27:0/754119350 >> 172.21.15.27:0/754119350 conn(0x7f805c077f40 msgr2=0x7f805c07a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.483+0000 7f8064800640 1 -- 172.21.15.27:0/754119350 shutdown_connections 2024-04-26T22:55:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.483+0000 7f8064800640 1 -- 172.21.15.27:0/754119350 wait complete. 2024-04-26T22:55:50.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.484+0000 7f8064800640 1 Processor -- start 2024-04-26T22:55:50.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.484+0000 7f8064800640 1 -- start start 2024-04-26T22:55:50.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.484+0000 7f8064800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f805c108f90 0x7f805c19f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:50.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8064800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f805c10a180 0x7f805c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:50.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8064800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f805c10ab40 0x7f805c1a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:50.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8064800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f805c114b20 con 0x7f805c10ab40 2024-04-26T22:55:50.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8064800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f805c1149a0 con 0x7f805c10a180 2024-04-26T22:55:50.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8064800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f805c114ca0 con 0x7f805c108f90 2024-04-26T22:55:50.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8061400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f805c10a180 0x7f805c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:50.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8061e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f805c108f90 0x7f805c19f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:50.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8062800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f805c10ab40 0x7f805c1a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:50.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8061400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f805c10a180 0x7f805c19f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:35320/0 (socket says 172.21.15.27:35320) 2024-04-26T22:55:50.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8061e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f805c108f90 0x7f805c19f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:49462/0 (socket says 172.21.15.27:49462) 2024-04-26T22:55:50.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8061400640 1 -- 172.21.15.27:0/3115759039 learned_addr learned my addr 172.21.15.27:0/3115759039 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:50.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8061400640 1 -- 172.21.15.27:0/3115759039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f805c108f90 msgr2=0x7f805c19f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:50.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8061400640 1 --2- 172.21.15.27:0/3115759039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f805c108f90 0x7f805c19f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8061400640 1 -- 172.21.15.27:0/3115759039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f805c10ab40 msgr2=0x7f805c1a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8061400640 1 --2- 172.21.15.27:0/3115759039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f805c10ab40 0x7f805c1a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8061400640 1 -- 172.21.15.27:0/3115759039 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805c1a6e20 con 0x7f805c10a180 2024-04-26T22:55:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.485+0000 7f8062800640 1 --2- 172.21.15.27:0/3115759039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f805c10ab40 0x7f805c1a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.486+0000 7f8061400640 1 --2- 172.21.15.27:0/3115759039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f805c10a180 0x7f805c19f740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f804c00ea60 tx=0x7f804c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.486+0000 7f8056a00640 1 -- 172.21.15.27:0/3115759039 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c00cde0 con 0x7f805c10a180 2024-04-26T22:55:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.486+0000 7f8056a00640 1 -- 172.21.15.27:0/3115759039 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f804c0035f0 con 0x7f805c10a180 2024-04-26T22:55:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.486+0000 7f8064800640 1 -- 172.21.15.27:0/3115759039 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805c1a7110 con 0x7f805c10a180 2024-04-26T22:55:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.486+0000 7f8056a00640 1 -- 172.21.15.27:0/3115759039 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c010690 con 0x7f805c10a180 2024-04-26T22:55:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.486+0000 7f8064800640 1 -- 172.21.15.27:0/3115759039 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f805c1a7660 con 0x7f805c10a180 2024-04-26T22:55:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.487+0000 7f8064800640 1 -- 172.21.15.27:0/3115759039 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f805c110380 con 0x7f805c10a180 2024-04-26T22:55:50.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.489+0000 7f8056a00640 1 -- 172.21.15.27:0/3115759039 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f804c0037a0 con 0x7f805c10a180 2024-04-26T22:55:50.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.491+0000 7f8056a00640 1 --2- 172.21.15.27:0/3115759039 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f80440a3bb0 0x7f80440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:50.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.491+0000 7f8056a00640 1 -- 172.21.15.27:0/3115759039 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f804c0d0930 con 0x7f805c10a180 2024-04-26T22:55:50.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.491+0000 7f8061e00640 1 --2- 172.21.15.27:0/3115759039 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f80440a3bb0 0x7f80440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:50.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.492+0000 7f8061e00640 1 --2- 172.21.15.27:0/3115759039 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f80440a3bb0 0x7f80440a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8050005ab0 tx=0x7f8050005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:50.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.495+0000 7f8056a00640 1 -- 172.21.15.27:0/3115759039 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f804c09d070 con 0x7f805c10a180 2024-04-26T22:55:50.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:50.745+0000 7f8064800640 1 -- 172.21.15.27:0/3115759039 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f805c063530 con 0x7f805c10a180 2024-04-26T22:55:50.822 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:50 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/503041458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:50.822 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:50 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1318140122' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:51.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:50 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/503041458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:51.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:50 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1318140122' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:51.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:50 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/503041458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:51.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:50 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1318140122' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:51.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.593+0000 7f8056a00640 1 -- 172.21.15.27:0/3115759039 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f804c09c970 con 0x7f805c10a180 2024-04-26T22:55:51.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.597+0000 7f8064800640 1 -- 172.21.15.27:0/3115759039 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f80440a3bb0 msgr2=0x7f80440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:51.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.597+0000 7f8064800640 1 --2- 172.21.15.27:0/3115759039 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f80440a3bb0 0x7f80440a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8050005ab0 tx=0x7f8050005a20 comp rx=0 tx=0).stop 2024-04-26T22:55:51.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.598+0000 7f8064800640 1 -- 172.21.15.27:0/3115759039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f805c10a180 msgr2=0x7f805c19f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:51.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.598+0000 7f8064800640 1 --2- 172.21.15.27:0/3115759039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f805c10a180 0x7f805c19f740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f804c00ea60 tx=0x7f804c00ef20 comp rx=0 tx=0).stop 2024-04-26T22:55:51.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.598+0000 7f8064800640 1 -- 172.21.15.27:0/3115759039 shutdown_connections 2024-04-26T22:55:51.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.598+0000 7f8064800640 1 --2- 172.21.15.27:0/3115759039 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f80440a3bb0 0x7f80440a6070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:51.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.598+0000 7f8064800640 1 --2- 172.21.15.27:0/3115759039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f805c10a180 0x7f805c19f740 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:51.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.598+0000 7f8064800640 1 --2- 172.21.15.27:0/3115759039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f805c10ab40 0x7f805c1a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:51.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.598+0000 7f8064800640 1 --2- 172.21.15.27:0/3115759039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f805c108f90 0x7f805c19f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:51.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.598+0000 7f8064800640 1 -- 172.21.15.27:0/3115759039 >> 172.21.15.27:0/3115759039 conn(0x7f805c077f40 msgr2=0x7f805c10d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:51.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.598+0000 7f8064800640 1 -- 172.21.15.27:0/3115759039 shutdown_connections 2024-04-26T22:55:51.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.598+0000 7f8064800640 1 -- 172.21.15.27:0/3115759039 wait complete. 2024-04-26T22:55:51.621 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-26T22:55:51.621 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-26T22:55:51.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.850+0000 7f8dbf400640 1 -- 172.21.15.27:0/3807188419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8db81033f0 msgr2=0x7f8db81057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:51.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.850+0000 7f8dbf400640 1 --2- 172.21.15.27:0/3807188419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8db81033f0 0x7f8db81057e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8dac009900 tx=0x7f8dac02a410 comp rx=0 tx=0).stop 2024-04-26T22:55:51.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.850+0000 7f8dbf400640 1 -- 172.21.15.27:0/3807188419 shutdown_connections 2024-04-26T22:55:51.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.850+0000 7f8dbf400640 1 --2- 172.21.15.27:0/3807188419 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8db8105d20 0x7f8db8108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:51.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.850+0000 7f8dbf400640 1 --2- 172.21.15.27:0/3807188419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8db8108650 0x7f8db810aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:51.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.850+0000 7f8dbf400640 1 --2- 172.21.15.27:0/3807188419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8db81033f0 0x7f8db81057e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:51.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.850+0000 7f8dbf400640 1 -- 172.21.15.27:0/3807188419 >> 172.21.15.27:0/3807188419 conn(0x7f8db80fce00 msgr2=0x7f8db80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:51.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.850+0000 7f8dbf400640 1 -- 172.21.15.27:0/3807188419 shutdown_connections 2024-04-26T22:55:51.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.850+0000 7f8dbf400640 1 -- 172.21.15.27:0/3807188419 wait complete. 2024-04-26T22:55:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.851+0000 7f8dbf400640 1 Processor -- start 2024-04-26T22:55:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.851+0000 7f8dbf400640 1 -- start start 2024-04-26T22:55:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.852+0000 7f8dbf400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8db81033f0 0x7f8db8102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.852+0000 7f8dbf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8db8105d20 0x7f8db81013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.852+0000 7f8dbf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8db8108650 0x7f8db81018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.852+0000 7f8dbf400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8db810e4f0 con 0x7f8db8108650 2024-04-26T22:55:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.852+0000 7f8dbf400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8db810e370 con 0x7f8db81033f0 2024-04-26T22:55:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.852+0000 7f8dbf400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8db810e670 con 0x7f8db8105d20 2024-04-26T22:55:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.852+0000 7f8dbca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8db81033f0 0x7f8db8102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.852+0000 7f8dbca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8db81033f0 0x7f8db8102d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:35346/0 (socket says 172.21.15.27:35346) 2024-04-26T22:55:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.852+0000 7f8dbca00640 1 -- 172.21.15.27:0/1420805958 learned_addr learned my addr 172.21.15.27:0/1420805958 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.852+0000 7f8dbd400640 1 --2- 172.21.15.27:0/1420805958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8db8108650 0x7f8db81018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:51.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.852+0000 7f8db7e00640 1 --2- 172.21.15.27:0/1420805958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8db8105d20 0x7f8db81013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:51.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.853+0000 7f8dbca00640 1 -- 172.21.15.27:0/1420805958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8db8105d20 msgr2=0x7f8db81013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:51.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.853+0000 7f8dbca00640 1 --2- 172.21.15.27:0/1420805958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8db8105d20 0x7f8db81013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:51.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.853+0000 7f8dbca00640 1 -- 172.21.15.27:0/1420805958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8db8108650 msgr2=0x7f8db81018e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:51.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.853+0000 7f8dbca00640 1 --2- 172.21.15.27:0/1420805958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8db8108650 0x7f8db81018e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:51.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.853+0000 7f8dbca00640 1 -- 172.21.15.27:0/1420805958 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db8102170 con 0x7f8db81033f0 2024-04-26T22:55:51.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.853+0000 7f8dbca00640 1 --2- 172.21.15.27:0/1420805958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8db81033f0 0x7f8db8102d60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8dac005ab0 tx=0x7f8dac003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:51.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.853+0000 7f8db5600640 1 -- 172.21.15.27:0/1420805958 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac03f070 con 0x7f8db81033f0 2024-04-26T22:55:51.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.853+0000 7f8db5600640 1 -- 172.21.15.27:0/1420805958 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dac031c50 con 0x7f8db81033f0 2024-04-26T22:55:51.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.853+0000 7f8dbf400640 1 -- 172.21.15.27:0/1420805958 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db81a2cb0 con 0x7f8db81033f0 2024-04-26T22:55:51.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.853+0000 7f8db5600640 1 -- 172.21.15.27:0/1420805958 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac035030 con 0x7f8db81033f0 2024-04-26T22:55:51.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.853+0000 7f8dbf400640 1 -- 172.21.15.27:0/1420805958 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db81a31a0 con 0x7f8db81033f0 2024-04-26T22:55:51.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.854+0000 7f8dbf400640 1 -- 172.21.15.27:0/1420805958 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8db80fe8f0 con 0x7f8db81033f0 2024-04-26T22:55:51.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.857+0000 7f8db5600640 1 -- 172.21.15.27:0/1420805958 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8dac042070 con 0x7f8db81033f0 2024-04-26T22:55:51.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.858+0000 7f8db5600640 1 --2- 172.21.15.27:0/1420805958 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8da00a3c30 0x7f8da00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:51.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.858+0000 7f8db5600640 1 -- 172.21.15.27:0/1420805958 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8dac0f0d30 con 0x7f8db81033f0 2024-04-26T22:55:51.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.858+0000 7f8db7e00640 1 --2- 172.21.15.27:0/1420805958 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8da00a3c30 0x7f8da00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:51.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.859+0000 7f8db7e00640 1 --2- 172.21.15.27:0/1420805958 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8da00a3c30 0x7f8da00a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8db8102ad0 tx=0x7f8da4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:51.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:51.862+0000 7f8db5600640 1 -- 172.21.15.27:0/1420805958 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8dac0bce20 con 0x7f8db81033f0 2024-04-26T22:55:51.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:51 smithi027 ceph-mon[28034]: pgmap v343: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:55:51.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:51 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3115759039' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T22:55:51.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:51 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T22:55:52.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:51 smithi080 ceph-mon[32324]: pgmap v343: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:55:52.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:51 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3115759039' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T22:55:52.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:51 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T22:55:52.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:51 smithi094 ceph-mon[32889]: pgmap v343: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:55:52.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:51 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3115759039' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T22:55:52.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:51 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T22:55:52.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.111+0000 7f8dbf400640 1 -- 172.21.15.27:0/1420805958 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f8db80635d0 con 0x7f8db81033f0 2024-04-26T22:55:52.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.597+0000 7f8db5600640 1 -- 172.21.15.27:0/1420805958 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f8dac0c1120 con 0x7f8db81033f0 2024-04-26T22:55:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.601+0000 7f8dbf400640 1 -- 172.21.15.27:0/1420805958 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8da00a3c30 msgr2=0x7f8da00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.601+0000 7f8dbf400640 1 --2- 172.21.15.27:0/1420805958 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8da00a3c30 0x7f8da00a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8db8102ad0 tx=0x7f8da4005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.601+0000 7f8dbf400640 1 -- 172.21.15.27:0/1420805958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8db81033f0 msgr2=0x7f8db8102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.601+0000 7f8dbf400640 1 --2- 172.21.15.27:0/1420805958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8db81033f0 0x7f8db8102d60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8dac005ab0 tx=0x7f8dac003b90 comp rx=0 tx=0).stop 2024-04-26T22:55:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.602+0000 7f8dbf400640 1 -- 172.21.15.27:0/1420805958 shutdown_connections 2024-04-26T22:55:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.602+0000 7f8dbf400640 1 --2- 172.21.15.27:0/1420805958 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8da00a3c30 0x7f8da00a60f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.602+0000 7f8dbf400640 1 --2- 172.21.15.27:0/1420805958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8db8108650 0x7f8db81018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.602+0000 7f8dbf400640 1 --2- 172.21.15.27:0/1420805958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8db8105d20 0x7f8db81013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.602+0000 7f8dbf400640 1 --2- 172.21.15.27:0/1420805958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8db81033f0 0x7f8db8102d60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.602+0000 7f8dbf400640 1 -- 172.21.15.27:0/1420805958 >> 172.21.15.27:0/1420805958 conn(0x7f8db80fce00 msgr2=0x7f8db8105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.602+0000 7f8dbf400640 1 -- 172.21.15.27:0/1420805958 shutdown_connections 2024-04-26T22:55:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.602+0000 7f8dbf400640 1 -- 172.21.15.27:0/1420805958 wait complete. 2024-04-26T22:55:52.633 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-26T22:55:52.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.829+0000 7f6435e00640 1 -- 172.21.15.27:0/3534249256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6430075390 msgr2=0x7f6430075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:52.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.829+0000 7f6435e00640 1 --2- 172.21.15.27:0/3534249256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6430075390 0x7f6430075810 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f641c009900 tx=0x7f641c02a3f0 comp rx=0 tx=0).stop 2024-04-26T22:55:52.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.829+0000 7f6435e00640 1 -- 172.21.15.27:0/3534249256 shutdown_connections 2024-04-26T22:55:52.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.829+0000 7f6435e00640 1 --2- 172.21.15.27:0/3534249256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643010ef50 0x7f6430111340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:52.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.829+0000 7f6435e00640 1 --2- 172.21.15.27:0/3534249256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6430075390 0x7f6430075810 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:52.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.829+0000 7f6435e00640 1 --2- 172.21.15.27:0/3534249256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6430076970 0x7f6430074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:52.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.829+0000 7f6435e00640 1 -- 172.21.15.27:0/3534249256 >> 172.21.15.27:0/3534249256 conn(0x7f64300fdef0 msgr2=0x7f6430100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:52.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.830+0000 7f6435e00640 1 -- 172.21.15.27:0/3534249256 shutdown_connections 2024-04-26T22:55:52.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.830+0000 7f6435e00640 1 -- 172.21.15.27:0/3534249256 wait complete. 2024-04-26T22:55:52.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.831+0000 7f6435e00640 1 Processor -- start 2024-04-26T22:55:52.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.831+0000 7f6435e00640 1 -- start start 2024-04-26T22:55:52.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.831+0000 7f6435e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6430075390 0x7f643019f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:52.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.831+0000 7f6435e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6430076970 0x7f643019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:52.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.831+0000 7f6435e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643010ef50 0x7f64301a6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:52.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.831+0000 7f6435e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6430114a80 con 0x7f6430076970 2024-04-26T22:55:52.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.831+0000 7f6435e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6430114900 con 0x7f643010ef50 2024-04-26T22:55:52.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.831+0000 7f6435e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6430114c00 con 0x7f6430075390 2024-04-26T22:55:52.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.832+0000 7f642ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6430075390 0x7f643019f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:52.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.832+0000 7f642e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6430076970 0x7f643019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:52.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.832+0000 7f642f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643010ef50 0x7f64301a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:52.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.832+0000 7f642ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6430075390 0x7f643019f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:49498/0 (socket says 172.21.15.27:49498) 2024-04-26T22:55:52.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.832+0000 7f642ea00640 1 -- 172.21.15.27:0/1616183088 learned_addr learned my addr 172.21.15.27:0/1616183088 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:52.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.832+0000 7f642f400640 1 -- 172.21.15.27:0/1616183088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6430075390 msgr2=0x7f643019f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:52.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.832+0000 7f642f400640 1 --2- 172.21.15.27:0/1616183088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6430075390 0x7f643019f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:52.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.832+0000 7f642f400640 1 -- 172.21.15.27:0/1616183088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6430076970 msgr2=0x7f643019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:52.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.832+0000 7f642f400640 1 --2- 172.21.15.27:0/1616183088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6430076970 0x7f643019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:52.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.832+0000 7f642f400640 1 -- 172.21.15.27:0/1616183088 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64301a6df0 con 0x7f643010ef50 2024-04-26T22:55:52.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.832+0000 7f642ea00640 1 --2- 172.21.15.27:0/1616183088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6430075390 0x7f643019f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:52.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.833+0000 7f642f400640 1 --2- 172.21.15.27:0/1616183088 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643010ef50 0x7f64301a6780 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6428009fd0 tx=0x7f642800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:52.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.833+0000 7f6434a00640 1 -- 172.21.15.27:0/1616183088 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642800ce20 con 0x7f643010ef50 2024-04-26T22:55:52.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.833+0000 7f6434a00640 1 -- 172.21.15.27:0/1616183088 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6428005020 con 0x7f643010ef50 2024-04-26T22:55:52.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.833+0000 7f6434a00640 1 -- 172.21.15.27:0/1616183088 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428010690 con 0x7f643010ef50 2024-04-26T22:55:52.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.833+0000 7f6435e00640 1 -- 172.21.15.27:0/1616183088 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64301a70e0 con 0x7f643010ef50 2024-04-26T22:55:52.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.833+0000 7f6435e00640 1 -- 172.21.15.27:0/1616183088 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64301a75b0 con 0x7f643010ef50 2024-04-26T22:55:52.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.834+0000 7f6435e00640 1 -- 172.21.15.27:0/1616183088 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6430105120 con 0x7f643010ef50 2024-04-26T22:55:52.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.837+0000 7f6434a00640 1 -- 172.21.15.27:0/1616183088 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f64280054d0 con 0x7f643010ef50 2024-04-26T22:55:52.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.838+0000 7f6434a00640 1 --2- 172.21.15.27:0/1616183088 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64140a3c30 0x7f64140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:52.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.838+0000 7f6434a00640 1 -- 172.21.15.27:0/1616183088 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f64280d0c30 con 0x7f643010ef50 2024-04-26T22:55:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.838+0000 7f642ea00640 1 --2- 172.21.15.27:0/1616183088 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64140a3c30 0x7f64140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.839+0000 7f642ea00640 1 --2- 172.21.15.27:0/1616183088 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64140a3c30 0x7f64140a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6420005fd0 tx=0x7f642000b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:52.840+0000 7f6434a00640 1 -- 172.21.15.27:0/1616183088 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f642809d370 con 0x7f643010ef50 2024-04-26T22:55:52.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:52 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T22:55:52.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:52 smithi027 ceph-mon[28034]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:55:52.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:52 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1420805958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T22:55:52.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:52 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T22:55:52.936 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:52 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T22:55:52.936 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:52 smithi094 ceph-mon[32889]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:55:52.936 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:52 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1420805958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T22:55:52.936 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:52 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T22:55:53.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:52 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T22:55:53.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:52 smithi080 ceph-mon[32324]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:55:53.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:52 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1420805958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T22:55:53.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:52 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T22:55:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.092+0000 7f6435e00640 1 -- 172.21.15.27:0/1616183088 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f643002dad0 con 0x7f643010ef50 2024-04-26T22:55:53.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.621+0000 7f6434a00640 1 -- 172.21.15.27:0/1616183088 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f642809cc70 con 0x7f643010ef50 2024-04-26T22:55:53.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.625+0000 7f6435e00640 1 -- 172.21.15.27:0/1616183088 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64140a3c30 msgr2=0x7f64140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:53.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.625+0000 7f6435e00640 1 --2- 172.21.15.27:0/1616183088 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64140a3c30 0x7f64140a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6420005fd0 tx=0x7f642000b4a0 comp rx=0 tx=0).stop 2024-04-26T22:55:53.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.625+0000 7f6435e00640 1 -- 172.21.15.27:0/1616183088 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643010ef50 msgr2=0x7f64301a6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:53.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.625+0000 7f6435e00640 1 --2- 172.21.15.27:0/1616183088 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643010ef50 0x7f64301a6780 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6428009fd0 tx=0x7f642800ef10 comp rx=0 tx=0).stop 2024-04-26T22:55:53.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.625+0000 7f6435e00640 1 -- 172.21.15.27:0/1616183088 shutdown_connections 2024-04-26T22:55:53.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.625+0000 7f6435e00640 1 --2- 172.21.15.27:0/1616183088 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64140a3c30 0x7f64140a60f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:53.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.625+0000 7f6435e00640 1 --2- 172.21.15.27:0/1616183088 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643010ef50 0x7f64301a6780 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:53.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.625+0000 7f6435e00640 1 --2- 172.21.15.27:0/1616183088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6430076970 0x7f643019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:53.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.625+0000 7f6435e00640 1 --2- 172.21.15.27:0/1616183088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6430075390 0x7f643019f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:53.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.625+0000 7f6435e00640 1 -- 172.21.15.27:0/1616183088 >> 172.21.15.27:0/1616183088 conn(0x7f64300fdef0 msgr2=0x7f64301022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:53.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.625+0000 7f6435e00640 1 -- 172.21.15.27:0/1616183088 shutdown_connections 2024-04-26T22:55:53.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.625+0000 7f6435e00640 1 -- 172.21.15.27:0/1616183088 wait complete. 2024-04-26T22:55:53.653 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-26T22:55:53.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.844+0000 7f7bee400640 1 -- 172.21.15.27:0/1519454222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7be8105c10 msgr2=0x7f7be8108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:53.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.844+0000 7f7bee400640 1 --2- 172.21.15.27:0/1519454222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7be8105c10 0x7f7be8108000 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009900 tx=0x7f7bd002a3f0 comp rx=0 tx=0).stop 2024-04-26T22:55:53.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.844+0000 7f7bee400640 1 -- 172.21.15.27:0/1519454222 shutdown_connections 2024-04-26T22:55:53.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.844+0000 7f7bee400640 1 --2- 172.21.15.27:0/1519454222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7be8105c10 0x7f7be8108000 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:53.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.844+0000 7f7bee400640 1 --2- 172.21.15.27:0/1519454222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7be8108540 0x7f7be810a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:53.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.844+0000 7f7bee400640 1 --2- 172.21.15.27:0/1519454222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7be81032e0 0x7f7be81056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:53.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.844+0000 7f7bee400640 1 -- 172.21.15.27:0/1519454222 >> 172.21.15.27:0/1519454222 conn(0x7f7be80fcc10 msgr2=0x7f7be80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:53.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.844+0000 7f7bee400640 1 -- 172.21.15.27:0/1519454222 shutdown_connections 2024-04-26T22:55:53.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.844+0000 7f7bee400640 1 -- 172.21.15.27:0/1519454222 wait complete. 2024-04-26T22:55:53.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.845+0000 7f7bee400640 1 Processor -- start 2024-04-26T22:55:53.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.845+0000 7f7bee400640 1 -- start start 2024-04-26T22:55:53.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7bee400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7be81032e0 0x7f7be8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:53.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7bee400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7be8105c10 0x7f7be8198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:53.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7bee400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7be8108540 0x7f7be819ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:53.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7bee400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7be810e330 con 0x7f7be8108540 2024-04-26T22:55:53.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7bee400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7be810e1b0 con 0x7f7be81032e0 2024-04-26T22:55:53.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7bee400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7be810e4b0 con 0x7f7be8105c10 2024-04-26T22:55:53.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7bed000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7be81032e0 0x7f7be8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:53.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7bed000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7be81032e0 0x7f7be8198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:35388/0 (socket says 172.21.15.27:35388) 2024-04-26T22:55:53.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7bed000640 1 -- 172.21.15.27:0/326512779 learned_addr learned my addr 172.21.15.27:0/326512779 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:53.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7beda00640 1 --2- 172.21.15.27:0/326512779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7be8108540 0x7f7be819ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:53.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7bdfe00640 1 --2- 172.21.15.27:0/326512779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7be8105c10 0x7f7be8198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:53.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7bed000640 1 -- 172.21.15.27:0/326512779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7be8105c10 msgr2=0x7f7be8198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:53.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7bed000640 1 --2- 172.21.15.27:0/326512779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7be8105c10 0x7f7be8198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:53.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7bed000640 1 -- 172.21.15.27:0/326512779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7be8108540 msgr2=0x7f7be819ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:53.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7bed000640 1 --2- 172.21.15.27:0/326512779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7be8108540 0x7f7be819ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:53.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7bed000640 1 -- 172.21.15.27:0/326512779 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be81a0620 con 0x7f7be81032e0 2024-04-26T22:55:53.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.846+0000 7f7beda00640 1 --2- 172.21.15.27:0/326512779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7be8108540 0x7f7be819ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:53.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.847+0000 7f7bed000640 1 --2- 172.21.15.27:0/326512779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7be81032e0 0x7f7be8198a00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800e9f0 tx=0x7f7bd800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:53.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.847+0000 7f7be6000640 1 -- 172.21.15.27:0/326512779 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd800cd80 con 0x7f7be81032e0 2024-04-26T22:55:53.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.847+0000 7f7be6000640 1 -- 172.21.15.27:0/326512779 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bd80108e0 con 0x7f7be81032e0 2024-04-26T22:55:53.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.847+0000 7f7be6000640 1 -- 172.21.15.27:0/326512779 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8018980 con 0x7f7be81032e0 2024-04-26T22:55:53.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.847+0000 7f7bee400640 1 -- 172.21.15.27:0/326512779 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be81a0910 con 0x7f7be81032e0 2024-04-26T22:55:53.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.847+0000 7f7bee400640 1 -- 172.21.15.27:0/326512779 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be81a0de0 con 0x7f7be81032e0 2024-04-26T22:55:53.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.848+0000 7f7bee400640 1 -- 172.21.15.27:0/326512779 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be80fe700 con 0x7f7be81032e0 2024-04-26T22:55:53.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.851+0000 7f7be6000640 1 -- 172.21.15.27:0/326512779 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7bd8010430 con 0x7f7be81032e0 2024-04-26T22:55:53.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.852+0000 7f7be6000640 1 --2- 172.21.15.27:0/326512779 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7bcc0a3c30 0x7f7bcc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:53.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.852+0000 7f7be6000640 1 -- 172.21.15.27:0/326512779 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7bd80d0960 con 0x7f7be81032e0 2024-04-26T22:55:53.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.852+0000 7f7bdfe00640 1 --2- 172.21.15.27:0/326512779 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7bcc0a3c30 0x7f7bcc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:53.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.853+0000 7f7bdfe00640 1 --2- 172.21.15.27:0/326512779 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7bcc0a3c30 0x7f7bcc0a60f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0005ab0 tx=0x7f7bd0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:53.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:53.856+0000 7f7be6000640 1 -- 172.21.15.27:0/326512779 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7bd809ca50 con 0x7f7be81032e0 2024-04-26T22:55:53.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:53 smithi027 ceph-mon[28034]: pgmap v344: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-26T22:55:53.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:53 smithi027 ceph-mon[28034]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T22:55:53.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:53 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T22:55:53.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:53 smithi027 ceph-mon[28034]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T22:55:53.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:53 smithi027 ceph-mon[28034]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:55:53.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:53 smithi027 ceph-mon[28034]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-26T22:55:53.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:53 smithi027 ceph-mon[28034]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-26T22:55:53.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:53 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1616183088' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T22:55:53.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:53 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T22:55:53.921 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:53 smithi080 ceph-mon[32324]: pgmap v344: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-26T22:55:53.921 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:53 smithi080 ceph-mon[32324]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T22:55:53.921 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:53 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T22:55:53.921 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:53 smithi080 ceph-mon[32324]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T22:55:53.921 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:53 smithi080 ceph-mon[32324]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:55:53.921 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:53 smithi080 ceph-mon[32324]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-26T22:55:53.921 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:53 smithi080 ceph-mon[32324]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-26T22:55:53.921 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:53 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1616183088' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T22:55:53.922 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:53 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T22:55:53.954 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:53 smithi094 ceph-mon[32889]: pgmap v344: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-26T22:55:53.954 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:53 smithi094 ceph-mon[32889]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T22:55:53.954 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:53 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T22:55:53.955 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:53 smithi094 ceph-mon[32889]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T22:55:53.955 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:53 smithi094 ceph-mon[32889]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:55:53.955 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:53 smithi094 ceph-mon[32889]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-26T22:55:53.955 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:53 smithi094 ceph-mon[32889]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-26T22:55:53.955 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:53 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1616183088' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T22:55:53.955 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:53 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T22:55:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.096+0000 7f7bee400640 1 -- 172.21.15.27:0/326512779 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f7be802dad0 con 0x7f7be81032e0 2024-04-26T22:55:54.271 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 22:55:53 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-b[63021]: 2024-04-26T22:55:53.921+0000 7f6a2c29f700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T22:55:54.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.642+0000 7f7be6000640 1 -- 172.21.15.27:0/326512779 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f7bd80a0d50 con 0x7f7be81032e0 2024-04-26T22:55:54.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.645+0000 7f7bee400640 1 -- 172.21.15.27:0/326512779 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7bcc0a3c30 msgr2=0x7f7bcc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:54.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.645+0000 7f7bee400640 1 --2- 172.21.15.27:0/326512779 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7bcc0a3c30 0x7f7bcc0a60f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0005ab0 tx=0x7f7bd0005a20 comp rx=0 tx=0).stop 2024-04-26T22:55:54.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.645+0000 7f7bee400640 1 -- 172.21.15.27:0/326512779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7be81032e0 msgr2=0x7f7be8198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:54.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.645+0000 7f7bee400640 1 --2- 172.21.15.27:0/326512779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7be81032e0 0x7f7be8198a00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800e9f0 tx=0x7f7bd800eeb0 comp rx=0 tx=0).stop 2024-04-26T22:55:54.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.645+0000 7f7bee400640 1 -- 172.21.15.27:0/326512779 shutdown_connections 2024-04-26T22:55:54.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.645+0000 7f7bee400640 1 --2- 172.21.15.27:0/326512779 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7bcc0a3c30 0x7f7bcc0a60f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:54.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.645+0000 7f7bee400640 1 --2- 172.21.15.27:0/326512779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7be8108540 0x7f7be819ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:54.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.645+0000 7f7bee400640 1 --2- 172.21.15.27:0/326512779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7be8105c10 0x7f7be8198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:54.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.645+0000 7f7bee400640 1 --2- 172.21.15.27:0/326512779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7be81032e0 0x7f7be8198a00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:54.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.645+0000 7f7bee400640 1 -- 172.21.15.27:0/326512779 >> 172.21.15.27:0/326512779 conn(0x7f7be80fcc10 msgr2=0x7f7be8103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:54.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.646+0000 7f7bee400640 1 -- 172.21.15.27:0/326512779 shutdown_connections 2024-04-26T22:55:54.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.646+0000 7f7bee400640 1 -- 172.21.15.27:0/326512779 wait complete. 2024-04-26T22:55:54.669 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-26T22:55:54.669 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-26T22:55:54.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.872+0000 7f253e600640 1 -- 172.21.15.27:0/3977743606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25381009d0 msgr2=0x7f2538100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:54.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.872+0000 7f253e600640 1 --2- 172.21.15.27:0/3977743606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25381009d0 0x7f2538100df0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f252c009900 tx=0x7f252c02a3f0 comp rx=0 tx=0).stop 2024-04-26T22:55:54.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.872+0000 7f253e600640 1 -- 172.21.15.27:0/3977743606 shutdown_connections 2024-04-26T22:55:54.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.872+0000 7f253e600640 1 --2- 172.21.15.27:0/3977743606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2538102580 0x7f2538108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:54.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.872+0000 7f253e600640 1 --2- 172.21.15.27:0/3977743606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2538101bc0 0x7f2538102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:54.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.872+0000 7f253e600640 1 --2- 172.21.15.27:0/3977743606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25381009d0 0x7f2538100df0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:54.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.872+0000 7f253e600640 1 -- 172.21.15.27:0/3977743606 >> 172.21.15.27:0/3977743606 conn(0x7f25380fbf40 msgr2=0x7f25380fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:54.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.872+0000 7f253e600640 1 -- 172.21.15.27:0/3977743606 shutdown_connections 2024-04-26T22:55:54.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.872+0000 7f253e600640 1 -- 172.21.15.27:0/3977743606 wait complete. 2024-04-26T22:55:54.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.874+0000 7f253e600640 1 Processor -- start 2024-04-26T22:55:54.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.874+0000 7f253e600640 1 -- start start 2024-04-26T22:55:54.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.875+0000 7f253e600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25381009d0 0x7f253819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:54.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.875+0000 7f253e600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2538101bc0 0x7f253819b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:54.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.875+0000 7f253e600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2538102580 0x7f25381a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.875+0000 7f253e600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f253810e320 con 0x7f2538101bc0 2024-04-26T22:55:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.875+0000 7f253e600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f253810e1a0 con 0x7f25381009d0 2024-04-26T22:55:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.875+0000 7f253e600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f253810e4a0 con 0x7f2538102580 2024-04-26T22:55:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.875+0000 7f253d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25381009d0 0x7f253819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.875+0000 7f2537e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2538101bc0 0x7f253819b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.875+0000 7f253d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25381009d0 0x7f253819ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:35412/0 (socket says 172.21.15.27:35412) 2024-04-26T22:55:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.875+0000 7f253d200640 1 -- 172.21.15.27:0/3718638625 learned_addr learned my addr 172.21.15.27:0/3718638625 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.875+0000 7f253dc00640 1 --2- 172.21.15.27:0/3718638625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2538102580 0x7f25381a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.875+0000 7f253d200640 1 -- 172.21.15.27:0/3718638625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2538102580 msgr2=0x7f25381a2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.875+0000 7f253d200640 1 --2- 172.21.15.27:0/3718638625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2538102580 0x7f25381a2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.875+0000 7f253d200640 1 -- 172.21.15.27:0/3718638625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2538101bc0 msgr2=0x7f253819b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.875+0000 7f253d200640 1 --2- 172.21.15.27:0/3718638625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2538101bc0 0x7f253819b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.875+0000 7f253d200640 1 -- 172.21.15.27:0/3718638625 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25381a28c0 con 0x7f25381009d0 2024-04-26T22:55:54.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.876+0000 7f253d200640 1 --2- 172.21.15.27:0/3718638625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25381009d0 0x7f253819ac80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f252c031500 tx=0x7f252c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:54.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.876+0000 7f2535600640 1 -- 172.21.15.27:0/3718638625 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c03f070 con 0x7f25381009d0 2024-04-26T22:55:54.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.876+0000 7f2535600640 1 -- 172.21.15.27:0/3718638625 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f252c031c50 con 0x7f25381009d0 2024-04-26T22:55:54.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.876+0000 7f2535600640 1 -- 172.21.15.27:0/3718638625 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c035030 con 0x7f25381009d0 2024-04-26T22:55:54.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.876+0000 7f253e600640 1 -- 172.21.15.27:0/3718638625 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25381a2b50 con 0x7f25381009d0 2024-04-26T22:55:54.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.876+0000 7f253e600640 1 -- 172.21.15.27:0/3718638625 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25381a2ff0 con 0x7f25381009d0 2024-04-26T22:55:54.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.877+0000 7f253e600640 1 -- 172.21.15.27:0/3718638625 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25381a3220 con 0x7f25381009d0 2024-04-26T22:55:54.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.879+0000 7f2535600640 1 -- 172.21.15.27:0/3718638625 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f252c042070 con 0x7f25381009d0 2024-04-26T22:55:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.881+0000 7f2535600640 1 --2- 172.21.15.27:0/3718638625 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f25200a3c30 0x7f25200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.881+0000 7f2535600640 1 -- 172.21.15.27:0/3718638625 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f252c0f0d80 con 0x7f25381009d0 2024-04-26T22:55:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.881+0000 7f2537e00640 1 --2- 172.21.15.27:0/3718638625 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f25200a3c30 0x7f25200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:54.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.882+0000 7f2537e00640 1 --2- 172.21.15.27:0/3718638625 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f25200a3c30 0x7f25200a60f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f253819c180 tx=0x7f2528005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:54.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:54.884+0000 7f2535600640 1 -- 172.21.15.27:0/3718638625 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f252c0bce70 con 0x7f25381009d0 2024-04-26T22:55:55.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:54 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T22:55:55.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:54 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] up:active 2024-04-26T22:55:55.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:54 smithi080 ceph-mon[32324]: daemon mds.a assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T22:55:55.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:54 smithi080 ceph-mon[32324]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T22:55:55.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:54 smithi080 ceph-mon[32324]: Cluster is now healthy 2024-04-26T22:55:55.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:54 smithi080 ceph-mon[32324]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-26T22:55:55.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:54 smithi080 ceph-mon[32324]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:creating} 9 up:standby 2024-04-26T22:55:55.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:54 smithi080 ceph-mon[32324]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-26T22:55:55.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:54 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/326512779' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T22:55:55.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:54 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T22:55:55.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:54 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T22:55:55.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:54 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] up:active 2024-04-26T22:55:55.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:54 smithi094 ceph-mon[32889]: daemon mds.a assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T22:55:55.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:54 smithi094 ceph-mon[32889]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T22:55:55.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:54 smithi094 ceph-mon[32889]: Cluster is now healthy 2024-04-26T22:55:55.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:54 smithi094 ceph-mon[32889]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-26T22:55:55.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:54 smithi094 ceph-mon[32889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:creating} 9 up:standby 2024-04-26T22:55:55.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:54 smithi094 ceph-mon[32889]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-26T22:55:55.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:54 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/326512779' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T22:55:55.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:54 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T22:55:55.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:54 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T22:55:55.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:54 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] up:active 2024-04-26T22:55:55.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:54 smithi027 ceph-mon[28034]: daemon mds.a assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T22:55:55.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:54 smithi027 ceph-mon[28034]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T22:55:55.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:54 smithi027 ceph-mon[28034]: Cluster is now healthy 2024-04-26T22:55:55.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:54 smithi027 ceph-mon[28034]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-26T22:55:55.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:54 smithi027 ceph-mon[28034]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:creating} 9 up:standby 2024-04-26T22:55:55.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:54 smithi027 ceph-mon[28034]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-26T22:55:55.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:54 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/326512779' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T22:55:55.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:54 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T22:55:55.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.121+0000 7f253e600640 1 -- 172.21.15.27:0/3718638625 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f253810c410 con 0x7f25200a3c30 2024-04-26T22:55:55.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.155+0000 7f2535600640 1 -- 172.21.15.27:0/3718638625 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f253810c410 con 0x7f25200a3c30 2024-04-26T22:55:55.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.157+0000 7f253e600640 1 -- 172.21.15.27:0/3718638625 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f25200a3c30 msgr2=0x7f25200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:55.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.157+0000 7f253e600640 1 --2- 172.21.15.27:0/3718638625 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f25200a3c30 0x7f25200a60f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f253819c180 tx=0x7f2528005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:55.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.157+0000 7f253e600640 1 -- 172.21.15.27:0/3718638625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25381009d0 msgr2=0x7f253819ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:55.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.157+0000 7f253e600640 1 --2- 172.21.15.27:0/3718638625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25381009d0 0x7f253819ac80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f252c031500 tx=0x7f252c003c10 comp rx=0 tx=0).stop 2024-04-26T22:55:55.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.157+0000 7f253e600640 1 -- 172.21.15.27:0/3718638625 shutdown_connections 2024-04-26T22:55:55.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.157+0000 7f253e600640 1 --2- 172.21.15.27:0/3718638625 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f25200a3c30 0x7f25200a60f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.157+0000 7f253e600640 1 --2- 172.21.15.27:0/3718638625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2538102580 0x7f25381a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.157+0000 7f253e600640 1 --2- 172.21.15.27:0/3718638625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2538101bc0 0x7f253819b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.157+0000 7f253e600640 1 --2- 172.21.15.27:0/3718638625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25381009d0 0x7f253819ac80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.157+0000 7f253e600640 1 -- 172.21.15.27:0/3718638625 >> 172.21.15.27:0/3718638625 conn(0x7f25380fbf40 msgr2=0x7f253810acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:55.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.157+0000 7f253e600640 1 -- 172.21.15.27:0/3718638625 shutdown_connections 2024-04-26T22:55:55.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.157+0000 7f253e600640 1 -- 172.21.15.27:0/3718638625 wait complete. 2024-04-26T22:55:55.177 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-26T22:55:55.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.378+0000 7ff026200640 1 -- 172.21.15.27:0/3313211425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff020109fa0 msgr2=0x7ff02010a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:55.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.378+0000 7ff026200640 1 --2- 172.21.15.27:0/3313211425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff020109fa0 0x7ff02010a420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff00c009900 tx=0x7ff00c02a400 comp rx=0 tx=0).stop 2024-04-26T22:55:55.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.378+0000 7ff026200640 1 -- 172.21.15.27:0/3313211425 shutdown_connections 2024-04-26T22:55:55.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.379+0000 7ff026200640 1 --2- 172.21.15.27:0/3313211425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff020109fa0 0x7ff02010a420 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.379+0000 7ff026200640 1 --2- 172.21.15.27:0/3313211425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff02010a960 0x7ff0201111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.379+0000 7ff026200640 1 --2- 172.21.15.27:0/3313211425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff020108db0 0x7ff0201091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.379+0000 7ff026200640 1 -- 172.21.15.27:0/3313211425 >> 172.21.15.27:0/3313211425 conn(0x7ff020077d70 msgr2=0x7ff02007a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:55.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.379+0000 7ff026200640 1 -- 172.21.15.27:0/3313211425 shutdown_connections 2024-04-26T22:55:55.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.379+0000 7ff026200640 1 -- 172.21.15.27:0/3313211425 wait complete. 2024-04-26T22:55:55.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.380+0000 7ff026200640 1 Processor -- start 2024-04-26T22:55:55.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.380+0000 7ff026200640 1 -- start start 2024-04-26T22:55:55.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.380+0000 7ff026200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff020108db0 0x7ff02019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:55.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.380+0000 7ff026200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff020109fa0 0x7ff02019f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:55.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.380+0000 7ff026200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff02010a960 0x7ff0201a6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:55.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.380+0000 7ff026200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff020114910 con 0x7ff02010a960 2024-04-26T22:55:55.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.380+0000 7ff026200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff020114790 con 0x7ff020108db0 2024-04-26T22:55:55.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.380+0000 7ff026200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff020114a90 con 0x7ff020109fa0 2024-04-26T22:55:55.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.380+0000 7ff024e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff020108db0 0x7ff02019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:55.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.380+0000 7ff024e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff020108db0 0x7ff02019f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:35454/0 (socket says 172.21.15.27:35454) 2024-04-26T22:55:55.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.380+0000 7ff01fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff020109fa0 0x7ff02019f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:55.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.380+0000 7ff024e00640 1 -- 172.21.15.27:0/2572051929 learned_addr learned my addr 172.21.15.27:0/2572051929 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:55.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.380+0000 7ff025800640 1 --2- 172.21.15.27:0/2572051929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff02010a960 0x7ff0201a6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:55.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.381+0000 7ff024e00640 1 -- 172.21.15.27:0/2572051929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff020109fa0 msgr2=0x7ff02019f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:55.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.381+0000 7ff024e00640 1 --2- 172.21.15.27:0/2572051929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff020109fa0 0x7ff02019f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.381+0000 7ff024e00640 1 -- 172.21.15.27:0/2572051929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff02010a960 msgr2=0x7ff0201a6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:55.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.381+0000 7ff024e00640 1 --2- 172.21.15.27:0/2572051929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff02010a960 0x7ff0201a6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.381+0000 7ff024e00640 1 -- 172.21.15.27:0/2572051929 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0201a6bc0 con 0x7ff020108db0 2024-04-26T22:55:55.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.381+0000 7ff025800640 1 --2- 172.21.15.27:0/2572051929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff02010a960 0x7ff0201a6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:55:55.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.381+0000 7ff024e00640 1 --2- 172.21.15.27:0/2572051929 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff020108db0 0x7ff02019f040 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff01400d950 tx=0x7ff01400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:55.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.381+0000 7ff01d600640 1 -- 172.21.15.27:0/2572051929 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014017020 con 0x7ff020108db0 2024-04-26T22:55:55.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.381+0000 7ff01d600640 1 -- 172.21.15.27:0/2572051929 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff014009c30 con 0x7ff020108db0 2024-04-26T22:55:55.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.381+0000 7ff01d600640 1 -- 172.21.15.27:0/2572051929 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01400f690 con 0x7ff020108db0 2024-04-26T22:55:55.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.381+0000 7ff026200640 1 -- 172.21.15.27:0/2572051929 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0201a6eb0 con 0x7ff020108db0 2024-04-26T22:55:55.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.382+0000 7ff026200640 1 -- 172.21.15.27:0/2572051929 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0201a72c0 con 0x7ff020108db0 2024-04-26T22:55:55.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.383+0000 7ff026200640 1 -- 172.21.15.27:0/2572051929 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0201100d0 con 0x7ff020108db0 2024-04-26T22:55:55.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.385+0000 7ff01d600640 1 -- 172.21.15.27:0/2572051929 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff01401a020 con 0x7ff020108db0 2024-04-26T22:55:55.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.386+0000 7ff01d600640 1 --2- 172.21.15.27:0/2572051929 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff0080a3c30 0x7ff0080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:55.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.386+0000 7ff01d600640 1 -- 172.21.15.27:0/2572051929 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff0140d0520 con 0x7ff020108db0 2024-04-26T22:55:55.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.386+0000 7ff01fe00640 1 --2- 172.21.15.27:0/2572051929 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff0080a3c30 0x7ff0080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:55.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.387+0000 7ff01fe00640 1 --2- 172.21.15.27:0/2572051929 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff0080a3c30 0x7ff0080a60f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff00c031500 tx=0x7ff00c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:55.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.390+0000 7ff01d600640 1 -- 172.21.15.27:0/2572051929 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff014021020 con 0x7ff020108db0 2024-04-26T22:55:55.615 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:55:55 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:55:55.135+0000 7f5b22a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:55:55.615 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:55:55 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:55:55.135+0000 7f5b22a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:55:55.615 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:55:55 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:55:55.135+0000 7f5b22a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:55:55.615 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:55:55 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:55:55.135+0000 7f5b22a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:55:55.615 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:55:55 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:55:55.135+0000 7f5b22a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:55:55.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.628+0000 7ff026200640 1 -- 172.21.15.27:0/2572051929 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff020063760 con 0x7ff0080a3c30 2024-04-26T22:55:55.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.663+0000 7ff01d600640 1 -- 172.21.15.27:0/2572051929 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff020063760 con 0x7ff0080a3c30 2024-04-26T22:55:55.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.665+0000 7ff026200640 1 -- 172.21.15.27:0/2572051929 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff0080a3c30 msgr2=0x7ff0080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:55.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.666+0000 7ff026200640 1 --2- 172.21.15.27:0/2572051929 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff0080a3c30 0x7ff0080a60f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff00c031500 tx=0x7ff00c005a20 comp rx=0 tx=0).stop 2024-04-26T22:55:55.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.666+0000 7ff026200640 1 -- 172.21.15.27:0/2572051929 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff020108db0 msgr2=0x7ff02019f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:55.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.666+0000 7ff026200640 1 --2- 172.21.15.27:0/2572051929 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff020108db0 0x7ff02019f040 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff01400d950 tx=0x7ff01400de10 comp rx=0 tx=0).stop 2024-04-26T22:55:55.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.666+0000 7ff026200640 1 -- 172.21.15.27:0/2572051929 shutdown_connections 2024-04-26T22:55:55.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.666+0000 7ff026200640 1 --2- 172.21.15.27:0/2572051929 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff0080a3c30 0x7ff0080a60f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.666+0000 7ff026200640 1 --2- 172.21.15.27:0/2572051929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff02010a960 0x7ff0201a6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.666+0000 7ff026200640 1 --2- 172.21.15.27:0/2572051929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff020109fa0 0x7ff02019f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.666+0000 7ff026200640 1 --2- 172.21.15.27:0/2572051929 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff020108db0 0x7ff02019f040 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.666+0000 7ff026200640 1 -- 172.21.15.27:0/2572051929 >> 172.21.15.27:0/2572051929 conn(0x7ff020077d70 msgr2=0x7ff02010d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:55.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.666+0000 7ff026200640 1 -- 172.21.15.27:0/2572051929 shutdown_connections 2024-04-26T22:55:55.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.666+0000 7ff026200640 1 -- 172.21.15.27:0/2572051929 wait complete. 2024-04-26T22:55:55.687 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-26T22:55:55.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.900+0000 7fbfae800640 1 -- 172.21.15.27:0/3593477838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfa8105b70 msgr2=0x7fbfa8107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:55.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.900+0000 7fbfae800640 1 --2- 172.21.15.27:0/3593477838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfa8105b70 0x7fbfa8107f60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009860 tx=0x7fbf9402a340 comp rx=0 tx=0).stop 2024-04-26T22:55:55.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.901+0000 7fbfae800640 1 -- 172.21.15.27:0/3593477838 shutdown_connections 2024-04-26T22:55:55.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.901+0000 7fbfae800640 1 --2- 172.21.15.27:0/3593477838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfa8105b70 0x7fbfa8107f60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.901+0000 7fbfae800640 1 --2- 172.21.15.27:0/3593477838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfa81084a0 0x7fbfa810a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.901+0000 7fbfae800640 1 --2- 172.21.15.27:0/3593477838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8103240 0x7fbfa8105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.901+0000 7fbfae800640 1 -- 172.21.15.27:0/3593477838 >> 172.21.15.27:0/3593477838 conn(0x7fbfa80fcc50 msgr2=0x7fbfa80ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:55.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.901+0000 7fbfae800640 1 -- 172.21.15.27:0/3593477838 shutdown_connections 2024-04-26T22:55:55.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.901+0000 7fbfae800640 1 -- 172.21.15.27:0/3593477838 wait complete. 2024-04-26T22:55:55.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.902+0000 7fbfae800640 1 Processor -- start 2024-04-26T22:55:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.903+0000 7fbfae800640 1 -- start start 2024-04-26T22:55:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.903+0000 7fbfae800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfa8103240 0x7fbfa819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.903+0000 7fbfae800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8105b70 0x7fbfa819b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.903+0000 7fbfae800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfa81084a0 0x7fbfa81a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.903+0000 7fbfae800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa810e2f0 con 0x7fbfa8105b70 2024-04-26T22:55:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.903+0000 7fbfae800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa810e170 con 0x7fbfa81084a0 2024-04-26T22:55:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.903+0000 7fbfae800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa810e470 con 0x7fbfa8103240 2024-04-26T22:55:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.903+0000 7fbfad400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfa8103240 0x7fbfa819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.903+0000 7fbfad400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfa8103240 0x7fbfa819ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:49578/0 (socket says 172.21.15.27:49578) 2024-04-26T22:55:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.903+0000 7fbfad400640 1 -- 172.21.15.27:0/2204710214 learned_addr learned my addr 172.21.15.27:0/2204710214 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.903+0000 7fbfaca00640 1 --2- 172.21.15.27:0/2204710214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8105b70 0x7fbfa819b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.903+0000 7fbfade00640 1 --2- 172.21.15.27:0/2204710214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfa81084a0 0x7fbfa81a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:55.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.904+0000 7fbfad400640 1 -- 172.21.15.27:0/2204710214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfa81084a0 msgr2=0x7fbfa81a21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:55.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.904+0000 7fbfad400640 1 --2- 172.21.15.27:0/2204710214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfa81084a0 0x7fbfa81a21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.904+0000 7fbfad400640 1 -- 172.21.15.27:0/2204710214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8105b70 msgr2=0x7fbfa819b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:55.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.904+0000 7fbfad400640 1 --2- 172.21.15.27:0/2204710214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8105b70 0x7fbfa819b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:55.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.904+0000 7fbfad400640 1 -- 172.21.15.27:0/2204710214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa81a2870 con 0x7fbfa8103240 2024-04-26T22:55:55.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.904+0000 7fbfade00640 1 --2- 172.21.15.27:0/2204710214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfa81084a0 0x7fbfa81a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:55.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.904+0000 7fbfad400640 1 --2- 172.21.15.27:0/2204710214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfa8103240 0x7fbfa819ac60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00efb0 tx=0x7fbf9c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:55.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.904+0000 7fbfa6000640 1 -- 172.21.15.27:0/2204710214 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c01b070 con 0x7fbfa8103240 2024-04-26T22:55:55.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.905+0000 7fbfa6000640 1 -- 172.21.15.27:0/2204710214 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf9c00ba20 con 0x7fbfa8103240 2024-04-26T22:55:55.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.905+0000 7fbfa6000640 1 -- 172.21.15.27:0/2204710214 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c016b30 con 0x7fbfa8103240 2024-04-26T22:55:55.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.905+0000 7fbfae800640 1 -- 172.21.15.27:0/2204710214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa81a2b60 con 0x7fbfa8103240 2024-04-26T22:55:55.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.905+0000 7fbfae800640 1 -- 172.21.15.27:0/2204710214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa81a3100 con 0x7fbfa8103240 2024-04-26T22:55:55.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.906+0000 7fbfae800640 1 -- 172.21.15.27:0/2204710214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa8109ff0 con 0x7fbfa8103240 2024-04-26T22:55:55.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.908+0000 7fbfa6000640 1 -- 172.21.15.27:0/2204710214 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbf9c01e070 con 0x7fbfa8103240 2024-04-26T22:55:55.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.910+0000 7fbfa6000640 1 --2- 172.21.15.27:0/2204710214 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbf900a3c30 0x7fbf900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:55.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.910+0000 7fbfa6000640 1 -- 172.21.15.27:0/2204710214 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbf9c0d0930 con 0x7fbfa8103240 2024-04-26T22:55:55.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.910+0000 7fbfaca00640 1 --2- 172.21.15.27:0/2204710214 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbf900a3c30 0x7fbf900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:55.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.911+0000 7fbfaca00640 1 --2- 172.21.15.27:0/2204710214 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbf900a3c30 0x7fbf900a60f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009830 tx=0x7fbf94030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:55.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:55.912+0000 7fbfa6000640 1 -- 172.21.15.27:0/2204710214 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbf9c09ca20 con 0x7fbfa8103240 2024-04-26T22:55:56.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:55 smithi080 ceph-mon[32324]: pgmap v345: 129 pgs: 129 active+clean; 582 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-04-26T22:55:56.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:55 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T22:55:56.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:55 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] up:active 2024-04-26T22:55:56.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:55 smithi080 ceph-mon[32324]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 9 up:standby 2024-04-26T22:55:56.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:55 smithi080 ceph-mon[32324]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T22:55:56.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:55 smithi080 ceph-mon[32324]: from='client.24818 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:55:56.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:55 smithi094 ceph-mon[32889]: pgmap v345: 129 pgs: 129 active+clean; 582 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-04-26T22:55:56.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:55 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T22:55:56.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:55 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] up:active 2024-04-26T22:55:56.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:55 smithi094 ceph-mon[32889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 9 up:standby 2024-04-26T22:55:56.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:55 smithi094 ceph-mon[32889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T22:55:56.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:55 smithi094 ceph-mon[32889]: from='client.24818 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:55:56.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:55 smithi027 ceph-mon[28034]: pgmap v345: 129 pgs: 129 active+clean; 582 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-04-26T22:55:56.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:55 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T22:55:56.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:55 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] up:active 2024-04-26T22:55:56.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:55 smithi027 ceph-mon[28034]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 9 up:standby 2024-04-26T22:55:56.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:55 smithi027 ceph-mon[28034]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T22:55:56.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:55 smithi027 ceph-mon[28034]: from='client.24818 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:55:56.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.152+0000 7fbfae800640 1 -- 172.21.15.27:0/2204710214 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fbfa8062d70 con 0x7fbf900a3c30 2024-04-26T22:55:56.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.162+0000 7fbfa6000640 1 -- 172.21.15.27:0/2204710214 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fbfa8062d70 con 0x7fbf900a3c30 2024-04-26T22:55:56.163 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/qa/sv_0/7a457be9-6787-42e4-b79d-c42282aef5ee 2024-04-26T22:55:56.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.165+0000 7fbfae800640 1 -- 172.21.15.27:0/2204710214 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbf900a3c30 msgr2=0x7fbf900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:56.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.165+0000 7fbfae800640 1 --2- 172.21.15.27:0/2204710214 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbf900a3c30 0x7fbf900a60f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009830 tx=0x7fbf94030450 comp rx=0 tx=0).stop 2024-04-26T22:55:56.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.165+0000 7fbfae800640 1 -- 172.21.15.27:0/2204710214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfa8103240 msgr2=0x7fbfa819ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:56.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.165+0000 7fbfae800640 1 --2- 172.21.15.27:0/2204710214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfa8103240 0x7fbfa819ac60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00efb0 tx=0x7fbf9c00c800 comp rx=0 tx=0).stop 2024-04-26T22:55:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.165+0000 7fbfae800640 1 -- 172.21.15.27:0/2204710214 shutdown_connections 2024-04-26T22:55:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.165+0000 7fbfae800640 1 --2- 172.21.15.27:0/2204710214 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbf900a3c30 0x7fbf900a60f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.165+0000 7fbfae800640 1 --2- 172.21.15.27:0/2204710214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfa81084a0 0x7fbfa81a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.165+0000 7fbfae800640 1 --2- 172.21.15.27:0/2204710214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8105b70 0x7fbfa819b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.165+0000 7fbfae800640 1 --2- 172.21.15.27:0/2204710214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfa8103240 0x7fbfa819ac60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.165+0000 7fbfae800640 1 -- 172.21.15.27:0/2204710214 >> 172.21.15.27:0/2204710214 conn(0x7fbfa80fcc50 msgr2=0x7fbfa81029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:56.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.166+0000 7fbfae800640 1 -- 172.21.15.27:0/2204710214 shutdown_connections 2024-04-26T22:55:56.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.166+0000 7fbfae800640 1 -- 172.21.15.27:0/2204710214 wait complete. 2024-04-26T22:55:56.187 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-26T22:55:56.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.406+0000 7fab0fc00640 1 -- 172.21.15.27:0/1109958017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab08103400 msgr2=0x7fab081057f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:56.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.406+0000 7fab0fc00640 1 --2- 172.21.15.27:0/1109958017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab08103400 0x7fab081057f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009ca0 tx=0x7faaf802a8c0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.406+0000 7fab0fc00640 1 -- 172.21.15.27:0/1109958017 shutdown_connections 2024-04-26T22:55:56.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.406+0000 7fab0fc00640 1 --2- 172.21.15.27:0/1109958017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab08108660 0x7fab0810aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.406+0000 7fab0fc00640 1 --2- 172.21.15.27:0/1109958017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab08105d30 0x7fab08108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.406+0000 7fab0fc00640 1 --2- 172.21.15.27:0/1109958017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab08103400 0x7fab081057f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.406+0000 7fab0fc00640 1 -- 172.21.15.27:0/1109958017 >> 172.21.15.27:0/1109958017 conn(0x7fab080fce10 msgr2=0x7fab080ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:56.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.406+0000 7fab0fc00640 1 -- 172.21.15.27:0/1109958017 shutdown_connections 2024-04-26T22:55:56.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.406+0000 7fab0fc00640 1 -- 172.21.15.27:0/1109958017 wait complete. 2024-04-26T22:55:56.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.407+0000 7fab0fc00640 1 Processor -- start 2024-04-26T22:55:56.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.407+0000 7fab0fc00640 1 -- start start 2024-04-26T22:55:56.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.408+0000 7fab0fc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab08103400 0x7fab08198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:56.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.408+0000 7fab0fc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab08105d30 0x7fab08199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:56.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.408+0000 7fab0fc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab08108660 0x7fab081a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:56.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.408+0000 7fab0fc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab0810e500 con 0x7fab08108660 2024-04-26T22:55:56.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.408+0000 7fab0fc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fab0810e380 con 0x7fab08105d30 2024-04-26T22:55:56.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.408+0000 7fab0fc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab0810e680 con 0x7fab08103400 2024-04-26T22:55:56.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.408+0000 7faaffe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab08105d30 0x7fab08199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:56.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.408+0000 7fab0d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab08103400 0x7fab08198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:56.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.408+0000 7faaffe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab08105d30 0x7fab08199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:35490/0 (socket says 172.21.15.27:35490) 2024-04-26T22:55:56.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.408+0000 7faaffe00640 1 -- 172.21.15.27:0/2797397782 learned_addr learned my addr 172.21.15.27:0/2797397782 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:56.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.408+0000 7fab0d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab08103400 0x7fab08198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:49592/0 (socket says 172.21.15.27:49592) 2024-04-26T22:55:56.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.408+0000 7fab0dc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab08108660 0x7fab081a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:56.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.409+0000 7faaffe00640 1 -- 172.21.15.27:0/2797397782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab08103400 msgr2=0x7fab08198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:56.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.409+0000 7faaffe00640 1 --2- 172.21.15.27:0/2797397782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab08103400 0x7fab08198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.409+0000 7faaffe00640 1 -- 172.21.15.27:0/2797397782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab08108660 msgr2=0x7fab081a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:56.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.409+0000 7faaffe00640 1 --2- 172.21.15.27:0/2797397782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab08108660 0x7fab081a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.409+0000 7faaffe00640 1 -- 172.21.15.27:0/2797397782 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab081a0830 con 0x7fab08105d30 2024-04-26T22:55:56.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.409+0000 7faaffe00640 1 --2- 172.21.15.27:0/2797397782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab08105d30 0x7fab08199160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faaf000d770 tx=0x7faaf000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:56.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.409+0000 7fab06000640 1 -- 172.21.15.27:0/2797397782 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0017070 con 0x7fab08105d30 2024-04-26T22:55:56.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.409+0000 7fab06000640 1 -- 172.21.15.27:0/2797397782 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaf0009c30 con 0x7fab08105d30 2024-04-26T22:55:56.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.409+0000 7fab0fc00640 1 -- 172.21.15.27:0/2797397782 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab081a0b20 con 0x7fab08105d30 2024-04-26T22:55:56.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.409+0000 7fab06000640 1 -- 172.21.15.27:0/2797397782 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf000f620 con 0x7fab08105d30 2024-04-26T22:55:56.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.410+0000 7fab0fc00640 1 -- 172.21.15.27:0/2797397782 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab081a1040 con 0x7fab08105d30 2024-04-26T22:55:56.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.411+0000 7fab0fc00640 1 -- 172.21.15.27:0/2797397782 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab080fe900 con 0x7fab08105d30 2024-04-26T22:55:56.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.413+0000 7fab06000640 1 -- 172.21.15.27:0/2797397782 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faaf001a070 con 0x7fab08105d30 2024-04-26T22:55:56.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.414+0000 7fab06000640 1 --2- 172.21.15.27:0/2797397782 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faaec0a3bb0 0x7faaec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:56.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.414+0000 7fab06000640 1 -- 172.21.15.27:0/2797397782 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7faaf00d01e0 con 0x7fab08105d30 2024-04-26T22:55:56.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.415+0000 7fab0d200640 1 --2- 172.21.15.27:0/2797397782 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faaec0a3bb0 0x7faaec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.416+0000 7fab0d200640 1 --2- 172.21.15.27:0/2797397782 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faaec0a3bb0 0x7faaec0a6070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faaf8005e30 tx=0x7faaf8005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:56.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.419+0000 7fab06000640 1 -- 172.21.15.27:0/2797397782 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faaf0009de0 con 0x7fab08105d30 2024-04-26T22:55:56.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.668+0000 7fab0fc00640 1 -- 172.21.15.27:0/2797397782 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fab08062e00 con 0x7faaec0a3bb0 2024-04-26T22:55:56.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.672+0000 7fab06000640 1 -- 172.21.15.27:0/2797397782 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faaf00988b0 con 0x7fab08105d30 2024-04-26T22:55:56.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.718+0000 7fab06000640 1 -- 172.21.15.27:0/2797397782 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fab08062e00 con 0x7faaec0a3bb0 2024-04-26T22:55:56.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.721+0000 7fab0fc00640 1 -- 172.21.15.27:0/2797397782 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faaec0a3bb0 msgr2=0x7faaec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:56.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.721+0000 7fab0fc00640 1 --2- 172.21.15.27:0/2797397782 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faaec0a3bb0 0x7faaec0a6070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faaf8005e30 tx=0x7faaf8005f70 comp rx=0 tx=0).stop 2024-04-26T22:55:56.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.721+0000 7fab0fc00640 1 -- 172.21.15.27:0/2797397782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab08105d30 msgr2=0x7fab08199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:56.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.721+0000 7fab0fc00640 1 --2- 172.21.15.27:0/2797397782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab08105d30 0x7fab08199160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faaf000d770 tx=0x7faaf000dc30 comp rx=0 tx=0).stop 2024-04-26T22:55:56.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.721+0000 7fab0fc00640 1 -- 172.21.15.27:0/2797397782 shutdown_connections 2024-04-26T22:55:56.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.721+0000 7fab0fc00640 1 --2- 172.21.15.27:0/2797397782 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faaec0a3bb0 0x7faaec0a6070 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.721+0000 7fab0fc00640 1 --2- 172.21.15.27:0/2797397782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab08105d30 0x7fab08199160 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.721+0000 7fab0fc00640 1 --2- 172.21.15.27:0/2797397782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab08108660 0x7fab081a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.721+0000 7fab0fc00640 1 --2- 172.21.15.27:0/2797397782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab08103400 0x7fab08198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.721+0000 7fab0fc00640 1 -- 172.21.15.27:0/2797397782 >> 172.21.15.27:0/2797397782 conn(0x7fab080fce10 msgr2=0x7fab08103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:56.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.722+0000 7fab0fc00640 1 -- 172.21.15.27:0/2797397782 shutdown_connections 2024-04-26T22:55:56.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.722+0000 7fab0fc00640 1 -- 172.21.15.27:0/2797397782 wait complete. 2024-04-26T22:55:56.743 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-26T22:55:56.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.958+0000 7f8257200640 1 -- 172.21.15.27:0/224839320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8258101010 msgr2=0x7f8258101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:56.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.958+0000 7f8257200640 1 --2- 172.21.15.27:0/224839320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8258101010 0x7f8258101430 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8248009900 tx=0x7f824802a3f0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.958+0000 7f8257200640 1 -- 172.21.15.27:0/224839320 shutdown_connections 2024-04-26T22:55:56.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.958+0000 7f8257200640 1 --2- 172.21.15.27:0/224839320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8258102310 0x7f8258102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.958+0000 7f8257200640 1 --2- 172.21.15.27:0/224839320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258101970 0x7f8258101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.958+0000 7f8257200640 1 --2- 172.21.15.27:0/224839320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8258101010 0x7f8258101430 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.958+0000 7f8257200640 1 -- 172.21.15.27:0/224839320 >> 172.21.15.27:0/224839320 conn(0x7f82580fcc10 msgr2=0x7f82580ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:56.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.958+0000 7f8257200640 1 -- 172.21.15.27:0/224839320 shutdown_connections 2024-04-26T22:55:56.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.958+0000 7f8257200640 1 -- 172.21.15.27:0/224839320 wait complete. 2024-04-26T22:55:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.960+0000 7f8257200640 1 Processor -- start 2024-04-26T22:55:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.960+0000 7f8257200640 1 -- start start 2024-04-26T22:55:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.960+0000 7f8257200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258101010 0x7f825819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.960+0000 7f8257200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8258101970 0x7f825819b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:56.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.960+0000 7f8257200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8258102310 0x7f82581a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:56.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.960+0000 7f8257200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f82581104e0 con 0x7f8258102310 2024-04-26T22:55:56.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.960+0000 7f8257200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8258110360 con 0x7f8258101970 2024-04-26T22:55:56.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.960+0000 7f8257200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8258110660 con 0x7f8258101010 2024-04-26T22:55:56.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.960+0000 7f8256800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8258102310 0x7f82581a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:56.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.960+0000 7f8255400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8258101970 0x7f825819b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:56.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.960+0000 7f8256800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8258102310 0x7f82581a2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50644/0 (socket says 172.21.15.27:50644) 2024-04-26T22:55:56.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.960+0000 7f8255e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258101010 0x7f825819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:56.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.960+0000 7f8256800640 1 -- 172.21.15.27:0/2833607977 learned_addr learned my addr 172.21.15.27:0/2833607977 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:56.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.961+0000 7f8255e00640 1 -- 172.21.15.27:0/2833607977 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8258101970 msgr2=0x7f825819b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:56.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.961+0000 7f8255e00640 1 --2- 172.21.15.27:0/2833607977 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8258101970 0x7f825819b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.961+0000 7f8255e00640 1 -- 172.21.15.27:0/2833607977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8258102310 msgr2=0x7f82581a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:56.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.961+0000 7f8255e00640 1 --2- 172.21.15.27:0/2833607977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8258102310 0x7f82581a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:56.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.961+0000 7f8255e00640 1 -- 172.21.15.27:0/2833607977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82581a27f0 con 0x7f8258101010 2024-04-26T22:55:56.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.961+0000 7f8256800640 1 --2- 172.21.15.27:0/2833607977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8258102310 0x7f82581a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:55:56.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.961+0000 7f8255e00640 1 --2- 172.21.15.27:0/2833607977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258101010 0x7f825819abd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f824802a9e0 tx=0x7f8248003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:56.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.962+0000 7f824ea00640 1 -- 172.21.15.27:0/2833607977 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824803f070 con 0x7f8258101010 2024-04-26T22:55:56.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.962+0000 7f8257200640 1 -- 172.21.15.27:0/2833607977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82581a2a80 con 0x7f8258101010 2024-04-26T22:55:56.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.962+0000 7f824ea00640 1 -- 172.21.15.27:0/2833607977 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8248031c50 con 0x7f8258101010 2024-04-26T22:55:56.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.962+0000 7f824ea00640 1 -- 172.21.15.27:0/2833607977 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8248035030 con 0x7f8258101010 2024-04-26T22:55:56.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.962+0000 7f8257200640 1 -- 172.21.15.27:0/2833607977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82581a2f90 con 0x7f8258101010 2024-04-26T22:55:56.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.963+0000 7f8257200640 1 -- 172.21.15.27:0/2833607977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825810bd60 con 0x7f8258101010 2024-04-26T22:55:56.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.965+0000 7f824ea00640 1 -- 172.21.15.27:0/2833607977 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8248042070 con 0x7f8258101010 2024-04-26T22:55:56.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.966+0000 7f824ea00640 1 --2- 172.21.15.27:0/2833607977 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f823c0a3c40 0x7f823c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:56.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.967+0000 7f824ea00640 1 -- 172.21.15.27:0/2833607977 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f82480f0ea0 con 0x7f8258101010 2024-04-26T22:55:56.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.967+0000 7f8255400640 1 --2- 172.21.15.27:0/2833607977 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f823c0a3c40 0x7f823c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:56.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.967+0000 7f8255400640 1 --2- 172.21.15.27:0/2833607977 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f823c0a3c40 0x7f823c0a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f82440095f0 tx=0x7f8244005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:56.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:56.971+0000 7f824ea00640 1 -- 172.21.15.27:0/2833607977 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f82480bce30 con 0x7f8258101010 2024-04-26T22:55:57.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:56 smithi080 ceph-mon[32324]: from='client.24830 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:55:57.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:56 smithi080 ceph-mon[32324]: from='client.24862 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:55:57.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:56 smithi094 ceph-mon[32889]: from='client.24830 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:55:57.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:56 smithi094 ceph-mon[32889]: from='client.24862 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:55:57.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:56 smithi027 ceph-mon[28034]: from='client.24830 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:55:57.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:56 smithi027 ceph-mon[28034]: from='client.24862 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:55:57.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.214+0000 7f8257200640 1 -- 172.21.15.27:0/2833607977 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f8258062e00 con 0x7f823c0a3c40 2024-04-26T22:55:57.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.222+0000 7f824ea00640 1 -- 172.21.15.27:0/2833607977 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8258062e00 con 0x7f823c0a3c40 2024-04-26T22:55:57.223 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/qa/sv_1/432cadfa-e601-4549-861d-e29e1f305178 2024-04-26T22:55:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.224+0000 7f8257200640 1 -- 172.21.15.27:0/2833607977 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f823c0a3c40 msgr2=0x7f823c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.224+0000 7f8257200640 1 --2- 172.21.15.27:0/2833607977 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f823c0a3c40 0x7f823c0a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f82440095f0 tx=0x7f8244005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.224+0000 7f8257200640 1 -- 172.21.15.27:0/2833607977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258101010 msgr2=0x7f825819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.224+0000 7f8257200640 1 --2- 172.21.15.27:0/2833607977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258101010 0x7f825819abd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f824802a9e0 tx=0x7f8248003c10 comp rx=0 tx=0).stop 2024-04-26T22:55:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.225+0000 7f8257200640 1 -- 172.21.15.27:0/2833607977 shutdown_connections 2024-04-26T22:55:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.225+0000 7f8257200640 1 --2- 172.21.15.27:0/2833607977 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f823c0a3c40 0x7f823c0a6100 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.225+0000 7f8257200640 1 --2- 172.21.15.27:0/2833607977 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8258101970 0x7f825819b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.225+0000 7f8257200640 1 --2- 172.21.15.27:0/2833607977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8258102310 0x7f82581a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.225+0000 7f8257200640 1 --2- 172.21.15.27:0/2833607977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258101010 0x7f825819abd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.225+0000 7f8257200640 1 -- 172.21.15.27:0/2833607977 >> 172.21.15.27:0/2833607977 conn(0x7f82580fcc10 msgr2=0x7f82580fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.225+0000 7f8257200640 1 -- 172.21.15.27:0/2833607977 shutdown_connections 2024-04-26T22:55:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.225+0000 7f8257200640 1 -- 172.21.15.27:0/2833607977 wait complete. 2024-04-26T22:55:57.245 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:55:57.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.471+0000 7f2de2c00640 1 -- 172.21.15.27:0/3364123194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ddc1011d0 msgr2=0x7f2ddc103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:57.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.471+0000 7f2de2c00640 1 --2- 172.21.15.27:0/3364123194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ddc1011d0 0x7f2ddc103600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009900 tx=0x7f2dcc02a410 comp rx=0 tx=0).stop 2024-04-26T22:55:57.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.471+0000 7f2de2c00640 1 -- 172.21.15.27:0/3364123194 shutdown_connections 2024-04-26T22:55:57.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.471+0000 7f2de2c00640 1 --2- 172.21.15.27:0/3364123194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ddc1064f0 0x7f2ddc106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.471+0000 7f2de2c00640 1 --2- 172.21.15.27:0/3364123194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ddc103b40 0x7f2ddc105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.471+0000 7f2de2c00640 1 --2- 172.21.15.27:0/3364123194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ddc1011d0 0x7f2ddc103600 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.471+0000 7f2de2c00640 1 -- 172.21.15.27:0/3364123194 >> 172.21.15.27:0/3364123194 conn(0x7f2ddc0fcdd0 msgr2=0x7f2ddc0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:57.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.471+0000 7f2de2c00640 1 -- 172.21.15.27:0/3364123194 shutdown_connections 2024-04-26T22:55:57.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.471+0000 7f2de2c00640 1 -- 172.21.15.27:0/3364123194 wait complete. 2024-04-26T22:55:57.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.473+0000 7f2de2c00640 1 Processor -- start 2024-04-26T22:55:57.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.473+0000 7f2de2c00640 1 -- start start 2024-04-26T22:55:57.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.473+0000 7f2de2c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ddc1011d0 0x7f2ddc19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:57.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.473+0000 7f2de2c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ddc103b40 0x7f2ddc19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:57.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.473+0000 7f2de2c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ddc1064f0 0x7f2ddc1a6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:57.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.473+0000 7f2de2c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc114ab0 con 0x7f2ddc1064f0 2024-04-26T22:55:57.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.473+0000 7f2de2c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc114930 con 0x7f2ddc1011d0 2024-04-26T22:55:57.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.473+0000 7f2de2c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc114c30 con 0x7f2ddc103b40 2024-04-26T22:55:57.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.474+0000 7f2de0c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ddc1064f0 0x7f2ddc1a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:57.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.474+0000 7f2ddbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ddc1011d0 0x7f2ddc19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:57.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.474+0000 7f2ddb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ddc103b40 0x7f2ddc19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:57.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.474+0000 7f2ddbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ddc1011d0 0x7f2ddc19f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:35510/0 (socket says 172.21.15.27:35510) 2024-04-26T22:55:57.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.474+0000 7f2ddb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ddc103b40 0x7f2ddc19f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:49618/0 (socket says 172.21.15.27:49618) 2024-04-26T22:55:57.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.474+0000 7f2ddbe00640 1 -- 172.21.15.27:0/1682083791 learned_addr learned my addr 172.21.15.27:0/1682083791 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:57.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.474+0000 7f2ddb400640 1 -- 172.21.15.27:0/1682083791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ddc1011d0 msgr2=0x7f2ddc19f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:57.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.474+0000 7f2ddb400640 1 --2- 172.21.15.27:0/1682083791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ddc1011d0 0x7f2ddc19f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.474+0000 7f2ddb400640 1 -- 172.21.15.27:0/1682083791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ddc1064f0 msgr2=0x7f2ddc1a6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:57.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.474+0000 7f2ddb400640 1 --2- 172.21.15.27:0/1682083791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ddc1064f0 0x7f2ddc1a6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.474+0000 7f2ddb400640 1 -- 172.21.15.27:0/1682083791 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ddc1a6d70 con 0x7f2ddc103b40 2024-04-26T22:55:57.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.474+0000 7f2ddb400640 1 --2- 172.21.15.27:0/1682083791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ddc103b40 0x7f2ddc19f690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400ec80 tx=0x7f2dc400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:57.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.475+0000 7f2dd8c00640 1 -- 172.21.15.27:0/1682083791 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4003b70 con 0x7f2ddc103b40 2024-04-26T22:55:57.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.475+0000 7f2de2c00640 1 -- 172.21.15.27:0/1682083791 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ddc1a7060 con 0x7f2ddc103b40 2024-04-26T22:55:57.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.475+0000 7f2de2c00640 1 -- 172.21.15.27:0/1682083791 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ddc1a7600 con 0x7f2ddc103b40 2024-04-26T22:55:57.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.475+0000 7f2dd8c00640 1 -- 172.21.15.27:0/1682083791 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dc4003d10 con 0x7f2ddc103b40 2024-04-26T22:55:57.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.475+0000 7f2dd8c00640 1 -- 172.21.15.27:0/1682083791 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc40108b0 con 0x7f2ddc103b40 2024-04-26T22:55:57.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.476+0000 7f2de2c00640 1 -- 172.21.15.27:0/1682083791 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ddc0fe8c0 con 0x7f2ddc103b40 2024-04-26T22:55:57.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.478+0000 7f2dd8c00640 1 -- 172.21.15.27:0/1682083791 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2dc4004560 con 0x7f2ddc103b40 2024-04-26T22:55:57.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.480+0000 7f2dd8c00640 1 --2- 172.21.15.27:0/1682083791 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2dbc0a3c40 0x7f2dbc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:57.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.480+0000 7f2dd8c00640 1 -- 172.21.15.27:0/1682083791 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2dc40d0ba0 con 0x7f2ddc103b40 2024-04-26T22:55:57.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.480+0000 7f2ddbe00640 1 --2- 172.21.15.27:0/1682083791 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2dbc0a3c40 0x7f2dbc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:57.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.481+0000 7f2ddbe00640 1 --2- 172.21.15.27:0/1682083791 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2dbc0a3c40 0x7f2dbc0a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009230 tx=0x7f2dcc0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:57.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.484+0000 7f2dd8c00640 1 -- 172.21.15.27:0/1682083791 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2dc409d180 con 0x7f2ddc103b40 2024-04-26T22:55:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.739+0000 7f2de2c00640 1 -- 172.21.15.27:0/1682083791 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ddc02dcd0 con 0x7f2ddc103b40 2024-04-26T22:55:57.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.740+0000 7f2dd8c00640 1 -- 172.21.15.27:0/1682083791 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11850 (secure 0 0 0) 0x7f2dc409ca80 con 0x7f2ddc103b40 2024-04-26T22:55:57.741 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:57.741 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T22:55:25.265331+0000","modified":"2024-04-26T22:55:54.648644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24578,"mds_1":24577,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/1620677319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1620677319},{"type":"v1","addr":"172.21.15.94:6839","nonce":1620677319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.27:6841/1571355196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1571355196},{"type":"v1","addr":"172.21.15.27:6841","nonce":1571355196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24577":{"gid":24577,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.27:6839/3193530521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3193530521},{"type":"v1","addr":"172.21.15.27:6839","nonce":3193530521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/95952000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":95952000},{"type":"v1","addr":"172.21.15.80:6839","nonce":95952000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24578,"qdb_cluster":[24560,24577,24578]},"id":1}]} 2024-04-26T22:55:57.741 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-04-26T22:55:57.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.744+0000 7f2de2c00640 1 -- 172.21.15.27:0/1682083791 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2dbc0a3c40 msgr2=0x7f2dbc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:57.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.744+0000 7f2de2c00640 1 --2- 172.21.15.27:0/1682083791 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2dbc0a3c40 0x7f2dbc0a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009230 tx=0x7f2dcc0058d0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.744+0000 7f2de2c00640 1 -- 172.21.15.27:0/1682083791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ddc103b40 msgr2=0x7f2ddc19f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.744+0000 7f2de2c00640 1 --2- 172.21.15.27:0/1682083791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ddc103b40 0x7f2ddc19f690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400ec80 tx=0x7f2dc400c6a0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.744+0000 7f2de2c00640 1 -- 172.21.15.27:0/1682083791 shutdown_connections 2024-04-26T22:55:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.744+0000 7f2de2c00640 1 --2- 172.21.15.27:0/1682083791 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2dbc0a3c40 0x7f2dbc0a6100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.744+0000 7f2de2c00640 1 --2- 172.21.15.27:0/1682083791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ddc1064f0 0x7f2ddc1a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.744+0000 7f2de2c00640 1 --2- 172.21.15.27:0/1682083791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ddc103b40 0x7f2ddc19f690 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.744+0000 7f2de2c00640 1 --2- 172.21.15.27:0/1682083791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ddc1011d0 0x7f2ddc19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.744+0000 7f2de2c00640 1 -- 172.21.15.27:0/1682083791 >> 172.21.15.27:0/1682083791 conn(0x7f2ddc0fcdd0 msgr2=0x7f2ddc104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.744+0000 7f2de2c00640 1 -- 172.21.15.27:0/1682083791 shutdown_connections 2024-04-26T22:55:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.744+0000 7f2de2c00640 1 -- 172.21.15.27:0/1682083791 wait complete. 2024-04-26T22:55:57.764 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:55:57.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.981+0000 7f0e23200640 1 -- 172.21.15.27:0/1242078960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e24105b20 msgr2=0x7f0e24107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:57.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.981+0000 7f0e23200640 1 --2- 172.21.15.27:0/1242078960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e24105b20 0x7f0e24107f10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009900 tx=0x7f0e0c02a400 comp rx=0 tx=0).stop 2024-04-26T22:55:57.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.981+0000 7f0e23200640 1 -- 172.21.15.27:0/1242078960 shutdown_connections 2024-04-26T22:55:57.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.981+0000 7f0e23200640 1 --2- 172.21.15.27:0/1242078960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e24108450 0x7f0e2410a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.981+0000 7f0e23200640 1 --2- 172.21.15.27:0/1242078960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e24105b20 0x7f0e24107f10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.981+0000 7f0e23200640 1 --2- 172.21.15.27:0/1242078960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e241031f0 0x7f0e241055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.981+0000 7f0e23200640 1 -- 172.21.15.27:0/1242078960 >> 172.21.15.27:0/1242078960 conn(0x7f0e240fcbe0 msgr2=0x7f0e240ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:57.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.981+0000 7f0e23200640 1 -- 172.21.15.27:0/1242078960 shutdown_connections 2024-04-26T22:55:57.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.981+0000 7f0e23200640 1 -- 172.21.15.27:0/1242078960 wait complete. 2024-04-26T22:55:57.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.982+0000 7f0e23200640 1 Processor -- start 2024-04-26T22:55:57.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.982+0000 7f0e23200640 1 -- start start 2024-04-26T22:55:57.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.983+0000 7f0e23200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e241031f0 0x7f0e2419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:57.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.983+0000 7f0e23200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e24105b20 0x7f0e2419b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:57.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.983+0000 7f0e23200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e24108450 0x7f0e241a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:57.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.983+0000 7f0e23200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2410e300 con 0x7f0e241031f0 2024-04-26T22:55:57.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.983+0000 7f0e23200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2410e180 con 0x7f0e24108450 2024-04-26T22:55:57.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.983+0000 7f0e23200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2410e480 con 0x7f0e24105b20 2024-04-26T22:55:57.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.983+0000 7f0e21e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e241031f0 0x7f0e2419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:57.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.983+0000 7f0e21e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e241031f0 0x7f0e2419aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33894/0 (socket says 172.21.15.27:33894) 2024-04-26T22:55:57.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.983+0000 7f0e21e00640 1 -- 172.21.15.27:0/395131158 learned_addr learned my addr 172.21.15.27:0/395131158 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:57.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.983+0000 7f0e19400640 1 --2- 172.21.15.27:0/395131158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e24105b20 0x7f0e2419b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:57.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.983+0000 7f0e22800640 1 --2- 172.21.15.27:0/395131158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e24108450 0x7f0e241a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:57.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.984+0000 7f0e21e00640 1 -- 172.21.15.27:0/395131158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e24105b20 msgr2=0x7f0e2419b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:57.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.984+0000 7f0e21e00640 1 --2- 172.21.15.27:0/395131158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e24105b20 0x7f0e2419b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.984+0000 7f0e21e00640 1 -- 172.21.15.27:0/395131158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e24108450 msgr2=0x7f0e241a20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:57.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.984+0000 7f0e21e00640 1 --2- 172.21.15.27:0/395131158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e24108450 0x7f0e241a20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:57.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.984+0000 7f0e21e00640 1 -- 172.21.15.27:0/395131158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e241a2710 con 0x7f0e241031f0 2024-04-26T22:55:57.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.984+0000 7f0e21e00640 1 --2- 172.21.15.27:0/395131158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e241031f0 0x7f0e2419aaf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400e9f0 tx=0x7f0e1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:57.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.984+0000 7f0e1b400640 1 -- 172.21.15.27:0/395131158 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400cd80 con 0x7f0e241031f0 2024-04-26T22:55:57.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.984+0000 7f0e1b400640 1 -- 172.21.15.27:0/395131158 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e140054d0 con 0x7f0e241031f0 2024-04-26T22:55:57.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.984+0000 7f0e1b400640 1 -- 172.21.15.27:0/395131158 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14010690 con 0x7f0e241031f0 2024-04-26T22:55:57.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.984+0000 7f0e23200640 1 -- 172.21.15.27:0/395131158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e241a2a00 con 0x7f0e241031f0 2024-04-26T22:55:57.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.984+0000 7f0e23200640 1 -- 172.21.15.27:0/395131158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e241a2ed0 con 0x7f0e241031f0 2024-04-26T22:55:57.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.985+0000 7f0e23200640 1 -- 172.21.15.27:0/395131158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e24109fa0 con 0x7f0e241031f0 2024-04-26T22:55:57.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.988+0000 7f0e1b400640 1 -- 172.21.15.27:0/395131158 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0e14004e90 con 0x7f0e241031f0 2024-04-26T22:55:57.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.989+0000 7f0e1b400640 1 --2- 172.21.15.27:0/395131158 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0e080a3cf0 0x7f0e080a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:57.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.989+0000 7f0e1b400640 1 -- 172.21.15.27:0/395131158 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0e140d0c10 con 0x7f0e241031f0 2024-04-26T22:55:57.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.989+0000 7f0e19400640 1 --2- 172.21.15.27:0/395131158 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0e080a3cf0 0x7f0e080a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:57.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.990+0000 7f0e19400640 1 --2- 172.21.15.27:0/395131158 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0e080a3cf0 0x7f0e080a61b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c005ab0 tx=0x7f0e0c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:57.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:57.994+0000 7f0e1b400640 1 -- 172.21.15.27:0/395131158 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0e1409d1f0 con 0x7f0e241031f0 2024-04-26T22:55:58.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:57 smithi080 ceph-mon[32324]: pgmap v346: 129 pgs: 129 active+clean; 602 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-04-26T22:55:58.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:57 smithi080 ceph-mon[32324]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-26T22:55:58.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:57 smithi080 ceph-mon[32324]: from='client.24842 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:55:58.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:57 smithi080 ceph-mon[32324]: from='client.24874 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:55:58.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:57 smithi094 ceph-mon[32889]: pgmap v346: 129 pgs: 129 active+clean; 602 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-04-26T22:55:58.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:57 smithi094 ceph-mon[32889]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-26T22:55:58.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:57 smithi094 ceph-mon[32889]: from='client.24842 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:55:58.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:57 smithi094 ceph-mon[32889]: from='client.24874 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:55:58.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:57 smithi027 ceph-mon[28034]: pgmap v346: 129 pgs: 129 active+clean; 602 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-04-26T22:55:58.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:57 smithi027 ceph-mon[28034]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-26T22:55:58.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:57 smithi027 ceph-mon[28034]: from='client.24842 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:55:58.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:57 smithi027 ceph-mon[28034]: from='client.24874 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:55:58.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.209+0000 7f0e23200640 1 -- 172.21.15.27:0/395131158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e2402dd10 con 0x7f0e241031f0 2024-04-26T22:55:58.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.213+0000 7f0e1b400640 1 -- 172.21.15.27:0/395131158 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24009 (secure 0 0 0) 0x7f0e1409caf0 con 0x7f0e241031f0 2024-04-26T22:55:58.214 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:58.214 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:55:30.886974+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:55:58.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.216+0000 7f0e23200640 1 -- 172.21.15.27:0/395131158 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0e080a3cf0 msgr2=0x7f0e080a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:58.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.216+0000 7f0e23200640 1 --2- 172.21.15.27:0/395131158 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0e080a3cf0 0x7f0e080a61b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c005ab0 tx=0x7f0e0c005a20 comp rx=0 tx=0).stop 2024-04-26T22:55:58.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.216+0000 7f0e23200640 1 -- 172.21.15.27:0/395131158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e241031f0 msgr2=0x7f0e2419aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.216+0000 7f0e23200640 1 --2- 172.21.15.27:0/395131158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e241031f0 0x7f0e2419aaf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400e9f0 tx=0x7f0e1400edf0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.216+0000 7f0e23200640 1 -- 172.21.15.27:0/395131158 shutdown_connections 2024-04-26T22:55:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.216+0000 7f0e23200640 1 --2- 172.21.15.27:0/395131158 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0e080a3cf0 0x7f0e080a61b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.216+0000 7f0e23200640 1 --2- 172.21.15.27:0/395131158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e24108450 0x7f0e241a20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.216+0000 7f0e23200640 1 --2- 172.21.15.27:0/395131158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e24105b20 0x7f0e2419b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.216+0000 7f0e23200640 1 --2- 172.21.15.27:0/395131158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e241031f0 0x7f0e2419aaf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.216+0000 7f0e23200640 1 -- 172.21.15.27:0/395131158 >> 172.21.15.27:0/395131158 conn(0x7f0e240fcbe0 msgr2=0x7f0e24102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.216+0000 7f0e23200640 1 -- 172.21.15.27:0/395131158 shutdown_connections 2024-04-26T22:55:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.216+0000 7f0e23200640 1 -- 172.21.15.27:0/395131158 wait complete. 2024-04-26T22:55:58.237 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T22:55:58.238 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:55:58.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.456+0000 7f5ce7a00640 1 -- 172.21.15.27:0/2315189992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce00768f0 msgr2=0x7f5ce0074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:58.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.456+0000 7f5ce7a00640 1 --2- 172.21.15.27:0/2315189992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce00768f0 0x7f5ce0074d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009900 tx=0x7f5cc802a480 comp rx=0 tx=0).stop 2024-04-26T22:55:58.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.456+0000 7f5ce7a00640 1 -- 172.21.15.27:0/2315189992 shutdown_connections 2024-04-26T22:55:58.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.456+0000 7f5ce7a00640 1 --2- 172.21.15.27:0/2315189992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ce010a840 0x7f5ce010ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.456+0000 7f5ce7a00640 1 --2- 172.21.15.27:0/2315189992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce0075310 0x7f5ce0075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.456+0000 7f5ce7a00640 1 --2- 172.21.15.27:0/2315189992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce00768f0 0x7f5ce0074d40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.456+0000 7f5ce7a00640 1 -- 172.21.15.27:0/2315189992 >> 172.21.15.27:0/2315189992 conn(0x7f5ce00fdf00 msgr2=0x7f5ce0100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:58.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.456+0000 7f5ce7a00640 1 -- 172.21.15.27:0/2315189992 shutdown_connections 2024-04-26T22:55:58.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.456+0000 7f5ce7a00640 1 -- 172.21.15.27:0/2315189992 wait complete. 2024-04-26T22:55:58.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.457+0000 7f5ce7a00640 1 Processor -- start 2024-04-26T22:55:58.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.457+0000 7f5ce7a00640 1 -- start start 2024-04-26T22:55:58.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.457+0000 7f5ce7a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce0075310 0x7f5ce019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:58.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.457+0000 7f5ce7a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ce00768f0 0x7f5ce019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:58.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.457+0000 7f5ce7a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce010a840 0x7f5ce01a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:58.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.458+0000 7f5ce7a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01106c0 con 0x7f5ce00768f0 2024-04-26T22:55:58.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.458+0000 7f5ce7a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0110540 con 0x7f5ce0075310 2024-04-26T22:55:58.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.458+0000 7f5ce7a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0110840 con 0x7f5ce010a840 2024-04-26T22:55:58.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.458+0000 7f5ce5000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce0075310 0x7f5ce019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:58.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.458+0000 7f5ce5000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce0075310 0x7f5ce019ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:60134/0 (socket says 172.21.15.27:60134) 2024-04-26T22:55:58.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.458+0000 7f5ce5000640 1 -- 172.21.15.27:0/1103369738 learned_addr learned my addr 172.21.15.27:0/1103369738 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:58.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.458+0000 7f5cdfe00640 1 --2- 172.21.15.27:0/1103369738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ce00768f0 0x7f5ce019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:58.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.458+0000 7f5ce5a00640 1 --2- 172.21.15.27:0/1103369738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce010a840 0x7f5ce01a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:58.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.458+0000 7f5ce5000640 1 -- 172.21.15.27:0/1103369738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce010a840 msgr2=0x7f5ce01a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:58.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.458+0000 7f5ce5000640 1 --2- 172.21.15.27:0/1103369738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce010a840 0x7f5ce01a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.458+0000 7f5ce5000640 1 -- 172.21.15.27:0/1103369738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ce00768f0 msgr2=0x7f5ce019b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:58.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.458+0000 7f5ce5000640 1 --2- 172.21.15.27:0/1103369738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ce00768f0 0x7f5ce019b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.458+0000 7f5ce5000640 1 -- 172.21.15.27:0/1103369738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce01a2960 con 0x7f5ce0075310 2024-04-26T22:55:58.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.459+0000 7f5ce5000640 1 --2- 172.21.15.27:0/1103369738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce0075310 0x7f5ce019ad50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8005940 tx=0x7f5cc8031a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:58.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.459+0000 7f5cdd600640 1 -- 172.21.15.27:0/1103369738 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc803f030 con 0x7f5ce0075310 2024-04-26T22:55:58.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.459+0000 7f5cdd600640 1 -- 172.21.15.27:0/1103369738 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cc8003aa0 con 0x7f5ce0075310 2024-04-26T22:55:58.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.459+0000 7f5cdd600640 1 -- 172.21.15.27:0/1103369738 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8035070 con 0x7f5ce0075310 2024-04-26T22:55:58.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.459+0000 7f5ce7a00640 1 -- 172.21.15.27:0/1103369738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce01a2bf0 con 0x7f5ce0075310 2024-04-26T22:55:58.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.459+0000 7f5ce7a00640 1 -- 172.21.15.27:0/1103369738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce01a3130 con 0x7f5ce0075310 2024-04-26T22:55:58.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.460+0000 7f5ce7a00640 1 -- 172.21.15.27:0/1103369738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce0109600 con 0x7f5ce0075310 2024-04-26T22:55:58.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.462+0000 7f5cdd600640 1 -- 172.21.15.27:0/1103369738 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5cc8042030 con 0x7f5ce0075310 2024-04-26T22:55:58.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.464+0000 7f5cdd600640 1 --2- 172.21.15.27:0/1103369738 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5cc40a3cc0 0x7f5cc40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:58.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.464+0000 7f5cdd600640 1 -- 172.21.15.27:0/1103369738 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5cc80f0dd0 con 0x7f5ce0075310 2024-04-26T22:55:58.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.464+0000 7f5cdfe00640 1 --2- 172.21.15.27:0/1103369738 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5cc40a3cc0 0x7f5cc40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:58.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.465+0000 7f5cdfe00640 1 --2- 172.21.15.27:0/1103369738 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5cc40a3cc0 0x7f5cc40a6180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0005fd0 tx=0x7f5cd0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:58.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.468+0000 7f5cdd600640 1 -- 172.21.15.27:0/1103369738 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5cc80bcd60 con 0x7f5ce0075310 2024-04-26T22:55:58.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.673+0000 7f5ce7a00640 1 -- 172.21.15.27:0/1103369738 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ce0062d70 con 0x7f5cc40a3cc0 2024-04-26T22:55:58.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.696+0000 7f5cdd600640 1 -- 172.21.15.27:0/1103369738 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340636 (secure 0 0 0) 0x7f5ce0062d70 con 0x7f5cc40a3cc0 2024-04-26T22:55:58.702 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:58.704 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-26T22:55:58.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.704+0000 7f5ce7a00640 1 -- 172.21.15.27:0/1103369738 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5cc40a3cc0 msgr2=0x7f5cc40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:58.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.704+0000 7f5ce7a00640 1 --2- 172.21.15.27:0/1103369738 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5cc40a3cc0 0x7f5cc40a6180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0005fd0 tx=0x7f5cd0009500 comp rx=0 tx=0).stop 2024-04-26T22:55:58.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.704+0000 7f5ce7a00640 1 -- 172.21.15.27:0/1103369738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce0075310 msgr2=0x7f5ce019ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:58.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.704+0000 7f5ce7a00640 1 --2- 172.21.15.27:0/1103369738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce0075310 0x7f5ce019ad50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8005940 tx=0x7f5cc8031a50 comp rx=0 tx=0).stop 2024-04-26T22:55:58.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.704+0000 7f5ce7a00640 1 -- 172.21.15.27:0/1103369738 shutdown_connections 2024-04-26T22:55:58.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.704+0000 7f5ce7a00640 1 --2- 172.21.15.27:0/1103369738 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f5cc40a3cc0 0x7f5cc40a6180 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.704+0000 7f5ce7a00640 1 --2- 172.21.15.27:0/1103369738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ce010a840 0x7f5ce01a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.704+0000 7f5ce7a00640 1 --2- 172.21.15.27:0/1103369738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ce00768f0 0x7f5ce019b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.704+0000 7f5ce7a00640 1 --2- 172.21.15.27:0/1103369738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce0075310 0x7f5ce019ad50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.704+0000 7f5ce7a00640 1 -- 172.21.15.27:0/1103369738 >> 172.21.15.27:0/1103369738 conn(0x7f5ce00fdf00 msgr2=0x7f5ce01067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:58.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.704+0000 7f5ce7a00640 1 -- 172.21.15.27:0/1103369738 shutdown_connections 2024-04-26T22:55:58.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.704+0000 7f5ce7a00640 1 -- 172.21.15.27:0/1103369738 wait complete. 2024-04-26T22:55:58.732 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-04-26T22:55:58.461174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":617510,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":167,"num_read_kb":165,"num_write":317,"num_write_kb":3004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":257,"ondisk_log_size":257,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381772,"kb_used_data":11820,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560052,"statfs":{"total":1151940427776,"available":1151549493248,"internally_reserved":0,"allocated":12103680,"data_stored":7629666,"data_compressed":284946,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22643,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":31,"num_read_kb":46,"num_write":88,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006774"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.804096+0000","last_change":"2024-04-26T22:55:25.404468+0000","last_active":"2024-04-26T22:55:32.804096+0000","last_peered":"2024-04-26T22:55:32.804096+0000","last_clean":"2024-04-26T22:55:32.804096+0000","last_became_active":"2024-04-26T22:55:25.404178+0000","last_became_peered":"2024-04-26T22:55:25.404178+0000","last_unstale":"2024-04-26T22:55:32.804096+0000","last_undegraded":"2024-04-26T22:55:32.804096+0000","last_fullsized":"2024-04-26T22:55:32.804096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:46.249569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.597005+0000","last_change":"2024-04-26T22:55:25.401044+0000","last_active":"2024-04-26T22:55:31.597005+0000","last_peered":"2024-04-26T22:55:31.597005+0000","last_clean":"2024-04-26T22:55:31.597005+0000","last_became_active":"2024-04-26T22:55:25.397925+0000","last_became_peered":"2024-04-26T22:55:25.397925+0000","last_unstale":"2024-04-26T22:55:31.597005+0000","last_undegraded":"2024-04-26T22:55:31.597005+0000","last_fullsized":"2024-04-26T22:55:31.597005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:28.503480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.596799+0000","last_change":"2024-04-26T22:55:25.396780+0000","last_active":"2024-04-26T22:55:31.596799+0000","last_peered":"2024-04-26T22:55:31.596799+0000","last_clean":"2024-04-26T22:55:31.596799+0000","last_became_active":"2024-04-26T22:55:25.395401+0000","last_became_peered":"2024-04-26T22:55:25.395401+0000","last_unstale":"2024-04-26T22:55:31.596799+0000","last_undegraded":"2024-04-26T22:55:31.596799+0000","last_fullsized":"2024-04-26T22:55:31.596799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:32.295042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412115+0000","last_change":"2024-04-26T22:55:25.373994+0000","last_active":"2024-04-26T22:55:31.412115+0000","last_peered":"2024-04-26T22:55:31.412115+0000","last_clean":"2024-04-26T22:55:31.412115+0000","last_became_active":"2024-04-26T22:55:25.373653+0000","last_became_peered":"2024-04-26T22:55:25.373653+0000","last_unstale":"2024-04-26T22:55:31.412115+0000","last_undegraded":"2024-04-26T22:55:31.412115+0000","last_fullsized":"2024-04-26T22:55:31.412115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:58.460947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.854991+0000","last_change":"2024-04-26T22:55:25.391341+0000","last_active":"2024-04-26T22:55:31.854991+0000","last_peered":"2024-04-26T22:55:31.854991+0000","last_clean":"2024-04-26T22:55:31.854991+0000","last_became_active":"2024-04-26T22:55:25.391061+0000","last_became_peered":"2024-04-26T22:55:25.391061+0000","last_unstale":"2024-04-26T22:55:31.854991+0000","last_undegraded":"2024-04-26T22:55:31.854991+0000","last_fullsized":"2024-04-26T22:55:31.854991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:08.391492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.012537+0000","last_change":"2024-04-26T22:55:25.388412+0000","last_active":"2024-04-26T22:55:32.012537+0000","last_peered":"2024-04-26T22:55:32.012537+0000","last_clean":"2024-04-26T22:55:32.012537+0000","last_became_active":"2024-04-26T22:55:25.388010+0000","last_became_peered":"2024-04-26T22:55:25.388010+0000","last_unstale":"2024-04-26T22:55:32.012537+0000","last_undegraded":"2024-04-26T22:55:32.012537+0000","last_fullsized":"2024-04-26T22:55:32.012537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:45.851109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.961851+0000","last_change":"2024-04-26T22:55:25.386174+0000","last_active":"2024-04-26T22:55:31.961851+0000","last_peered":"2024-04-26T22:55:31.961851+0000","last_clean":"2024-04-26T22:55:31.961851+0000","last_became_active":"2024-04-26T22:55:25.385800+0000","last_became_peered":"2024-04-26T22:55:25.385800+0000","last_unstale":"2024-04-26T22:55:31.961851+0000","last_undegraded":"2024-04-26T22:55:31.961851+0000","last_fullsized":"2024-04-26T22:55:31.961851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:55.833414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.816834+0000","last_change":"2024-04-26T22:55:25.408952+0000","last_active":"2024-04-26T22:55:32.816834+0000","last_peered":"2024-04-26T22:55:32.816834+0000","last_clean":"2024-04-26T22:55:32.816834+0000","last_became_active":"2024-04-26T22:55:25.408485+0000","last_became_peered":"2024-04-26T22:55:25.408485+0000","last_unstale":"2024-04-26T22:55:32.816834+0000","last_undegraded":"2024-04-26T22:55:32.816834+0000","last_fullsized":"2024-04-26T22:55:32.816834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.043114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.814577+0000","last_change":"2024-04-26T22:55:25.377377+0000","last_active":"2024-04-26T22:55:32.814577+0000","last_peered":"2024-04-26T22:55:32.814577+0000","last_clean":"2024-04-26T22:55:32.814577+0000","last_became_active":"2024-04-26T22:55:25.377015+0000","last_became_peered":"2024-04-26T22:55:25.377015+0000","last_unstale":"2024-04-26T22:55:32.814577+0000","last_undegraded":"2024-04-26T22:55:32.814577+0000","last_fullsized":"2024-04-26T22:55:32.814577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:25.676469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.012956+0000","last_change":"2024-04-26T22:55:25.375549+0000","last_active":"2024-04-26T22:55:32.012956+0000","last_peered":"2024-04-26T22:55:32.012956+0000","last_clean":"2024-04-26T22:55:32.012956+0000","last_became_active":"2024-04-26T22:55:25.375232+0000","last_became_peered":"2024-04-26T22:55:25.375232+0000","last_unstale":"2024-04-26T22:55:32.012956+0000","last_undegraded":"2024-04-26T22:55:32.012956+0000","last_fullsized":"2024-04-26T22:55:32.012956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:22.561623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.962444+0000","last_change":"2024-04-26T22:55:25.374224+0000","last_active":"2024-04-26T22:55:31.962444+0000","last_peered":"2024-04-26T22:55:31.962444+0000","last_clean":"2024-04-26T22:55:31.962444+0000","last_became_active":"2024-04-26T22:55:25.373813+0000","last_became_peered":"2024-04-26T22:55:25.373813+0000","last_unstale":"2024-04-26T22:55:31.962444+0000","last_undegraded":"2024-04-26T22:55:31.962444+0000","last_fullsized":"2024-04-26T22:55:31.962444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:34.874888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412118+0000","last_change":"2024-04-26T22:55:25.376666+0000","last_active":"2024-04-26T22:55:31.412118+0000","last_peered":"2024-04-26T22:55:31.412118+0000","last_clean":"2024-04-26T22:55:31.412118+0000","last_became_active":"2024-04-26T22:55:25.376328+0000","last_became_peered":"2024-04-26T22:55:25.376328+0000","last_unstale":"2024-04-26T22:55:31.412118+0000","last_undegraded":"2024-04-26T22:55:31.412118+0000","last_fullsized":"2024-04-26T22:55:31.412118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:32:34.287984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.819314+0000","last_change":"2024-04-26T22:55:25.377220+0000","last_active":"2024-04-26T22:55:32.819314+0000","last_peered":"2024-04-26T22:55:32.819314+0000","last_clean":"2024-04-26T22:55:32.819314+0000","last_became_active":"2024-04-26T22:55:25.376637+0000","last_became_peered":"2024-04-26T22:55:25.376637+0000","last_unstale":"2024-04-26T22:55:32.819314+0000","last_undegraded":"2024-04-26T22:55:32.819314+0000","last_fullsized":"2024-04-26T22:55:32.819314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:57.934912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.596020+0000","last_change":"2024-04-26T22:55:25.404669+0000","last_active":"2024-04-26T22:55:31.596020+0000","last_peered":"2024-04-26T22:55:31.596020+0000","last_clean":"2024-04-26T22:55:31.596020+0000","last_became_active":"2024-04-26T22:55:25.402804+0000","last_became_peered":"2024-04-26T22:55:25.402804+0000","last_unstale":"2024-04-26T22:55:31.596020+0000","last_undegraded":"2024-04-26T22:55:31.596020+0000","last_fullsized":"2024-04-26T22:55:31.596020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:37.239667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.596020+0000","last_change":"2024-04-26T22:55:25.399877+0000","last_active":"2024-04-26T22:55:31.596020+0000","last_peered":"2024-04-26T22:55:31.596020+0000","last_clean":"2024-04-26T22:55:31.596020+0000","last_became_active":"2024-04-26T22:55:25.397476+0000","last_became_peered":"2024-04-26T22:55:25.397476+0000","last_unstale":"2024-04-26T22:55:31.596020+0000","last_undegraded":"2024-04-26T22:55:31.596020+0000","last_fullsized":"2024-04-26T22:55:31.596020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:26.732300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.813892+0000","last_change":"2024-04-26T22:55:25.393682+0000","last_active":"2024-04-26T22:55:32.813892+0000","last_peered":"2024-04-26T22:55:32.813892+0000","last_clean":"2024-04-26T22:55:32.813892+0000","last_became_active":"2024-04-26T22:55:25.393349+0000","last_became_peered":"2024-04-26T22:55:25.393349+0000","last_unstale":"2024-04-26T22:55:32.813892+0000","last_undegraded":"2024-04-26T22:55:32.813892+0000","last_fullsized":"2024-04-26T22:55:32.813892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:48.065904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.803972+0000","last_change":"2024-04-26T22:55:25.404449+0000","last_active":"2024-04-26T22:55:32.803972+0000","last_peered":"2024-04-26T22:55:32.803972+0000","last_clean":"2024-04-26T22:55:32.803972+0000","last_became_active":"2024-04-26T22:55:25.404170+0000","last_became_peered":"2024-04-26T22:55:25.404170+0000","last_unstale":"2024-04-26T22:55:32.803972+0000","last_undegraded":"2024-04-26T22:55:32.803972+0000","last_fullsized":"2024-04-26T22:55:32.803972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:04.731208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.414210+0000","last_change":"2024-04-26T22:55:25.390368+0000","last_active":"2024-04-26T22:55:31.414210+0000","last_peered":"2024-04-26T22:55:31.414210+0000","last_clean":"2024-04-26T22:55:31.414210+0000","last_became_active":"2024-04-26T22:55:25.389201+0000","last_became_peered":"2024-04-26T22:55:25.389201+0000","last_unstale":"2024-04-26T22:55:31.414210+0000","last_undegraded":"2024-04-26T22:55:31.414210+0000","last_fullsized":"2024-04-26T22:55:31.414210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:35.985419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.813847+0000","last_change":"2024-04-26T22:55:25.387616+0000","last_active":"2024-04-26T22:55:32.813847+0000","last_peered":"2024-04-26T22:55:32.813847+0000","last_clean":"2024-04-26T22:55:32.813847+0000","last_became_active":"2024-04-26T22:55:25.387360+0000","last_became_peered":"2024-04-26T22:55:25.387360+0000","last_unstale":"2024-04-26T22:55:32.813847+0000","last_undegraded":"2024-04-26T22:55:32.813847+0000","last_fullsized":"2024-04-26T22:55:32.813847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:49.378287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.813775+0000","last_change":"2024-04-26T22:55:30.425407+0000","last_active":"2024-04-26T22:55:32.813775+0000","last_peered":"2024-04-26T22:55:32.813775+0000","last_clean":"2024-04-26T22:55:32.813775+0000","last_became_active":"2024-04-26T22:55:30.424796+0000","last_became_peered":"2024-04-26T22:55:30.424796+0000","last_unstale":"2024-04-26T22:55:32.813775+0000","last_undegraded":"2024-04-26T22:55:32.813775+0000","last_fullsized":"2024-04-26T22:55:32.813775+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:42.539415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.648372+0000","last_change":"2024-04-26T22:55:24.776253+0000","last_active":"2024-04-26T22:55:53.648372+0000","last_peered":"2024-04-26T22:55:53.648372+0000","last_clean":"2024-04-26T22:55:53.648372+0000","last_became_active":"2024-04-26T22:55:24.775700+0000","last_became_peered":"2024-04-26T22:55:24.775700+0000","last_unstale":"2024-04-26T22:55:53.648372+0000","last_undegraded":"2024-04-26T22:55:53.648372+0000","last_fullsized":"2024-04-26T22:55:53.648372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:45.608363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.817143+0000","last_change":"2024-04-26T22:55:25.390690+0000","last_active":"2024-04-26T22:55:32.817143+0000","last_peered":"2024-04-26T22:55:32.817143+0000","last_clean":"2024-04-26T22:55:32.817143+0000","last_became_active":"2024-04-26T22:55:25.390289+0000","last_became_peered":"2024-04-26T22:55:25.390289+0000","last_unstale":"2024-04-26T22:55:32.817143+0000","last_undegraded":"2024-04-26T22:55:32.817143+0000","last_fullsized":"2024-04-26T22:55:32.817143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:54.637992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411801+0000","last_change":"2024-04-26T22:55:30.433555+0000","last_active":"2024-04-26T22:55:31.411801+0000","last_peered":"2024-04-26T22:55:31.411801+0000","last_clean":"2024-04-26T22:55:31.411801+0000","last_became_active":"2024-04-26T22:55:30.433240+0000","last_became_peered":"2024-04-26T22:55:30.433240+0000","last_unstale":"2024-04-26T22:55:31.411801+0000","last_undegraded":"2024-04-26T22:55:31.411801+0000","last_fullsized":"2024-04-26T22:55:31.411801+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:55:21.215073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.815147+0000","last_change":"2024-04-26T22:55:24.788218+0000","last_active":"2024-04-26T22:55:32.815147+0000","last_peered":"2024-04-26T22:55:32.815147+0000","last_clean":"2024-04-26T22:55:32.815147+0000","last_became_active":"2024-04-26T22:55:24.787762+0000","last_became_peered":"2024-04-26T22:55:24.787762+0000","last_unstale":"2024-04-26T22:55:32.815147+0000","last_undegraded":"2024-04-26T22:55:32.815147+0000","last_fullsized":"2024-04-26T22:55:32.815147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:42.264617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.597068+0000","last_change":"2024-04-26T22:55:25.400356+0000","last_active":"2024-04-26T22:55:31.597068+0000","last_peered":"2024-04-26T22:55:31.597068+0000","last_clean":"2024-04-26T22:55:31.597068+0000","last_became_active":"2024-04-26T22:55:25.399907+0000","last_became_peered":"2024-04-26T22:55:25.399907+0000","last_unstale":"2024-04-26T22:55:31.597068+0000","last_undegraded":"2024-04-26T22:55:31.597068+0000","last_fullsized":"2024-04-26T22:55:31.597068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:50:50.804112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411980+0000","last_change":"2024-04-26T22:55:30.426429+0000","last_active":"2024-04-26T22:55:31.411980+0000","last_peered":"2024-04-26T22:55:31.411980+0000","last_clean":"2024-04-26T22:55:31.411980+0000","last_became_active":"2024-04-26T22:55:30.425862+0000","last_became_peered":"2024-04-26T22:55:30.425862+0000","last_unstale":"2024-04-26T22:55:31.411980+0000","last_undegraded":"2024-04-26T22:55:31.411980+0000","last_fullsized":"2024-04-26T22:55:31.411980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:49.646336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.643721+0000","last_change":"2024-04-26T22:55:24.763945+0000","last_active":"2024-04-26T22:55:53.643721+0000","last_peered":"2024-04-26T22:55:53.643721+0000","last_clean":"2024-04-26T22:55:53.643721+0000","last_became_active":"2024-04-26T22:55:24.763580+0000","last_became_peered":"2024-04-26T22:55:24.763580+0000","last_unstale":"2024-04-26T22:55:53.643721+0000","last_undegraded":"2024-04-26T22:55:53.643721+0000","last_fullsized":"2024-04-26T22:55:53.643721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:21:52.298738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.814288+0000","last_change":"2024-04-26T22:55:25.381896+0000","last_active":"2024-04-26T22:55:32.814288+0000","last_peered":"2024-04-26T22:55:32.814288+0000","last_clean":"2024-04-26T22:55:32.814288+0000","last_became_active":"2024-04-26T22:55:25.381411+0000","last_became_peered":"2024-04-26T22:55:25.381411+0000","last_unstale":"2024-04-26T22:55:32.814288+0000","last_undegraded":"2024-04-26T22:55:32.814288+0000","last_fullsized":"2024-04-26T22:55:32.814288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:54.731407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412051+0000","last_change":"2024-04-26T22:55:30.419870+0000","last_active":"2024-04-26T22:55:31.412051+0000","last_peered":"2024-04-26T22:55:31.412051+0000","last_clean":"2024-04-26T22:55:31.412051+0000","last_became_active":"2024-04-26T22:55:30.419533+0000","last_became_peered":"2024-04-26T22:55:30.419533+0000","last_unstale":"2024-04-26T22:55:31.412051+0000","last_undegraded":"2024-04-26T22:55:31.412051+0000","last_fullsized":"2024-04-26T22:55:31.412051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:37:47.647953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.855098+0000","last_change":"2024-04-26T22:55:24.756017+0000","last_active":"2024-04-26T22:55:31.855098+0000","last_peered":"2024-04-26T22:55:31.855098+0000","last_clean":"2024-04-26T22:55:31.855098+0000","last_became_active":"2024-04-26T22:55:24.755661+0000","last_became_peered":"2024-04-26T22:55:24.755661+0000","last_unstale":"2024-04-26T22:55:31.855098+0000","last_undegraded":"2024-04-26T22:55:31.855098+0000","last_fullsized":"2024-04-26T22:55:31.855098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:53.909567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411492+0000","last_change":"2024-04-26T22:55:25.375929+0000","last_active":"2024-04-26T22:55:31.411492+0000","last_peered":"2024-04-26T22:55:31.411492+0000","last_clean":"2024-04-26T22:55:31.411492+0000","last_became_active":"2024-04-26T22:55:25.375435+0000","last_became_peered":"2024-04-26T22:55:25.375435+0000","last_unstale":"2024-04-26T22:55:31.411492+0000","last_undegraded":"2024-04-26T22:55:31.411492+0000","last_fullsized":"2024-04-26T22:55:31.411492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:43.336889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.813934+0000","last_change":"2024-04-26T22:55:30.409074+0000","last_active":"2024-04-26T22:55:32.813934+0000","last_peered":"2024-04-26T22:55:32.813934+0000","last_clean":"2024-04-26T22:55:32.813934+0000","last_became_active":"2024-04-26T22:55:30.408613+0000","last_became_peered":"2024-04-26T22:55:30.408613+0000","last_unstale":"2024-04-26T22:55:32.813934+0000","last_undegraded":"2024-04-26T22:55:32.813934+0000","last_fullsized":"2024-04-26T22:55:32.813934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:24.312971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:54.658023+0000","last_change":"2024-04-26T22:55:24.763787+0000","last_active":"2024-04-26T22:55:54.658023+0000","last_peered":"2024-04-26T22:55:54.658023+0000","last_clean":"2024-04-26T22:55:54.658023+0000","last_became_active":"2024-04-26T22:55:24.763480+0000","last_became_peered":"2024-04-26T22:55:24.763480+0000","last_unstale":"2024-04-26T22:55:54.658023+0000","last_undegraded":"2024-04-26T22:55:54.658023+0000","last_fullsized":"2024-04-26T22:55:54.658023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:18.591009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.855024+0000","last_change":"2024-04-26T22:55:25.388313+0000","last_active":"2024-04-26T22:55:31.855024+0000","last_peered":"2024-04-26T22:55:31.855024+0000","last_clean":"2024-04-26T22:55:31.855024+0000","last_became_active":"2024-04-26T22:55:25.387974+0000","last_became_peered":"2024-04-26T22:55:25.387974+0000","last_unstale":"2024-04-26T22:55:31.855024+0000","last_undegraded":"2024-04-26T22:55:31.855024+0000","last_fullsized":"2024-04-26T22:55:31.855024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:23.237387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.813766+0000","last_change":"2024-04-26T22:55:30.425891+0000","last_active":"2024-04-26T22:55:32.813766+0000","last_peered":"2024-04-26T22:55:32.813766+0000","last_clean":"2024-04-26T22:55:32.813766+0000","last_became_active":"2024-04-26T22:55:30.424742+0000","last_became_peered":"2024-04-26T22:55:30.424742+0000","last_unstale":"2024-04-26T22:55:32.813766+0000","last_undegraded":"2024-04-26T22:55:32.813766+0000","last_fullsized":"2024-04-26T22:55:32.813766+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:04.876670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.646351+0000","last_change":"2024-04-26T22:55:24.771979+0000","last_active":"2024-04-26T22:55:53.646351+0000","last_peered":"2024-04-26T22:55:53.646351+0000","last_clean":"2024-04-26T22:55:53.646351+0000","last_became_active":"2024-04-26T22:55:24.771622+0000","last_became_peered":"2024-04-26T22:55:24.771622+0000","last_unstale":"2024-04-26T22:55:53.646351+0000","last_undegraded":"2024-04-26T22:55:53.646351+0000","last_fullsized":"2024-04-26T22:55:53.646351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:40.210406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411568+0000","last_change":"2024-04-26T22:55:25.399892+0000","last_active":"2024-04-26T22:55:31.411568+0000","last_peered":"2024-04-26T22:55:31.411568+0000","last_clean":"2024-04-26T22:55:31.411568+0000","last_became_active":"2024-04-26T22:55:25.399515+0000","last_became_peered":"2024-04-26T22:55:25.399515+0000","last_unstale":"2024-04-26T22:55:31.411568+0000","last_undegraded":"2024-04-26T22:55:31.411568+0000","last_fullsized":"2024-04-26T22:55:31.411568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.472103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.804079+0000","last_change":"2024-04-26T22:55:30.422932+0000","last_active":"2024-04-26T22:55:32.804079+0000","last_peered":"2024-04-26T22:55:32.804079+0000","last_clean":"2024-04-26T22:55:32.804079+0000","last_became_active":"2024-04-26T22:55:30.422481+0000","last_became_peered":"2024-04-26T22:55:30.422481+0000","last_unstale":"2024-04-26T22:55:32.804079+0000","last_undegraded":"2024-04-26T22:55:32.804079+0000","last_fullsized":"2024-04-26T22:55:32.804079+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:11.469515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.649536+0000","last_change":"2024-04-26T22:55:24.780973+0000","last_active":"2024-04-26T22:55:53.649536+0000","last_peered":"2024-04-26T22:55:53.649536+0000","last_clean":"2024-04-26T22:55:53.649536+0000","last_became_active":"2024-04-26T22:55:24.779810+0000","last_became_peered":"2024-04-26T22:55:24.779810+0000","last_unstale":"2024-04-26T22:55:53.649536+0000","last_undegraded":"2024-04-26T22:55:53.649536+0000","last_fullsized":"2024-04-26T22:55:53.649536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:33.708588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.816600+0000","last_change":"2024-04-26T22:55:25.402067+0000","last_active":"2024-04-26T22:55:32.816600+0000","last_peered":"2024-04-26T22:55:32.816600+0000","last_clean":"2024-04-26T22:55:32.816600+0000","last_became_active":"2024-04-26T22:55:25.401717+0000","last_became_peered":"2024-04-26T22:55:25.401717+0000","last_unstale":"2024-04-26T22:55:32.816600+0000","last_undegraded":"2024-04-26T22:55:32.816600+0000","last_fullsized":"2024-04-26T22:55:32.816600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:16.260620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.597202+0000","last_change":"2024-04-26T22:55:30.429430+0000","last_active":"2024-04-26T22:55:31.597202+0000","last_peered":"2024-04-26T22:55:31.597202+0000","last_clean":"2024-04-26T22:55:31.597202+0000","last_became_active":"2024-04-26T22:55:30.428800+0000","last_became_peered":"2024-04-26T22:55:30.428800+0000","last_unstale":"2024-04-26T22:55:31.597202+0000","last_undegraded":"2024-04-26T22:55:31.597202+0000","last_fullsized":"2024-04-26T22:55:31.597202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:05.501004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411118+0000","last_change":"2024-04-26T22:55:24.765181+0000","last_active":"2024-04-26T22:55:31.411118+0000","last_peered":"2024-04-26T22:55:31.411118+0000","last_clean":"2024-04-26T22:55:31.411118+0000","last_became_active":"2024-04-26T22:55:24.764847+0000","last_became_peered":"2024-04-26T22:55:24.764847+0000","last_unstale":"2024-04-26T22:55:31.411118+0000","last_undegraded":"2024-04-26T22:55:31.411118+0000","last_fullsized":"2024-04-26T22:55:31.411118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:25.949304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412633+0000","last_change":"2024-04-26T22:55:25.401228+0000","last_active":"2024-04-26T22:55:31.412633+0000","last_peered":"2024-04-26T22:55:31.412633+0000","last_clean":"2024-04-26T22:55:31.412633+0000","last_became_active":"2024-04-26T22:55:25.400756+0000","last_became_peered":"2024-04-26T22:55:25.400756+0000","last_unstale":"2024-04-26T22:55:31.412633+0000","last_undegraded":"2024-04-26T22:55:31.412633+0000","last_fullsized":"2024-04-26T22:55:31.412633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:00.220165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.815845+0000","last_change":"2024-04-26T22:55:30.424537+0000","last_active":"2024-04-26T22:55:32.815845+0000","last_peered":"2024-04-26T22:55:32.815845+0000","last_clean":"2024-04-26T22:55:32.815845+0000","last_became_active":"2024-04-26T22:55:30.423262+0000","last_became_peered":"2024-04-26T22:55:30.423262+0000","last_unstale":"2024-04-26T22:55:32.815845+0000","last_undegraded":"2024-04-26T22:55:32.815845+0000","last_fullsized":"2024-04-26T22:55:32.815845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:48.666453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:54.656607+0000","last_change":"2024-04-26T22:55:24.776580+0000","last_active":"2024-04-26T22:55:54.656607+0000","last_peered":"2024-04-26T22:55:54.656607+0000","last_clean":"2024-04-26T22:55:54.656607+0000","last_became_active":"2024-04-26T22:55:24.776143+0000","last_became_peered":"2024-04-26T22:55:24.776143+0000","last_unstale":"2024-04-26T22:55:54.656607+0000","last_undegraded":"2024-04-26T22:55:54.656607+0000","last_fullsized":"2024-04-26T22:55:54.656607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:02.291237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.814567+0000","last_change":"2024-04-26T22:55:25.385645+0000","last_active":"2024-04-26T22:55:32.814567+0000","last_peered":"2024-04-26T22:55:32.814567+0000","last_clean":"2024-04-26T22:55:32.814567+0000","last_became_active":"2024-04-26T22:55:25.385289+0000","last_became_peered":"2024-04-26T22:55:25.385289+0000","last_unstale":"2024-04-26T22:55:32.814567+0000","last_undegraded":"2024-04-26T22:55:32.814567+0000","last_fullsized":"2024-04-26T22:55:32.814567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:34.662117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412619+0000","last_change":"2024-04-26T22:55:30.437915+0000","last_active":"2024-04-26T22:55:31.412619+0000","last_peered":"2024-04-26T22:55:31.412619+0000","last_clean":"2024-04-26T22:55:31.412619+0000","last_became_active":"2024-04-26T22:55:30.437148+0000","last_became_peered":"2024-04-26T22:55:30.437148+0000","last_unstale":"2024-04-26T22:55:31.412619+0000","last_undegraded":"2024-04-26T22:55:31.412619+0000","last_fullsized":"2024-04-26T22:55:31.412619+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:11.647498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:52.617793+0000","last_change":"2024-04-26T22:55:24.763281+0000","last_active":"2024-04-26T22:55:52.617793+0000","last_peered":"2024-04-26T22:55:52.617793+0000","last_clean":"2024-04-26T22:55:52.617793+0000","last_became_active":"2024-04-26T22:55:24.762901+0000","last_became_peered":"2024-04-26T22:55:24.762901+0000","last_unstale":"2024-04-26T22:55:52.617793+0000","last_undegraded":"2024-04-26T22:55:52.617793+0000","last_fullsized":"2024-04-26T22:55:52.617793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:10.092887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.597067+0000","last_change":"2024-04-26T22:55:25.401057+0000","last_active":"2024-04-26T22:55:31.597067+0000","last_peered":"2024-04-26T22:55:31.597067+0000","last_clean":"2024-04-26T22:55:31.597067+0000","last_became_active":"2024-04-26T22:55:25.398265+0000","last_became_peered":"2024-04-26T22:55:25.398265+0000","last_unstale":"2024-04-26T22:55:31.597067+0000","last_undegraded":"2024-04-26T22:55:31.597067+0000","last_fullsized":"2024-04-26T22:55:31.597067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:49.967012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412258+0000","last_change":"2024-04-26T22:55:30.426682+0000","last_active":"2024-04-26T22:55:31.412258+0000","last_peered":"2024-04-26T22:55:31.412258+0000","last_clean":"2024-04-26T22:55:31.412258+0000","last_became_active":"2024-04-26T22:55:30.425918+0000","last_became_peered":"2024-04-26T22:55:30.425918+0000","last_unstale":"2024-04-26T22:55:31.412258+0000","last_undegraded":"2024-04-26T22:55:31.412258+0000","last_fullsized":"2024-04-26T22:55:31.412258+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:44.230083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:52.620416+0000","last_change":"2024-04-26T22:55:24.773497+0000","last_active":"2024-04-26T22:55:52.620416+0000","last_peered":"2024-04-26T22:55:52.620416+0000","last_clean":"2024-04-26T22:55:52.620416+0000","last_became_active":"2024-04-26T22:55:24.773183+0000","last_became_peered":"2024-04-26T22:55:24.773183+0000","last_unstale":"2024-04-26T22:55:52.620416+0000","last_undegraded":"2024-04-26T22:55:52.620416+0000","last_fullsized":"2024-04-26T22:55:52.620416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:58:07.157785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.814353+0000","last_change":"2024-04-26T22:55:25.378420+0000","last_active":"2024-04-26T22:55:32.814353+0000","last_peered":"2024-04-26T22:55:32.814353+0000","last_clean":"2024-04-26T22:55:32.814353+0000","last_became_active":"2024-04-26T22:55:25.378068+0000","last_became_peered":"2024-04-26T22:55:25.378068+0000","last_unstale":"2024-04-26T22:55:32.814353+0000","last_undegraded":"2024-04-26T22:55:32.814353+0000","last_fullsized":"2024-04-26T22:55:32.814353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:10.910541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412908+0000","last_change":"2024-04-26T22:55:30.436973+0000","last_active":"2024-04-26T22:55:31.412908+0000","last_peered":"2024-04-26T22:55:31.412908+0000","last_clean":"2024-04-26T22:55:31.412908+0000","last_became_active":"2024-04-26T22:55:30.436327+0000","last_became_peered":"2024-04-26T22:55:30.436327+0000","last_unstale":"2024-04-26T22:55:31.412908+0000","last_undegraded":"2024-04-26T22:55:31.412908+0000","last_fullsized":"2024-04-26T22:55:31.412908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:36.616328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'19","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:55.660082+0000","last_change":"2024-04-26T22:55:24.760508+0000","last_active":"2024-04-26T22:55:55.660082+0000","last_peered":"2024-04-26T22:55:55.660082+0000","last_clean":"2024-04-26T22:55:55.660082+0000","last_became_active":"2024-04-26T22:55:24.760180+0000","last_became_peered":"2024-04-26T22:55:24.760180+0000","last_unstale":"2024-04-26T22:55:55.660082+0000","last_undegraded":"2024-04-26T22:55:55.660082+0000","last_fullsized":"2024-04-26T22:55:55.660082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:50.094352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":26,"num_write":20,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.855288+0000","last_change":"2024-04-26T22:55:25.376091+0000","last_active":"2024-04-26T22:55:31.855288+0000","last_peered":"2024-04-26T22:55:31.855288+0000","last_clean":"2024-04-26T22:55:31.855288+0000","last_became_active":"2024-04-26T22:55:25.375623+0000","last_became_peered":"2024-04-26T22:55:25.375623+0000","last_unstale":"2024-04-26T22:55:31.855288+0000","last_undegraded":"2024-04-26T22:55:31.855288+0000","last_fullsized":"2024-04-26T22:55:31.855288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:09.466719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411983+0000","last_change":"2024-04-26T22:55:30.413534+0000","last_active":"2024-04-26T22:55:31.411983+0000","last_peered":"2024-04-26T22:55:31.411983+0000","last_clean":"2024-04-26T22:55:31.411983+0000","last_became_active":"2024-04-26T22:55:30.413120+0000","last_became_peered":"2024-04-26T22:55:30.413120+0000","last_unstale":"2024-04-26T22:55:31.411983+0000","last_undegraded":"2024-04-26T22:55:31.411983+0000","last_fullsized":"2024-04-26T22:55:31.411983+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:13.829733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.644303+0000","last_change":"2024-04-26T22:55:24.781446+0000","last_active":"2024-04-26T22:55:53.644303+0000","last_peered":"2024-04-26T22:55:53.644303+0000","last_clean":"2024-04-26T22:55:53.644303+0000","last_became_active":"2024-04-26T22:55:24.781098+0000","last_became_peered":"2024-04-26T22:55:24.781098+0000","last_unstale":"2024-04-26T22:55:53.644303+0000","last_undegraded":"2024-04-26T22:55:53.644303+0000","last_fullsized":"2024-04-26T22:55:53.644303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:40.939755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.818603+0000","last_change":"2024-04-26T22:55:25.394147+0000","last_active":"2024-04-26T22:55:32.818603+0000","last_peered":"2024-04-26T22:55:32.818603+0000","last_clean":"2024-04-26T22:55:32.818603+0000","last_became_active":"2024-04-26T22:55:25.393654+0000","last_became_peered":"2024-04-26T22:55:25.393654+0000","last_unstale":"2024-04-26T22:55:32.818603+0000","last_undegraded":"2024-04-26T22:55:32.818603+0000","last_fullsized":"2024-04-26T22:55:32.818603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:16.335027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.803903+0000","last_change":"2024-04-26T22:55:30.431692+0000","last_active":"2024-04-26T22:55:32.803903+0000","last_peered":"2024-04-26T22:55:32.803903+0000","last_clean":"2024-04-26T22:55:32.803903+0000","last_became_active":"2024-04-26T22:55:30.431380+0000","last_became_peered":"2024-04-26T22:55:30.431380+0000","last_unstale":"2024-04-26T22:55:32.803903+0000","last_undegraded":"2024-04-26T22:55:32.803903+0000","last_fullsized":"2024-04-26T22:55:32.803903+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:14.556048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:52.623594+0000","last_change":"2024-04-26T22:55:24.788121+0000","last_active":"2024-04-26T22:55:52.623594+0000","last_peered":"2024-04-26T22:55:52.623594+0000","last_clean":"2024-04-26T22:55:52.623594+0000","last_became_active":"2024-04-26T22:55:24.787759+0000","last_became_peered":"2024-04-26T22:55:24.787759+0000","last_unstale":"2024-04-26T22:55:52.623594+0000","last_undegraded":"2024-04-26T22:55:52.623594+0000","last_fullsized":"2024-04-26T22:55:52.623594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:49.734675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.962118+0000","last_change":"2024-04-26T22:55:25.377576+0000","last_active":"2024-04-26T22:55:31.962118+0000","last_peered":"2024-04-26T22:55:31.962118+0000","last_clean":"2024-04-26T22:55:31.962118+0000","last_became_active":"2024-04-26T22:55:25.377203+0000","last_became_peered":"2024-04-26T22:55:25.377203+0000","last_unstale":"2024-04-26T22:55:31.962118+0000","last_undegraded":"2024-04-26T22:55:31.962118+0000","last_fullsized":"2024-04-26T22:55:31.962118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:39.361118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.414737+0000","last_change":"2024-04-26T22:55:30.436672+0000","last_active":"2024-04-26T22:55:31.414737+0000","last_peered":"2024-04-26T22:55:31.414737+0000","last_clean":"2024-04-26T22:55:31.414737+0000","last_became_active":"2024-04-26T22:55:30.435178+0000","last_became_peered":"2024-04-26T22:55:30.435178+0000","last_unstale":"2024-04-26T22:55:31.414737+0000","last_undegraded":"2024-04-26T22:55:31.414737+0000","last_fullsized":"2024-04-26T22:55:31.414737+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:34.435290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:55.659807+0000","last_change":"2024-04-26T22:55:24.760965+0000","last_active":"2024-04-26T22:55:55.659807+0000","last_peered":"2024-04-26T22:55:55.659807+0000","last_clean":"2024-04-26T22:55:55.659807+0000","last_became_active":"2024-04-26T22:55:24.760589+0000","last_became_peered":"2024-04-26T22:55:24.760589+0000","last_unstale":"2024-04-26T22:55:55.659807+0000","last_undegraded":"2024-04-26T22:55:55.659807+0000","last_fullsized":"2024-04-26T22:55:55.659807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:25.493209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.961939+0000","last_change":"2024-04-26T22:55:25.375433+0000","last_active":"2024-04-26T22:55:31.961939+0000","last_peered":"2024-04-26T22:55:31.961939+0000","last_clean":"2024-04-26T22:55:31.961939+0000","last_became_active":"2024-04-26T22:55:25.375062+0000","last_became_peered":"2024-04-26T22:55:25.375062+0000","last_unstale":"2024-04-26T22:55:31.961939+0000","last_undegraded":"2024-04-26T22:55:31.961939+0000","last_fullsized":"2024-04-26T22:55:31.961939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:55.423666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.820568+0000","last_change":"2024-04-26T22:55:30.430002+0000","last_active":"2024-04-26T22:55:32.820568+0000","last_peered":"2024-04-26T22:55:32.820568+0000","last_clean":"2024-04-26T22:55:32.820568+0000","last_became_active":"2024-04-26T22:55:30.427608+0000","last_became_peered":"2024-04-26T22:55:30.427608+0000","last_unstale":"2024-04-26T22:55:32.820568+0000","last_undegraded":"2024-04-26T22:55:32.820568+0000","last_fullsized":"2024-04-26T22:55:32.820568+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:23.286696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.645464+0000","last_change":"2024-04-26T22:55:24.773551+0000","last_active":"2024-04-26T22:55:53.645464+0000","last_peered":"2024-04-26T22:55:53.645464+0000","last_clean":"2024-04-26T22:55:53.645464+0000","last_became_active":"2024-04-26T22:55:24.773202+0000","last_became_peered":"2024-04-26T22:55:24.773202+0000","last_unstale":"2024-04-26T22:55:53.645464+0000","last_undegraded":"2024-04-26T22:55:53.645464+0000","last_fullsized":"2024-04-26T22:55:53.645464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:08.271682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411051+0000","last_change":"2024-04-26T22:55:25.372954+0000","last_active":"2024-04-26T22:55:31.411051+0000","last_peered":"2024-04-26T22:55:31.411051+0000","last_clean":"2024-04-26T22:55:31.411051+0000","last_became_active":"2024-04-26T22:55:25.372625+0000","last_became_peered":"2024-04-26T22:55:25.372625+0000","last_unstale":"2024-04-26T22:55:31.411051+0000","last_undegraded":"2024-04-26T22:55:31.411051+0000","last_fullsized":"2024-04-26T22:55:31.411051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:57.325359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411505+0000","last_change":"2024-04-26T22:55:30.422835+0000","last_active":"2024-04-26T22:55:31.411505+0000","last_peered":"2024-04-26T22:55:31.411505+0000","last_clean":"2024-04-26T22:55:31.411505+0000","last_became_active":"2024-04-26T22:55:30.422349+0000","last_became_peered":"2024-04-26T22:55:30.422349+0000","last_unstale":"2024-04-26T22:55:31.411505+0000","last_undegraded":"2024-04-26T22:55:31.411505+0000","last_fullsized":"2024-04-26T22:55:31.411505+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:26:33.613861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:54.655057+0000","last_change":"2024-04-26T22:55:24.756236+0000","last_active":"2024-04-26T22:55:54.655057+0000","last_peered":"2024-04-26T22:55:54.655057+0000","last_clean":"2024-04-26T22:55:54.655057+0000","last_became_active":"2024-04-26T22:55:24.755858+0000","last_became_peered":"2024-04-26T22:55:24.755858+0000","last_unstale":"2024-04-26T22:55:54.655057+0000","last_undegraded":"2024-04-26T22:55:54.655057+0000","last_fullsized":"2024-04-26T22:55:54.655057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:55.257152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.803863+0000","last_change":"2024-04-26T22:55:25.374367+0000","last_active":"2024-04-26T22:55:32.803863+0000","last_peered":"2024-04-26T22:55:32.803863+0000","last_clean":"2024-04-26T22:55:32.803863+0000","last_became_active":"2024-04-26T22:55:25.373186+0000","last_became_peered":"2024-04-26T22:55:25.373186+0000","last_unstale":"2024-04-26T22:55:32.803863+0000","last_undegraded":"2024-04-26T22:55:32.803863+0000","last_fullsized":"2024-04-26T22:55:32.803863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:31:55.780924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.855081+0000","last_change":"2024-04-26T22:55:30.425818+0000","last_active":"2024-04-26T22:55:31.855081+0000","last_peered":"2024-04-26T22:55:31.855081+0000","last_clean":"2024-04-26T22:55:31.855081+0000","last_became_active":"2024-04-26T22:55:30.425400+0000","last_became_peered":"2024-04-26T22:55:30.425400+0000","last_unstale":"2024-04-26T22:55:31.855081+0000","last_undegraded":"2024-04-26T22:55:31.855081+0000","last_fullsized":"2024-04-26T22:55:31.855081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:36.856545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.413862+0000","last_change":"2024-04-26T22:55:24.780607+0000","last_active":"2024-04-26T22:55:31.413862+0000","last_peered":"2024-04-26T22:55:31.413862+0000","last_clean":"2024-04-26T22:55:31.413862+0000","last_became_active":"2024-04-26T22:55:24.780271+0000","last_became_peered":"2024-04-26T22:55:24.780271+0000","last_unstale":"2024-04-26T22:55:31.413862+0000","last_undegraded":"2024-04-26T22:55:31.413862+0000","last_fullsized":"2024-04-26T22:55:31.413862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:25.057816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.012806+0000","last_change":"2024-04-26T22:55:25.399709+0000","last_active":"2024-04-26T22:55:32.012806+0000","last_peered":"2024-04-26T22:55:32.012806+0000","last_clean":"2024-04-26T22:55:32.012806+0000","last_became_active":"2024-04-26T22:55:25.399392+0000","last_became_peered":"2024-04-26T22:55:25.399392+0000","last_unstale":"2024-04-26T22:55:32.012806+0000","last_undegraded":"2024-04-26T22:55:32.012806+0000","last_fullsized":"2024-04-26T22:55:32.012806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:22.821809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.819736+0000","last_change":"2024-04-26T22:55:30.424070+0000","last_active":"2024-04-26T22:55:32.819736+0000","last_peered":"2024-04-26T22:55:32.819736+0000","last_clean":"2024-04-26T22:55:32.819736+0000","last_became_active":"2024-04-26T22:55:30.423637+0000","last_became_peered":"2024-04-26T22:55:30.423637+0000","last_unstale":"2024-04-26T22:55:32.819736+0000","last_undegraded":"2024-04-26T22:55:32.819736+0000","last_fullsized":"2024-04-26T22:55:32.819736+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:48.020079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.962035+0000","last_change":"2024-04-26T22:55:24.771881+0000","last_active":"2024-04-26T22:55:31.962035+0000","last_peered":"2024-04-26T22:55:31.962035+0000","last_clean":"2024-04-26T22:55:31.962035+0000","last_became_active":"2024-04-26T22:55:24.771563+0000","last_became_peered":"2024-04-26T22:55:24.771563+0000","last_unstale":"2024-04-26T22:55:31.962035+0000","last_undegraded":"2024-04-26T22:55:31.962035+0000","last_fullsized":"2024-04-26T22:55:31.962035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:14.306677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.012535+0000","last_change":"2024-04-26T22:55:25.399789+0000","last_active":"2024-04-26T22:55:32.012535+0000","last_peered":"2024-04-26T22:55:32.012535+0000","last_clean":"2024-04-26T22:55:32.012535+0000","last_became_active":"2024-04-26T22:55:25.399493+0000","last_became_peered":"2024-04-26T22:55:25.399493+0000","last_unstale":"2024-04-26T22:55:32.012535+0000","last_undegraded":"2024-04-26T22:55:32.012535+0000","last_fullsized":"2024-04-26T22:55:32.012535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:45.096556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.820271+0000","last_change":"2024-04-26T22:55:30.428107+0000","last_active":"2024-04-26T22:55:32.820271+0000","last_peered":"2024-04-26T22:55:32.820271+0000","last_clean":"2024-04-26T22:55:32.820271+0000","last_became_active":"2024-04-26T22:55:30.421955+0000","last_became_peered":"2024-04-26T22:55:30.421955+0000","last_unstale":"2024-04-26T22:55:32.820271+0000","last_undegraded":"2024-04-26T22:55:32.820271+0000","last_fullsized":"2024-04-26T22:55:32.820271+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:52.430809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411204+0000","last_change":"2024-04-26T22:55:25.373730+0000","last_active":"2024-04-26T22:55:31.411204+0000","last_peered":"2024-04-26T22:55:31.411204+0000","last_clean":"2024-04-26T22:55:31.411204+0000","last_became_active":"2024-04-26T22:55:25.373416+0000","last_became_peered":"2024-04-26T22:55:25.373416+0000","last_unstale":"2024-04-26T22:55:31.411204+0000","last_undegraded":"2024-04-26T22:55:31.411204+0000","last_fullsized":"2024-04-26T22:55:31.411204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:31:44.146244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.649853+0000","last_change":"2024-04-26T22:55:24.763998+0000","last_active":"2024-04-26T22:55:53.649853+0000","last_peered":"2024-04-26T22:55:53.649853+0000","last_clean":"2024-04-26T22:55:53.649853+0000","last_became_active":"2024-04-26T22:55:24.763561+0000","last_became_peered":"2024-04-26T22:55:24.763561+0000","last_unstale":"2024-04-26T22:55:53.649853+0000","last_undegraded":"2024-04-26T22:55:53.649853+0000","last_fullsized":"2024-04-26T22:55:53.649853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:25.270310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.962227+0000","last_change":"2024-04-26T22:55:30.424930+0000","last_active":"2024-04-26T22:55:31.962227+0000","last_peered":"2024-04-26T22:55:31.962227+0000","last_clean":"2024-04-26T22:55:31.962227+0000","last_became_active":"2024-04-26T22:55:30.424660+0000","last_became_peered":"2024-04-26T22:55:30.424660+0000","last_unstale":"2024-04-26T22:55:31.962227+0000","last_undegraded":"2024-04-26T22:55:31.962227+0000","last_fullsized":"2024-04-26T22:55:31.962227+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:29.483956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.815853+0000","last_change":"2024-04-26T22:55:25.390547+0000","last_active":"2024-04-26T22:55:32.815853+0000","last_peered":"2024-04-26T22:55:32.815853+0000","last_clean":"2024-04-26T22:55:32.815853+0000","last_became_active":"2024-04-26T22:55:25.386566+0000","last_became_peered":"2024-04-26T22:55:25.386566+0000","last_unstale":"2024-04-26T22:55:32.815853+0000","last_undegraded":"2024-04-26T22:55:32.815853+0000","last_fullsized":"2024-04-26T22:55:32.815853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:43.809125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:54.654143+0000","last_change":"2024-04-26T22:55:24.778254+0000","last_active":"2024-04-26T22:55:54.654143+0000","last_peered":"2024-04-26T22:55:54.654143+0000","last_clean":"2024-04-26T22:55:54.654143+0000","last_became_active":"2024-04-26T22:55:24.777850+0000","last_became_peered":"2024-04-26T22:55:24.777850+0000","last_unstale":"2024-04-26T22:55:54.654143+0000","last_undegraded":"2024-04-26T22:55:54.654143+0000","last_fullsized":"2024-04-26T22:55:54.654143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:48.053712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.961765+0000","last_change":"2024-04-26T22:53:59.233875+0000","last_active":"2024-04-26T22:55:31.961765+0000","last_peered":"2024-04-26T22:55:31.961765+0000","last_clean":"2024-04-26T22:55:31.961765+0000","last_became_active":"2024-04-26T22:53:59.233488+0000","last_became_peered":"2024-04-26T22:53:59.233488+0000","last_unstale":"2024-04-26T22:55:31.961765+0000","last_undegraded":"2024-04-26T22:55:31.961765+0000","last_fullsized":"2024-04-26T22:55:31.961765+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:49:55.319714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:49:55.319714+0000","last_clean_scrub_stamp":"2024-04-26T22:49:55.319714+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:37:32.885519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411660+0000","last_change":"2024-04-26T22:55:30.427936+0000","last_active":"2024-04-26T22:55:31.411660+0000","last_peered":"2024-04-26T22:55:31.411660+0000","last_clean":"2024-04-26T22:55:31.411660+0000","last_became_active":"2024-04-26T22:55:30.427483+0000","last_became_peered":"2024-04-26T22:55:30.427483+0000","last_unstale":"2024-04-26T22:55:31.411660+0000","last_undegraded":"2024-04-26T22:55:31.411660+0000","last_fullsized":"2024-04-26T22:55:31.411660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:00.856153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.817632+0000","last_change":"2024-04-26T22:55:25.378277+0000","last_active":"2024-04-26T22:55:32.817632+0000","last_peered":"2024-04-26T22:55:32.817632+0000","last_clean":"2024-04-26T22:55:32.817632+0000","last_became_active":"2024-04-26T22:55:25.377650+0000","last_became_peered":"2024-04-26T22:55:25.377650+0000","last_unstale":"2024-04-26T22:55:32.817632+0000","last_undegraded":"2024-04-26T22:55:32.817632+0000","last_fullsized":"2024-04-26T22:55:32.817632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:47.537306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.643559+0000","last_change":"2024-04-26T22:55:24.762439+0000","last_active":"2024-04-26T22:55:53.643559+0000","last_peered":"2024-04-26T22:55:53.643559+0000","last_clean":"2024-04-26T22:55:53.643559+0000","last_became_active":"2024-04-26T22:55:24.760732+0000","last_became_peered":"2024-04-26T22:55:24.760732+0000","last_unstale":"2024-04-26T22:55:53.643559+0000","last_undegraded":"2024-04-26T22:55:53.643559+0000","last_fullsized":"2024-04-26T22:55:53.643559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:33.857025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.803857+0000","last_change":"2024-04-26T22:55:30.411883+0000","last_active":"2024-04-26T22:55:32.803857+0000","last_peered":"2024-04-26T22:55:32.803857+0000","last_clean":"2024-04-26T22:55:32.803857+0000","last_became_active":"2024-04-26T22:55:30.411419+0000","last_became_peered":"2024-04-26T22:55:30.411419+0000","last_unstale":"2024-04-26T22:55:32.803857+0000","last_undegraded":"2024-04-26T22:55:32.803857+0000","last_fullsized":"2024-04-26T22:55:32.803857+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:14.977947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:52.624023+0000","last_change":"2024-04-26T22:55:24.779481+0000","last_active":"2024-04-26T22:55:52.624023+0000","last_peered":"2024-04-26T22:55:52.624023+0000","last_clean":"2024-04-26T22:55:52.624023+0000","last_became_active":"2024-04-26T22:55:24.779182+0000","last_became_peered":"2024-04-26T22:55:24.779182+0000","last_unstale":"2024-04-26T22:55:52.624023+0000","last_undegraded":"2024-04-26T22:55:52.624023+0000","last_fullsized":"2024-04-26T22:55:52.624023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:52.076060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.820035+0000","last_change":"2024-04-26T22:55:25.390938+0000","last_active":"2024-04-26T22:55:32.820035+0000","last_peered":"2024-04-26T22:55:32.820035+0000","last_clean":"2024-04-26T22:55:32.820035+0000","last_became_active":"2024-04-26T22:55:25.390566+0000","last_became_peered":"2024-04-26T22:55:25.390566+0000","last_unstale":"2024-04-26T22:55:32.820035+0000","last_undegraded":"2024-04-26T22:55:32.820035+0000","last_fullsized":"2024-04-26T22:55:32.820035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:48.079024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.815850+0000","last_change":"2024-04-26T22:55:30.430926+0000","last_active":"2024-04-26T22:55:32.815850+0000","last_peered":"2024-04-26T22:55:32.815850+0000","last_clean":"2024-04-26T22:55:32.815850+0000","last_became_active":"2024-04-26T22:55:30.428286+0000","last_became_peered":"2024-04-26T22:55:30.428286+0000","last_unstale":"2024-04-26T22:55:32.815850+0000","last_undegraded":"2024-04-26T22:55:32.815850+0000","last_fullsized":"2024-04-26T22:55:32.815850+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:26.095658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.647318+0000","last_change":"2024-04-26T22:55:24.784960+0000","last_active":"2024-04-26T22:55:53.647318+0000","last_peered":"2024-04-26T22:55:53.647318+0000","last_clean":"2024-04-26T22:55:53.647318+0000","last_became_active":"2024-04-26T22:55:24.784550+0000","last_became_peered":"2024-04-26T22:55:24.784550+0000","last_unstale":"2024-04-26T22:55:53.647318+0000","last_undegraded":"2024-04-26T22:55:53.647318+0000","last_fullsized":"2024-04-26T22:55:53.647318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:05.058172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.817222+0000","last_change":"2024-04-26T22:55:25.392352+0000","last_active":"2024-04-26T22:55:32.817222+0000","last_peered":"2024-04-26T22:55:32.817222+0000","last_clean":"2024-04-26T22:55:32.817222+0000","last_became_active":"2024-04-26T22:55:25.391983+0000","last_became_peered":"2024-04-26T22:55:25.391983+0000","last_unstale":"2024-04-26T22:55:32.817222+0000","last_undegraded":"2024-04-26T22:55:32.817222+0000","last_fullsized":"2024-04-26T22:55:32.817222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:37.192419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.813510+0000","last_change":"2024-04-26T22:55:30.409951+0000","last_active":"2024-04-26T22:55:32.813510+0000","last_peered":"2024-04-26T22:55:32.813510+0000","last_clean":"2024-04-26T22:55:32.813510+0000","last_became_active":"2024-04-26T22:55:30.409502+0000","last_became_peered":"2024-04-26T22:55:30.409502+0000","last_unstale":"2024-04-26T22:55:32.813510+0000","last_undegraded":"2024-04-26T22:55:32.813510+0000","last_fullsized":"2024-04-26T22:55:32.813510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:40:48.416697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:52.621372+0000","last_change":"2024-04-26T22:55:24.770999+0000","last_active":"2024-04-26T22:55:52.621372+0000","last_peered":"2024-04-26T22:55:52.621372+0000","last_clean":"2024-04-26T22:55:52.621372+0000","last_became_active":"2024-04-26T22:55:24.770630+0000","last_became_peered":"2024-04-26T22:55:24.770630+0000","last_unstale":"2024-04-26T22:55:52.621372+0000","last_undegraded":"2024-04-26T22:55:52.621372+0000","last_fullsized":"2024-04-26T22:55:52.621372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:01.651705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.597311+0000","last_change":"2024-04-26T22:55:25.401845+0000","last_active":"2024-04-26T22:55:31.597311+0000","last_peered":"2024-04-26T22:55:31.597311+0000","last_clean":"2024-04-26T22:55:31.597311+0000","last_became_active":"2024-04-26T22:55:25.398759+0000","last_became_peered":"2024-04-26T22:55:25.398759+0000","last_unstale":"2024-04-26T22:55:31.597311+0000","last_undegraded":"2024-04-26T22:55:31.597311+0000","last_fullsized":"2024-04-26T22:55:31.597311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:14.158710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412403+0000","last_change":"2024-04-26T22:55:30.426206+0000","last_active":"2024-04-26T22:55:31.412403+0000","last_peered":"2024-04-26T22:55:31.412403+0000","last_clean":"2024-04-26T22:55:31.412403+0000","last_became_active":"2024-04-26T22:55:30.425678+0000","last_became_peered":"2024-04-26T22:55:30.425678+0000","last_unstale":"2024-04-26T22:55:31.412403+0000","last_undegraded":"2024-04-26T22:55:31.412403+0000","last_fullsized":"2024-04-26T22:55:31.412403+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:48.913899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:52.619236+0000","last_change":"2024-04-26T22:55:24.772775+0000","last_active":"2024-04-26T22:55:52.619236+0000","last_peered":"2024-04-26T22:55:52.619236+0000","last_clean":"2024-04-26T22:55:52.619236+0000","last_became_active":"2024-04-26T22:55:24.772434+0000","last_became_peered":"2024-04-26T22:55:24.772434+0000","last_unstale":"2024-04-26T22:55:52.619236+0000","last_undegraded":"2024-04-26T22:55:52.619236+0000","last_fullsized":"2024-04-26T22:55:52.619236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:23.309545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.854873+0000","last_change":"2024-04-26T22:55:25.372051+0000","last_active":"2024-04-26T22:55:31.854873+0000","last_peered":"2024-04-26T22:55:31.854873+0000","last_clean":"2024-04-26T22:55:31.854873+0000","last_became_active":"2024-04-26T22:55:25.371681+0000","last_became_peered":"2024-04-26T22:55:25.371681+0000","last_unstale":"2024-04-26T22:55:31.854873+0000","last_undegraded":"2024-04-26T22:55:31.854873+0000","last_fullsized":"2024-04-26T22:55:31.854873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:34:39.412974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.414753+0000","last_change":"2024-04-26T22:55:30.427160+0000","last_active":"2024-04-26T22:55:31.414753+0000","last_peered":"2024-04-26T22:55:31.414753+0000","last_clean":"2024-04-26T22:55:31.414753+0000","last_became_active":"2024-04-26T22:55:30.426555+0000","last_became_peered":"2024-04-26T22:55:30.426555+0000","last_unstale":"2024-04-26T22:55:31.414753+0000","last_undegraded":"2024-04-26T22:55:31.414753+0000","last_fullsized":"2024-04-26T22:55:31.414753+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:31.688532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:52.624396+0000","last_change":"2024-04-26T22:55:24.785265+0000","last_active":"2024-04-26T22:55:52.624396+0000","last_peered":"2024-04-26T22:55:52.624396+0000","last_clean":"2024-04-26T22:55:52.624396+0000","last_became_active":"2024-04-26T22:55:24.784896+0000","last_became_peered":"2024-04-26T22:55:24.784896+0000","last_unstale":"2024-04-26T22:55:52.624396+0000","last_undegraded":"2024-04-26T22:55:52.624396+0000","last_fullsized":"2024-04-26T22:55:52.624396+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:28.815337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.803575+0000","last_change":"2024-04-26T22:55:25.399606+0000","last_active":"2024-04-26T22:55:32.803575+0000","last_peered":"2024-04-26T22:55:32.803575+0000","last_clean":"2024-04-26T22:55:32.803575+0000","last_became_active":"2024-04-26T22:55:25.399195+0000","last_became_peered":"2024-04-26T22:55:25.399195+0000","last_unstale":"2024-04-26T22:55:32.803575+0000","last_undegraded":"2024-04-26T22:55:32.803575+0000","last_fullsized":"2024-04-26T22:55:32.803575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:53:53.618431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411952+0000","last_change":"2024-04-26T22:55:30.427381+0000","last_active":"2024-04-26T22:55:31.411952+0000","last_peered":"2024-04-26T22:55:31.411952+0000","last_clean":"2024-04-26T22:55:31.411952+0000","last_became_active":"2024-04-26T22:55:30.426736+0000","last_became_peered":"2024-04-26T22:55:30.426736+0000","last_unstale":"2024-04-26T22:55:31.411952+0000","last_undegraded":"2024-04-26T22:55:31.411952+0000","last_fullsized":"2024-04-26T22:55:31.411952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:07.106991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.812991+0000","last_change":"2024-04-26T22:55:24.781483+0000","last_active":"2024-04-26T22:55:32.812991+0000","last_peered":"2024-04-26T22:55:32.812991+0000","last_clean":"2024-04-26T22:55:32.812991+0000","last_became_active":"2024-04-26T22:55:24.781174+0000","last_became_peered":"2024-04-26T22:55:24.781174+0000","last_unstale":"2024-04-26T22:55:32.812991+0000","last_undegraded":"2024-04-26T22:55:32.812991+0000","last_fullsized":"2024-04-26T22:55:32.812991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:58:05.550242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.816238+0000","last_change":"2024-04-26T22:55:25.406764+0000","last_active":"2024-04-26T22:55:32.816238+0000","last_peered":"2024-04-26T22:55:32.816238+0000","last_clean":"2024-04-26T22:55:32.816238+0000","last_became_active":"2024-04-26T22:55:25.405948+0000","last_became_peered":"2024-04-26T22:55:25.405948+0000","last_unstale":"2024-04-26T22:55:32.816238+0000","last_undegraded":"2024-04-26T22:55:32.816238+0000","last_fullsized":"2024-04-26T22:55:32.816238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:58.414530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412300+0000","last_change":"2024-04-26T22:55:30.422989+0000","last_active":"2024-04-26T22:55:31.412300+0000","last_peered":"2024-04-26T22:55:31.412300+0000","last_clean":"2024-04-26T22:55:31.412300+0000","last_became_active":"2024-04-26T22:55:30.422529+0000","last_became_peered":"2024-04-26T22:55:30.422529+0000","last_unstale":"2024-04-26T22:55:31.412300+0000","last_undegraded":"2024-04-26T22:55:31.412300+0000","last_fullsized":"2024-04-26T22:55:31.412300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:08.312667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:52.618832+0000","last_change":"2024-04-26T22:55:24.763338+0000","last_active":"2024-04-26T22:55:52.618832+0000","last_peered":"2024-04-26T22:55:52.618832+0000","last_clean":"2024-04-26T22:55:52.618832+0000","last_became_active":"2024-04-26T22:55:24.762975+0000","last_became_peered":"2024-04-26T22:55:24.762975+0000","last_unstale":"2024-04-26T22:55:52.618832+0000","last_undegraded":"2024-04-26T22:55:52.618832+0000","last_fullsized":"2024-04-26T22:55:52.618832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:45.529970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.962137+0000","last_change":"2024-04-26T22:55:25.371428+0000","last_active":"2024-04-26T22:55:31.962137+0000","last_peered":"2024-04-26T22:55:31.962137+0000","last_clean":"2024-04-26T22:55:31.962137+0000","last_became_active":"2024-04-26T22:55:25.371088+0000","last_became_peered":"2024-04-26T22:55:25.371088+0000","last_unstale":"2024-04-26T22:55:31.962137+0000","last_undegraded":"2024-04-26T22:55:31.962137+0000","last_fullsized":"2024-04-26T22:55:31.962137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:55:15.066665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.814181+0000","last_change":"2024-04-26T22:55:30.411966+0000","last_active":"2024-04-26T22:55:32.814181+0000","last_peered":"2024-04-26T22:55:32.814181+0000","last_clean":"2024-04-26T22:55:32.814181+0000","last_became_active":"2024-04-26T22:55:30.411486+0000","last_became_peered":"2024-04-26T22:55:30.411486+0000","last_unstale":"2024-04-26T22:55:32.814181+0000","last_undegraded":"2024-04-26T22:55:32.814181+0000","last_fullsized":"2024-04-26T22:55:32.814181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:08.207340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.645958+0000","last_change":"2024-04-26T22:55:24.771435+0000","last_active":"2024-04-26T22:55:53.645958+0000","last_peered":"2024-04-26T22:55:53.645958+0000","last_clean":"2024-04-26T22:55:53.645958+0000","last_became_active":"2024-04-26T22:55:24.771156+0000","last_became_peered":"2024-04-26T22:55:24.771156+0000","last_unstale":"2024-04-26T22:55:53.645958+0000","last_undegraded":"2024-04-26T22:55:53.645958+0000","last_fullsized":"2024-04-26T22:55:53.645958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:15.130786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.962476+0000","last_change":"2024-04-26T22:55:25.374051+0000","last_active":"2024-04-26T22:55:31.962476+0000","last_peered":"2024-04-26T22:55:31.962476+0000","last_clean":"2024-04-26T22:55:31.962476+0000","last_became_active":"2024-04-26T22:55:25.373348+0000","last_became_peered":"2024-04-26T22:55:25.373348+0000","last_unstale":"2024-04-26T22:55:31.962476+0000","last_undegraded":"2024-04-26T22:55:31.962476+0000","last_fullsized":"2024-04-26T22:55:31.962476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:30:31.116829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.597427+0000","last_change":"2024-04-26T22:55:30.425497+0000","last_active":"2024-04-26T22:55:31.597427+0000","last_peered":"2024-04-26T22:55:31.597427+0000","last_clean":"2024-04-26T22:55:31.597427+0000","last_became_active":"2024-04-26T22:55:30.425174+0000","last_became_peered":"2024-04-26T22:55:30.425174+0000","last_unstale":"2024-04-26T22:55:31.597427+0000","last_undegraded":"2024-04-26T22:55:31.597427+0000","last_fullsized":"2024-04-26T22:55:31.597427+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:22.555486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411764+0000","last_change":"2024-04-26T22:55:24.788057+0000","last_active":"2024-04-26T22:55:31.411764+0000","last_peered":"2024-04-26T22:55:31.411764+0000","last_clean":"2024-04-26T22:55:31.411764+0000","last_became_active":"2024-04-26T22:55:24.787753+0000","last_became_peered":"2024-04-26T22:55:24.787753+0000","last_unstale":"2024-04-26T22:55:31.411764+0000","last_undegraded":"2024-04-26T22:55:31.411764+0000","last_fullsized":"2024-04-26T22:55:31.411764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:53.054665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.962264+0000","last_change":"2024-04-26T22:55:25.400227+0000","last_active":"2024-04-26T22:55:31.962264+0000","last_peered":"2024-04-26T22:55:31.962264+0000","last_clean":"2024-04-26T22:55:31.962264+0000","last_became_active":"2024-04-26T22:55:25.399738+0000","last_became_peered":"2024-04-26T22:55:25.399738+0000","last_unstale":"2024-04-26T22:55:31.962264+0000","last_undegraded":"2024-04-26T22:55:31.962264+0000","last_fullsized":"2024-04-26T22:55:31.962264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:22.224654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.410867+0000","last_change":"2024-04-26T22:55:30.410479+0000","last_active":"2024-04-26T22:55:31.410867+0000","last_peered":"2024-04-26T22:55:31.410867+0000","last_clean":"2024-04-26T22:55:31.410867+0000","last_became_active":"2024-04-26T22:55:30.410049+0000","last_became_peered":"2024-04-26T22:55:30.410049+0000","last_unstale":"2024-04-26T22:55:31.410867+0000","last_undegraded":"2024-04-26T22:55:31.410867+0000","last_fullsized":"2024-04-26T22:55:31.410867+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:04.412005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.820389+0000","last_change":"2024-04-26T22:55:24.780741+0000","last_active":"2024-04-26T22:55:32.820389+0000","last_peered":"2024-04-26T22:55:32.820389+0000","last_clean":"2024-04-26T22:55:32.820389+0000","last_became_active":"2024-04-26T22:55:24.780400+0000","last_became_peered":"2024-04-26T22:55:24.780400+0000","last_unstale":"2024-04-26T22:55:32.820389+0000","last_undegraded":"2024-04-26T22:55:32.820389+0000","last_fullsized":"2024-04-26T22:55:32.820389+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:52.274456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.962381+0000","last_change":"2024-04-26T22:55:25.369449+0000","last_active":"2024-04-26T22:55:31.962381+0000","last_peered":"2024-04-26T22:55:31.962381+0000","last_clean":"2024-04-26T22:55:31.962381+0000","last_became_active":"2024-04-26T22:55:25.369106+0000","last_became_peered":"2024-04-26T22:55:25.369106+0000","last_unstale":"2024-04-26T22:55:31.962381+0000","last_undegraded":"2024-04-26T22:55:31.962381+0000","last_fullsized":"2024-04-26T22:55:31.962381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:21.347385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.803752+0000","last_change":"2024-04-26T22:55:25.371959+0000","last_active":"2024-04-26T22:55:32.803752+0000","last_peered":"2024-04-26T22:55:32.803752+0000","last_clean":"2024-04-26T22:55:32.803752+0000","last_became_active":"2024-04-26T22:55:25.371580+0000","last_became_peered":"2024-04-26T22:55:25.371580+0000","last_unstale":"2024-04-26T22:55:32.803752+0000","last_undegraded":"2024-04-26T22:55:32.803752+0000","last_fullsized":"2024-04-26T22:55:32.803752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:11.001269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.813650+0000","last_change":"2024-04-26T22:55:25.373979+0000","last_active":"2024-04-26T22:55:32.813650+0000","last_peered":"2024-04-26T22:55:32.813650+0000","last_clean":"2024-04-26T22:55:32.813650+0000","last_became_active":"2024-04-26T22:55:25.373591+0000","last_became_peered":"2024-04-26T22:55:25.373591+0000","last_unstale":"2024-04-26T22:55:32.813650+0000","last_undegraded":"2024-04-26T22:55:32.813650+0000","last_fullsized":"2024-04-26T22:55:32.813650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:44.295357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.813506+0000","last_change":"2024-04-26T22:55:25.398778+0000","last_active":"2024-04-26T22:55:32.813506+0000","last_peered":"2024-04-26T22:55:32.813506+0000","last_clean":"2024-04-26T22:55:32.813506+0000","last_became_active":"2024-04-26T22:55:25.398460+0000","last_became_peered":"2024-04-26T22:55:25.398460+0000","last_unstale":"2024-04-26T22:55:32.813506+0000","last_undegraded":"2024-04-26T22:55:32.813506+0000","last_fullsized":"2024-04-26T22:55:32.813506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.873418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.597218+0000","last_change":"2024-04-26T22:55:25.396548+0000","last_active":"2024-04-26T22:55:31.597218+0000","last_peered":"2024-04-26T22:55:31.597218+0000","last_clean":"2024-04-26T22:55:31.597218+0000","last_became_active":"2024-04-26T22:55:25.395983+0000","last_became_peered":"2024-04-26T22:55:25.395983+0000","last_unstale":"2024-04-26T22:55:31.597218+0000","last_undegraded":"2024-04-26T22:55:31.597218+0000","last_fullsized":"2024-04-26T22:55:31.597218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:21:34.690636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.819811+0000","last_change":"2024-04-26T22:55:25.396742+0000","last_active":"2024-04-26T22:55:32.819811+0000","last_peered":"2024-04-26T22:55:32.819811+0000","last_clean":"2024-04-26T22:55:32.819811+0000","last_became_active":"2024-04-26T22:55:25.396372+0000","last_became_peered":"2024-04-26T22:55:25.396372+0000","last_unstale":"2024-04-26T22:55:32.819811+0000","last_undegraded":"2024-04-26T22:55:32.819811+0000","last_fullsized":"2024-04-26T22:55:32.819811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:32:12.068358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.803895+0000","last_change":"2024-04-26T22:55:25.372059+0000","last_active":"2024-04-26T22:55:32.803895+0000","last_peered":"2024-04-26T22:55:32.803895+0000","last_clean":"2024-04-26T22:55:32.803895+0000","last_became_active":"2024-04-26T22:55:25.371750+0000","last_became_peered":"2024-04-26T22:55:25.371750+0000","last_unstale":"2024-04-26T22:55:32.803895+0000","last_undegraded":"2024-04-26T22:55:32.803895+0000","last_fullsized":"2024-04-26T22:55:32.803895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:10.230585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.814077+0000","last_change":"2024-04-26T22:55:25.374904+0000","last_active":"2024-04-26T22:55:32.814077+0000","last_peered":"2024-04-26T22:55:32.814077+0000","last_clean":"2024-04-26T22:55:32.814077+0000","last_became_active":"2024-04-26T22:55:25.374600+0000","last_became_peered":"2024-04-26T22:55:25.374600+0000","last_unstale":"2024-04-26T22:55:32.814077+0000","last_undegraded":"2024-04-26T22:55:32.814077+0000","last_fullsized":"2024-04-26T22:55:32.814077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:49.124981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.817369+0000","last_change":"2024-04-26T22:55:25.391026+0000","last_active":"2024-04-26T22:55:32.817369+0000","last_peered":"2024-04-26T22:55:32.817369+0000","last_clean":"2024-04-26T22:55:32.817369+0000","last_became_active":"2024-04-26T22:55:25.389836+0000","last_became_peered":"2024-04-26T22:55:25.389836+0000","last_unstale":"2024-04-26T22:55:32.817369+0000","last_undegraded":"2024-04-26T22:55:32.817369+0000","last_fullsized":"2024-04-26T22:55:32.817369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:37.102439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411107+0000","last_change":"2024-04-26T22:55:25.406584+0000","last_active":"2024-04-26T22:55:31.411107+0000","last_peered":"2024-04-26T22:55:31.411107+0000","last_clean":"2024-04-26T22:55:31.411107+0000","last_became_active":"2024-04-26T22:55:25.406352+0000","last_became_peered":"2024-04-26T22:55:25.406352+0000","last_unstale":"2024-04-26T22:55:31.411107+0000","last_undegraded":"2024-04-26T22:55:31.411107+0000","last_fullsized":"2024-04-26T22:55:31.411107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:43.594712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.816766+0000","last_change":"2024-04-26T22:55:25.399578+0000","last_active":"2024-04-26T22:55:32.816766+0000","last_peered":"2024-04-26T22:55:32.816766+0000","last_clean":"2024-04-26T22:55:32.816766+0000","last_became_active":"2024-04-26T22:55:25.396555+0000","last_became_peered":"2024-04-26T22:55:25.396555+0000","last_unstale":"2024-04-26T22:55:32.816766+0000","last_undegraded":"2024-04-26T22:55:32.816766+0000","last_fullsized":"2024-04-26T22:55:32.816766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:07:50.082978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.814186+0000","last_change":"2024-04-26T22:55:25.393034+0000","last_active":"2024-04-26T22:55:32.814186+0000","last_peered":"2024-04-26T22:55:32.814186+0000","last_clean":"2024-04-26T22:55:32.814186+0000","last_became_active":"2024-04-26T22:55:25.392730+0000","last_became_peered":"2024-04-26T22:55:25.392730+0000","last_unstale":"2024-04-26T22:55:32.814186+0000","last_undegraded":"2024-04-26T22:55:32.814186+0000","last_fullsized":"2024-04-26T22:55:32.814186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:11.040311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.819317+0000","last_change":"2024-04-26T22:55:25.398224+0000","last_active":"2024-04-26T22:55:32.819317+0000","last_peered":"2024-04-26T22:55:32.819317+0000","last_clean":"2024-04-26T22:55:32.819317+0000","last_became_active":"2024-04-26T22:55:25.397744+0000","last_became_peered":"2024-04-26T22:55:25.397744+0000","last_unstale":"2024-04-26T22:55:32.819317+0000","last_undegraded":"2024-04-26T22:55:32.819317+0000","last_fullsized":"2024-04-26T22:55:32.819317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:35.107942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.803820+0000","last_change":"2024-04-26T22:55:25.374076+0000","last_active":"2024-04-26T22:55:32.803820+0000","last_peered":"2024-04-26T22:55:32.803820+0000","last_clean":"2024-04-26T22:55:32.803820+0000","last_became_active":"2024-04-26T22:55:25.373736+0000","last_became_peered":"2024-04-26T22:55:25.373736+0000","last_unstale":"2024-04-26T22:55:32.803820+0000","last_undegraded":"2024-04-26T22:55:32.803820+0000","last_fullsized":"2024-04-26T22:55:32.803820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:01:56.096004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282639,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":27142,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":31,"num_read_kb":46,"num_write":136,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":42828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383701,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":490072,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645338,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1088173,"data_compressed":50705,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808864,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":486212,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070503,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":486132,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332139,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1077554,"data_compressed":50705,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":485484,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":489236,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":486872,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":490586,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":485678,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":486429,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":1077238,"data_compressed":50705,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34303,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34303,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34303,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:55:58.738 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:55:58.923 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:58 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1682083791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:58.923 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:58 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/395131158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:58.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.995+0000 7f5236200640 1 -- 172.21.15.27:0/882501178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523010a630 msgr2=0x7f523010cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:58.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.995+0000 7f5236200640 1 --2- 172.21.15.27:0/882501178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523010a630 0x7f523010cad0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5228009ca0 tx=0x7f522802a950 comp rx=0 tx=0).stop 2024-04-26T22:55:58.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.995+0000 7f5236200640 1 -- 172.21.15.27:0/882501178 shutdown_connections 2024-04-26T22:55:58.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.995+0000 7f5236200640 1 --2- 172.21.15.27:0/882501178 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5230075330 0x7f5230075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.995+0000 7f5236200640 1 --2- 172.21.15.27:0/882501178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523010a630 0x7f523010cad0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.995+0000 7f5236200640 1 --2- 172.21.15.27:0/882501178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5230076910 0x7f5230074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.995+0000 7f5236200640 1 -- 172.21.15.27:0/882501178 >> 172.21.15.27:0/882501178 conn(0x7f52300fdd10 msgr2=0x7f5230100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:58.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.995+0000 7f5236200640 1 -- 172.21.15.27:0/882501178 shutdown_connections 2024-04-26T22:55:58.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.996+0000 7f5236200640 1 -- 172.21.15.27:0/882501178 wait complete. 2024-04-26T22:55:58.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.997+0000 7f5236200640 1 Processor -- start 2024-04-26T22:55:58.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.997+0000 7f5236200640 1 -- start start 2024-04-26T22:55:58.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.997+0000 7f5236200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5230075330 0x7f523019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:58.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.997+0000 7f5236200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5230076910 0x7f523019af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:58.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.997+0000 7f5236200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f523010a630 0x7f52301a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.997+0000 7f5236200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5230110520 con 0x7f523010a630 2024-04-26T22:55:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.997+0000 7f5236200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f52301103a0 con 0x7f5230076910 2024-04-26T22:55:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.997+0000 7f5236200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f52301106a0 con 0x7f5230075330 2024-04-26T22:55:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.998+0000 7f5235800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f523010a630 0x7f52301a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.998+0000 7f522fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5230076910 0x7f523019af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.998+0000 7f5234e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5230075330 0x7f523019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.998+0000 7f5234e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5230075330 0x7f523019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:51414/0 (socket says 172.21.15.27:51414) 2024-04-26T22:55:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.998+0000 7f5234e00640 1 -- 172.21.15.27:0/1904118675 learned_addr learned my addr 172.21.15.27:0/1904118675 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:55:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.998+0000 7f5235800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f523010a630 0x7f52301a1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33930/0 (socket says 172.21.15.27:33930) 2024-04-26T22:55:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.998+0000 7f522fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5230076910 0x7f523019af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:60160/0 (socket says 172.21.15.27:60160) 2024-04-26T22:55:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.998+0000 7f5234e00640 1 -- 172.21.15.27:0/1904118675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5230076910 msgr2=0x7f523019af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.998+0000 7f5234e00640 1 --2- 172.21.15.27:0/1904118675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5230076910 0x7f523019af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.998+0000 7f5234e00640 1 -- 172.21.15.27:0/1904118675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f523010a630 msgr2=0x7f52301a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:59.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.998+0000 7f5234e00640 1 --2- 172.21.15.27:0/1904118675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f523010a630 0x7f52301a1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:59.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.998+0000 7f5234e00640 1 -- 172.21.15.27:0/1904118675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52301a2660 con 0x7f5230075330 2024-04-26T22:55:59.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.999+0000 7f5234e00640 1 --2- 172.21.15.27:0/1904118675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5230075330 0x7f523019aa40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f522400e9f0 tx=0x7f522400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:59.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.999+0000 7f522d600640 1 -- 172.21.15.27:0/1904118675 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522400cd80 con 0x7f5230075330 2024-04-26T22:55:59.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.999+0000 7f522d600640 1 -- 172.21.15.27:0/1904118675 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52240108e0 con 0x7f5230075330 2024-04-26T22:55:59.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.999+0000 7f522d600640 1 -- 172.21.15.27:0/1904118675 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224018980 con 0x7f5230075330 2024-04-26T22:55:59.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.999+0000 7f5236200640 1 -- 172.21.15.27:0/1904118675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52301a2950 con 0x7f5230075330 2024-04-26T22:55:59.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:58.999+0000 7f5236200640 1 -- 172.21.15.27:0/1904118675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52301a2ef0 con 0x7f5230075330 2024-04-26T22:55:59.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.000+0000 7f5236200640 1 -- 172.21.15.27:0/1904118675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52301093f0 con 0x7f5230075330 2024-04-26T22:55:59.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.003+0000 7f522d600640 1 -- 172.21.15.27:0/1904118675 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5224010430 con 0x7f5230075330 2024-04-26T22:55:59.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.004+0000 7f522d600640 1 --2- 172.21.15.27:0/1904118675 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f52180a3d70 0x7f52180a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:55:59.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.005+0000 7f522d600640 1 -- 172.21.15.27:0/1904118675 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f52240d0a90 con 0x7f5230075330 2024-04-26T22:55:59.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.005+0000 7f522fe00640 1 --2- 172.21.15.27:0/1904118675 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f52180a3d70 0x7f52180a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:55:59.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.005+0000 7f522fe00640 1 --2- 172.21.15.27:0/1904118675 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f52180a3d70 0x7f52180a6230 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f523019bf50 tx=0x7f521c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:55:59.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.009+0000 7f522d600640 1 -- 172.21.15.27:0/1904118675 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f522409ca80 con 0x7f5230075330 2024-04-26T22:55:59.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:58 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1682083791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:59.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:58 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/395131158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:59.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:58 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1682083791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:55:59.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:58 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/395131158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:55:59.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.218+0000 7f5236200640 1 -- 172.21.15.27:0/1904118675 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5230062d70 con 0x7f52180a3d70 2024-04-26T22:55:59.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.234+0000 7f522d600640 1 -- 172.21.15.27:0/1904118675 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340636 (secure 0 0 0) 0x7f5230062d70 con 0x7f52180a3d70 2024-04-26T22:55:59.243 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:55:59.245 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-26T22:55:59.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.242+0000 7f5236200640 1 -- 172.21.15.27:0/1904118675 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f52180a3d70 msgr2=0x7f52180a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:59.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.242+0000 7f5236200640 1 --2- 172.21.15.27:0/1904118675 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f52180a3d70 0x7f52180a6230 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f523019bf50 tx=0x7f521c005f90 comp rx=0 tx=0).stop 2024-04-26T22:55:59.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.242+0000 7f5236200640 1 -- 172.21.15.27:0/1904118675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5230075330 msgr2=0x7f523019aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:55:59.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.242+0000 7f5236200640 1 --2- 172.21.15.27:0/1904118675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5230075330 0x7f523019aa40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f522400e9f0 tx=0x7f522400eeb0 comp rx=0 tx=0).stop 2024-04-26T22:55:59.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.242+0000 7f5236200640 1 -- 172.21.15.27:0/1904118675 shutdown_connections 2024-04-26T22:55:59.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.242+0000 7f5236200640 1 --2- 172.21.15.27:0/1904118675 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f52180a3d70 0x7f52180a6230 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:59.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.242+0000 7f5236200640 1 --2- 172.21.15.27:0/1904118675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5230076910 0x7f523019af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:59.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.242+0000 7f5236200640 1 --2- 172.21.15.27:0/1904118675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f523010a630 0x7f52301a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:59.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.242+0000 7f5236200640 1 --2- 172.21.15.27:0/1904118675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5230075330 0x7f523019aa40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:55:59.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.242+0000 7f5236200640 1 -- 172.21.15.27:0/1904118675 >> 172.21.15.27:0/1904118675 conn(0x7f52300fdd10 msgr2=0x7f52301065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:55:59.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.242+0000 7f5236200640 1 -- 172.21.15.27:0/1904118675 shutdown_connections 2024-04-26T22:55:59.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:55:59.242+0000 7f5236200640 1 -- 172.21.15.27:0/1904118675 wait complete. 2024-04-26T22:55:59.265 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-04-26T22:55:58.461174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":617510,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":167,"num_read_kb":165,"num_write":317,"num_write_kb":3004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":257,"ondisk_log_size":257,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381772,"kb_used_data":11820,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560052,"statfs":{"total":1151940427776,"available":1151549493248,"internally_reserved":0,"allocated":12103680,"data_stored":7629666,"data_compressed":284946,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22643,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":31,"num_read_kb":46,"num_write":88,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006774"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.804096+0000","last_change":"2024-04-26T22:55:25.404468+0000","last_active":"2024-04-26T22:55:32.804096+0000","last_peered":"2024-04-26T22:55:32.804096+0000","last_clean":"2024-04-26T22:55:32.804096+0000","last_became_active":"2024-04-26T22:55:25.404178+0000","last_became_peered":"2024-04-26T22:55:25.404178+0000","last_unstale":"2024-04-26T22:55:32.804096+0000","last_undegraded":"2024-04-26T22:55:32.804096+0000","last_fullsized":"2024-04-26T22:55:32.804096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:46.249569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.597005+0000","last_change":"2024-04-26T22:55:25.401044+0000","last_active":"2024-04-26T22:55:31.597005+0000","last_peered":"2024-04-26T22:55:31.597005+0000","last_clean":"2024-04-26T22:55:31.597005+0000","last_became_active":"2024-04-26T22:55:25.397925+0000","last_became_peered":"2024-04-26T22:55:25.397925+0000","last_unstale":"2024-04-26T22:55:31.597005+0000","last_undegraded":"2024-04-26T22:55:31.597005+0000","last_fullsized":"2024-04-26T22:55:31.597005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:28.503480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.596799+0000","last_change":"2024-04-26T22:55:25.396780+0000","last_active":"2024-04-26T22:55:31.596799+0000","last_peered":"2024-04-26T22:55:31.596799+0000","last_clean":"2024-04-26T22:55:31.596799+0000","last_became_active":"2024-04-26T22:55:25.395401+0000","last_became_peered":"2024-04-26T22:55:25.395401+0000","last_unstale":"2024-04-26T22:55:31.596799+0000","last_undegraded":"2024-04-26T22:55:31.596799+0000","last_fullsized":"2024-04-26T22:55:31.596799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:32.295042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412115+0000","last_change":"2024-04-26T22:55:25.373994+0000","last_active":"2024-04-26T22:55:31.412115+0000","last_peered":"2024-04-26T22:55:31.412115+0000","last_clean":"2024-04-26T22:55:31.412115+0000","last_became_active":"2024-04-26T22:55:25.373653+0000","last_became_peered":"2024-04-26T22:55:25.373653+0000","last_unstale":"2024-04-26T22:55:31.412115+0000","last_undegraded":"2024-04-26T22:55:31.412115+0000","last_fullsized":"2024-04-26T22:55:31.412115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:58.460947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.854991+0000","last_change":"2024-04-26T22:55:25.391341+0000","last_active":"2024-04-26T22:55:31.854991+0000","last_peered":"2024-04-26T22:55:31.854991+0000","last_clean":"2024-04-26T22:55:31.854991+0000","last_became_active":"2024-04-26T22:55:25.391061+0000","last_became_peered":"2024-04-26T22:55:25.391061+0000","last_unstale":"2024-04-26T22:55:31.854991+0000","last_undegraded":"2024-04-26T22:55:31.854991+0000","last_fullsized":"2024-04-26T22:55:31.854991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:08.391492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.012537+0000","last_change":"2024-04-26T22:55:25.388412+0000","last_active":"2024-04-26T22:55:32.012537+0000","last_peered":"2024-04-26T22:55:32.012537+0000","last_clean":"2024-04-26T22:55:32.012537+0000","last_became_active":"2024-04-26T22:55:25.388010+0000","last_became_peered":"2024-04-26T22:55:25.388010+0000","last_unstale":"2024-04-26T22:55:32.012537+0000","last_undegraded":"2024-04-26T22:55:32.012537+0000","last_fullsized":"2024-04-26T22:55:32.012537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:45.851109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.961851+0000","last_change":"2024-04-26T22:55:25.386174+0000","last_active":"2024-04-26T22:55:31.961851+0000","last_peered":"2024-04-26T22:55:31.961851+0000","last_clean":"2024-04-26T22:55:31.961851+0000","last_became_active":"2024-04-26T22:55:25.385800+0000","last_became_peered":"2024-04-26T22:55:25.385800+0000","last_unstale":"2024-04-26T22:55:31.961851+0000","last_undegraded":"2024-04-26T22:55:31.961851+0000","last_fullsized":"2024-04-26T22:55:31.961851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:55.833414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.816834+0000","last_change":"2024-04-26T22:55:25.408952+0000","last_active":"2024-04-26T22:55:32.816834+0000","last_peered":"2024-04-26T22:55:32.816834+0000","last_clean":"2024-04-26T22:55:32.816834+0000","last_became_active":"2024-04-26T22:55:25.408485+0000","last_became_peered":"2024-04-26T22:55:25.408485+0000","last_unstale":"2024-04-26T22:55:32.816834+0000","last_undegraded":"2024-04-26T22:55:32.816834+0000","last_fullsized":"2024-04-26T22:55:32.816834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.043114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.814577+0000","last_change":"2024-04-26T22:55:25.377377+0000","last_active":"2024-04-26T22:55:32.814577+0000","last_peered":"2024-04-26T22:55:32.814577+0000","last_clean":"2024-04-26T22:55:32.814577+0000","last_became_active":"2024-04-26T22:55:25.377015+0000","last_became_peered":"2024-04-26T22:55:25.377015+0000","last_unstale":"2024-04-26T22:55:32.814577+0000","last_undegraded":"2024-04-26T22:55:32.814577+0000","last_fullsized":"2024-04-26T22:55:32.814577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:25.676469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.012956+0000","last_change":"2024-04-26T22:55:25.375549+0000","last_active":"2024-04-26T22:55:32.012956+0000","last_peered":"2024-04-26T22:55:32.012956+0000","last_clean":"2024-04-26T22:55:32.012956+0000","last_became_active":"2024-04-26T22:55:25.375232+0000","last_became_peered":"2024-04-26T22:55:25.375232+0000","last_unstale":"2024-04-26T22:55:32.012956+0000","last_undegraded":"2024-04-26T22:55:32.012956+0000","last_fullsized":"2024-04-26T22:55:32.012956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:22.561623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.962444+0000","last_change":"2024-04-26T22:55:25.374224+0000","last_active":"2024-04-26T22:55:31.962444+0000","last_peered":"2024-04-26T22:55:31.962444+0000","last_clean":"2024-04-26T22:55:31.962444+0000","last_became_active":"2024-04-26T22:55:25.373813+0000","last_became_peered":"2024-04-26T22:55:25.373813+0000","last_unstale":"2024-04-26T22:55:31.962444+0000","last_undegraded":"2024-04-26T22:55:31.962444+0000","last_fullsized":"2024-04-26T22:55:31.962444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:34.874888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412118+0000","last_change":"2024-04-26T22:55:25.376666+0000","last_active":"2024-04-26T22:55:31.412118+0000","last_peered":"2024-04-26T22:55:31.412118+0000","last_clean":"2024-04-26T22:55:31.412118+0000","last_became_active":"2024-04-26T22:55:25.376328+0000","last_became_peered":"2024-04-26T22:55:25.376328+0000","last_unstale":"2024-04-26T22:55:31.412118+0000","last_undegraded":"2024-04-26T22:55:31.412118+0000","last_fullsized":"2024-04-26T22:55:31.412118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:32:34.287984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.819314+0000","last_change":"2024-04-26T22:55:25.377220+0000","last_active":"2024-04-26T22:55:32.819314+0000","last_peered":"2024-04-26T22:55:32.819314+0000","last_clean":"2024-04-26T22:55:32.819314+0000","last_became_active":"2024-04-26T22:55:25.376637+0000","last_became_peered":"2024-04-26T22:55:25.376637+0000","last_unstale":"2024-04-26T22:55:32.819314+0000","last_undegraded":"2024-04-26T22:55:32.819314+0000","last_fullsized":"2024-04-26T22:55:32.819314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:57.934912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.596020+0000","last_change":"2024-04-26T22:55:25.404669+0000","last_active":"2024-04-26T22:55:31.596020+0000","last_peered":"2024-04-26T22:55:31.596020+0000","last_clean":"2024-04-26T22:55:31.596020+0000","last_became_active":"2024-04-26T22:55:25.402804+0000","last_became_peered":"2024-04-26T22:55:25.402804+0000","last_unstale":"2024-04-26T22:55:31.596020+0000","last_undegraded":"2024-04-26T22:55:31.596020+0000","last_fullsized":"2024-04-26T22:55:31.596020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:37.239667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.596020+0000","last_change":"2024-04-26T22:55:25.399877+0000","last_active":"2024-04-26T22:55:31.596020+0000","last_peered":"2024-04-26T22:55:31.596020+0000","last_clean":"2024-04-26T22:55:31.596020+0000","last_became_active":"2024-04-26T22:55:25.397476+0000","last_became_peered":"2024-04-26T22:55:25.397476+0000","last_unstale":"2024-04-26T22:55:31.596020+0000","last_undegraded":"2024-04-26T22:55:31.596020+0000","last_fullsized":"2024-04-26T22:55:31.596020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:26.732300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.813892+0000","last_change":"2024-04-26T22:55:25.393682+0000","last_active":"2024-04-26T22:55:32.813892+0000","last_peered":"2024-04-26T22:55:32.813892+0000","last_clean":"2024-04-26T22:55:32.813892+0000","last_became_active":"2024-04-26T22:55:25.393349+0000","last_became_peered":"2024-04-26T22:55:25.393349+0000","last_unstale":"2024-04-26T22:55:32.813892+0000","last_undegraded":"2024-04-26T22:55:32.813892+0000","last_fullsized":"2024-04-26T22:55:32.813892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:48.065904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.803972+0000","last_change":"2024-04-26T22:55:25.404449+0000","last_active":"2024-04-26T22:55:32.803972+0000","last_peered":"2024-04-26T22:55:32.803972+0000","last_clean":"2024-04-26T22:55:32.803972+0000","last_became_active":"2024-04-26T22:55:25.404170+0000","last_became_peered":"2024-04-26T22:55:25.404170+0000","last_unstale":"2024-04-26T22:55:32.803972+0000","last_undegraded":"2024-04-26T22:55:32.803972+0000","last_fullsized":"2024-04-26T22:55:32.803972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:04.731208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.414210+0000","last_change":"2024-04-26T22:55:25.390368+0000","last_active":"2024-04-26T22:55:31.414210+0000","last_peered":"2024-04-26T22:55:31.414210+0000","last_clean":"2024-04-26T22:55:31.414210+0000","last_became_active":"2024-04-26T22:55:25.389201+0000","last_became_peered":"2024-04-26T22:55:25.389201+0000","last_unstale":"2024-04-26T22:55:31.414210+0000","last_undegraded":"2024-04-26T22:55:31.414210+0000","last_fullsized":"2024-04-26T22:55:31.414210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:35.985419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.813847+0000","last_change":"2024-04-26T22:55:25.387616+0000","last_active":"2024-04-26T22:55:32.813847+0000","last_peered":"2024-04-26T22:55:32.813847+0000","last_clean":"2024-04-26T22:55:32.813847+0000","last_became_active":"2024-04-26T22:55:25.387360+0000","last_became_peered":"2024-04-26T22:55:25.387360+0000","last_unstale":"2024-04-26T22:55:32.813847+0000","last_undegraded":"2024-04-26T22:55:32.813847+0000","last_fullsized":"2024-04-26T22:55:32.813847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:49.378287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.813775+0000","last_change":"2024-04-26T22:55:30.425407+0000","last_active":"2024-04-26T22:55:32.813775+0000","last_peered":"2024-04-26T22:55:32.813775+0000","last_clean":"2024-04-26T22:55:32.813775+0000","last_became_active":"2024-04-26T22:55:30.424796+0000","last_became_peered":"2024-04-26T22:55:30.424796+0000","last_unstale":"2024-04-26T22:55:32.813775+0000","last_undegraded":"2024-04-26T22:55:32.813775+0000","last_fullsized":"2024-04-26T22:55:32.813775+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:42.539415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.648372+0000","last_change":"2024-04-26T22:55:24.776253+0000","last_active":"2024-04-26T22:55:53.648372+0000","last_peered":"2024-04-26T22:55:53.648372+0000","last_clean":"2024-04-26T22:55:53.648372+0000","last_became_active":"2024-04-26T22:55:24.775700+0000","last_became_peered":"2024-04-26T22:55:24.775700+0000","last_unstale":"2024-04-26T22:55:53.648372+0000","last_undegraded":"2024-04-26T22:55:53.648372+0000","last_fullsized":"2024-04-26T22:55:53.648372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:45.608363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.817143+0000","last_change":"2024-04-26T22:55:25.390690+0000","last_active":"2024-04-26T22:55:32.817143+0000","last_peered":"2024-04-26T22:55:32.817143+0000","last_clean":"2024-04-26T22:55:32.817143+0000","last_became_active":"2024-04-26T22:55:25.390289+0000","last_became_peered":"2024-04-26T22:55:25.390289+0000","last_unstale":"2024-04-26T22:55:32.817143+0000","last_undegraded":"2024-04-26T22:55:32.817143+0000","last_fullsized":"2024-04-26T22:55:32.817143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:54.637992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411801+0000","last_change":"2024-04-26T22:55:30.433555+0000","last_active":"2024-04-26T22:55:31.411801+0000","last_peered":"2024-04-26T22:55:31.411801+0000","last_clean":"2024-04-26T22:55:31.411801+0000","last_became_active":"2024-04-26T22:55:30.433240+0000","last_became_peered":"2024-04-26T22:55:30.433240+0000","last_unstale":"2024-04-26T22:55:31.411801+0000","last_undegraded":"2024-04-26T22:55:31.411801+0000","last_fullsized":"2024-04-26T22:55:31.411801+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:55:21.215073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.815147+0000","last_change":"2024-04-26T22:55:24.788218+0000","last_active":"2024-04-26T22:55:32.815147+0000","last_peered":"2024-04-26T22:55:32.815147+0000","last_clean":"2024-04-26T22:55:32.815147+0000","last_became_active":"2024-04-26T22:55:24.787762+0000","last_became_peered":"2024-04-26T22:55:24.787762+0000","last_unstale":"2024-04-26T22:55:32.815147+0000","last_undegraded":"2024-04-26T22:55:32.815147+0000","last_fullsized":"2024-04-26T22:55:32.815147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:42.264617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.597068+0000","last_change":"2024-04-26T22:55:25.400356+0000","last_active":"2024-04-26T22:55:31.597068+0000","last_peered":"2024-04-26T22:55:31.597068+0000","last_clean":"2024-04-26T22:55:31.597068+0000","last_became_active":"2024-04-26T22:55:25.399907+0000","last_became_peered":"2024-04-26T22:55:25.399907+0000","last_unstale":"2024-04-26T22:55:31.597068+0000","last_undegraded":"2024-04-26T22:55:31.597068+0000","last_fullsized":"2024-04-26T22:55:31.597068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:50:50.804112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411980+0000","last_change":"2024-04-26T22:55:30.426429+0000","last_active":"2024-04-26T22:55:31.411980+0000","last_peered":"2024-04-26T22:55:31.411980+0000","last_clean":"2024-04-26T22:55:31.411980+0000","last_became_active":"2024-04-26T22:55:30.425862+0000","last_became_peered":"2024-04-26T22:55:30.425862+0000","last_unstale":"2024-04-26T22:55:31.411980+0000","last_undegraded":"2024-04-26T22:55:31.411980+0000","last_fullsized":"2024-04-26T22:55:31.411980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:49.646336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.643721+0000","last_change":"2024-04-26T22:55:24.763945+0000","last_active":"2024-04-26T22:55:53.643721+0000","last_peered":"2024-04-26T22:55:53.643721+0000","last_clean":"2024-04-26T22:55:53.643721+0000","last_became_active":"2024-04-26T22:55:24.763580+0000","last_became_peered":"2024-04-26T22:55:24.763580+0000","last_unstale":"2024-04-26T22:55:53.643721+0000","last_undegraded":"2024-04-26T22:55:53.643721+0000","last_fullsized":"2024-04-26T22:55:53.643721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:21:52.298738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.814288+0000","last_change":"2024-04-26T22:55:25.381896+0000","last_active":"2024-04-26T22:55:32.814288+0000","last_peered":"2024-04-26T22:55:32.814288+0000","last_clean":"2024-04-26T22:55:32.814288+0000","last_became_active":"2024-04-26T22:55:25.381411+0000","last_became_peered":"2024-04-26T22:55:25.381411+0000","last_unstale":"2024-04-26T22:55:32.814288+0000","last_undegraded":"2024-04-26T22:55:32.814288+0000","last_fullsized":"2024-04-26T22:55:32.814288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:54.731407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412051+0000","last_change":"2024-04-26T22:55:30.419870+0000","last_active":"2024-04-26T22:55:31.412051+0000","last_peered":"2024-04-26T22:55:31.412051+0000","last_clean":"2024-04-26T22:55:31.412051+0000","last_became_active":"2024-04-26T22:55:30.419533+0000","last_became_peered":"2024-04-26T22:55:30.419533+0000","last_unstale":"2024-04-26T22:55:31.412051+0000","last_undegraded":"2024-04-26T22:55:31.412051+0000","last_fullsized":"2024-04-26T22:55:31.412051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:37:47.647953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.855098+0000","last_change":"2024-04-26T22:55:24.756017+0000","last_active":"2024-04-26T22:55:31.855098+0000","last_peered":"2024-04-26T22:55:31.855098+0000","last_clean":"2024-04-26T22:55:31.855098+0000","last_became_active":"2024-04-26T22:55:24.755661+0000","last_became_peered":"2024-04-26T22:55:24.755661+0000","last_unstale":"2024-04-26T22:55:31.855098+0000","last_undegraded":"2024-04-26T22:55:31.855098+0000","last_fullsized":"2024-04-26T22:55:31.855098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:53.909567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411492+0000","last_change":"2024-04-26T22:55:25.375929+0000","last_active":"2024-04-26T22:55:31.411492+0000","last_peered":"2024-04-26T22:55:31.411492+0000","last_clean":"2024-04-26T22:55:31.411492+0000","last_became_active":"2024-04-26T22:55:25.375435+0000","last_became_peered":"2024-04-26T22:55:25.375435+0000","last_unstale":"2024-04-26T22:55:31.411492+0000","last_undegraded":"2024-04-26T22:55:31.411492+0000","last_fullsized":"2024-04-26T22:55:31.411492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:43.336889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.813934+0000","last_change":"2024-04-26T22:55:30.409074+0000","last_active":"2024-04-26T22:55:32.813934+0000","last_peered":"2024-04-26T22:55:32.813934+0000","last_clean":"2024-04-26T22:55:32.813934+0000","last_became_active":"2024-04-26T22:55:30.408613+0000","last_became_peered":"2024-04-26T22:55:30.408613+0000","last_unstale":"2024-04-26T22:55:32.813934+0000","last_undegraded":"2024-04-26T22:55:32.813934+0000","last_fullsized":"2024-04-26T22:55:32.813934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:24.312971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:54.658023+0000","last_change":"2024-04-26T22:55:24.763787+0000","last_active":"2024-04-26T22:55:54.658023+0000","last_peered":"2024-04-26T22:55:54.658023+0000","last_clean":"2024-04-26T22:55:54.658023+0000","last_became_active":"2024-04-26T22:55:24.763480+0000","last_became_peered":"2024-04-26T22:55:24.763480+0000","last_unstale":"2024-04-26T22:55:54.658023+0000","last_undegraded":"2024-04-26T22:55:54.658023+0000","last_fullsized":"2024-04-26T22:55:54.658023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:18.591009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.855024+0000","last_change":"2024-04-26T22:55:25.388313+0000","last_active":"2024-04-26T22:55:31.855024+0000","last_peered":"2024-04-26T22:55:31.855024+0000","last_clean":"2024-04-26T22:55:31.855024+0000","last_became_active":"2024-04-26T22:55:25.387974+0000","last_became_peered":"2024-04-26T22:55:25.387974+0000","last_unstale":"2024-04-26T22:55:31.855024+0000","last_undegraded":"2024-04-26T22:55:31.855024+0000","last_fullsized":"2024-04-26T22:55:31.855024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:23.237387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.813766+0000","last_change":"2024-04-26T22:55:30.425891+0000","last_active":"2024-04-26T22:55:32.813766+0000","last_peered":"2024-04-26T22:55:32.813766+0000","last_clean":"2024-04-26T22:55:32.813766+0000","last_became_active":"2024-04-26T22:55:30.424742+0000","last_became_peered":"2024-04-26T22:55:30.424742+0000","last_unstale":"2024-04-26T22:55:32.813766+0000","last_undegraded":"2024-04-26T22:55:32.813766+0000","last_fullsized":"2024-04-26T22:55:32.813766+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:04.876670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.646351+0000","last_change":"2024-04-26T22:55:24.771979+0000","last_active":"2024-04-26T22:55:53.646351+0000","last_peered":"2024-04-26T22:55:53.646351+0000","last_clean":"2024-04-26T22:55:53.646351+0000","last_became_active":"2024-04-26T22:55:24.771622+0000","last_became_peered":"2024-04-26T22:55:24.771622+0000","last_unstale":"2024-04-26T22:55:53.646351+0000","last_undegraded":"2024-04-26T22:55:53.646351+0000","last_fullsized":"2024-04-26T22:55:53.646351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:40.210406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411568+0000","last_change":"2024-04-26T22:55:25.399892+0000","last_active":"2024-04-26T22:55:31.411568+0000","last_peered":"2024-04-26T22:55:31.411568+0000","last_clean":"2024-04-26T22:55:31.411568+0000","last_became_active":"2024-04-26T22:55:25.399515+0000","last_became_peered":"2024-04-26T22:55:25.399515+0000","last_unstale":"2024-04-26T22:55:31.411568+0000","last_undegraded":"2024-04-26T22:55:31.411568+0000","last_fullsized":"2024-04-26T22:55:31.411568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.472103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.804079+0000","last_change":"2024-04-26T22:55:30.422932+0000","last_active":"2024-04-26T22:55:32.804079+0000","last_peered":"2024-04-26T22:55:32.804079+0000","last_clean":"2024-04-26T22:55:32.804079+0000","last_became_active":"2024-04-26T22:55:30.422481+0000","last_became_peered":"2024-04-26T22:55:30.422481+0000","last_unstale":"2024-04-26T22:55:32.804079+0000","last_undegraded":"2024-04-26T22:55:32.804079+0000","last_fullsized":"2024-04-26T22:55:32.804079+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:11.469515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.649536+0000","last_change":"2024-04-26T22:55:24.780973+0000","last_active":"2024-04-26T22:55:53.649536+0000","last_peered":"2024-04-26T22:55:53.649536+0000","last_clean":"2024-04-26T22:55:53.649536+0000","last_became_active":"2024-04-26T22:55:24.779810+0000","last_became_peered":"2024-04-26T22:55:24.779810+0000","last_unstale":"2024-04-26T22:55:53.649536+0000","last_undegraded":"2024-04-26T22:55:53.649536+0000","last_fullsized":"2024-04-26T22:55:53.649536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:33.708588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.816600+0000","last_change":"2024-04-26T22:55:25.402067+0000","last_active":"2024-04-26T22:55:32.816600+0000","last_peered":"2024-04-26T22:55:32.816600+0000","last_clean":"2024-04-26T22:55:32.816600+0000","last_became_active":"2024-04-26T22:55:25.401717+0000","last_became_peered":"2024-04-26T22:55:25.401717+0000","last_unstale":"2024-04-26T22:55:32.816600+0000","last_undegraded":"2024-04-26T22:55:32.816600+0000","last_fullsized":"2024-04-26T22:55:32.816600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:16.260620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.597202+0000","last_change":"2024-04-26T22:55:30.429430+0000","last_active":"2024-04-26T22:55:31.597202+0000","last_peered":"2024-04-26T22:55:31.597202+0000","last_clean":"2024-04-26T22:55:31.597202+0000","last_became_active":"2024-04-26T22:55:30.428800+0000","last_became_peered":"2024-04-26T22:55:30.428800+0000","last_unstale":"2024-04-26T22:55:31.597202+0000","last_undegraded":"2024-04-26T22:55:31.597202+0000","last_fullsized":"2024-04-26T22:55:31.597202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:05.501004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411118+0000","last_change":"2024-04-26T22:55:24.765181+0000","last_active":"2024-04-26T22:55:31.411118+0000","last_peered":"2024-04-26T22:55:31.411118+0000","last_clean":"2024-04-26T22:55:31.411118+0000","last_became_active":"2024-04-26T22:55:24.764847+0000","last_became_peered":"2024-04-26T22:55:24.764847+0000","last_unstale":"2024-04-26T22:55:31.411118+0000","last_undegraded":"2024-04-26T22:55:31.411118+0000","last_fullsized":"2024-04-26T22:55:31.411118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:25.949304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412633+0000","last_change":"2024-04-26T22:55:25.401228+0000","last_active":"2024-04-26T22:55:31.412633+0000","last_peered":"2024-04-26T22:55:31.412633+0000","last_clean":"2024-04-26T22:55:31.412633+0000","last_became_active":"2024-04-26T22:55:25.400756+0000","last_became_peered":"2024-04-26T22:55:25.400756+0000","last_unstale":"2024-04-26T22:55:31.412633+0000","last_undegraded":"2024-04-26T22:55:31.412633+0000","last_fullsized":"2024-04-26T22:55:31.412633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:00.220165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.815845+0000","last_change":"2024-04-26T22:55:30.424537+0000","last_active":"2024-04-26T22:55:32.815845+0000","last_peered":"2024-04-26T22:55:32.815845+0000","last_clean":"2024-04-26T22:55:32.815845+0000","last_became_active":"2024-04-26T22:55:30.423262+0000","last_became_peered":"2024-04-26T22:55:30.423262+0000","last_unstale":"2024-04-26T22:55:32.815845+0000","last_undegraded":"2024-04-26T22:55:32.815845+0000","last_fullsized":"2024-04-26T22:55:32.815845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:48.666453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:54.656607+0000","last_change":"2024-04-26T22:55:24.776580+0000","last_active":"2024-04-26T22:55:54.656607+0000","last_peered":"2024-04-26T22:55:54.656607+0000","last_clean":"2024-04-26T22:55:54.656607+0000","last_became_active":"2024-04-26T22:55:24.776143+0000","last_became_peered":"2024-04-26T22:55:24.776143+0000","last_unstale":"2024-04-26T22:55:54.656607+0000","last_undegraded":"2024-04-26T22:55:54.656607+0000","last_fullsized":"2024-04-26T22:55:54.656607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:02.291237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.814567+0000","last_change":"2024-04-26T22:55:25.385645+0000","last_active":"2024-04-26T22:55:32.814567+0000","last_peered":"2024-04-26T22:55:32.814567+0000","last_clean":"2024-04-26T22:55:32.814567+0000","last_became_active":"2024-04-26T22:55:25.385289+0000","last_became_peered":"2024-04-26T22:55:25.385289+0000","last_unstale":"2024-04-26T22:55:32.814567+0000","last_undegraded":"2024-04-26T22:55:32.814567+0000","last_fullsized":"2024-04-26T22:55:32.814567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:34.662117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412619+0000","last_change":"2024-04-26T22:55:30.437915+0000","last_active":"2024-04-26T22:55:31.412619+0000","last_peered":"2024-04-26T22:55:31.412619+0000","last_clean":"2024-04-26T22:55:31.412619+0000","last_became_active":"2024-04-26T22:55:30.437148+0000","last_became_peered":"2024-04-26T22:55:30.437148+0000","last_unstale":"2024-04-26T22:55:31.412619+0000","last_undegraded":"2024-04-26T22:55:31.412619+0000","last_fullsized":"2024-04-26T22:55:31.412619+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:11.647498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:52.617793+0000","last_change":"2024-04-26T22:55:24.763281+0000","last_active":"2024-04-26T22:55:52.617793+0000","last_peered":"2024-04-26T22:55:52.617793+0000","last_clean":"2024-04-26T22:55:52.617793+0000","last_became_active":"2024-04-26T22:55:24.762901+0000","last_became_peered":"2024-04-26T22:55:24.762901+0000","last_unstale":"2024-04-26T22:55:52.617793+0000","last_undegraded":"2024-04-26T22:55:52.617793+0000","last_fullsized":"2024-04-26T22:55:52.617793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:10.092887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.597067+0000","last_change":"2024-04-26T22:55:25.401057+0000","last_active":"2024-04-26T22:55:31.597067+0000","last_peered":"2024-04-26T22:55:31.597067+0000","last_clean":"2024-04-26T22:55:31.597067+0000","last_became_active":"2024-04-26T22:55:25.398265+0000","last_became_peered":"2024-04-26T22:55:25.398265+0000","last_unstale":"2024-04-26T22:55:31.597067+0000","last_undegraded":"2024-04-26T22:55:31.597067+0000","last_fullsized":"2024-04-26T22:55:31.597067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:49.967012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412258+0000","last_change":"2024-04-26T22:55:30.426682+0000","last_active":"2024-04-26T22:55:31.412258+0000","last_peered":"2024-04-26T22:55:31.412258+0000","last_clean":"2024-04-26T22:55:31.412258+0000","last_became_active":"2024-04-26T22:55:30.425918+0000","last_became_peered":"2024-04-26T22:55:30.425918+0000","last_unstale":"2024-04-26T22:55:31.412258+0000","last_undegraded":"2024-04-26T22:55:31.412258+0000","last_fullsized":"2024-04-26T22:55:31.412258+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:44.230083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:52.620416+0000","last_change":"2024-04-26T22:55:24.773497+0000","last_active":"2024-04-26T22:55:52.620416+0000","last_peered":"2024-04-26T22:55:52.620416+0000","last_clean":"2024-04-26T22:55:52.620416+0000","last_became_active":"2024-04-26T22:55:24.773183+0000","last_became_peered":"2024-04-26T22:55:24.773183+0000","last_unstale":"2024-04-26T22:55:52.620416+0000","last_undegraded":"2024-04-26T22:55:52.620416+0000","last_fullsized":"2024-04-26T22:55:52.620416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:58:07.157785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.814353+0000","last_change":"2024-04-26T22:55:25.378420+0000","last_active":"2024-04-26T22:55:32.814353+0000","last_peered":"2024-04-26T22:55:32.814353+0000","last_clean":"2024-04-26T22:55:32.814353+0000","last_became_active":"2024-04-26T22:55:25.378068+0000","last_became_peered":"2024-04-26T22:55:25.378068+0000","last_unstale":"2024-04-26T22:55:32.814353+0000","last_undegraded":"2024-04-26T22:55:32.814353+0000","last_fullsized":"2024-04-26T22:55:32.814353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:10.910541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412908+0000","last_change":"2024-04-26T22:55:30.436973+0000","last_active":"2024-04-26T22:55:31.412908+0000","last_peered":"2024-04-26T22:55:31.412908+0000","last_clean":"2024-04-26T22:55:31.412908+0000","last_became_active":"2024-04-26T22:55:30.436327+0000","last_became_peered":"2024-04-26T22:55:30.436327+0000","last_unstale":"2024-04-26T22:55:31.412908+0000","last_undegraded":"2024-04-26T22:55:31.412908+0000","last_fullsized":"2024-04-26T22:55:31.412908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:36.616328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'19","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:55.660082+0000","last_change":"2024-04-26T22:55:24.760508+0000","last_active":"2024-04-26T22:55:55.660082+0000","last_peered":"2024-04-26T22:55:55.660082+0000","last_clean":"2024-04-26T22:55:55.660082+0000","last_became_active":"2024-04-26T22:55:24.760180+0000","last_became_peered":"2024-04-26T22:55:24.760180+0000","last_unstale":"2024-04-26T22:55:55.660082+0000","last_undegraded":"2024-04-26T22:55:55.660082+0000","last_fullsized":"2024-04-26T22:55:55.660082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:50.094352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":26,"num_write":20,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.855288+0000","last_change":"2024-04-26T22:55:25.376091+0000","last_active":"2024-04-26T22:55:31.855288+0000","last_peered":"2024-04-26T22:55:31.855288+0000","last_clean":"2024-04-26T22:55:31.855288+0000","last_became_active":"2024-04-26T22:55:25.375623+0000","last_became_peered":"2024-04-26T22:55:25.375623+0000","last_unstale":"2024-04-26T22:55:31.855288+0000","last_undegraded":"2024-04-26T22:55:31.855288+0000","last_fullsized":"2024-04-26T22:55:31.855288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:09.466719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411983+0000","last_change":"2024-04-26T22:55:30.413534+0000","last_active":"2024-04-26T22:55:31.411983+0000","last_peered":"2024-04-26T22:55:31.411983+0000","last_clean":"2024-04-26T22:55:31.411983+0000","last_became_active":"2024-04-26T22:55:30.413120+0000","last_became_peered":"2024-04-26T22:55:30.413120+0000","last_unstale":"2024-04-26T22:55:31.411983+0000","last_undegraded":"2024-04-26T22:55:31.411983+0000","last_fullsized":"2024-04-26T22:55:31.411983+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:13.829733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.644303+0000","last_change":"2024-04-26T22:55:24.781446+0000","last_active":"2024-04-26T22:55:53.644303+0000","last_peered":"2024-04-26T22:55:53.644303+0000","last_clean":"2024-04-26T22:55:53.644303+0000","last_became_active":"2024-04-26T22:55:24.781098+0000","last_became_peered":"2024-04-26T22:55:24.781098+0000","last_unstale":"2024-04-26T22:55:53.644303+0000","last_undegraded":"2024-04-26T22:55:53.644303+0000","last_fullsized":"2024-04-26T22:55:53.644303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:40.939755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.818603+0000","last_change":"2024-04-26T22:55:25.394147+0000","last_active":"2024-04-26T22:55:32.818603+0000","last_peered":"2024-04-26T22:55:32.818603+0000","last_clean":"2024-04-26T22:55:32.818603+0000","last_became_active":"2024-04-26T22:55:25.393654+0000","last_became_peered":"2024-04-26T22:55:25.393654+0000","last_unstale":"2024-04-26T22:55:32.818603+0000","last_undegraded":"2024-04-26T22:55:32.818603+0000","last_fullsized":"2024-04-26T22:55:32.818603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:16.335027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.803903+0000","last_change":"2024-04-26T22:55:30.431692+0000","last_active":"2024-04-26T22:55:32.803903+0000","last_peered":"2024-04-26T22:55:32.803903+0000","last_clean":"2024-04-26T22:55:32.803903+0000","last_became_active":"2024-04-26T22:55:30.431380+0000","last_became_peered":"2024-04-26T22:55:30.431380+0000","last_unstale":"2024-04-26T22:55:32.803903+0000","last_undegraded":"2024-04-26T22:55:32.803903+0000","last_fullsized":"2024-04-26T22:55:32.803903+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:14.556048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:52.623594+0000","last_change":"2024-04-26T22:55:24.788121+0000","last_active":"2024-04-26T22:55:52.623594+0000","last_peered":"2024-04-26T22:55:52.623594+0000","last_clean":"2024-04-26T22:55:52.623594+0000","last_became_active":"2024-04-26T22:55:24.787759+0000","last_became_peered":"2024-04-26T22:55:24.787759+0000","last_unstale":"2024-04-26T22:55:52.623594+0000","last_undegraded":"2024-04-26T22:55:52.623594+0000","last_fullsized":"2024-04-26T22:55:52.623594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:49.734675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.962118+0000","last_change":"2024-04-26T22:55:25.377576+0000","last_active":"2024-04-26T22:55:31.962118+0000","last_peered":"2024-04-26T22:55:31.962118+0000","last_clean":"2024-04-26T22:55:31.962118+0000","last_became_active":"2024-04-26T22:55:25.377203+0000","last_became_peered":"2024-04-26T22:55:25.377203+0000","last_unstale":"2024-04-26T22:55:31.962118+0000","last_undegraded":"2024-04-26T22:55:31.962118+0000","last_fullsized":"2024-04-26T22:55:31.962118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:39.361118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.414737+0000","last_change":"2024-04-26T22:55:30.436672+0000","last_active":"2024-04-26T22:55:31.414737+0000","last_peered":"2024-04-26T22:55:31.414737+0000","last_clean":"2024-04-26T22:55:31.414737+0000","last_became_active":"2024-04-26T22:55:30.435178+0000","last_became_peered":"2024-04-26T22:55:30.435178+0000","last_unstale":"2024-04-26T22:55:31.414737+0000","last_undegraded":"2024-04-26T22:55:31.414737+0000","last_fullsized":"2024-04-26T22:55:31.414737+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:34.435290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:55.659807+0000","last_change":"2024-04-26T22:55:24.760965+0000","last_active":"2024-04-26T22:55:55.659807+0000","last_peered":"2024-04-26T22:55:55.659807+0000","last_clean":"2024-04-26T22:55:55.659807+0000","last_became_active":"2024-04-26T22:55:24.760589+0000","last_became_peered":"2024-04-26T22:55:24.760589+0000","last_unstale":"2024-04-26T22:55:55.659807+0000","last_undegraded":"2024-04-26T22:55:55.659807+0000","last_fullsized":"2024-04-26T22:55:55.659807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:25.493209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.961939+0000","last_change":"2024-04-26T22:55:25.375433+0000","last_active":"2024-04-26T22:55:31.961939+0000","last_peered":"2024-04-26T22:55:31.961939+0000","last_clean":"2024-04-26T22:55:31.961939+0000","last_became_active":"2024-04-26T22:55:25.375062+0000","last_became_peered":"2024-04-26T22:55:25.375062+0000","last_unstale":"2024-04-26T22:55:31.961939+0000","last_undegraded":"2024-04-26T22:55:31.961939+0000","last_fullsized":"2024-04-26T22:55:31.961939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:55.423666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.820568+0000","last_change":"2024-04-26T22:55:30.430002+0000","last_active":"2024-04-26T22:55:32.820568+0000","last_peered":"2024-04-26T22:55:32.820568+0000","last_clean":"2024-04-26T22:55:32.820568+0000","last_became_active":"2024-04-26T22:55:30.427608+0000","last_became_peered":"2024-04-26T22:55:30.427608+0000","last_unstale":"2024-04-26T22:55:32.820568+0000","last_undegraded":"2024-04-26T22:55:32.820568+0000","last_fullsized":"2024-04-26T22:55:32.820568+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:23.286696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.645464+0000","last_change":"2024-04-26T22:55:24.773551+0000","last_active":"2024-04-26T22:55:53.645464+0000","last_peered":"2024-04-26T22:55:53.645464+0000","last_clean":"2024-04-26T22:55:53.645464+0000","last_became_active":"2024-04-26T22:55:24.773202+0000","last_became_peered":"2024-04-26T22:55:24.773202+0000","last_unstale":"2024-04-26T22:55:53.645464+0000","last_undegraded":"2024-04-26T22:55:53.645464+0000","last_fullsized":"2024-04-26T22:55:53.645464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:08.271682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411051+0000","last_change":"2024-04-26T22:55:25.372954+0000","last_active":"2024-04-26T22:55:31.411051+0000","last_peered":"2024-04-26T22:55:31.411051+0000","last_clean":"2024-04-26T22:55:31.411051+0000","last_became_active":"2024-04-26T22:55:25.372625+0000","last_became_peered":"2024-04-26T22:55:25.372625+0000","last_unstale":"2024-04-26T22:55:31.411051+0000","last_undegraded":"2024-04-26T22:55:31.411051+0000","last_fullsized":"2024-04-26T22:55:31.411051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:57.325359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411505+0000","last_change":"2024-04-26T22:55:30.422835+0000","last_active":"2024-04-26T22:55:31.411505+0000","last_peered":"2024-04-26T22:55:31.411505+0000","last_clean":"2024-04-26T22:55:31.411505+0000","last_became_active":"2024-04-26T22:55:30.422349+0000","last_became_peered":"2024-04-26T22:55:30.422349+0000","last_unstale":"2024-04-26T22:55:31.411505+0000","last_undegraded":"2024-04-26T22:55:31.411505+0000","last_fullsized":"2024-04-26T22:55:31.411505+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:26:33.613861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:54.655057+0000","last_change":"2024-04-26T22:55:24.756236+0000","last_active":"2024-04-26T22:55:54.655057+0000","last_peered":"2024-04-26T22:55:54.655057+0000","last_clean":"2024-04-26T22:55:54.655057+0000","last_became_active":"2024-04-26T22:55:24.755858+0000","last_became_peered":"2024-04-26T22:55:24.755858+0000","last_unstale":"2024-04-26T22:55:54.655057+0000","last_undegraded":"2024-04-26T22:55:54.655057+0000","last_fullsized":"2024-04-26T22:55:54.655057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:55.257152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.803863+0000","last_change":"2024-04-26T22:55:25.374367+0000","last_active":"2024-04-26T22:55:32.803863+0000","last_peered":"2024-04-26T22:55:32.803863+0000","last_clean":"2024-04-26T22:55:32.803863+0000","last_became_active":"2024-04-26T22:55:25.373186+0000","last_became_peered":"2024-04-26T22:55:25.373186+0000","last_unstale":"2024-04-26T22:55:32.803863+0000","last_undegraded":"2024-04-26T22:55:32.803863+0000","last_fullsized":"2024-04-26T22:55:32.803863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:31:55.780924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.855081+0000","last_change":"2024-04-26T22:55:30.425818+0000","last_active":"2024-04-26T22:55:31.855081+0000","last_peered":"2024-04-26T22:55:31.855081+0000","last_clean":"2024-04-26T22:55:31.855081+0000","last_became_active":"2024-04-26T22:55:30.425400+0000","last_became_peered":"2024-04-26T22:55:30.425400+0000","last_unstale":"2024-04-26T22:55:31.855081+0000","last_undegraded":"2024-04-26T22:55:31.855081+0000","last_fullsized":"2024-04-26T22:55:31.855081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:36.856545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.413862+0000","last_change":"2024-04-26T22:55:24.780607+0000","last_active":"2024-04-26T22:55:31.413862+0000","last_peered":"2024-04-26T22:55:31.413862+0000","last_clean":"2024-04-26T22:55:31.413862+0000","last_became_active":"2024-04-26T22:55:24.780271+0000","last_became_peered":"2024-04-26T22:55:24.780271+0000","last_unstale":"2024-04-26T22:55:31.413862+0000","last_undegraded":"2024-04-26T22:55:31.413862+0000","last_fullsized":"2024-04-26T22:55:31.413862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:25.057816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.012806+0000","last_change":"2024-04-26T22:55:25.399709+0000","last_active":"2024-04-26T22:55:32.012806+0000","last_peered":"2024-04-26T22:55:32.012806+0000","last_clean":"2024-04-26T22:55:32.012806+0000","last_became_active":"2024-04-26T22:55:25.399392+0000","last_became_peered":"2024-04-26T22:55:25.399392+0000","last_unstale":"2024-04-26T22:55:32.012806+0000","last_undegraded":"2024-04-26T22:55:32.012806+0000","last_fullsized":"2024-04-26T22:55:32.012806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:22.821809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.819736+0000","last_change":"2024-04-26T22:55:30.424070+0000","last_active":"2024-04-26T22:55:32.819736+0000","last_peered":"2024-04-26T22:55:32.819736+0000","last_clean":"2024-04-26T22:55:32.819736+0000","last_became_active":"2024-04-26T22:55:30.423637+0000","last_became_peered":"2024-04-26T22:55:30.423637+0000","last_unstale":"2024-04-26T22:55:32.819736+0000","last_undegraded":"2024-04-26T22:55:32.819736+0000","last_fullsized":"2024-04-26T22:55:32.819736+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:48.020079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.962035+0000","last_change":"2024-04-26T22:55:24.771881+0000","last_active":"2024-04-26T22:55:31.962035+0000","last_peered":"2024-04-26T22:55:31.962035+0000","last_clean":"2024-04-26T22:55:31.962035+0000","last_became_active":"2024-04-26T22:55:24.771563+0000","last_became_peered":"2024-04-26T22:55:24.771563+0000","last_unstale":"2024-04-26T22:55:31.962035+0000","last_undegraded":"2024-04-26T22:55:31.962035+0000","last_fullsized":"2024-04-26T22:55:31.962035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:14.306677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.012535+0000","last_change":"2024-04-26T22:55:25.399789+0000","last_active":"2024-04-26T22:55:32.012535+0000","last_peered":"2024-04-26T22:55:32.012535+0000","last_clean":"2024-04-26T22:55:32.012535+0000","last_became_active":"2024-04-26T22:55:25.399493+0000","last_became_peered":"2024-04-26T22:55:25.399493+0000","last_unstale":"2024-04-26T22:55:32.012535+0000","last_undegraded":"2024-04-26T22:55:32.012535+0000","last_fullsized":"2024-04-26T22:55:32.012535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:45.096556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.820271+0000","last_change":"2024-04-26T22:55:30.428107+0000","last_active":"2024-04-26T22:55:32.820271+0000","last_peered":"2024-04-26T22:55:32.820271+0000","last_clean":"2024-04-26T22:55:32.820271+0000","last_became_active":"2024-04-26T22:55:30.421955+0000","last_became_peered":"2024-04-26T22:55:30.421955+0000","last_unstale":"2024-04-26T22:55:32.820271+0000","last_undegraded":"2024-04-26T22:55:32.820271+0000","last_fullsized":"2024-04-26T22:55:32.820271+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:52.430809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411204+0000","last_change":"2024-04-26T22:55:25.373730+0000","last_active":"2024-04-26T22:55:31.411204+0000","last_peered":"2024-04-26T22:55:31.411204+0000","last_clean":"2024-04-26T22:55:31.411204+0000","last_became_active":"2024-04-26T22:55:25.373416+0000","last_became_peered":"2024-04-26T22:55:25.373416+0000","last_unstale":"2024-04-26T22:55:31.411204+0000","last_undegraded":"2024-04-26T22:55:31.411204+0000","last_fullsized":"2024-04-26T22:55:31.411204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:31:44.146244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.649853+0000","last_change":"2024-04-26T22:55:24.763998+0000","last_active":"2024-04-26T22:55:53.649853+0000","last_peered":"2024-04-26T22:55:53.649853+0000","last_clean":"2024-04-26T22:55:53.649853+0000","last_became_active":"2024-04-26T22:55:24.763561+0000","last_became_peered":"2024-04-26T22:55:24.763561+0000","last_unstale":"2024-04-26T22:55:53.649853+0000","last_undegraded":"2024-04-26T22:55:53.649853+0000","last_fullsized":"2024-04-26T22:55:53.649853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:25.270310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.962227+0000","last_change":"2024-04-26T22:55:30.424930+0000","last_active":"2024-04-26T22:55:31.962227+0000","last_peered":"2024-04-26T22:55:31.962227+0000","last_clean":"2024-04-26T22:55:31.962227+0000","last_became_active":"2024-04-26T22:55:30.424660+0000","last_became_peered":"2024-04-26T22:55:30.424660+0000","last_unstale":"2024-04-26T22:55:31.962227+0000","last_undegraded":"2024-04-26T22:55:31.962227+0000","last_fullsized":"2024-04-26T22:55:31.962227+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:29.483956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.815853+0000","last_change":"2024-04-26T22:55:25.390547+0000","last_active":"2024-04-26T22:55:32.815853+0000","last_peered":"2024-04-26T22:55:32.815853+0000","last_clean":"2024-04-26T22:55:32.815853+0000","last_became_active":"2024-04-26T22:55:25.386566+0000","last_became_peered":"2024-04-26T22:55:25.386566+0000","last_unstale":"2024-04-26T22:55:32.815853+0000","last_undegraded":"2024-04-26T22:55:32.815853+0000","last_fullsized":"2024-04-26T22:55:32.815853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:43.809125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:54.654143+0000","last_change":"2024-04-26T22:55:24.778254+0000","last_active":"2024-04-26T22:55:54.654143+0000","last_peered":"2024-04-26T22:55:54.654143+0000","last_clean":"2024-04-26T22:55:54.654143+0000","last_became_active":"2024-04-26T22:55:24.777850+0000","last_became_peered":"2024-04-26T22:55:24.777850+0000","last_unstale":"2024-04-26T22:55:54.654143+0000","last_undegraded":"2024-04-26T22:55:54.654143+0000","last_fullsized":"2024-04-26T22:55:54.654143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:48.053712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.961765+0000","last_change":"2024-04-26T22:53:59.233875+0000","last_active":"2024-04-26T22:55:31.961765+0000","last_peered":"2024-04-26T22:55:31.961765+0000","last_clean":"2024-04-26T22:55:31.961765+0000","last_became_active":"2024-04-26T22:53:59.233488+0000","last_became_peered":"2024-04-26T22:53:59.233488+0000","last_unstale":"2024-04-26T22:55:31.961765+0000","last_undegraded":"2024-04-26T22:55:31.961765+0000","last_fullsized":"2024-04-26T22:55:31.961765+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:49:55.319714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:49:55.319714+0000","last_clean_scrub_stamp":"2024-04-26T22:49:55.319714+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:37:32.885519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411660+0000","last_change":"2024-04-26T22:55:30.427936+0000","last_active":"2024-04-26T22:55:31.411660+0000","last_peered":"2024-04-26T22:55:31.411660+0000","last_clean":"2024-04-26T22:55:31.411660+0000","last_became_active":"2024-04-26T22:55:30.427483+0000","last_became_peered":"2024-04-26T22:55:30.427483+0000","last_unstale":"2024-04-26T22:55:31.411660+0000","last_undegraded":"2024-04-26T22:55:31.411660+0000","last_fullsized":"2024-04-26T22:55:31.411660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:00.856153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.817632+0000","last_change":"2024-04-26T22:55:25.378277+0000","last_active":"2024-04-26T22:55:32.817632+0000","last_peered":"2024-04-26T22:55:32.817632+0000","last_clean":"2024-04-26T22:55:32.817632+0000","last_became_active":"2024-04-26T22:55:25.377650+0000","last_became_peered":"2024-04-26T22:55:25.377650+0000","last_unstale":"2024-04-26T22:55:32.817632+0000","last_undegraded":"2024-04-26T22:55:32.817632+0000","last_fullsized":"2024-04-26T22:55:32.817632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:47.537306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.643559+0000","last_change":"2024-04-26T22:55:24.762439+0000","last_active":"2024-04-26T22:55:53.643559+0000","last_peered":"2024-04-26T22:55:53.643559+0000","last_clean":"2024-04-26T22:55:53.643559+0000","last_became_active":"2024-04-26T22:55:24.760732+0000","last_became_peered":"2024-04-26T22:55:24.760732+0000","last_unstale":"2024-04-26T22:55:53.643559+0000","last_undegraded":"2024-04-26T22:55:53.643559+0000","last_fullsized":"2024-04-26T22:55:53.643559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:33.857025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.803857+0000","last_change":"2024-04-26T22:55:30.411883+0000","last_active":"2024-04-26T22:55:32.803857+0000","last_peered":"2024-04-26T22:55:32.803857+0000","last_clean":"2024-04-26T22:55:32.803857+0000","last_became_active":"2024-04-26T22:55:30.411419+0000","last_became_peered":"2024-04-26T22:55:30.411419+0000","last_unstale":"2024-04-26T22:55:32.803857+0000","last_undegraded":"2024-04-26T22:55:32.803857+0000","last_fullsized":"2024-04-26T22:55:32.803857+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:14.977947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:52.624023+0000","last_change":"2024-04-26T22:55:24.779481+0000","last_active":"2024-04-26T22:55:52.624023+0000","last_peered":"2024-04-26T22:55:52.624023+0000","last_clean":"2024-04-26T22:55:52.624023+0000","last_became_active":"2024-04-26T22:55:24.779182+0000","last_became_peered":"2024-04-26T22:55:24.779182+0000","last_unstale":"2024-04-26T22:55:52.624023+0000","last_undegraded":"2024-04-26T22:55:52.624023+0000","last_fullsized":"2024-04-26T22:55:52.624023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:52.076060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.820035+0000","last_change":"2024-04-26T22:55:25.390938+0000","last_active":"2024-04-26T22:55:32.820035+0000","last_peered":"2024-04-26T22:55:32.820035+0000","last_clean":"2024-04-26T22:55:32.820035+0000","last_became_active":"2024-04-26T22:55:25.390566+0000","last_became_peered":"2024-04-26T22:55:25.390566+0000","last_unstale":"2024-04-26T22:55:32.820035+0000","last_undegraded":"2024-04-26T22:55:32.820035+0000","last_fullsized":"2024-04-26T22:55:32.820035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:48.079024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.815850+0000","last_change":"2024-04-26T22:55:30.430926+0000","last_active":"2024-04-26T22:55:32.815850+0000","last_peered":"2024-04-26T22:55:32.815850+0000","last_clean":"2024-04-26T22:55:32.815850+0000","last_became_active":"2024-04-26T22:55:30.428286+0000","last_became_peered":"2024-04-26T22:55:30.428286+0000","last_unstale":"2024-04-26T22:55:32.815850+0000","last_undegraded":"2024-04-26T22:55:32.815850+0000","last_fullsized":"2024-04-26T22:55:32.815850+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:26.095658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.647318+0000","last_change":"2024-04-26T22:55:24.784960+0000","last_active":"2024-04-26T22:55:53.647318+0000","last_peered":"2024-04-26T22:55:53.647318+0000","last_clean":"2024-04-26T22:55:53.647318+0000","last_became_active":"2024-04-26T22:55:24.784550+0000","last_became_peered":"2024-04-26T22:55:24.784550+0000","last_unstale":"2024-04-26T22:55:53.647318+0000","last_undegraded":"2024-04-26T22:55:53.647318+0000","last_fullsized":"2024-04-26T22:55:53.647318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:05.058172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.817222+0000","last_change":"2024-04-26T22:55:25.392352+0000","last_active":"2024-04-26T22:55:32.817222+0000","last_peered":"2024-04-26T22:55:32.817222+0000","last_clean":"2024-04-26T22:55:32.817222+0000","last_became_active":"2024-04-26T22:55:25.391983+0000","last_became_peered":"2024-04-26T22:55:25.391983+0000","last_unstale":"2024-04-26T22:55:32.817222+0000","last_undegraded":"2024-04-26T22:55:32.817222+0000","last_fullsized":"2024-04-26T22:55:32.817222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:37.192419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.813510+0000","last_change":"2024-04-26T22:55:30.409951+0000","last_active":"2024-04-26T22:55:32.813510+0000","last_peered":"2024-04-26T22:55:32.813510+0000","last_clean":"2024-04-26T22:55:32.813510+0000","last_became_active":"2024-04-26T22:55:30.409502+0000","last_became_peered":"2024-04-26T22:55:30.409502+0000","last_unstale":"2024-04-26T22:55:32.813510+0000","last_undegraded":"2024-04-26T22:55:32.813510+0000","last_fullsized":"2024-04-26T22:55:32.813510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:40:48.416697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:52.621372+0000","last_change":"2024-04-26T22:55:24.770999+0000","last_active":"2024-04-26T22:55:52.621372+0000","last_peered":"2024-04-26T22:55:52.621372+0000","last_clean":"2024-04-26T22:55:52.621372+0000","last_became_active":"2024-04-26T22:55:24.770630+0000","last_became_peered":"2024-04-26T22:55:24.770630+0000","last_unstale":"2024-04-26T22:55:52.621372+0000","last_undegraded":"2024-04-26T22:55:52.621372+0000","last_fullsized":"2024-04-26T22:55:52.621372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:01.651705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.597311+0000","last_change":"2024-04-26T22:55:25.401845+0000","last_active":"2024-04-26T22:55:31.597311+0000","last_peered":"2024-04-26T22:55:31.597311+0000","last_clean":"2024-04-26T22:55:31.597311+0000","last_became_active":"2024-04-26T22:55:25.398759+0000","last_became_peered":"2024-04-26T22:55:25.398759+0000","last_unstale":"2024-04-26T22:55:31.597311+0000","last_undegraded":"2024-04-26T22:55:31.597311+0000","last_fullsized":"2024-04-26T22:55:31.597311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:14.158710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412403+0000","last_change":"2024-04-26T22:55:30.426206+0000","last_active":"2024-04-26T22:55:31.412403+0000","last_peered":"2024-04-26T22:55:31.412403+0000","last_clean":"2024-04-26T22:55:31.412403+0000","last_became_active":"2024-04-26T22:55:30.425678+0000","last_became_peered":"2024-04-26T22:55:30.425678+0000","last_unstale":"2024-04-26T22:55:31.412403+0000","last_undegraded":"2024-04-26T22:55:31.412403+0000","last_fullsized":"2024-04-26T22:55:31.412403+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:48.913899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:52.619236+0000","last_change":"2024-04-26T22:55:24.772775+0000","last_active":"2024-04-26T22:55:52.619236+0000","last_peered":"2024-04-26T22:55:52.619236+0000","last_clean":"2024-04-26T22:55:52.619236+0000","last_became_active":"2024-04-26T22:55:24.772434+0000","last_became_peered":"2024-04-26T22:55:24.772434+0000","last_unstale":"2024-04-26T22:55:52.619236+0000","last_undegraded":"2024-04-26T22:55:52.619236+0000","last_fullsized":"2024-04-26T22:55:52.619236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:23.309545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.854873+0000","last_change":"2024-04-26T22:55:25.372051+0000","last_active":"2024-04-26T22:55:31.854873+0000","last_peered":"2024-04-26T22:55:31.854873+0000","last_clean":"2024-04-26T22:55:31.854873+0000","last_became_active":"2024-04-26T22:55:25.371681+0000","last_became_peered":"2024-04-26T22:55:25.371681+0000","last_unstale":"2024-04-26T22:55:31.854873+0000","last_undegraded":"2024-04-26T22:55:31.854873+0000","last_fullsized":"2024-04-26T22:55:31.854873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:34:39.412974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.414753+0000","last_change":"2024-04-26T22:55:30.427160+0000","last_active":"2024-04-26T22:55:31.414753+0000","last_peered":"2024-04-26T22:55:31.414753+0000","last_clean":"2024-04-26T22:55:31.414753+0000","last_became_active":"2024-04-26T22:55:30.426555+0000","last_became_peered":"2024-04-26T22:55:30.426555+0000","last_unstale":"2024-04-26T22:55:31.414753+0000","last_undegraded":"2024-04-26T22:55:31.414753+0000","last_fullsized":"2024-04-26T22:55:31.414753+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:31.688532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:52.624396+0000","last_change":"2024-04-26T22:55:24.785265+0000","last_active":"2024-04-26T22:55:52.624396+0000","last_peered":"2024-04-26T22:55:52.624396+0000","last_clean":"2024-04-26T22:55:52.624396+0000","last_became_active":"2024-04-26T22:55:24.784896+0000","last_became_peered":"2024-04-26T22:55:24.784896+0000","last_unstale":"2024-04-26T22:55:52.624396+0000","last_undegraded":"2024-04-26T22:55:52.624396+0000","last_fullsized":"2024-04-26T22:55:52.624396+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:28.815337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.803575+0000","last_change":"2024-04-26T22:55:25.399606+0000","last_active":"2024-04-26T22:55:32.803575+0000","last_peered":"2024-04-26T22:55:32.803575+0000","last_clean":"2024-04-26T22:55:32.803575+0000","last_became_active":"2024-04-26T22:55:25.399195+0000","last_became_peered":"2024-04-26T22:55:25.399195+0000","last_unstale":"2024-04-26T22:55:32.803575+0000","last_undegraded":"2024-04-26T22:55:32.803575+0000","last_fullsized":"2024-04-26T22:55:32.803575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:53:53.618431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411952+0000","last_change":"2024-04-26T22:55:30.427381+0000","last_active":"2024-04-26T22:55:31.411952+0000","last_peered":"2024-04-26T22:55:31.411952+0000","last_clean":"2024-04-26T22:55:31.411952+0000","last_became_active":"2024-04-26T22:55:30.426736+0000","last_became_peered":"2024-04-26T22:55:30.426736+0000","last_unstale":"2024-04-26T22:55:31.411952+0000","last_undegraded":"2024-04-26T22:55:31.411952+0000","last_fullsized":"2024-04-26T22:55:31.411952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:07.106991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.812991+0000","last_change":"2024-04-26T22:55:24.781483+0000","last_active":"2024-04-26T22:55:32.812991+0000","last_peered":"2024-04-26T22:55:32.812991+0000","last_clean":"2024-04-26T22:55:32.812991+0000","last_became_active":"2024-04-26T22:55:24.781174+0000","last_became_peered":"2024-04-26T22:55:24.781174+0000","last_unstale":"2024-04-26T22:55:32.812991+0000","last_undegraded":"2024-04-26T22:55:32.812991+0000","last_fullsized":"2024-04-26T22:55:32.812991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:58:05.550242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.816238+0000","last_change":"2024-04-26T22:55:25.406764+0000","last_active":"2024-04-26T22:55:32.816238+0000","last_peered":"2024-04-26T22:55:32.816238+0000","last_clean":"2024-04-26T22:55:32.816238+0000","last_became_active":"2024-04-26T22:55:25.405948+0000","last_became_peered":"2024-04-26T22:55:25.405948+0000","last_unstale":"2024-04-26T22:55:32.816238+0000","last_undegraded":"2024-04-26T22:55:32.816238+0000","last_fullsized":"2024-04-26T22:55:32.816238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:58.414530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.412300+0000","last_change":"2024-04-26T22:55:30.422989+0000","last_active":"2024-04-26T22:55:31.412300+0000","last_peered":"2024-04-26T22:55:31.412300+0000","last_clean":"2024-04-26T22:55:31.412300+0000","last_became_active":"2024-04-26T22:55:30.422529+0000","last_became_peered":"2024-04-26T22:55:30.422529+0000","last_unstale":"2024-04-26T22:55:31.412300+0000","last_undegraded":"2024-04-26T22:55:31.412300+0000","last_fullsized":"2024-04-26T22:55:31.412300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:08.312667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:52.618832+0000","last_change":"2024-04-26T22:55:24.763338+0000","last_active":"2024-04-26T22:55:52.618832+0000","last_peered":"2024-04-26T22:55:52.618832+0000","last_clean":"2024-04-26T22:55:52.618832+0000","last_became_active":"2024-04-26T22:55:24.762975+0000","last_became_peered":"2024-04-26T22:55:24.762975+0000","last_unstale":"2024-04-26T22:55:52.618832+0000","last_undegraded":"2024-04-26T22:55:52.618832+0000","last_fullsized":"2024-04-26T22:55:52.618832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:45.529970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.962137+0000","last_change":"2024-04-26T22:55:25.371428+0000","last_active":"2024-04-26T22:55:31.962137+0000","last_peered":"2024-04-26T22:55:31.962137+0000","last_clean":"2024-04-26T22:55:31.962137+0000","last_became_active":"2024-04-26T22:55:25.371088+0000","last_became_peered":"2024-04-26T22:55:25.371088+0000","last_unstale":"2024-04-26T22:55:31.962137+0000","last_undegraded":"2024-04-26T22:55:31.962137+0000","last_fullsized":"2024-04-26T22:55:31.962137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:55:15.066665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.814181+0000","last_change":"2024-04-26T22:55:30.411966+0000","last_active":"2024-04-26T22:55:32.814181+0000","last_peered":"2024-04-26T22:55:32.814181+0000","last_clean":"2024-04-26T22:55:32.814181+0000","last_became_active":"2024-04-26T22:55:30.411486+0000","last_became_peered":"2024-04-26T22:55:30.411486+0000","last_unstale":"2024-04-26T22:55:32.814181+0000","last_undegraded":"2024-04-26T22:55:32.814181+0000","last_fullsized":"2024-04-26T22:55:32.814181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:08.207340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:53.645958+0000","last_change":"2024-04-26T22:55:24.771435+0000","last_active":"2024-04-26T22:55:53.645958+0000","last_peered":"2024-04-26T22:55:53.645958+0000","last_clean":"2024-04-26T22:55:53.645958+0000","last_became_active":"2024-04-26T22:55:24.771156+0000","last_became_peered":"2024-04-26T22:55:24.771156+0000","last_unstale":"2024-04-26T22:55:53.645958+0000","last_undegraded":"2024-04-26T22:55:53.645958+0000","last_fullsized":"2024-04-26T22:55:53.645958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:15.130786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.962476+0000","last_change":"2024-04-26T22:55:25.374051+0000","last_active":"2024-04-26T22:55:31.962476+0000","last_peered":"2024-04-26T22:55:31.962476+0000","last_clean":"2024-04-26T22:55:31.962476+0000","last_became_active":"2024-04-26T22:55:25.373348+0000","last_became_peered":"2024-04-26T22:55:25.373348+0000","last_unstale":"2024-04-26T22:55:31.962476+0000","last_undegraded":"2024-04-26T22:55:31.962476+0000","last_fullsized":"2024-04-26T22:55:31.962476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:30:31.116829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.597427+0000","last_change":"2024-04-26T22:55:30.425497+0000","last_active":"2024-04-26T22:55:31.597427+0000","last_peered":"2024-04-26T22:55:31.597427+0000","last_clean":"2024-04-26T22:55:31.597427+0000","last_became_active":"2024-04-26T22:55:30.425174+0000","last_became_peered":"2024-04-26T22:55:30.425174+0000","last_unstale":"2024-04-26T22:55:31.597427+0000","last_undegraded":"2024-04-26T22:55:31.597427+0000","last_fullsized":"2024-04-26T22:55:31.597427+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:22.555486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411764+0000","last_change":"2024-04-26T22:55:24.788057+0000","last_active":"2024-04-26T22:55:31.411764+0000","last_peered":"2024-04-26T22:55:31.411764+0000","last_clean":"2024-04-26T22:55:31.411764+0000","last_became_active":"2024-04-26T22:55:24.787753+0000","last_became_peered":"2024-04-26T22:55:24.787753+0000","last_unstale":"2024-04-26T22:55:31.411764+0000","last_undegraded":"2024-04-26T22:55:31.411764+0000","last_fullsized":"2024-04-26T22:55:31.411764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:53.054665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.962264+0000","last_change":"2024-04-26T22:55:25.400227+0000","last_active":"2024-04-26T22:55:31.962264+0000","last_peered":"2024-04-26T22:55:31.962264+0000","last_clean":"2024-04-26T22:55:31.962264+0000","last_became_active":"2024-04-26T22:55:25.399738+0000","last_became_peered":"2024-04-26T22:55:25.399738+0000","last_unstale":"2024-04-26T22:55:31.962264+0000","last_undegraded":"2024-04-26T22:55:31.962264+0000","last_fullsized":"2024-04-26T22:55:31.962264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:22.224654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.410867+0000","last_change":"2024-04-26T22:55:30.410479+0000","last_active":"2024-04-26T22:55:31.410867+0000","last_peered":"2024-04-26T22:55:31.410867+0000","last_clean":"2024-04-26T22:55:31.410867+0000","last_became_active":"2024-04-26T22:55:30.410049+0000","last_became_peered":"2024-04-26T22:55:30.410049+0000","last_unstale":"2024-04-26T22:55:31.410867+0000","last_undegraded":"2024-04-26T22:55:31.410867+0000","last_fullsized":"2024-04-26T22:55:31.410867+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:04.412005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.820389+0000","last_change":"2024-04-26T22:55:24.780741+0000","last_active":"2024-04-26T22:55:32.820389+0000","last_peered":"2024-04-26T22:55:32.820389+0000","last_clean":"2024-04-26T22:55:32.820389+0000","last_became_active":"2024-04-26T22:55:24.780400+0000","last_became_peered":"2024-04-26T22:55:24.780400+0000","last_unstale":"2024-04-26T22:55:32.820389+0000","last_undegraded":"2024-04-26T22:55:32.820389+0000","last_fullsized":"2024-04-26T22:55:32.820389+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:52.274456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.962381+0000","last_change":"2024-04-26T22:55:25.369449+0000","last_active":"2024-04-26T22:55:31.962381+0000","last_peered":"2024-04-26T22:55:31.962381+0000","last_clean":"2024-04-26T22:55:31.962381+0000","last_became_active":"2024-04-26T22:55:25.369106+0000","last_became_peered":"2024-04-26T22:55:25.369106+0000","last_unstale":"2024-04-26T22:55:31.962381+0000","last_undegraded":"2024-04-26T22:55:31.962381+0000","last_fullsized":"2024-04-26T22:55:31.962381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:21.347385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.803752+0000","last_change":"2024-04-26T22:55:25.371959+0000","last_active":"2024-04-26T22:55:32.803752+0000","last_peered":"2024-04-26T22:55:32.803752+0000","last_clean":"2024-04-26T22:55:32.803752+0000","last_became_active":"2024-04-26T22:55:25.371580+0000","last_became_peered":"2024-04-26T22:55:25.371580+0000","last_unstale":"2024-04-26T22:55:32.803752+0000","last_undegraded":"2024-04-26T22:55:32.803752+0000","last_fullsized":"2024-04-26T22:55:32.803752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:11.001269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.813650+0000","last_change":"2024-04-26T22:55:25.373979+0000","last_active":"2024-04-26T22:55:32.813650+0000","last_peered":"2024-04-26T22:55:32.813650+0000","last_clean":"2024-04-26T22:55:32.813650+0000","last_became_active":"2024-04-26T22:55:25.373591+0000","last_became_peered":"2024-04-26T22:55:25.373591+0000","last_unstale":"2024-04-26T22:55:32.813650+0000","last_undegraded":"2024-04-26T22:55:32.813650+0000","last_fullsized":"2024-04-26T22:55:32.813650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:44.295357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.813506+0000","last_change":"2024-04-26T22:55:25.398778+0000","last_active":"2024-04-26T22:55:32.813506+0000","last_peered":"2024-04-26T22:55:32.813506+0000","last_clean":"2024-04-26T22:55:32.813506+0000","last_became_active":"2024-04-26T22:55:25.398460+0000","last_became_peered":"2024-04-26T22:55:25.398460+0000","last_unstale":"2024-04-26T22:55:32.813506+0000","last_undegraded":"2024-04-26T22:55:32.813506+0000","last_fullsized":"2024-04-26T22:55:32.813506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.873418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.597218+0000","last_change":"2024-04-26T22:55:25.396548+0000","last_active":"2024-04-26T22:55:31.597218+0000","last_peered":"2024-04-26T22:55:31.597218+0000","last_clean":"2024-04-26T22:55:31.597218+0000","last_became_active":"2024-04-26T22:55:25.395983+0000","last_became_peered":"2024-04-26T22:55:25.395983+0000","last_unstale":"2024-04-26T22:55:31.597218+0000","last_undegraded":"2024-04-26T22:55:31.597218+0000","last_fullsized":"2024-04-26T22:55:31.597218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:21:34.690636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.819811+0000","last_change":"2024-04-26T22:55:25.396742+0000","last_active":"2024-04-26T22:55:32.819811+0000","last_peered":"2024-04-26T22:55:32.819811+0000","last_clean":"2024-04-26T22:55:32.819811+0000","last_became_active":"2024-04-26T22:55:25.396372+0000","last_became_peered":"2024-04-26T22:55:25.396372+0000","last_unstale":"2024-04-26T22:55:32.819811+0000","last_undegraded":"2024-04-26T22:55:32.819811+0000","last_fullsized":"2024-04-26T22:55:32.819811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:32:12.068358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.803895+0000","last_change":"2024-04-26T22:55:25.372059+0000","last_active":"2024-04-26T22:55:32.803895+0000","last_peered":"2024-04-26T22:55:32.803895+0000","last_clean":"2024-04-26T22:55:32.803895+0000","last_became_active":"2024-04-26T22:55:25.371750+0000","last_became_peered":"2024-04-26T22:55:25.371750+0000","last_unstale":"2024-04-26T22:55:32.803895+0000","last_undegraded":"2024-04-26T22:55:32.803895+0000","last_fullsized":"2024-04-26T22:55:32.803895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:10.230585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.814077+0000","last_change":"2024-04-26T22:55:25.374904+0000","last_active":"2024-04-26T22:55:32.814077+0000","last_peered":"2024-04-26T22:55:32.814077+0000","last_clean":"2024-04-26T22:55:32.814077+0000","last_became_active":"2024-04-26T22:55:25.374600+0000","last_became_peered":"2024-04-26T22:55:25.374600+0000","last_unstale":"2024-04-26T22:55:32.814077+0000","last_undegraded":"2024-04-26T22:55:32.814077+0000","last_fullsized":"2024-04-26T22:55:32.814077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:49.124981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.817369+0000","last_change":"2024-04-26T22:55:25.391026+0000","last_active":"2024-04-26T22:55:32.817369+0000","last_peered":"2024-04-26T22:55:32.817369+0000","last_clean":"2024-04-26T22:55:32.817369+0000","last_became_active":"2024-04-26T22:55:25.389836+0000","last_became_peered":"2024-04-26T22:55:25.389836+0000","last_unstale":"2024-04-26T22:55:32.817369+0000","last_undegraded":"2024-04-26T22:55:32.817369+0000","last_fullsized":"2024-04-26T22:55:32.817369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:37.102439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:31.411107+0000","last_change":"2024-04-26T22:55:25.406584+0000","last_active":"2024-04-26T22:55:31.411107+0000","last_peered":"2024-04-26T22:55:31.411107+0000","last_clean":"2024-04-26T22:55:31.411107+0000","last_became_active":"2024-04-26T22:55:25.406352+0000","last_became_peered":"2024-04-26T22:55:25.406352+0000","last_unstale":"2024-04-26T22:55:31.411107+0000","last_undegraded":"2024-04-26T22:55:31.411107+0000","last_fullsized":"2024-04-26T22:55:31.411107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:43.594712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.816766+0000","last_change":"2024-04-26T22:55:25.399578+0000","last_active":"2024-04-26T22:55:32.816766+0000","last_peered":"2024-04-26T22:55:32.816766+0000","last_clean":"2024-04-26T22:55:32.816766+0000","last_became_active":"2024-04-26T22:55:25.396555+0000","last_became_peered":"2024-04-26T22:55:25.396555+0000","last_unstale":"2024-04-26T22:55:32.816766+0000","last_undegraded":"2024-04-26T22:55:32.816766+0000","last_fullsized":"2024-04-26T22:55:32.816766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:07:50.082978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.814186+0000","last_change":"2024-04-26T22:55:25.393034+0000","last_active":"2024-04-26T22:55:32.814186+0000","last_peered":"2024-04-26T22:55:32.814186+0000","last_clean":"2024-04-26T22:55:32.814186+0000","last_became_active":"2024-04-26T22:55:25.392730+0000","last_became_peered":"2024-04-26T22:55:25.392730+0000","last_unstale":"2024-04-26T22:55:32.814186+0000","last_undegraded":"2024-04-26T22:55:32.814186+0000","last_fullsized":"2024-04-26T22:55:32.814186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:11.040311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.819317+0000","last_change":"2024-04-26T22:55:25.398224+0000","last_active":"2024-04-26T22:55:32.819317+0000","last_peered":"2024-04-26T22:55:32.819317+0000","last_clean":"2024-04-26T22:55:32.819317+0000","last_became_active":"2024-04-26T22:55:25.397744+0000","last_became_peered":"2024-04-26T22:55:25.397744+0000","last_unstale":"2024-04-26T22:55:32.819317+0000","last_undegraded":"2024-04-26T22:55:32.819317+0000","last_fullsized":"2024-04-26T22:55:32.819317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:35.107942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:55:32.803820+0000","last_change":"2024-04-26T22:55:25.374076+0000","last_active":"2024-04-26T22:55:32.803820+0000","last_peered":"2024-04-26T22:55:32.803820+0000","last_clean":"2024-04-26T22:55:32.803820+0000","last_became_active":"2024-04-26T22:55:25.373736+0000","last_became_peered":"2024-04-26T22:55:25.373736+0000","last_unstale":"2024-04-26T22:55:32.803820+0000","last_undegraded":"2024-04-26T22:55:32.803820+0000","last_fullsized":"2024-04-26T22:55:32.803820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:01:56.096004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282639,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":27142,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":31,"num_read_kb":46,"num_write":136,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":42828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383701,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":490072,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645338,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1088173,"data_compressed":50705,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808864,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":486212,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070503,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":486132,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332139,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1077554,"data_compressed":50705,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":485484,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":489236,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":486872,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":490586,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":485678,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":486429,"data_compressed":14759,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":1077238,"data_compressed":50705,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34303,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34303,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34303,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:55:59.271 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T22:55:59.271 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T22:55:59.272 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T22:55:59.794 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:55:59.946 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:59 smithi027 ceph-mon[28034]: pgmap v347: 129 pgs: 129 active+clean; 603 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-04-26T22:55:59.946 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:55:59 smithi027 ceph-mon[28034]: from='client.24866 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:56:00.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:59 smithi080 ceph-mon[32324]: pgmap v347: 129 pgs: 129 active+clean; 603 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-04-26T22:56:00.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:55:59 smithi080 ceph-mon[32324]: from='client.24866 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:56:00.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:59 smithi094 ceph-mon[32889]: pgmap v347: 129 pgs: 129 active+clean; 603 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-04-26T22:56:00.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:55:59 smithi094 ceph-mon[32889]: from='client.24866 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:56:01.015 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:00 smithi027 ceph-mon[28034]: from='client.24898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:56:01.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:00 smithi080 ceph-mon[32324]: from='client.24898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:56:01.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:00 smithi094 ceph-mon[32889]: from='client.24898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:56:01.975 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:01 smithi027 ceph-mon[28034]: pgmap v348: 129 pgs: 129 active+clean; 603 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-04-26T22:56:02.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:01 smithi080 ceph-mon[32324]: pgmap v348: 129 pgs: 129 active+clean; 603 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-04-26T22:56:02.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:01 smithi094 ceph-mon[32889]: pgmap v348: 129 pgs: 129 active+clean; 603 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-04-26T22:56:02.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.342+0000 7fe6d0e00700 1 -- 172.21.15.27:0/2846098896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6cc1056d0 msgr2=0x7fe6cc105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:02.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.342+0000 7fe6d0e00700 1 --2- 172.21.15.27:0/2846098896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6cc1056d0 0x7fe6cc105af0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009990 tx=0x7fe6bc009ca0 comp rx=0 tx=0).stop 2024-04-26T22:56:02.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.342+0000 7fe6d0e00700 1 -- 172.21.15.27:0/2846098896 shutdown_connections 2024-04-26T22:56:02.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.342+0000 7fe6d0e00700 1 --2- 172.21.15.27:0/2846098896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6cc107270 0x7fe6cc10db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:02.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.342+0000 7fe6d0e00700 1 --2- 172.21.15.27:0/2846098896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6cc1056d0 0x7fe6cc105af0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:02.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.342+0000 7fe6d0e00700 1 --2- 172.21.15.27:0/2846098896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6cc1068d0 0x7fe6cc106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:02.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.342+0000 7fe6d0e00700 1 -- 172.21.15.27:0/2846098896 >> 172.21.15.27:0/2846098896 conn(0x7fe6cc100c70 msgr2=0x7fe6cc1030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:02.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.342+0000 7fe6d0e00700 1 -- 172.21.15.27:0/2846098896 shutdown_connections 2024-04-26T22:56:02.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.342+0000 7fe6d0e00700 1 -- 172.21.15.27:0/2846098896 wait complete. 2024-04-26T22:56:02.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.344+0000 7fe6d0e00700 1 Processor -- start 2024-04-26T22:56:02.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.345+0000 7fe6d0e00700 1 -- start start 2024-04-26T22:56:02.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.345+0000 7fe6d0e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6cc1056d0 0x7fe6cc19b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:02.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.345+0000 7fe6d0e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6cc1068d0 0x7fe6cc19bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:02.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.345+0000 7fe6d0e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6cc107270 0x7fe6cc1a2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:02.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.345+0000 7fe6d0e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc1111b0 con 0x7fe6cc1068d0 2024-04-26T22:56:02.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.345+0000 7fe6d0e00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc111030 con 0x7fe6cc107270 2024-04-26T22:56:02.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.345+0000 7fe6d0e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc111330 con 0x7fe6cc1056d0 2024-04-26T22:56:02.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.346+0000 7fe6c9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6cc1056d0 0x7fe6cc19b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:02.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.346+0000 7fe6c9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6cc1056d0 0x7fe6cc19b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:51420/0 (socket says 172.21.15.27:51420) 2024-04-26T22:56:02.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.346+0000 7fe6c9e00700 1 -- 172.21.15.27:0/1621253838 learned_addr learned my addr 172.21.15.27:0/1621253838 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:02.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.346+0000 7fe6c3e00700 1 --2- 172.21.15.27:0/1621253838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6cc1068d0 0x7fe6cc19bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:02.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.346+0000 7fe6ca800700 1 --2- 172.21.15.27:0/1621253838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6cc107270 0x7fe6cc1a2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:02.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.346+0000 7fe6c9e00700 1 -- 172.21.15.27:0/1621253838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6cc107270 msgr2=0x7fe6cc1a2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:02.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.346+0000 7fe6c9e00700 1 --2- 172.21.15.27:0/1621253838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6cc107270 0x7fe6cc1a2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:02.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.346+0000 7fe6c9e00700 1 -- 172.21.15.27:0/1621253838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6cc1068d0 msgr2=0x7fe6cc19bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:02.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.346+0000 7fe6c9e00700 1 --2- 172.21.15.27:0/1621253838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6cc1068d0 0x7fe6cc19bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:02.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.346+0000 7fe6c9e00700 1 -- 172.21.15.27:0/1621253838 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc1a3400 con 0x7fe6cc1056d0 2024-04-26T22:56:02.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.347+0000 7fe6c9e00700 1 --2- 172.21.15.27:0/1621253838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6cc1056d0 0x7fe6cc19b7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc000c00 tx=0x7fe6bc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:02.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.347+0000 7fe6c2a00700 1 -- 172.21.15.27:0/1621253838 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc026070 con 0x7fe6cc1056d0 2024-04-26T22:56:02.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.347+0000 7fe6c2a00700 1 -- 172.21.15.27:0/1621253838 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6bc021410 con 0x7fe6cc1056d0 2024-04-26T22:56:02.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.347+0000 7fe6c2a00700 1 -- 172.21.15.27:0/1621253838 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc017650 con 0x7fe6cc1056d0 2024-04-26T22:56:02.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.347+0000 7fe6d0e00700 1 -- 172.21.15.27:0/1621253838 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6cc1a3690 con 0x7fe6cc1056d0 2024-04-26T22:56:02.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.347+0000 7fe6d0e00700 1 -- 172.21.15.27:0/1621253838 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6cc1a3a90 con 0x7fe6cc1056d0 2024-04-26T22:56:02.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.348+0000 7fe6d0e00700 1 -- 172.21.15.27:0/1621253838 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6cc051960 con 0x7fe6cc1056d0 2024-04-26T22:56:02.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.351+0000 7fe6c2a00700 1 -- 172.21.15.27:0/1621253838 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe6bc029070 con 0x7fe6cc1056d0 2024-04-26T22:56:02.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.352+0000 7fe6c2a00700 1 --2- 172.21.15.27:0/1621253838 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6b00a3e00 0x7fe6b00a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:02.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.352+0000 7fe6c2a00700 1 -- 172.21.15.27:0/1621253838 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe6bc0d1820 con 0x7fe6cc1056d0 2024-04-26T22:56:02.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.352+0000 7fe6c3e00700 1 --2- 172.21.15.27:0/1621253838 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6b00a3e00 0x7fe6b00a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:02.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.353+0000 7fe6c3e00700 1 --2- 172.21.15.27:0/1621253838 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6b00a3e00 0x7fe6b00a62c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009c60 tx=0x7fe6b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.354+0000 7fe6c2a00700 1 -- 172.21.15.27:0/1621253838 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe6bc030080 con 0x7fe6cc1056d0 2024-04-26T22:56:02.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.690+0000 7fe6d0e00700 1 -- 172.21.15.27:0/1621253838 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fe6cc02e490 con 0x7fe6cc1056d0 2024-04-26T22:56:02.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.699+0000 7fe6c2a00700 1 -- 172.21.15.27:0/1621253838 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fe6bc013070 con 0x7fe6cc1056d0 2024-04-26T22:56:02.698 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-04-26T22:56:02.699 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQACMSxmIglOKRAAuJo4uexbmkyPaD8UC7J0xg== 2024-04-26T22:56:02.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.702+0000 7fe6d0e00700 1 -- 172.21.15.27:0/1621253838 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6b00a3e00 msgr2=0x7fe6b00a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:02.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.702+0000 7fe6d0e00700 1 --2- 172.21.15.27:0/1621253838 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6b00a3e00 0x7fe6b00a62c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009c60 tx=0x7fe6b4009260 comp rx=0 tx=0).stop 2024-04-26T22:56:02.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.702+0000 7fe6d0e00700 1 -- 172.21.15.27:0/1621253838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6cc1056d0 msgr2=0x7fe6cc19b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:02.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.702+0000 7fe6d0e00700 1 --2- 172.21.15.27:0/1621253838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6cc1056d0 0x7fe6cc19b7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc000c00 tx=0x7fe6bc009580 comp rx=0 tx=0).stop 2024-04-26T22:56:02.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.702+0000 7fe6d0e00700 1 -- 172.21.15.27:0/1621253838 shutdown_connections 2024-04-26T22:56:02.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.702+0000 7fe6d0e00700 1 --2- 172.21.15.27:0/1621253838 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6b00a3e00 0x7fe6b00a62c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:02.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.702+0000 7fe6d0e00700 1 --2- 172.21.15.27:0/1621253838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6cc107270 0x7fe6cc1a2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:02.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.702+0000 7fe6d0e00700 1 --2- 172.21.15.27:0/1621253838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6cc1056d0 0x7fe6cc19b7a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:02.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.702+0000 7fe6d0e00700 1 --2- 172.21.15.27:0/1621253838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6cc1068d0 0x7fe6cc19bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:02.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.702+0000 7fe6d0e00700 1 -- 172.21.15.27:0/1621253838 >> 172.21.15.27:0/1621253838 conn(0x7fe6cc100c70 msgr2=0x7fe6cc109bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:02.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.702+0000 7fe6d0e00700 1 -- 172.21.15.27:0/1621253838 shutdown_connections 2024-04-26T22:56:02.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:02.703+0000 7fe6d0e00700 1 -- 172.21.15.27:0/1621253838 wait complete. 2024-04-26T22:56:03.714 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:56:03.714 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T22:56:03.714 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T22:56:03.859 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T22:56:03.859 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T22:56:03.860 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T22:56:03.983 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:03 smithi027 ceph-mon[28034]: pgmap v349: 129 pgs: 129 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-04-26T22:56:03.983 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:03 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1621253838' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T22:56:03.983 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:03 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T22:56:03.983 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:03 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T22:56:04.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:03 smithi080 ceph-mon[32324]: pgmap v349: 129 pgs: 129 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-04-26T22:56:04.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:03 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1621253838' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T22:56:04.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:03 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T22:56:04.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:03 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T22:56:04.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:03 smithi094 ceph-mon[32889]: pgmap v349: 129 pgs: 129 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-04-26T22:56:04.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:03 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1621253838' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T22:56:04.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:03 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T22:56:04.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:03 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T22:56:04.372 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:04.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-26T22:56:04.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4, 7, 11]}]: dispatch 2024-04-26T22:56:04.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-26T22:56:04.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-26T22:56:04.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-26T22:56:04.786 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]: dispatch 2024-04-26T22:56:04.786 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-04-26T22:56:04.786 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-04-26T22:56:04.786 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-04-26T22:56:05.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-26T22:56:05.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4, 7, 11]}]: dispatch 2024-04-26T22:56:05.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-26T22:56:05.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-26T22:56:05.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-26T22:56:05.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]: dispatch 2024-04-26T22:56:05.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-04-26T22:56:05.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-04-26T22:56:05.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-04-26T22:56:05.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-26T22:56:05.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4, 7, 11]}]: dispatch 2024-04-26T22:56:05.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-26T22:56:05.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-26T22:56:05.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-26T22:56:05.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]: dispatch 2024-04-26T22:56:05.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-04-26T22:56:05.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-04-26T22:56:05.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-04-26T22:56:05.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:05 smithi027 ceph-mon[28034]: pgmap v350: 129 pgs: 129 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 KiB/s wr, 13 op/s 2024-04-26T22:56:05.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-26T22:56:05.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4, 7, 11]}]': finished 2024-04-26T22:56:05.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-26T22:56:05.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-26T22:56:05.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-26T22:56:05.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]': finished 2024-04-26T22:56:05.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-04-26T22:56:05.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-04-26T22:56:05.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-04-26T22:56:05.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:05 smithi027 ceph-mon[28034]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T22:56:06.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:05 smithi080 ceph-mon[32324]: pgmap v350: 129 pgs: 129 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 KiB/s wr, 13 op/s 2024-04-26T22:56:06.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-26T22:56:06.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4, 7, 11]}]': finished 2024-04-26T22:56:06.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-26T22:56:06.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-26T22:56:06.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-26T22:56:06.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]': finished 2024-04-26T22:56:06.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-04-26T22:56:06.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-04-26T22:56:06.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-04-26T22:56:06.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:05 smithi080 ceph-mon[32324]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T22:56:06.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:05 smithi094 ceph-mon[32889]: pgmap v350: 129 pgs: 129 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 KiB/s wr, 13 op/s 2024-04-26T22:56:06.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-26T22:56:06.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4, 7, 11]}]': finished 2024-04-26T22:56:06.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-26T22:56:06.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-26T22:56:06.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-26T22:56:06.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]': finished 2024-04-26T22:56:06.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-04-26T22:56:06.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-04-26T22:56:06.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-04-26T22:56:06.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:05 smithi094 ceph-mon[32889]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T22:56:06.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.795+0000 7f81b1600700 1 -- 172.21.15.27:0/2434587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81ac10fa50 msgr2=0x7f81ac10fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:06.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.795+0000 7f81b1600700 1 --2- 172.21.15.27:0/2434587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81ac10fa50 0x7f81ac10fe30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f819c009990 tx=0x7f819c009ca0 comp rx=0 tx=0).stop 2024-04-26T22:56:06.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.795+0000 7f81b1600700 1 -- 172.21.15.27:0/2434587 shutdown_connections 2024-04-26T22:56:06.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.795+0000 7f81b1600700 1 --2- 172.21.15.27:0/2434587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81ac10a410 0x7f81ac116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:06.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.795+0000 7f81b1600700 1 --2- 172.21.15.27:0/2434587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81ac10fa50 0x7f81ac10fe30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:06.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.795+0000 7f81b1600700 1 --2- 172.21.15.27:0/2434587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81ac109a50 0x7f81ac109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:06.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.795+0000 7f81b1600700 1 -- 172.21.15.27:0/2434587 >> 172.21.15.27:0/2434587 conn(0x7f81ac076bf0 msgr2=0x7f81ac108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:06.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.795+0000 7f81b1600700 1 -- 172.21.15.27:0/2434587 shutdown_connections 2024-04-26T22:56:06.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.795+0000 7f81b1600700 1 -- 172.21.15.27:0/2434587 wait complete. 2024-04-26T22:56:06.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.797+0000 7f81b1600700 1 Processor -- start 2024-04-26T22:56:06.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.797+0000 7f81b1600700 1 -- start start 2024-04-26T22:56:06.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.797+0000 7f81b1600700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81ac109a50 0x7f81ac1a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:06.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.797+0000 7f81b1600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81ac10a410 0x7f81ac1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:06.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.797+0000 7f81b1600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81ac10fa50 0x7f81ac1a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:06.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.797+0000 7f81b1600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac119ca0 con 0x7f81ac10a410 2024-04-26T22:56:06.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.797+0000 7f81b1600700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac119b20 con 0x7f81ac109a50 2024-04-26T22:56:06.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.797+0000 7f81b1600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac119e20 con 0x7f81ac10fa50 2024-04-26T22:56:06.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.798+0000 7f81aaa00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81ac109a50 0x7f81ac1a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:06.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.798+0000 7f81aaa00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81ac109a50 0x7f81ac1a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:60210/0 (socket says 172.21.15.27:60210) 2024-04-26T22:56:06.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.798+0000 7f81aaa00700 1 -- 172.21.15.27:0/2445858295 learned_addr learned my addr 172.21.15.27:0/2445858295 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:06.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.798+0000 7f81a2000700 1 --2- 172.21.15.27:0/2445858295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81ac10a410 0x7f81ac1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:06.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.798+0000 7f81ab400700 1 --2- 172.21.15.27:0/2445858295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81ac10fa50 0x7f81ac1a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.798+0000 7f81aaa00700 1 -- 172.21.15.27:0/2445858295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81ac10fa50 msgr2=0x7f81ac1a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.798+0000 7f81aaa00700 1 --2- 172.21.15.27:0/2445858295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81ac10fa50 0x7f81ac1a4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.798+0000 7f81aaa00700 1 -- 172.21.15.27:0/2445858295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81ac10a410 msgr2=0x7f81ac1a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.798+0000 7f81aaa00700 1 --2- 172.21.15.27:0/2445858295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81ac10a410 0x7f81ac1a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.798+0000 7f81aaa00700 1 -- 172.21.15.27:0/2445858295 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ac1a5200 con 0x7f81ac109a50 2024-04-26T22:56:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.798+0000 7f81ab400700 1 --2- 172.21.15.27:0/2445858295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81ac10fa50 0x7f81ac1a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:56:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.799+0000 7f81aaa00700 1 --2- 172.21.15.27:0/2445858295 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81ac109a50 0x7f81ac1a01f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f819c000c00 tx=0x7f819c0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.799+0000 7f81a3e00700 1 -- 172.21.15.27:0/2445858295 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c027070 con 0x7f81ac109a50 2024-04-26T22:56:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.799+0000 7f81b1600700 1 -- 172.21.15.27:0/2445858295 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ac1a5490 con 0x7f81ac109a50 2024-04-26T22:56:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.799+0000 7f81a3e00700 1 -- 172.21.15.27:0/2445858295 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f819c022470 con 0x7f81ac109a50 2024-04-26T22:56:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.799+0000 7f81a3e00700 1 -- 172.21.15.27:0/2445858295 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c017650 con 0x7f81ac109a50 2024-04-26T22:56:06.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.799+0000 7f81b1600700 1 -- 172.21.15.27:0/2445858295 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ac1acd70 con 0x7f81ac109a50 2024-04-26T22:56:06.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.800+0000 7f81b1600700 1 -- 172.21.15.27:0/2445858295 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81ac10b190 con 0x7f81ac109a50 2024-04-26T22:56:06.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.802+0000 7f81a3e00700 1 -- 172.21.15.27:0/2445858295 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f819c02a070 con 0x7f81ac109a50 2024-04-26T22:56:06.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.804+0000 7f81a3e00700 1 --2- 172.21.15.27:0/2445858295 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f81900a3cd0 0x7f81900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:06.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.804+0000 7f81a3e00700 1 -- 172.21.15.27:0/2445858295 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f819c0d1580 con 0x7f81ac109a50 2024-04-26T22:56:06.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.804+0000 7f81a2000700 1 --2- 172.21.15.27:0/2445858295 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f81900a3cd0 0x7f81900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:06.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.805+0000 7f81a2000700 1 --2- 172.21.15.27:0/2445858295 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f81900a3cd0 0x7f81900a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f81ac078bc0 tx=0x7f8194009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:06.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:06.808+0000 7f81a3e00700 1 -- 172.21.15.27:0/2445858295 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f819c013070 con 0x7f81ac109a50 2024-04-26T22:56:07.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:06 smithi080 ceph-mon[32324]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T22:56:07.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:06 smithi094 ceph-mon[32889]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T22:56:07.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:07.107+0000 7f81b1600700 1 -- 172.21.15.27:0/2445858295 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f81ac1a1550 con 0x7f81ac109a50 2024-04-26T22:56:07.108 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:06 smithi027 ceph-mon[28034]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T22:56:07.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:07.112+0000 7f81a3e00700 1 -- 172.21.15.27:0/2445858295 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264998 (secure 0 0 0) 0x7f819c09daf0 con 0x7f81ac109a50 2024-04-26T22:56:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:56:07.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:07.120+0000 7f81b1600700 1 -- 172.21.15.27:0/2445858295 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f81900a3cd0 msgr2=0x7f81900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:07.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:07.120+0000 7f81b1600700 1 --2- 172.21.15.27:0/2445858295 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f81900a3cd0 0x7f81900a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f81ac078bc0 tx=0x7f8194009260 comp rx=0 tx=0).stop 2024-04-26T22:56:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:07.120+0000 7f81b1600700 1 -- 172.21.15.27:0/2445858295 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81ac109a50 msgr2=0x7f81ac1a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:07.120+0000 7f81b1600700 1 --2- 172.21.15.27:0/2445858295 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81ac109a50 0x7f81ac1a01f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f819c000c00 tx=0x7f819c0057c0 comp rx=0 tx=0).stop 2024-04-26T22:56:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:07.120+0000 7f81b1600700 1 -- 172.21.15.27:0/2445858295 shutdown_connections 2024-04-26T22:56:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:07.120+0000 7f81b1600700 1 --2- 172.21.15.27:0/2445858295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81ac10fa50 0x7f81ac1a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:07.120+0000 7f81b1600700 1 --2- 172.21.15.27:0/2445858295 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f81900a3cd0 0x7f81900a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:07.120+0000 7f81b1600700 1 --2- 172.21.15.27:0/2445858295 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81ac109a50 0x7f81ac1a01f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:07.120+0000 7f81b1600700 1 --2- 172.21.15.27:0/2445858295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81ac10a410 0x7f81ac1a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:07.120+0000 7f81b1600700 1 -- 172.21.15.27:0/2445858295 >> 172.21.15.27:0/2445858295 conn(0x7f81ac076bf0 msgr2=0x7f81ac077250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:07.120+0000 7f81b1600700 1 -- 172.21.15.27:0/2445858295 shutdown_connections 2024-04-26T22:56:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:07.120+0000 7f81b1600700 1 -- 172.21.15.27:0/2445858295 wait complete. 2024-04-26T22:56:08.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:07 smithi080 ceph-mon[32324]: pgmap v353: 129 pgs: 2 peering, 127 active+clean; 630 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.7 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:56:08.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:07 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2445858295' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T22:56:08.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:07 smithi094 ceph-mon[32889]: pgmap v353: 129 pgs: 2 peering, 127 active+clean; 630 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.7 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:56:08.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:07 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2445858295' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T22:56:08.071 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:07 smithi027 ceph-mon[28034]: pgmap v353: 129 pgs: 2 peering, 127 active+clean; 630 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.7 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:56:08.071 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:07 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2445858295' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T22:56:08.121 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3636526714},{"type":"v1","addr":"172.21.15.27:6801","nonce":3636526714}]},"active_addr":"172.21.15.27:6801/3636526714","active_change":"2024-04-26T22:47:04.292588+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.27:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":3276968774}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":3449139270}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":1199904816}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":3746496955}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":4202829010}]}]} 2024-04-26T22:56:08.127 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T22:56:08.127 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T22:56:08.127 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T22:56:08.668 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:10.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:09 smithi080 ceph-mon[32324]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 630 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 KiB/s wr, 8 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T22:56:10.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:09 smithi094 ceph-mon[32889]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 630 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 KiB/s wr, 8 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T22:56:10.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:09 smithi027 ceph-mon[28034]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 630 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 KiB/s wr, 8 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T22:56:11.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.321+0000 7f64f3000700 1 -- 172.21.15.27:0/2372472180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ec06b130 msgr2=0x7f64ec06b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.321+0000 7f64f3000700 1 --2- 172.21.15.27:0/2372472180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ec06b130 0x7f64ec06b590 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009990 tx=0x7f64d4009ca0 comp rx=0 tx=0).stop 2024-04-26T22:56:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.321+0000 7f64f3000700 1 -- 172.21.15.27:0/2372472180 shutdown_connections 2024-04-26T22:56:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.321+0000 7f64f3000700 1 --2- 172.21.15.27:0/2372472180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64ec06bad0 0x7f64ec1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.321+0000 7f64f3000700 1 --2- 172.21.15.27:0/2372472180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64ec1094a0 0x7f64ec109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.321+0000 7f64f3000700 1 --2- 172.21.15.27:0/2372472180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ec06b130 0x7f64ec06b590 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.321+0000 7f64f3000700 1 -- 172.21.15.27:0/2372472180 >> 172.21.15.27:0/2372472180 conn(0x7f64ec0792d0 msgr2=0x7f64ec0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.321+0000 7f64f3000700 1 -- 172.21.15.27:0/2372472180 shutdown_connections 2024-04-26T22:56:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.321+0000 7f64f3000700 1 -- 172.21.15.27:0/2372472180 wait complete. 2024-04-26T22:56:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.322+0000 7f64f3000700 1 Processor -- start 2024-04-26T22:56:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.322+0000 7f64f3000700 1 -- start start 2024-04-26T22:56:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.323+0000 7f64f3000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ec06b130 0x7f64ec1023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.323+0000 7f64f3000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64ec06bad0 0x7f64ec102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.323+0000 7f64f3000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64ec1094a0 0x7f64ec102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.323+0000 7f64f3000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec113600 con 0x7f64ec06bad0 2024-04-26T22:56:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.323+0000 7f64f3000700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec113480 con 0x7f64ec1094a0 2024-04-26T22:56:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.323+0000 7f64f3000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec113780 con 0x7f64ec06b130 2024-04-26T22:56:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.323+0000 7f64f1000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64ec1094a0 0x7f64ec102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.323+0000 7f64e3400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64ec06bad0 0x7f64ec102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.323+0000 7f64ebe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ec06b130 0x7f64ec1023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.323+0000 7f64f1000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64ec1094a0 0x7f64ec102e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:38308/0 (socket says 172.21.15.27:38308) 2024-04-26T22:56:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.323+0000 7f64f1000700 1 -- 172.21.15.27:0/1240225415 learned_addr learned my addr 172.21.15.27:0/1240225415 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.324+0000 7f64f1000700 1 -- 172.21.15.27:0/1240225415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ec06b130 msgr2=0x7f64ec1023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.324+0000 7f64f1000700 1 --2- 172.21.15.27:0/1240225415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ec06b130 0x7f64ec1023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.324+0000 7f64f1000700 1 -- 172.21.15.27:0/1240225415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64ec06bad0 msgr2=0x7f64ec102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.324+0000 7f64f1000700 1 --2- 172.21.15.27:0/1240225415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64ec06bad0 0x7f64ec102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.324+0000 7f64f1000700 1 -- 172.21.15.27:0/1240225415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ec1a8120 con 0x7f64ec1094a0 2024-04-26T22:56:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.324+0000 7f64ebe00700 1 --2- 172.21.15.27:0/1240225415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ec06b130 0x7f64ec1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:56:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.324+0000 7f64f1000700 1 --2- 172.21.15.27:0/1240225415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64ec1094a0 0x7f64ec102e70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f64e400ef30 tx=0x7f64e400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.324+0000 7f64e9600700 1 -- 172.21.15.27:0/1240225415 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e40092e0 con 0x7f64ec1094a0 2024-04-26T22:56:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.324+0000 7f64e9600700 1 -- 172.21.15.27:0/1240225415 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64e4004bb0 con 0x7f64ec1094a0 2024-04-26T22:56:11.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.324+0000 7f64f3000700 1 -- 172.21.15.27:0/1240225415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ec1a83b0 con 0x7f64ec1094a0 2024-04-26T22:56:11.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.325+0000 7f64e9600700 1 -- 172.21.15.27:0/1240225415 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4004020 con 0x7f64ec1094a0 2024-04-26T22:56:11.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.325+0000 7f64f3000700 1 -- 172.21.15.27:0/1240225415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ec1a88f0 con 0x7f64ec1094a0 2024-04-26T22:56:11.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.326+0000 7f64f3000700 1 -- 172.21.15.27:0/1240225415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ec06a110 con 0x7f64ec1094a0 2024-04-26T22:56:11.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.328+0000 7f64e9600700 1 -- 172.21.15.27:0/1240225415 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f64e401b030 con 0x7f64ec1094a0 2024-04-26T22:56:11.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.329+0000 7f64e9600700 1 --2- 172.21.15.27:0/1240225415 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64d00a3cd0 0x7f64d00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:11.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.330+0000 7f64e9600700 1 -- 172.21.15.27:0/1240225415 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f64e40d0340 con 0x7f64ec1094a0 2024-04-26T22:56:11.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.330+0000 7f64ebe00700 1 --2- 172.21.15.27:0/1240225415 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64d00a3cd0 0x7f64d00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:11.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.330+0000 7f64ebe00700 1 --2- 172.21.15.27:0/1240225415 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64d00a3cd0 0x7f64d00a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009fd0 tx=0x7f64dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:11.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.334+0000 7f64e9600700 1 -- 172.21.15.27:0/1240225415 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f64e4022020 con 0x7f64ec1094a0 2024-04-26T22:56:11.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.571+0000 7f64f3000700 1 -- 172.21.15.27:0/1240225415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64ec02e490 con 0x7f64ec1094a0 2024-04-26T22:56:11.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.573+0000 7f64e9600700 1 -- 172.21.15.27:0/1240225415 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24508 (secure 0 0 0) 0x7f64e4010040 con 0x7f64ec1094a0 2024-04-26T22:56:11.574 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:56:11.574 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":92,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:56:05.706239+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4},{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:56:11.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.577+0000 7f64f3000700 1 -- 172.21.15.27:0/1240225415 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64d00a3cd0 msgr2=0x7f64d00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:11.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.577+0000 7f64f3000700 1 --2- 172.21.15.27:0/1240225415 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64d00a3cd0 0x7f64d00a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009fd0 tx=0x7f64dc009260 comp rx=0 tx=0).stop 2024-04-26T22:56:11.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.577+0000 7f64f3000700 1 -- 172.21.15.27:0/1240225415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64ec1094a0 msgr2=0x7f64ec102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:11.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.577+0000 7f64f3000700 1 --2- 172.21.15.27:0/1240225415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64ec1094a0 0x7f64ec102e70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f64e400ef30 tx=0x7f64e400c3a0 comp rx=0 tx=0).stop 2024-04-26T22:56:11.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.577+0000 7f64f3000700 1 -- 172.21.15.27:0/1240225415 shutdown_connections 2024-04-26T22:56:11.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.577+0000 7f64f3000700 1 --2- 172.21.15.27:0/1240225415 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f64d00a3cd0 0x7f64d00a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:11.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.577+0000 7f64f3000700 1 --2- 172.21.15.27:0/1240225415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64ec1094a0 0x7f64ec102e70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:11.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.577+0000 7f64f3000700 1 --2- 172.21.15.27:0/1240225415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64ec06b130 0x7f64ec1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:11.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.577+0000 7f64f3000700 1 --2- 172.21.15.27:0/1240225415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64ec06bad0 0x7f64ec102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:11.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.577+0000 7f64f3000700 1 -- 172.21.15.27:0/1240225415 >> 172.21.15.27:0/1240225415 conn(0x7f64ec0792d0 msgr2=0x7f64ec100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:11.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.577+0000 7f64f3000700 1 -- 172.21.15.27:0/1240225415 shutdown_connections 2024-04-26T22:56:11.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:11.578+0000 7f64f3000700 1 -- 172.21.15.27:0/1240225415 wait complete. 2024-04-26T22:56:11.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:11 smithi027 ceph-mon[28034]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 630 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T22:56:11.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:11 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1240225415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:56:12.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:11 smithi080 ceph-mon[32324]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 630 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T22:56:12.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:11 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1240225415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:56:12.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:11 smithi094 ceph-mon[32889]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 630 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T22:56:12.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:11 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1240225415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:56:12.531 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T22:56:12.532 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T22:56:13.014 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:14.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:13 smithi080 ceph-mon[32324]: pgmap v356: 129 pgs: 129 active+clean; 630 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T22:56:14.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:13 smithi094 ceph-mon[32889]: pgmap v356: 129 pgs: 129 active+clean; 630 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T22:56:14.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:13 smithi027 ceph-mon[28034]: pgmap v356: 129 pgs: 129 active+clean; 630 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T22:56:15.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.428+0000 7f00efc00700 1 -- 172.21.15.27:0/1237717457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00e8106000 msgr2=0x7f00e8112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:15.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.428+0000 7f00efc00700 1 --2- 172.21.15.27:0/1237717457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00e8106000 0x7f00e8112510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f00e000b0a0 tx=0x7f00e000b3b0 comp rx=0 tx=0).stop 2024-04-26T22:56:15.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.428+0000 7f00efc00700 1 -- 172.21.15.27:0/1237717457 shutdown_connections 2024-04-26T22:56:15.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.428+0000 7f00efc00700 1 --2- 172.21.15.27:0/1237717457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00e8106000 0x7f00e8112510 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:15.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.428+0000 7f00efc00700 1 --2- 172.21.15.27:0/1237717457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e810b640 0x7f00e810ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:15.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.428+0000 7f00efc00700 1 --2- 172.21.15.27:0/1237717457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00e8105640 0x7f00e8105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:15.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.428+0000 7f00efc00700 1 -- 172.21.15.27:0/1237717457 >> 172.21.15.27:0/1237717457 conn(0x7f00e8101130 msgr2=0x7f00e8103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:15.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.428+0000 7f00efc00700 1 -- 172.21.15.27:0/1237717457 shutdown_connections 2024-04-26T22:56:15.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.428+0000 7f00efc00700 1 -- 172.21.15.27:0/1237717457 wait complete. 2024-04-26T22:56:15.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.429+0000 7f00efc00700 1 Processor -- start 2024-04-26T22:56:15.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.430+0000 7f00efc00700 1 -- start start 2024-04-26T22:56:15.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.430+0000 7f00efc00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00e8105640 0x7f00e81a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:15.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.430+0000 7f00efc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e8106000 0x7f00e81a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:15.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.430+0000 7f00efc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00e810b640 0x7f00e819bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:15.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.430+0000 7f00efc00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81158d0 con 0x7f00e8106000 2024-04-26T22:56:15.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.430+0000 7f00efc00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8115750 con 0x7f00e8105640 2024-04-26T22:56:15.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.430+0000 7f00efc00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8115a50 con 0x7f00e810b640 2024-04-26T22:56:15.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.430+0000 7f00e7e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e8106000 0x7f00e81a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:15.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.430+0000 7f00ed200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00e8105640 0x7f00e81a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.430+0000 7f00e7e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e8106000 0x7f00e81a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40220/0 (socket says 172.21.15.27:40220) 2024-04-26T22:56:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.430+0000 7f00e7e00700 1 -- 172.21.15.27:0/2324552877 learned_addr learned my addr 172.21.15.27:0/2324552877 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.430+0000 7f00edc00700 1 --2- 172.21.15.27:0/2324552877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00e810b640 0x7f00e819bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.431+0000 7f00ed200700 1 -- 172.21.15.27:0/2324552877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00e810b640 msgr2=0x7f00e819bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.431+0000 7f00ed200700 1 --2- 172.21.15.27:0/2324552877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00e810b640 0x7f00e819bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.431+0000 7f00ed200700 1 -- 172.21.15.27:0/2324552877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e8106000 msgr2=0x7f00e81a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.431+0000 7f00ed200700 1 --2- 172.21.15.27:0/2324552877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e8106000 0x7f00e81a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.431+0000 7f00ed200700 1 -- 172.21.15.27:0/2324552877 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e819c230 con 0x7f00e8105640 2024-04-26T22:56:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.431+0000 7f00ed200700 1 --2- 172.21.15.27:0/2324552877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00e8105640 0x7f00e81a1910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00e9f0 tx=0x7f00dc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.431+0000 7f00e5600700 1 -- 172.21.15.27:0/2324552877 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00c9c0 con 0x7f00e8105640 2024-04-26T22:56:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.431+0000 7f00e5600700 1 -- 172.21.15.27:0/2324552877 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00dc00cb60 con 0x7f00e8105640 2024-04-26T22:56:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.431+0000 7f00e5600700 1 -- 172.21.15.27:0/2324552877 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc018680 con 0x7f00e8105640 2024-04-26T22:56:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.432+0000 7f00efc00700 1 -- 172.21.15.27:0/2324552877 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e819c520 con 0x7f00e8105640 2024-04-26T22:56:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.432+0000 7f00efc00700 1 -- 172.21.15.27:0/2324552877 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e81a87e0 con 0x7f00e8105640 2024-04-26T22:56:15.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.433+0000 7f00efc00700 1 -- 172.21.15.27:0/2324552877 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e806a110 con 0x7f00e8105640 2024-04-26T22:56:15.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.435+0000 7f00e5600700 1 -- 172.21.15.27:0/2324552877 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f00dc010430 con 0x7f00e8105640 2024-04-26T22:56:15.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.436+0000 7f00e5600700 1 --2- 172.21.15.27:0/2324552877 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f00d00a3d50 0x7f00d00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:15.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.436+0000 7f00e5600700 1 -- 172.21.15.27:0/2324552877 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f00dc0d0960 con 0x7f00e8105640 2024-04-26T22:56:15.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.437+0000 7f00e7e00700 1 --2- 172.21.15.27:0/2324552877 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f00d00a3d50 0x7f00d00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:15.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.437+0000 7f00e7e00700 1 --2- 172.21.15.27:0/2324552877 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f00d00a3d50 0x7f00d00a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f00e8103780 tx=0x7f00d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.441+0000 7f00e5600700 1 -- 172.21.15.27:0/2324552877 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f00dc09ce40 con 0x7f00e8105640 2024-04-26T22:56:15.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.673+0000 7f00efc00700 1 -- 172.21.15.27:0/2324552877 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00e802e490 con 0x7f00e8105640 2024-04-26T22:56:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.675+0000 7f00e5600700 1 -- 172.21.15.27:0/2324552877 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24508 (secure 0 0 0) 0x7f00dc09c740 con 0x7f00e8105640 2024-04-26T22:56:15.677 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:56:15.677 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":92,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:56:05.706239+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4},{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:56:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.679+0000 7f00efc00700 1 -- 172.21.15.27:0/2324552877 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f00d00a3d50 msgr2=0x7f00d00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.679+0000 7f00efc00700 1 --2- 172.21.15.27:0/2324552877 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f00d00a3d50 0x7f00d00a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f00e8103780 tx=0x7f00d4009260 comp rx=0 tx=0).stop 2024-04-26T22:56:15.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.679+0000 7f00efc00700 1 -- 172.21.15.27:0/2324552877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00e8105640 msgr2=0x7f00e81a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:15.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.679+0000 7f00efc00700 1 --2- 172.21.15.27:0/2324552877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00e8105640 0x7f00e81a1910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00e9f0 tx=0x7f00dc00ed00 comp rx=0 tx=0).stop 2024-04-26T22:56:15.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.679+0000 7f00efc00700 1 -- 172.21.15.27:0/2324552877 shutdown_connections 2024-04-26T22:56:15.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.679+0000 7f00efc00700 1 --2- 172.21.15.27:0/2324552877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00e810b640 0x7f00e819bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:15.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.679+0000 7f00efc00700 1 --2- 172.21.15.27:0/2324552877 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f00d00a3d50 0x7f00d00a6210 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:15.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.679+0000 7f00efc00700 1 --2- 172.21.15.27:0/2324552877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00e8105640 0x7f00e81a1910 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:15.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.679+0000 7f00efc00700 1 --2- 172.21.15.27:0/2324552877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e8106000 0x7f00e81a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:15.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.679+0000 7f00efc00700 1 -- 172.21.15.27:0/2324552877 >> 172.21.15.27:0/2324552877 conn(0x7f00e8101130 msgr2=0x7f00e81018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:15.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.679+0000 7f00efc00700 1 -- 172.21.15.27:0/2324552877 shutdown_connections 2024-04-26T22:56:15.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:15.679+0000 7f00efc00700 1 -- 172.21.15.27:0/2324552877 wait complete. 2024-04-26T22:56:16.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:15 smithi080 ceph-mon[32324]: pgmap v357: 129 pgs: 129 active+clean; 630 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 629 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T22:56:16.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:15 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2324552877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:56:16.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:15 smithi094 ceph-mon[32889]: pgmap v357: 129 pgs: 129 active+clean; 630 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 629 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T22:56:16.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:15 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2324552877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:56:16.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:15 smithi027 ceph-mon[28034]: pgmap v357: 129 pgs: 129 active+clean; 630 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 629 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T22:56:16.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:15 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2324552877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:56:16.611 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T22:56:16.612 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T22:56:16.612 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T22:56:16.613 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T22:56:16.613 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T22:56:16.614 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T22:56:16.614 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-26T22:56:16.614 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-26T22:56:16.615 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-26T22:56:16.615 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-26T22:56:16.616 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-26T22:56:16.616 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-26T22:56:17.797 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:17.888 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:17.977 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:18.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:17 smithi080 ceph-mon[32324]: pgmap v358: 129 pgs: 129 active+clean; 633 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.0 KiB/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T22:56:18.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:17 smithi094 ceph-mon[32889]: pgmap v358: 129 pgs: 129 active+clean; 633 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.0 KiB/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T22:56:18.048 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:17 smithi027 ceph-mon[28034]: pgmap v358: 129 pgs: 129 active+clean; 633 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.0 KiB/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T22:56:19.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:19 smithi027 ceph-mon[28034]: pgmap v359: 129 pgs: 129 active+clean; 633 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T22:56:20.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:19 smithi094 ceph-mon[32889]: pgmap v359: 129 pgs: 129 active+clean; 633 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T22:56:20.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:19 smithi080 ceph-mon[32324]: pgmap v359: 129 pgs: 129 active+clean; 633 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T22:56:21.657 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:21.936 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:21 smithi027 ceph-mon[28034]: pgmap v360: 129 pgs: 129 active+clean; 633 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-26T22:56:22.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:21 smithi094 ceph-mon[32889]: pgmap v360: 129 pgs: 129 active+clean; 633 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-26T22:56:22.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:21 smithi080 ceph-mon[32324]: pgmap v360: 129 pgs: 129 active+clean; 633 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-26T22:56:22.490 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:23.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.445+0000 7fbebdc00700 1 -- 172.21.15.27:0/2570542800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb81035b0 msgr2=0x7fbeb8103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.445+0000 7fbebdc00700 1 --2- 172.21.15.27:0/2570542800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb81035b0 0x7fbeb8103990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbea80099c0 tx=0x7fbea8009cd0 comp rx=0 tx=0).stop 2024-04-26T22:56:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.445+0000 7fbebdc00700 1 -- 172.21.15.27:0/2570542800 shutdown_connections 2024-04-26T22:56:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.445+0000 7fbebdc00700 1 --2- 172.21.15.27:0/2570542800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbeb8108690 0x7fbeb8110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.445+0000 7fbebdc00700 1 --2- 172.21.15.27:0/2570542800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb81035b0 0x7fbeb8103990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.445+0000 7fbebdc00700 1 --2- 172.21.15.27:0/2570542800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeb8103f60 0x7fbeb8107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.445+0000 7fbebdc00700 1 -- 172.21.15.27:0/2570542800 >> 172.21.15.27:0/2570542800 conn(0x7fbeb80feea0 msgr2=0x7fbeb81012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.445+0000 7fbebdc00700 1 -- 172.21.15.27:0/2570542800 shutdown_connections 2024-04-26T22:56:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.445+0000 7fbebdc00700 1 -- 172.21.15.27:0/2570542800 wait complete. 2024-04-26T22:56:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.446+0000 7fbebdc00700 1 Processor -- start 2024-04-26T22:56:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.446+0000 7fbebdc00700 1 -- start start 2024-04-26T22:56:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.447+0000 7fbebdc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb81035b0 0x7fbeb8075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.447+0000 7fbebdc00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbeb8103f60 0x7fbeb8075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.447+0000 7fbebdc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeb8108690 0x7fbeb80761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.447+0000 7fbebdc00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8113570 con 0x7fbeb8108690 2024-04-26T22:56:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.447+0000 7fbebdc00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81133f0 con 0x7fbeb8103f60 2024-04-26T22:56:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.447+0000 7fbebdc00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81136f0 con 0x7fbeb81035b0 2024-04-26T22:56:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.447+0000 7fbeb7e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb81035b0 0x7fbeb8075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.447+0000 7fbeb7400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbeb8103f60 0x7fbeb8075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.447+0000 7fbeb7e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb81035b0 0x7fbeb8075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:51348/0 (socket says 172.21.15.27:51348) 2024-04-26T22:56:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.447+0000 7fbeb7e00700 1 -- 172.21.15.27:0/592797892 learned_addr learned my addr 172.21.15.27:0/592797892 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.447+0000 7fbebd200700 1 --2- 172.21.15.27:0/592797892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeb8108690 0x7fbeb80761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.448+0000 7fbeb7e00700 1 -- 172.21.15.27:0/592797892 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbeb8103f60 msgr2=0x7fbeb8075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.448+0000 7fbeb7e00700 1 --2- 172.21.15.27:0/592797892 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbeb8103f60 0x7fbeb8075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.448+0000 7fbeb7e00700 1 -- 172.21.15.27:0/592797892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeb8108690 msgr2=0x7fbeb80761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.448+0000 7fbeb7e00700 1 --2- 172.21.15.27:0/592797892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeb8108690 0x7fbeb80761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.448+0000 7fbeb7e00700 1 -- 172.21.15.27:0/592797892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb81a7f70 con 0x7fbeb81035b0 2024-04-26T22:56:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.448+0000 7fbeb7400700 1 --2- 172.21.15.27:0/592797892 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbeb8103f60 0x7fbeb8075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:56:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.448+0000 7fbeb7e00700 1 --2- 172.21.15.27:0/592797892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb81035b0 0x7fbeb8075720 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbea8005910 tx=0x7fbea8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.449+0000 7fbeb4c00700 1 -- 172.21.15.27:0/592797892 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8026070 con 0x7fbeb81035b0 2024-04-26T22:56:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.449+0000 7fbeb4c00700 1 -- 172.21.15.27:0/592797892 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea8003c70 con 0x7fbeb81035b0 2024-04-26T22:56:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.449+0000 7fbebdc00700 1 -- 172.21.15.27:0/592797892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb81a8200 con 0x7fbeb81035b0 2024-04-26T22:56:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.449+0000 7fbeb4c00700 1 -- 172.21.15.27:0/592797892 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea800fa40 con 0x7fbeb81035b0 2024-04-26T22:56:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.449+0000 7fbebdc00700 1 -- 172.21.15.27:0/592797892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb81a86b0 con 0x7fbeb81035b0 2024-04-26T22:56:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.450+0000 7fbebdc00700 1 -- 172.21.15.27:0/592797892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbeb8063b30 con 0x7fbeb81035b0 2024-04-26T22:56:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.452+0000 7fbeb4c00700 1 -- 172.21.15.27:0/592797892 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbea8029070 con 0x7fbeb81035b0 2024-04-26T22:56:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.453+0000 7fbeb4c00700 1 --2- 172.21.15.27:0/592797892 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbe9c0a3d50 0x7fbe9c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:23.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.454+0000 7fbeb4c00700 1 -- 172.21.15.27:0/592797892 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fbea80d1820 con 0x7fbeb81035b0 2024-04-26T22:56:23.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.454+0000 7fbeb7400700 1 --2- 172.21.15.27:0/592797892 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbe9c0a3d50 0x7fbe9c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:23.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.454+0000 7fbeb4c00700 1 --2- 172.21.15.27:0/592797892 >> [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302] conn(0x7fbe9c0aee30 0x7fbe9c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:23.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.454+0000 7fbeb4c00700 1 -- 172.21.15.27:0/592797892 --> [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe9c0b19a0 con 0x7fbe9c0aee30 2024-04-26T22:56:23.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.454+0000 7fbeb4c00700 1 -- 172.21.15.27:0/592797892 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fbea80d1bd0 con 0x7fbeb81035b0 2024-04-26T22:56:23.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.454+0000 7fbebd200700 1 --2- 172.21.15.27:0/592797892 >> [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302] conn(0x7fbe9c0aee30 0x7fbe9c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:23.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.455+0000 7fbeb7400700 1 --2- 172.21.15.27:0/592797892 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbe9c0a3d50 0x7fbe9c0a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009680 tx=0x7fbea4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:23.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.455+0000 7fbebd200700 1 --2- 172.21.15.27:0/592797892 >> [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302] conn(0x7fbe9c0aee30 0x7fbe9c0b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:23.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.456+0000 7fbeb4c00700 1 -- 172.21.15.27:0/592797892 <== osd.5 v2:172.21.15.94:6808/396894302 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe9c0b19a0 con 0x7fbe9c0aee30 2024-04-26T22:56:23.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.474+0000 7fbebdc00700 1 -- 172.21.15.27:0/592797892 --> [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbeb8063fb0 con 0x7fbe9c0aee30 2024-04-26T22:56:23.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.475+0000 7fbeb4c00700 1 -- 172.21.15.27:0/592797892 <== osd.5 v2:172.21.15.94:6808/396894302 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbeb8063fb0 con 0x7fbe9c0aee30 2024-04-26T22:56:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.476+0000 7fbebdc00700 1 -- 172.21.15.27:0/592797892 >> [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302] conn(0x7fbe9c0aee30 msgr2=0x7fbe9c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.476+0000 7fbebdc00700 1 --2- 172.21.15.27:0/592797892 >> [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302] conn(0x7fbe9c0aee30 0x7fbe9c0b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.476+0000 7fbebdc00700 1 -- 172.21.15.27:0/592797892 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbe9c0a3d50 msgr2=0x7fbe9c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.476+0000 7fbebdc00700 1 --2- 172.21.15.27:0/592797892 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbe9c0a3d50 0x7fbe9c0a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009680 tx=0x7fbea4009260 comp rx=0 tx=0).stop 2024-04-26T22:56:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.476+0000 7fbebdc00700 1 -- 172.21.15.27:0/592797892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb81035b0 msgr2=0x7fbeb8075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.476+0000 7fbebdc00700 1 --2- 172.21.15.27:0/592797892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb81035b0 0x7fbeb8075720 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbea8005910 tx=0x7fbea8004270 comp rx=0 tx=0).stop 2024-04-26T22:56:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.476+0000 7fbebd200700 1 -- 172.21.15.27:0/592797892 reap_dead start 2024-04-26T22:56:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.476+0000 7fbebdc00700 1 -- 172.21.15.27:0/592797892 shutdown_connections 2024-04-26T22:56:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.476+0000 7fbebdc00700 1 -- 172.21.15.27:0/592797892 >> 172.21.15.27:0/592797892 conn(0x7fbeb80feea0 msgr2=0x7fbeb8107820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.476+0000 7fbebdc00700 1 -- 172.21.15.27:0/592797892 shutdown_connections 2024-04-26T22:56:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:23.476+0000 7fbebdc00700 1 -- 172.21.15.27:0/592797892 wait complete. 2024-04-26T22:56:23.827 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:23.842 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:23.843 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:23.845 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:23.846 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:23.847 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:24.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:23 smithi094 ceph-mon[32889]: pgmap v361: 129 pgs: 129 active+clean; 633 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T22:56:24.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:23 smithi027 ceph-mon[28034]: pgmap v361: 129 pgs: 129 active+clean; 633 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T22:56:24.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:23 smithi080 ceph-mon[32324]: pgmap v361: 129 pgs: 129 active+clean; 633 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T22:56:26.090 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:25 smithi027 ceph-mon[28034]: pgmap v362: 129 pgs: 129 active+clean; 633 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-26T22:56:26.090 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:56:26.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.143+0000 7fefa5000700 1 -- 172.21.15.27:0/1365793030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefa0102d80 msgr2=0x7fefa0103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:26.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.143+0000 7fefa5000700 1 --2- 172.21.15.27:0/1365793030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefa0102d80 0x7fefa0103160 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fef900099c0 tx=0x7fef90009cd0 comp rx=0 tx=0).stop 2024-04-26T22:56:26.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.143+0000 7fefa5000700 1 -- 172.21.15.27:0/1365793030 shutdown_connections 2024-04-26T22:56:26.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.143+0000 7fefa5000700 1 --2- 172.21.15.27:0/1365793030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa01076a0 0x7fefa010f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:26.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.143+0000 7fefa5000700 1 --2- 172.21.15.27:0/1365793030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefa0102d80 0x7fefa0103160 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:26.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.143+0000 7fefa5000700 1 --2- 172.21.15.27:0/1365793030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fefa0103730 0x7fefa0106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:26.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.143+0000 7fefa5000700 1 -- 172.21.15.27:0/1365793030 >> 172.21.15.27:0/1365793030 conn(0x7fefa0078990 msgr2=0x7fefa00ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:26.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.143+0000 7fefa5000700 1 -- 172.21.15.27:0/1365793030 shutdown_connections 2024-04-26T22:56:26.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.143+0000 7fefa5000700 1 -- 172.21.15.27:0/1365793030 wait complete. 2024-04-26T22:56:26.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.145+0000 7fefa5000700 1 Processor -- start 2024-04-26T22:56:26.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.145+0000 7fefa5000700 1 -- start start 2024-04-26T22:56:26.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.146+0000 7fefa5000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fefa0102d80 0x7fefa01a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:26.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.146+0000 7fefa5000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefa0103730 0x7fefa01a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:26.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.146+0000 7fefa5000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa01076a0 0x7fefa01a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:26.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.146+0000 7fefa5000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fefa01125f0 con 0x7fefa0103730 2024-04-26T22:56:26.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.146+0000 7fefa5000700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fefa0112470 con 0x7fefa0102d80 2024-04-26T22:56:26.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.146+0000 7fefa5000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fefa0112770 con 0x7fefa01076a0 2024-04-26T22:56:26.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.147+0000 7fef9d400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefa0103730 0x7fefa01a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:26.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.147+0000 7fef9de00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fefa0102d80 0x7fefa01a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:26.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.147+0000 7fef9e800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa01076a0 0x7fefa01a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:26.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.147+0000 7fef9e800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa01076a0 0x7fefa01a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:51366/0 (socket says 172.21.15.27:51366) 2024-04-26T22:56:26.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.147+0000 7fef9e800700 1 -- 172.21.15.27:0/2615774065 learned_addr learned my addr 172.21.15.27:0/2615774065 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:26.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.147+0000 7fef9e800700 1 -- 172.21.15.27:0/2615774065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fefa0102d80 msgr2=0x7fefa01a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:26.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.147+0000 7fef9e800700 1 --2- 172.21.15.27:0/2615774065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fefa0102d80 0x7fefa01a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:26.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.147+0000 7fef9e800700 1 -- 172.21.15.27:0/2615774065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefa0103730 msgr2=0x7fefa01a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:26.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.147+0000 7fef9e800700 1 --2- 172.21.15.27:0/2615774065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefa0103730 0x7fefa01a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:26.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.147+0000 7fef9e800700 1 -- 172.21.15.27:0/2615774065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa01a5240 con 0x7fefa01076a0 2024-04-26T22:56:26.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.148+0000 7fef9e800700 1 --2- 172.21.15.27:0/2615774065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa01076a0 0x7fefa01a4b70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fef9800ef30 tx=0x7fef9800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:26.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.148+0000 7fef96a00700 1 -- 172.21.15.27:0/2615774065 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef980092e0 con 0x7fefa01076a0 2024-04-26T22:56:26.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.148+0000 7fef96a00700 1 -- 172.21.15.27:0/2615774065 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef98004bb0 con 0x7fefa01076a0 2024-04-26T22:56:26.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.148+0000 7fefa5000700 1 -- 172.21.15.27:0/2615774065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa01a54d0 con 0x7fefa01076a0 2024-04-26T22:56:26.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.148+0000 7fefa5000700 1 -- 172.21.15.27:0/2615774065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa01ace20 con 0x7fefa01076a0 2024-04-26T22:56:26.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.148+0000 7fef96a00700 1 -- 172.21.15.27:0/2615774065 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98004280 con 0x7fefa01076a0 2024-04-26T22:56:26.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.150+0000 7fefa5000700 1 -- 172.21.15.27:0/2615774065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef80001030 con 0x7fefa01076a0 2024-04-26T22:56:26.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.154+0000 7fef96a00700 1 -- 172.21.15.27:0/2615774065 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fef9801b030 con 0x7fefa01076a0 2024-04-26T22:56:26.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.157+0000 7fef96a00700 1 --2- 172.21.15.27:0/2615774065 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fef840a3d50 0x7fef840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:26.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.157+0000 7fef96a00700 1 -- 172.21.15.27:0/2615774065 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fef980d0cd0 con 0x7fefa01076a0 2024-04-26T22:56:26.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.158+0000 7fef96a00700 1 --2- 172.21.15.27:0/2615774065 >> [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391] conn(0x7fef840aee30 0x7fef840b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:26.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.158+0000 7fef96a00700 1 -- 172.21.15.27:0/2615774065 --> [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef840b19a0 con 0x7fef840aee30 2024-04-26T22:56:26.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.158+0000 7fef96a00700 1 -- 172.21.15.27:0/2615774065 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fef980d1080 con 0x7fefa01076a0 2024-04-26T22:56:26.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.159+0000 7fef9d400700 1 --2- 172.21.15.27:0/2615774065 >> [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391] conn(0x7fef840aee30 0x7fef840b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:26.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.159+0000 7fef9de00700 1 --2- 172.21.15.27:0/2615774065 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fef840a3d50 0x7fef840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:26.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.160+0000 7fef9d400700 1 --2- 172.21.15.27:0/2615774065 >> [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391] conn(0x7fef840aee30 0x7fef840b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:26.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.160+0000 7fef9de00700 1 --2- 172.21.15.27:0/2615774065 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fef840a3d50 0x7fef840a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fef90003760 tx=0x7fef90005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:26.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.161+0000 7fef96a00700 1 -- 172.21.15.27:0/2615774065 <== osd.10 v2:172.21.15.80:6824/3521249391 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef840b19a0 con 0x7fef840aee30 2024-04-26T22:56:26.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.202+0000 7fefa5000700 1 -- 172.21.15.27:0/2615774065 --> [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fef80002e60 con 0x7fef840aee30 2024-04-26T22:56:26.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.202+0000 7fef96a00700 1 -- 172.21.15.27:0/2615774065 <== osd.10 v2:172.21.15.80:6824/3521249391 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fef80002e60 con 0x7fef840aee30 2024-04-26T22:56:26.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.203+0000 7fefa5000700 1 -- 172.21.15.27:0/2615774065 >> [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391] conn(0x7fef840aee30 msgr2=0x7fef840b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:26.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.203+0000 7fefa5000700 1 --2- 172.21.15.27:0/2615774065 >> [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391] conn(0x7fef840aee30 0x7fef840b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:26.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.203+0000 7fefa5000700 1 -- 172.21.15.27:0/2615774065 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fef840a3d50 msgr2=0x7fef840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:26.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.203+0000 7fefa5000700 1 --2- 172.21.15.27:0/2615774065 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fef840a3d50 0x7fef840a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fef90003760 tx=0x7fef90005fb0 comp rx=0 tx=0).stop 2024-04-26T22:56:26.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.203+0000 7fefa5000700 1 -- 172.21.15.27:0/2615774065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa01076a0 msgr2=0x7fefa01a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:26.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.203+0000 7fefa5000700 1 --2- 172.21.15.27:0/2615774065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa01076a0 0x7fefa01a4b70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fef9800ef30 tx=0x7fef9800c3a0 comp rx=0 tx=0).stop 2024-04-26T22:56:26.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.203+0000 7fef9e800700 1 -- 172.21.15.27:0/2615774065 reap_dead start 2024-04-26T22:56:26.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.203+0000 7fefa5000700 1 -- 172.21.15.27:0/2615774065 shutdown_connections 2024-04-26T22:56:26.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.203+0000 7fefa5000700 1 -- 172.21.15.27:0/2615774065 >> 172.21.15.27:0/2615774065 conn(0x7fefa0078990 msgr2=0x7fefa0101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:26.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.204+0000 7fefa5000700 1 -- 172.21.15.27:0/2615774065 shutdown_connections 2024-04-26T22:56:26.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:26.204+0000 7fefa5000700 1 -- 172.21.15.27:0/2615774065 wait complete. 2024-04-26T22:56:26.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:25 smithi080 ceph-mon[32324]: pgmap v362: 129 pgs: 129 active+clean; 633 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-26T22:56:26.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:56:26.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:25 smithi094 ceph-mon[32889]: pgmap v362: 129 pgs: 129 active+clean; 633 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-26T22:56:26.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:56:27.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:27 smithi027 ceph-mon[28034]: pgmap v363: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-26T22:56:27.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:56:27.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:56:27.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:56:27.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:27 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:56:28.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.346+0000 7f8a46200700 1 -- 172.21.15.27:0/2824271242 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a40109a50 msgr2=0x7f8a40109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:28.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.346+0000 7f8a46200700 1 --2- 172.21.15.27:0/2824271242 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a40109a50 0x7f8a40109ed0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8a28009990 tx=0x7f8a28009ca0 comp rx=0 tx=0).stop 2024-04-26T22:56:28.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.346+0000 7f8a46200700 1 -- 172.21.15.27:0/2824271242 shutdown_connections 2024-04-26T22:56:28.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.346+0000 7f8a46200700 1 --2- 172.21.15.27:0/2824271242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a4010a410 0x7f8a40116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:28.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.346+0000 7f8a46200700 1 --2- 172.21.15.27:0/2824271242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a4010fa50 0x7f8a4010fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:28.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.346+0000 7f8a46200700 1 --2- 172.21.15.27:0/2824271242 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a40109a50 0x7f8a40109ed0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:28.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.346+0000 7f8a46200700 1 -- 172.21.15.27:0/2824271242 >> 172.21.15.27:0/2824271242 conn(0x7f8a40076bf0 msgr2=0x7f8a40108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:28.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.346+0000 7f8a46200700 1 -- 172.21.15.27:0/2824271242 shutdown_connections 2024-04-26T22:56:28.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.346+0000 7f8a46200700 1 -- 172.21.15.27:0/2824271242 wait complete. 2024-04-26T22:56:28.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.348+0000 7f8a46200700 1 Processor -- start 2024-04-26T22:56:28.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.348+0000 7f8a46200700 1 -- start start 2024-04-26T22:56:28.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.348+0000 7f8a46200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a40109a50 0x7f8a401a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:28.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.348+0000 7f8a46200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a4010a410 0x7f8a401a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:28.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.348+0000 7f8a46200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a4010fa50 0x7f8a401a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:28.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.348+0000 7f8a46200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8a40119ce0 con 0x7f8a4010fa50 2024-04-26T22:56:28.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.348+0000 7f8a46200700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8a40119b60 con 0x7f8a4010a410 2024-04-26T22:56:28.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.348+0000 7f8a46200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8a40119e60 con 0x7f8a40109a50 2024-04-26T22:56:28.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.349+0000 7f8a3f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a40109a50 0x7f8a401a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:28.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.349+0000 7f8a36a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a4010a410 0x7f8a401a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:28.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.349+0000 7f8a3f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a40109a50 0x7f8a401a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:51380/0 (socket says 172.21.15.27:51380) 2024-04-26T22:56:28.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.349+0000 7f8a3f400700 1 -- 172.21.15.27:0/3672868757 learned_addr learned my addr 172.21.15.27:0/3672868757 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:28.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.349+0000 7f8a3fe00700 1 --2- 172.21.15.27:0/3672868757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a4010fa50 0x7f8a401a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:28.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.349+0000 7f8a3f400700 1 -- 172.21.15.27:0/3672868757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a4010a410 msgr2=0x7f8a401a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:28.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.349+0000 7f8a3f400700 1 --2- 172.21.15.27:0/3672868757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a4010a410 0x7f8a401a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:28.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.349+0000 7f8a3f400700 1 -- 172.21.15.27:0/3672868757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a4010fa50 msgr2=0x7f8a401a4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:28.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.349+0000 7f8a3f400700 1 --2- 172.21.15.27:0/3672868757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a4010fa50 0x7f8a401a4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:28.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.349+0000 7f8a3f400700 1 -- 172.21.15.27:0/3672868757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a401a51e0 con 0x7f8a40109a50 2024-04-26T22:56:28.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.349+0000 7f8a36a00700 1 --2- 172.21.15.27:0/3672868757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a4010a410 0x7f8a401a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:56:28.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.349+0000 7f8a3f400700 1 --2- 172.21.15.27:0/3672868757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a40109a50 0x7f8a401a01d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000ef30 tx=0x7f8a3000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:28.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.350+0000 7f8a3cc00700 1 -- 172.21.15.27:0/3672868757 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a300092e0 con 0x7f8a40109a50 2024-04-26T22:56:28.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.350+0000 7f8a3cc00700 1 -- 172.21.15.27:0/3672868757 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a30004bb0 con 0x7f8a40109a50 2024-04-26T22:56:28.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.350+0000 7f8a46200700 1 -- 172.21.15.27:0/3672868757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a401a5470 con 0x7f8a40109a50 2024-04-26T22:56:28.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.350+0000 7f8a3cc00700 1 -- 172.21.15.27:0/3672868757 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30004280 con 0x7f8a40109a50 2024-04-26T22:56:28.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.350+0000 7f8a46200700 1 -- 172.21.15.27:0/3672868757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a401acd50 con 0x7f8a40109a50 2024-04-26T22:56:28.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.351+0000 7f8a46200700 1 -- 172.21.15.27:0/3672868757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8a40063b30 con 0x7f8a40109a50 2024-04-26T22:56:28.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.353+0000 7f8a3cc00700 1 -- 172.21.15.27:0/3672868757 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8a3001b030 con 0x7f8a40109a50 2024-04-26T22:56:28.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.355+0000 7f8a3cc00700 1 --2- 172.21.15.27:0/3672868757 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8a240a3d50 0x7f8a240a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:28.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.355+0000 7f8a3cc00700 1 -- 172.21.15.27:0/3672868757 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f8a300d0380 con 0x7f8a40109a50 2024-04-26T22:56:28.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.355+0000 7f8a36a00700 1 --2- 172.21.15.27:0/3672868757 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8a240a3d50 0x7f8a240a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:28.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.355+0000 7f8a3cc00700 1 --2- 172.21.15.27:0/3672868757 >> [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649] conn(0x7f8a240aee30 0x7f8a240b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:28.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.355+0000 7f8a3cc00700 1 -- 172.21.15.27:0/3672868757 --> [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8a240b19a0 con 0x7f8a240aee30 2024-04-26T22:56:28.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.356+0000 7f8a3cc00700 1 -- 172.21.15.27:0/3672868757 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f8a300d7050 con 0x7f8a40109a50 2024-04-26T22:56:28.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.356+0000 7f8a3fe00700 1 --2- 172.21.15.27:0/3672868757 >> [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649] conn(0x7f8a240aee30 0x7f8a240b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:28.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.356+0000 7f8a36a00700 1 --2- 172.21.15.27:0/3672868757 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8a240a3d50 0x7f8a240a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8a40078bf0 tx=0x7f8a2801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:28.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.356+0000 7f8a3fe00700 1 --2- 172.21.15.27:0/3672868757 >> [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649] conn(0x7f8a240aee30 0x7f8a240b12d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:28.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.357+0000 7f8a3cc00700 1 -- 172.21.15.27:0/3672868757 <== osd.8 v2:172.21.15.94:6816/484486649 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8a240b19a0 con 0x7f8a240aee30 2024-04-26T22:56:28.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.376+0000 7f8a46200700 1 -- 172.21.15.27:0/3672868757 --> [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8a40063fb0 con 0x7f8a240aee30 2024-04-26T22:56:28.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.376+0000 7f8a3cc00700 1 -- 172.21.15.27:0/3672868757 <== osd.8 v2:172.21.15.94:6816/484486649 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8a40063fb0 con 0x7f8a240aee30 2024-04-26T22:56:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.377+0000 7f8a46200700 1 -- 172.21.15.27:0/3672868757 >> [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649] conn(0x7f8a240aee30 msgr2=0x7f8a240b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.377+0000 7f8a46200700 1 --2- 172.21.15.27:0/3672868757 >> [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649] conn(0x7f8a240aee30 0x7f8a240b12d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.377+0000 7f8a46200700 1 -- 172.21.15.27:0/3672868757 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8a240a3d50 msgr2=0x7f8a240a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.377+0000 7f8a46200700 1 --2- 172.21.15.27:0/3672868757 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f8a240a3d50 0x7f8a240a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8a40078bf0 tx=0x7f8a2801a040 comp rx=0 tx=0).stop 2024-04-26T22:56:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.377+0000 7f8a46200700 1 -- 172.21.15.27:0/3672868757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a40109a50 msgr2=0x7f8a401a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.377+0000 7f8a46200700 1 --2- 172.21.15.27:0/3672868757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a40109a50 0x7f8a401a01d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000ef30 tx=0x7f8a3000c3a0 comp rx=0 tx=0).stop 2024-04-26T22:56:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.377+0000 7f8a3fe00700 1 -- 172.21.15.27:0/3672868757 reap_dead start 2024-04-26T22:56:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.377+0000 7f8a46200700 1 -- 172.21.15.27:0/3672868757 shutdown_connections 2024-04-26T22:56:28.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.377+0000 7f8a46200700 1 -- 172.21.15.27:0/3672868757 >> 172.21.15.27:0/3672868757 conn(0x7f8a40076bf0 msgr2=0x7f8a400772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:28.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.377+0000 7f8a46200700 1 -- 172.21.15.27:0/3672868757 shutdown_connections 2024-04-26T22:56:28.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:27.377+0000 7f8a46200700 1 -- 172.21.15.27:0/3672868757 wait complete. 2024-04-26T22:56:28.172 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:27 smithi080 ceph-mon[32324]: pgmap v363: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-26T22:56:28.172 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:56:28.172 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:56:28.172 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:56:28.172 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:27 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:56:28.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:27 smithi094 ceph-mon[32889]: pgmap v363: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-26T22:56:28.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:56:28.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:56:28.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:56:28.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:27 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:56:29.641 INFO:teuthology.orchestra.run.smithi027.stdout:309237645345 2024-04-26T22:56:29.642 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-26T22:56:29.981 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:29 smithi027 ceph-mon[28034]: pgmap v364: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T22:56:30.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:29 smithi080 ceph-mon[32324]: pgmap v364: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T22:56:30.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:29 smithi094 ceph-mon[32889]: pgmap v364: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T22:56:30.764 INFO:teuthology.orchestra.run.smithi027.stdout:253403070510 2024-04-26T22:56:30.765 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-26T22:56:31.340 INFO:teuthology.orchestra.run.smithi027.stdout:163208757310 2024-04-26T22:56:31.341 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T22:56:32.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:31 smithi027 ceph-mon[28034]: pgmap v365: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T22:56:32.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:31 smithi080 ceph-mon[32324]: pgmap v365: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T22:56:32.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:31 smithi094 ceph-mon[32889]: pgmap v365: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T22:56:33.999 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:33 smithi027 ceph-mon[28034]: pgmap v366: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T22:56:34.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:33 smithi080 ceph-mon[32324]: pgmap v366: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T22:56:34.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:33 smithi094 ceph-mon[32889]: pgmap v366: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T22:56:36.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:35 smithi027 ceph-mon[28034]: pgmap v367: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T22:56:36.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.165+0000 7f8935c00700 1 -- 172.21.15.27:0/1085099117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8930075650 msgr2=0x7f8930075a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:36.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.165+0000 7f8935c00700 1 --2- 172.21.15.27:0/1085099117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8930075650 0x7f8930075a30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f892000b0a0 tx=0x7f892000b3b0 comp rx=0 tx=0).stop 2024-04-26T22:56:36.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.165+0000 7f8935c00700 1 -- 172.21.15.27:0/1085099117 shutdown_connections 2024-04-26T22:56:36.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.165+0000 7f8935c00700 1 --2- 172.21.15.27:0/1085099117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f893010fe20 0x7f8930112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:36.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.165+0000 7f8935c00700 1 --2- 172.21.15.27:0/1085099117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8930075650 0x7f8930075a30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:36.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.165+0000 7f8935c00700 1 --2- 172.21.15.27:0/1085099117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8930075f70 0x7f893010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:36.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.165+0000 7f8935c00700 1 -- 172.21.15.27:0/1085099117 >> 172.21.15.27:0/1085099117 conn(0x7f893007a580 msgr2=0x7f893007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:36.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.165+0000 7f8935c00700 1 -- 172.21.15.27:0/1085099117 shutdown_connections 2024-04-26T22:56:36.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.165+0000 7f8935c00700 1 -- 172.21.15.27:0/1085099117 wait complete. 2024-04-26T22:56:36.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.166+0000 7f8935c00700 1 Processor -- start 2024-04-26T22:56:36.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.167+0000 7f8935c00700 1 -- start start 2024-04-26T22:56:36.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.167+0000 7f8935c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8930075650 0x7f893019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:36.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.167+0000 7f8935c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8930075f70 0x7f893019c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:36.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.167+0000 7f8935c00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893010fe20 0x7f89301a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:36.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.167+0000 7f8935c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f89301157d0 con 0x7f8930075f70 2024-04-26T22:56:36.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.167+0000 7f8935c00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8930115650 con 0x7f893010fe20 2024-04-26T22:56:36.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.167+0000 7f8935c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8930115950 con 0x7f8930075650 2024-04-26T22:56:36.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.167+0000 7f892e000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8930075f70 0x7f893019c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:36.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.168+0000 7f892ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8930075650 0x7f893019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:36.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.168+0000 7f892ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8930075650 0x7f893019bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:48794/0 (socket says 172.21.15.27:48794) 2024-04-26T22:56:36.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.168+0000 7f892e000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8930075f70 0x7f893019c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38202/0 (socket says 172.21.15.27:38202) 2024-04-26T22:56:36.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.168+0000 7f892ea00700 1 -- 172.21.15.27:0/2488344572 learned_addr learned my addr 172.21.15.27:0/2488344572 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:36.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.168+0000 7f892f400700 1 --2- 172.21.15.27:0/2488344572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893010fe20 0x7f89301a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:36.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.168+0000 7f892f400700 1 -- 172.21.15.27:0/2488344572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8930075650 msgr2=0x7f893019bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:36.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.168+0000 7f892f400700 1 --2- 172.21.15.27:0/2488344572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8930075650 0x7f893019bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:36.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.168+0000 7f892f400700 1 -- 172.21.15.27:0/2488344572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8930075f70 msgr2=0x7f893019c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:36.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.168+0000 7f892f400700 1 --2- 172.21.15.27:0/2488344572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8930075f70 0x7f893019c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:36.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.168+0000 7f892f400700 1 -- 172.21.15.27:0/2488344572 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89301a0d30 con 0x7f893010fe20 2024-04-26T22:56:36.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.168+0000 7f892e000700 1 --2- 172.21.15.27:0/2488344572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8930075f70 0x7f893019c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:56:36.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.169+0000 7f892f400700 1 --2- 172.21.15.27:0/2488344572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893010fe20 0x7f89301a05f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f892800e8d0 tx=0x7f892800ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:36.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.169+0000 7f8927400700 1 -- 172.21.15.27:0/2488344572 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928004220 con 0x7f893010fe20 2024-04-26T22:56:36.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.169+0000 7f8935c00700 1 -- 172.21.15.27:0/2488344572 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89301a1020 con 0x7f893010fe20 2024-04-26T22:56:36.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.169+0000 7f8927400700 1 -- 172.21.15.27:0/2488344572 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89280043c0 con 0x7f893010fe20 2024-04-26T22:56:36.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.169+0000 7f8935c00700 1 -- 172.21.15.27:0/2488344572 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89301a88a0 con 0x7f893010fe20 2024-04-26T22:56:36.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.169+0000 7f8927400700 1 -- 172.21.15.27:0/2488344572 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928003d00 con 0x7f893010fe20 2024-04-26T22:56:36.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.170+0000 7f8935c00700 1 -- 172.21.15.27:0/2488344572 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8930063bb0 con 0x7f893010fe20 2024-04-26T22:56:36.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.172+0000 7f8927400700 1 -- 172.21.15.27:0/2488344572 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f892800c760 con 0x7f893010fe20 2024-04-26T22:56:36.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.174+0000 7f8927400700 1 --2- 172.21.15.27:0/2488344572 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f89140a3d50 0x7f89140a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:36.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.174+0000 7f8927400700 1 -- 172.21.15.27:0/2488344572 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f89280d08c0 con 0x7f893010fe20 2024-04-26T22:56:36.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.174+0000 7f892ea00700 1 --2- 172.21.15.27:0/2488344572 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f89140a3d50 0x7f89140a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:36.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.174+0000 7f8927400700 1 --2- 172.21.15.27:0/2488344572 >> [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976] conn(0x7f89140aee30 0x7f89140b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:36.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.174+0000 7f8927400700 1 -- 172.21.15.27:0/2488344572 --> [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f89140b19a0 con 0x7f89140aee30 2024-04-26T22:56:36.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.174+0000 7f8927400700 1 -- 172.21.15.27:0/2488344572 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f89280d52a0 con 0x7f893010fe20 2024-04-26T22:56:36.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.175+0000 7f892e000700 1 --2- 172.21.15.27:0/2488344572 >> [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976] conn(0x7f89140aee30 0x7f89140b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:36.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.175+0000 7f892ea00700 1 --2- 172.21.15.27:0/2488344572 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f89140a3d50 0x7f89140a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8920009e60 tx=0x7f892000b740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:36.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.175+0000 7f892e000700 1 --2- 172.21.15.27:0/2488344572 >> [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976] conn(0x7f89140aee30 0x7f89140b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:36.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.176+0000 7f8927400700 1 -- 172.21.15.27:0/2488344572 <== osd.9 v2:172.21.15.27:6826/2196150976 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f89140b19a0 con 0x7f89140aee30 2024-04-26T22:56:36.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.201+0000 7f8935c00700 1 -- 172.21.15.27:0/2488344572 --> [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8930064060 con 0x7f89140aee30 2024-04-26T22:56:36.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.201+0000 7f8927400700 1 -- 172.21.15.27:0/2488344572 <== osd.9 v2:172.21.15.27:6826/2196150976 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8930064060 con 0x7f89140aee30 2024-04-26T22:56:36.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.202+0000 7f8935c00700 1 -- 172.21.15.27:0/2488344572 >> [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976] conn(0x7f89140aee30 msgr2=0x7f89140b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:36.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.202+0000 7f8935c00700 1 --2- 172.21.15.27:0/2488344572 >> [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976] conn(0x7f89140aee30 0x7f89140b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:36.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.202+0000 7f8935c00700 1 -- 172.21.15.27:0/2488344572 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f89140a3d50 msgr2=0x7f89140a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:36.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.202+0000 7f8935c00700 1 --2- 172.21.15.27:0/2488344572 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f89140a3d50 0x7f89140a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8920009e60 tx=0x7f892000b740 comp rx=0 tx=0).stop 2024-04-26T22:56:36.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.202+0000 7f8935c00700 1 -- 172.21.15.27:0/2488344572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893010fe20 msgr2=0x7f89301a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:36.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.202+0000 7f8935c00700 1 --2- 172.21.15.27:0/2488344572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893010fe20 0x7f89301a05f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f892800e8d0 tx=0x7f892800ebe0 comp rx=0 tx=0).stop 2024-04-26T22:56:36.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.202+0000 7f892f400700 1 -- 172.21.15.27:0/2488344572 reap_dead start 2024-04-26T22:56:36.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.203+0000 7f8935c00700 1 -- 172.21.15.27:0/2488344572 shutdown_connections 2024-04-26T22:56:36.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.203+0000 7f8935c00700 1 -- 172.21.15.27:0/2488344572 >> 172.21.15.27:0/2488344572 conn(0x7f893007a580 msgr2=0x7f8930100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:36.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.203+0000 7f8935c00700 1 -- 172.21.15.27:0/2488344572 shutdown_connections 2024-04-26T22:56:36.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:36.203+0000 7f8935c00700 1 -- 172.21.15.27:0/2488344572 wait complete. 2024-04-26T22:56:36.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:35 smithi080 ceph-mon[32324]: pgmap v367: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T22:56:36.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:35 smithi094 ceph-mon[32889]: pgmap v367: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T22:56:38.078 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:37 smithi027 ceph-mon[28034]: pgmap v368: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T22:56:38.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:37 smithi080 ceph-mon[32324]: pgmap v368: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T22:56:38.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:37 smithi094 ceph-mon[32889]: pgmap v368: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T22:56:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.260+0000 7f8677e00700 1 -- 172.21.15.27:0/3773886345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f867010fa50 msgr2=0x7f867010fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.260+0000 7f8677e00700 1 --2- 172.21.15.27:0/3773886345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f867010fa50 0x7f867010fe30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8664009990 tx=0x7f8664009ca0 comp rx=0 tx=0).stop 2024-04-26T22:56:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.260+0000 7f8677e00700 1 -- 172.21.15.27:0/3773886345 shutdown_connections 2024-04-26T22:56:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.260+0000 7f8677e00700 1 --2- 172.21.15.27:0/3773886345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f867010a410 0x7f8670116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.260+0000 7f8677e00700 1 --2- 172.21.15.27:0/3773886345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f867010fa50 0x7f867010fe30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.260+0000 7f8677e00700 1 --2- 172.21.15.27:0/3773886345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8670109a50 0x7f8670109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.260+0000 7f8677e00700 1 -- 172.21.15.27:0/3773886345 >> 172.21.15.27:0/3773886345 conn(0x7f8670076bf0 msgr2=0x7f8670108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.260+0000 7f8677e00700 1 -- 172.21.15.27:0/3773886345 shutdown_connections 2024-04-26T22:56:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.260+0000 7f8677e00700 1 -- 172.21.15.27:0/3773886345 wait complete. 2024-04-26T22:56:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.262+0000 7f8677e00700 1 Processor -- start 2024-04-26T22:56:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.262+0000 7f8677e00700 1 -- start start 2024-04-26T22:56:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.262+0000 7f8677e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8670109a50 0x7f86701a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.262+0000 7f8677e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f867010a410 0x7f86701a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.262+0000 7f8677e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f867010fa50 0x7f86701a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.263+0000 7f8677e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8670119c50 con 0x7f8670109a50 2024-04-26T22:56:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.263+0000 7f8677e00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8670119ad0 con 0x7f867010fa50 2024-04-26T22:56:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.263+0000 7f8677e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8670119dd0 con 0x7f867010a410 2024-04-26T22:56:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.263+0000 7f8675400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8670109a50 0x7f86701a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.263+0000 7f8675400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8670109a50 0x7f86701a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36392/0 (socket says 172.21.15.27:36392) 2024-04-26T22:56:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.263+0000 7f8675e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f867010fa50 0x7f86701a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.263+0000 7f8675400700 1 -- 172.21.15.27:0/2581467789 learned_addr learned my addr 172.21.15.27:0/2581467789 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.263+0000 7f8674a00700 1 --2- 172.21.15.27:0/2581467789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f867010a410 0x7f86701a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.263+0000 7f8675400700 1 -- 172.21.15.27:0/2581467789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f867010a410 msgr2=0x7f86701a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.263+0000 7f8675400700 1 --2- 172.21.15.27:0/2581467789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f867010a410 0x7f86701a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.263+0000 7f8675400700 1 -- 172.21.15.27:0/2581467789 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f867010fa50 msgr2=0x7f86701a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.263+0000 7f8675400700 1 --2- 172.21.15.27:0/2581467789 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f867010fa50 0x7f86701a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.263+0000 7f8675400700 1 -- 172.21.15.27:0/2581467789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86701a50b0 con 0x7f8670109a50 2024-04-26T22:56:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.263+0000 7f8675e00700 1 --2- 172.21.15.27:0/2581467789 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f867010fa50 0x7f86701a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:56:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.264+0000 7f8675400700 1 --2- 172.21.15.27:0/2581467789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8670109a50 0x7f86701a00a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8664000c00 tx=0x7f866400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.264+0000 7f866e000700 1 -- 172.21.15.27:0/2581467789 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664027070 con 0x7f8670109a50 2024-04-26T22:56:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.264+0000 7f866e000700 1 -- 172.21.15.27:0/2581467789 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8664022470 con 0x7f8670109a50 2024-04-26T22:56:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.264+0000 7f8677e00700 1 -- 172.21.15.27:0/2581467789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86701a5340 con 0x7f8670109a50 2024-04-26T22:56:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.264+0000 7f866e000700 1 -- 172.21.15.27:0/2581467789 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664017650 con 0x7f8670109a50 2024-04-26T22:56:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.264+0000 7f8677e00700 1 -- 172.21.15.27:0/2581467789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86701acc20 con 0x7f8670109a50 2024-04-26T22:56:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.265+0000 7f8677e00700 1 -- 172.21.15.27:0/2581467789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8670063bb0 con 0x7f8670109a50 2024-04-26T22:56:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.267+0000 7f866e000700 1 -- 172.21.15.27:0/2581467789 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f866402a070 con 0x7f8670109a50 2024-04-26T22:56:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.269+0000 7f866e000700 1 --2- 172.21.15.27:0/2581467789 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f86580a3cd0 0x7f86580a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.269+0000 7f866e000700 1 -- 172.21.15.27:0/2581467789 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f86640d1db0 con 0x7f8670109a50 2024-04-26T22:56:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.269+0000 7f8674a00700 1 --2- 172.21.15.27:0/2581467789 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f86580a3cd0 0x7f86580a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.269+0000 7f866e000700 1 --2- 172.21.15.27:0/2581467789 >> [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880] conn(0x7f86580aedb0 0x7f86580b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.269+0000 7f866e000700 1 -- 172.21.15.27:0/2581467789 --> [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86640095d0 con 0x7f86580aedb0 2024-04-26T22:56:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.269+0000 7f866e000700 1 -- 172.21.15.27:0/2581467789 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f86640d2160 con 0x7f8670109a50 2024-04-26T22:56:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.270+0000 7f8675e00700 1 --2- 172.21.15.27:0/2581467789 >> [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880] conn(0x7f86580aedb0 0x7f86580b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.270+0000 7f8674a00700 1 --2- 172.21.15.27:0/2581467789 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f86580a3cd0 0x7f86580a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f865c009680 tx=0x7f865c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.270+0000 7f8675e00700 1 --2- 172.21.15.27:0/2581467789 >> [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880] conn(0x7f86580aedb0 0x7f86580b1250 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.271+0000 7f866e000700 1 -- 172.21.15.27:0/2581467789 <== osd.0 v2:172.21.15.27:6802/4133324880 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86640095d0 con 0x7f86580aedb0 2024-04-26T22:56:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.290+0000 7f8677e00700 1 -- 172.21.15.27:0/2581467789 --> [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8670064030 con 0x7f86580aedb0 2024-04-26T22:56:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.291+0000 7f866e000700 1 -- 172.21.15.27:0/2581467789 <== osd.0 v2:172.21.15.27:6802/4133324880 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8670064030 con 0x7f86580aedb0 2024-04-26T22:56:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.291+0000 7f8677e00700 1 -- 172.21.15.27:0/2581467789 >> [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880] conn(0x7f86580aedb0 msgr2=0x7f86580b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.292+0000 7f8677e00700 1 --2- 172.21.15.27:0/2581467789 >> [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880] conn(0x7f86580aedb0 0x7f86580b1250 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.292+0000 7f8677e00700 1 -- 172.21.15.27:0/2581467789 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f86580a3cd0 msgr2=0x7f86580a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.292+0000 7f8677e00700 1 --2- 172.21.15.27:0/2581467789 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f86580a3cd0 0x7f86580a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f865c009680 tx=0x7f865c009260 comp rx=0 tx=0).stop 2024-04-26T22:56:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.292+0000 7f8677e00700 1 -- 172.21.15.27:0/2581467789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8670109a50 msgr2=0x7f86701a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.292+0000 7f8677e00700 1 --2- 172.21.15.27:0/2581467789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8670109a50 0x7f86701a00a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8664000c00 tx=0x7f866400bfd0 comp rx=0 tx=0).stop 2024-04-26T22:56:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.292+0000 7f8675e00700 1 -- 172.21.15.27:0/2581467789 reap_dead start 2024-04-26T22:56:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.292+0000 7f8677e00700 1 -- 172.21.15.27:0/2581467789 shutdown_connections 2024-04-26T22:56:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.292+0000 7f8677e00700 1 -- 172.21.15.27:0/2581467789 >> 172.21.15.27:0/2581467789 conn(0x7f8670076bf0 msgr2=0x7f8670077210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.292+0000 7f8677e00700 1 -- 172.21.15.27:0/2581467789 shutdown_connections 2024-04-26T22:56:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:38.292+0000 7f8677e00700 1 -- 172.21.15.27:0/2581467789 wait complete. 2024-04-26T22:56:39.981 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:39 smithi027 ceph-mon[28034]: pgmap v369: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:56:40.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:39 smithi080 ceph-mon[32324]: pgmap v369: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:56:40.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:39 smithi094 ceph-mon[32889]: pgmap v369: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:56:40.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.468+0000 7f0cf7e00700 1 -- 172.21.15.27:0/2178243261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cf80757c0 msgr2=0x7f0cf8075ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:40.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.468+0000 7f0cf7e00700 1 --2- 172.21.15.27:0/2178243261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cf80757c0 0x7f0cf8075ba0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80098f0 tx=0x7f0ce8009c00 comp rx=0 tx=0).stop 2024-04-26T22:56:40.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.468+0000 7f0cf7e00700 1 -- 172.21.15.27:0/2178243261 shutdown_connections 2024-04-26T22:56:40.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.468+0000 7f0cf7e00700 1 --2- 172.21.15.27:0/2178243261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cf8114360 0x7f0cf8116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:40.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.468+0000 7f0cf7e00700 1 --2- 172.21.15.27:0/2178243261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cf80757c0 0x7f0cf8075ba0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:40.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.468+0000 7f0cf7e00700 1 --2- 172.21.15.27:0/2178243261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cf80760e0 0x7f0cf8113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:40.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.468+0000 7f0cf7e00700 1 -- 172.21.15.27:0/2178243261 >> 172.21.15.27:0/2178243261 conn(0x7f0cf80feea0 msgr2=0x7f0cf81012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.468+0000 7f0cf7e00700 1 -- 172.21.15.27:0/2178243261 shutdown_connections 2024-04-26T22:56:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.468+0000 7f0cf7e00700 1 -- 172.21.15.27:0/2178243261 wait complete. 2024-04-26T22:56:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.470+0000 7f0cf7e00700 1 Processor -- start 2024-04-26T22:56:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.470+0000 7f0cf7e00700 1 -- start start 2024-04-26T22:56:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.470+0000 7f0cf7e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cf80757c0 0x7f0cf81a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.470+0000 7f0cf7e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cf80760e0 0x7f0cf81a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.470+0000 7f0cf7e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cf8114360 0x7f0cf81a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.470+0000 7f0cf7e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8119b90 con 0x7f0cf80757c0 2024-04-26T22:56:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.470+0000 7f0cf7e00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8119a10 con 0x7f0cf80760e0 2024-04-26T22:56:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.470+0000 7f0cf7e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8119d10 con 0x7f0cf8114360 2024-04-26T22:56:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.471+0000 7f0cf6000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cf80760e0 0x7f0cf81a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.471+0000 7f0cf6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cf80757c0 0x7f0cf81a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.471+0000 7f0cf6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cf80757c0 0x7f0cf81a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36420/0 (socket says 172.21.15.27:36420) 2024-04-26T22:56:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.471+0000 7f0cf6a00700 1 -- 172.21.15.27:0/1006290114 learned_addr learned my addr 172.21.15.27:0/1006290114 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.471+0000 7f0cf6000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cf80760e0 0x7f0cf81a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:37410/0 (socket says 172.21.15.27:37410) 2024-04-26T22:56:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.471+0000 7f0cf7400700 1 --2- 172.21.15.27:0/1006290114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cf8114360 0x7f0cf81a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.471+0000 7f0cf6a00700 1 -- 172.21.15.27:0/1006290114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cf8114360 msgr2=0x7f0cf81a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.471+0000 7f0cf6a00700 1 --2- 172.21.15.27:0/1006290114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cf8114360 0x7f0cf81a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.471+0000 7f0cf6a00700 1 -- 172.21.15.27:0/1006290114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cf80760e0 msgr2=0x7f0cf81a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.471+0000 7f0cf6a00700 1 --2- 172.21.15.27:0/1006290114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cf80760e0 0x7f0cf81a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.471+0000 7f0cf6a00700 1 -- 172.21.15.27:0/1006290114 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf81a50b0 con 0x7f0cf80757c0 2024-04-26T22:56:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.472+0000 7f0cf6a00700 1 --2- 172.21.15.27:0/1006290114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cf80757c0 0x7f0cf81a00a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009230 tx=0x7f0ce800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.472+0000 7f0cef400700 1 -- 172.21.15.27:0/1006290114 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80050c0 con 0x7f0cf80757c0 2024-04-26T22:56:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.472+0000 7f0cef400700 1 -- 172.21.15.27:0/1006290114 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce800b560 con 0x7f0cf80757c0 2024-04-26T22:56:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.472+0000 7f0cf7e00700 1 -- 172.21.15.27:0/1006290114 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf81a5340 con 0x7f0cf80757c0 2024-04-26T22:56:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.472+0000 7f0cef400700 1 -- 172.21.15.27:0/1006290114 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8003a00 con 0x7f0cf80757c0 2024-04-26T22:56:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.472+0000 7f0cf7e00700 1 -- 172.21.15.27:0/1006290114 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf81acc20 con 0x7f0cf80757c0 2024-04-26T22:56:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.473+0000 7f0cf7e00700 1 -- 172.21.15.27:0/1006290114 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0cf8063bb0 con 0x7f0cf80757c0 2024-04-26T22:56:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.475+0000 7f0cef400700 1 -- 172.21.15.27:0/1006290114 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0ce800b710 con 0x7f0cf80757c0 2024-04-26T22:56:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.477+0000 7f0cef400700 1 --2- 172.21.15.27:0/1006290114 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0cdc0a3cd0 0x7f0cdc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.477+0000 7f0cef400700 1 -- 172.21.15.27:0/1006290114 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f0ce80d1c10 con 0x7f0cf80757c0 2024-04-26T22:56:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.477+0000 7f0cef400700 1 --2- 172.21.15.27:0/1006290114 >> [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925] conn(0x7f0cdc0aec80 0x7f0cdc0b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.477+0000 7f0cf6000700 1 --2- 172.21.15.27:0/1006290114 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0cdc0a3cd0 0x7f0cdc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.477+0000 7f0cef400700 1 -- 172.21.15.27:0/1006290114 --> [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0cdc0b17f0 con 0x7f0cdc0aec80 2024-04-26T22:56:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.477+0000 7f0cef400700 1 -- 172.21.15.27:0/1006290114 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f0ce80d1fc0 con 0x7f0cf80757c0 2024-04-26T22:56:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.477+0000 7f0cf7400700 1 --2- 172.21.15.27:0/1006290114 >> [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925] conn(0x7f0cdc0aec80 0x7f0cdc0b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.478+0000 7f0cf6000700 1 --2- 172.21.15.27:0/1006290114 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0cdc0a3cd0 0x7f0cdc0a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8107150 tx=0x7f0ce4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.478+0000 7f0cf7400700 1 --2- 172.21.15.27:0/1006290114 >> [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925] conn(0x7f0cdc0aec80 0x7f0cdc0b1120 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.479+0000 7f0cef400700 1 -- 172.21.15.27:0/1006290114 <== osd.11 v2:172.21.15.94:6824/2763583925 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0cdc0b17f0 con 0x7f0cdc0aec80 2024-04-26T22:56:40.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.497+0000 7f0cf7e00700 1 -- 172.21.15.27:0/1006290114 --> [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0cf8064030 con 0x7f0cdc0aec80 2024-04-26T22:56:40.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.498+0000 7f0cef400700 1 -- 172.21.15.27:0/1006290114 <== osd.11 v2:172.21.15.94:6824/2763583925 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0cf8064030 con 0x7f0cdc0aec80 2024-04-26T22:56:40.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.499+0000 7f0cf7e00700 1 -- 172.21.15.27:0/1006290114 >> [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925] conn(0x7f0cdc0aec80 msgr2=0x7f0cdc0b1120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:40.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.499+0000 7f0cf7e00700 1 --2- 172.21.15.27:0/1006290114 >> [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925] conn(0x7f0cdc0aec80 0x7f0cdc0b1120 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:40.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.499+0000 7f0cf7e00700 1 -- 172.21.15.27:0/1006290114 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0cdc0a3cd0 msgr2=0x7f0cdc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:40.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.499+0000 7f0cf7e00700 1 --2- 172.21.15.27:0/1006290114 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0cdc0a3cd0 0x7f0cdc0a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8107150 tx=0x7f0ce4009260 comp rx=0 tx=0).stop 2024-04-26T22:56:40.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.499+0000 7f0cf7e00700 1 -- 172.21.15.27:0/1006290114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cf80757c0 msgr2=0x7f0cf81a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.499+0000 7f0cf7e00700 1 --2- 172.21.15.27:0/1006290114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cf80757c0 0x7f0cf81a00a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009230 tx=0x7f0ce800fa70 comp rx=0 tx=0).stop 2024-04-26T22:56:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.499+0000 7f0cf7400700 1 -- 172.21.15.27:0/1006290114 reap_dead start 2024-04-26T22:56:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.499+0000 7f0cf7e00700 1 -- 172.21.15.27:0/1006290114 shutdown_connections 2024-04-26T22:56:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.499+0000 7f0cf7e00700 1 -- 172.21.15.27:0/1006290114 >> 172.21.15.27:0/1006290114 conn(0x7f0cf80feea0 msgr2=0x7f0cf80ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.499+0000 7f0cf7e00700 1 -- 172.21.15.27:0/1006290114 shutdown_connections 2024-04-26T22:56:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:39.499+0000 7f0cf7e00700 1 -- 172.21.15.27:0/1006290114 wait complete. 2024-04-26T22:56:41.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.800+0000 7f4393a00700 1 -- 172.21.15.27:0/2651950373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f438c075f70 msgr2=0x7f438c10f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:41.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.800+0000 7f4393a00700 1 --2- 172.21.15.27:0/2651950373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f438c075f70 0x7f438c10f940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4378009990 tx=0x7f4378009ca0 comp rx=0 tx=0).stop 2024-04-26T22:56:41.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.800+0000 7f4393a00700 1 -- 172.21.15.27:0/2651950373 shutdown_connections 2024-04-26T22:56:41.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.800+0000 7f4393a00700 1 --2- 172.21.15.27:0/2651950373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f438c10fe80 0x7f438c112270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:41.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.800+0000 7f4393a00700 1 --2- 172.21.15.27:0/2651950373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f438c075650 0x7f438c075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:41.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.800+0000 7f4393a00700 1 --2- 172.21.15.27:0/2651950373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f438c075f70 0x7f438c10f940 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:41.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.800+0000 7f4393a00700 1 -- 172.21.15.27:0/2651950373 >> 172.21.15.27:0/2651950373 conn(0x7f438c07a580 msgr2=0x7f438c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:41.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.800+0000 7f4393a00700 1 -- 172.21.15.27:0/2651950373 shutdown_connections 2024-04-26T22:56:41.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.800+0000 7f4393a00700 1 -- 172.21.15.27:0/2651950373 wait complete. 2024-04-26T22:56:41.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.801+0000 7f4393a00700 1 Processor -- start 2024-04-26T22:56:41.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.801+0000 7f4393a00700 1 -- start start 2024-04-26T22:56:41.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.801+0000 7f4393a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f438c075650 0x7f438c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:41.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.801+0000 7f4393a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f438c075f70 0x7f438c1a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:41.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.802+0000 7f4393a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f438c10fe80 0x7f438c19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:41.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.802+0000 7f4393a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f438c115840 con 0x7f438c10fe80 2024-04-26T22:56:41.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.802+0000 7f4393a00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f438c1156c0 con 0x7f438c075f70 2024-04-26T22:56:41.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.802+0000 7f4393a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f438c1159c0 con 0x7f438c075650 2024-04-26T22:56:41.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.802+0000 7f4383e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f438c075f70 0x7f438c1a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:41.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.802+0000 7f4391000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f438c075650 0x7f438c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:41.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.802+0000 7f4383e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f438c075f70 0x7f438c1a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:37416/0 (socket says 172.21.15.27:37416) 2024-04-26T22:56:41.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.802+0000 7f4383e00700 1 -- 172.21.15.27:0/1555304308 learned_addr learned my addr 172.21.15.27:0/1555304308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:41.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.802+0000 7f4391a00700 1 --2- 172.21.15.27:0/1555304308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f438c10fe80 0x7f438c19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:41.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.802+0000 7f4391000700 1 -- 172.21.15.27:0/1555304308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f438c075f70 msgr2=0x7f438c1a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:41.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.802+0000 7f4391000700 1 --2- 172.21.15.27:0/1555304308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f438c075f70 0x7f438c1a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:41.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.802+0000 7f4391000700 1 -- 172.21.15.27:0/1555304308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f438c10fe80 msgr2=0x7f438c19bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:41.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.802+0000 7f4391000700 1 --2- 172.21.15.27:0/1555304308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f438c10fe80 0x7f438c19bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:41.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.802+0000 7f4391000700 1 -- 172.21.15.27:0/1555304308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438c19c400 con 0x7f438c075650 2024-04-26T22:56:41.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.802+0000 7f4391a00700 1 --2- 172.21.15.27:0/1555304308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f438c10fe80 0x7f438c19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:56:41.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.802+0000 7f4383e00700 1 --2- 172.21.15.27:0/1555304308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f438c075f70 0x7f438c1a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:56:41.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.803+0000 7f4391000700 1 --2- 172.21.15.27:0/1555304308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f438c075650 0x7f438c1a1a50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f437c00e9f0 tx=0x7f437c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:41.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.803+0000 7f438a000700 1 -- 172.21.15.27:0/1555304308 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c00cbd0 con 0x7f438c075650 2024-04-26T22:56:41.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.803+0000 7f438a000700 1 -- 172.21.15.27:0/1555304308 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f437c00cd70 con 0x7f438c075650 2024-04-26T22:56:41.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.803+0000 7f4393a00700 1 -- 172.21.15.27:0/1555304308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438c19c6f0 con 0x7f438c075650 2024-04-26T22:56:41.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.803+0000 7f438a000700 1 -- 172.21.15.27:0/1555304308 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c018680 con 0x7f438c075650 2024-04-26T22:56:41.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.803+0000 7f4393a00700 1 -- 172.21.15.27:0/1555304308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f438c1a89d0 con 0x7f438c075650 2024-04-26T22:56:41.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.804+0000 7f4393a00700 1 -- 172.21.15.27:0/1555304308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f438c063b70 con 0x7f438c075650 2024-04-26T22:56:41.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.807+0000 7f438a000700 1 -- 172.21.15.27:0/1555304308 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f437c010430 con 0x7f438c075650 2024-04-26T22:56:41.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.808+0000 7f438a000700 1 --2- 172.21.15.27:0/1555304308 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f43700a3d50 0x7f43700a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:41.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.808+0000 7f438a000700 1 -- 172.21.15.27:0/1555304308 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f437c0d1260 con 0x7f438c075650 2024-04-26T22:56:41.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.808+0000 7f4383e00700 1 --2- 172.21.15.27:0/1555304308 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f43700a3d50 0x7f43700a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:41.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.809+0000 7f438a000700 1 --2- 172.21.15.27:0/1555304308 >> [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127] conn(0x7f43700aee30 0x7f43700b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:41.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.809+0000 7f438a000700 1 -- 172.21.15.27:0/1555304308 --> [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f437c027e50 con 0x7f43700aee30 2024-04-26T22:56:41.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.809+0000 7f438a000700 1 -- 172.21.15.27:0/1555304308 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f437c0d1610 con 0x7f438c075650 2024-04-26T22:56:41.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.809+0000 7f4391a00700 1 --2- 172.21.15.27:0/1555304308 >> [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127] conn(0x7f43700aee30 0x7f43700b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:41.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.809+0000 7f4383e00700 1 --2- 172.21.15.27:0/1555304308 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f43700a3d50 0x7f43700a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f438c19d270 tx=0x7f437801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:41.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.810+0000 7f4391a00700 1 --2- 172.21.15.27:0/1555304308 >> [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127] conn(0x7f43700aee30 0x7f43700b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:41.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.811+0000 7f438a000700 1 -- 172.21.15.27:0/1555304308 <== osd.7 v2:172.21.15.80:6816/534257127 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f437c027e50 con 0x7f43700aee30 2024-04-26T22:56:41.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.829+0000 7f4393a00700 1 -- 172.21.15.27:0/1555304308 --> [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f438c063f70 con 0x7f43700aee30 2024-04-26T22:56:41.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.830+0000 7f438a000700 1 -- 172.21.15.27:0/1555304308 <== osd.7 v2:172.21.15.80:6816/534257127 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f438c063f70 con 0x7f43700aee30 2024-04-26T22:56:41.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.831+0000 7f4393a00700 1 -- 172.21.15.27:0/1555304308 >> [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127] conn(0x7f43700aee30 msgr2=0x7f43700b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:41.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.831+0000 7f4393a00700 1 --2- 172.21.15.27:0/1555304308 >> [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127] conn(0x7f43700aee30 0x7f43700b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:41.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.831+0000 7f4393a00700 1 -- 172.21.15.27:0/1555304308 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f43700a3d50 msgr2=0x7f43700a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:41.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.831+0000 7f4393a00700 1 --2- 172.21.15.27:0/1555304308 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f43700a3d50 0x7f43700a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f438c19d270 tx=0x7f437801a040 comp rx=0 tx=0).stop 2024-04-26T22:56:41.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.831+0000 7f4393a00700 1 -- 172.21.15.27:0/1555304308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f438c075650 msgr2=0x7f438c1a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:41.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.831+0000 7f4393a00700 1 --2- 172.21.15.27:0/1555304308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f438c075650 0x7f438c1a1a50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f437c00e9f0 tx=0x7f437c00ed00 comp rx=0 tx=0).stop 2024-04-26T22:56:41.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.831+0000 7f4391a00700 1 -- 172.21.15.27:0/1555304308 reap_dead start 2024-04-26T22:56:41.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.831+0000 7f4393a00700 1 -- 172.21.15.27:0/1555304308 shutdown_connections 2024-04-26T22:56:41.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.831+0000 7f4393a00700 1 -- 172.21.15.27:0/1555304308 >> 172.21.15.27:0/1555304308 conn(0x7f438c07a580 msgr2=0x7f438c1006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:41.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.831+0000 7f4393a00700 1 -- 172.21.15.27:0/1555304308 shutdown_connections 2024-04-26T22:56:41.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:41.832+0000 7f4393a00700 1 -- 172.21.15.27:0/1555304308 wait complete. 2024-04-26T22:56:42.060 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:41 smithi027 ceph-mon[28034]: pgmap v370: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:56:42.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:41 smithi080 ceph-mon[32324]: pgmap v370: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:56:42.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:41 smithi094 ceph-mon[32889]: pgmap v370: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:56:42.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.060+0000 7f982d200700 1 -- 172.21.15.27:0/627607113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9828103e30 msgr2=0x7f9828110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.060+0000 7f982d200700 1 --2- 172.21.15.27:0/627607113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9828103e30 0x7f9828110290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f982000b0a0 tx=0x7f982000b3b0 comp rx=0 tx=0).stop 2024-04-26T22:56:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.060+0000 7f982d200700 1 -- 172.21.15.27:0/627607113 shutdown_connections 2024-04-26T22:56:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.061+0000 7f982d200700 1 --2- 172.21.15.27:0/627607113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9828103e30 0x7f9828110290 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.061+0000 7f982d200700 1 --2- 172.21.15.27:0/627607113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98281094c0 0x7f98281098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.061+0000 7f982d200700 1 --2- 172.21.15.27:0/627607113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9828103470 0x7f98281038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.061+0000 7f982d200700 1 -- 172.21.15.27:0/627607113 >> 172.21.15.27:0/627607113 conn(0x7f982807a580 msgr2=0x7f982807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.061+0000 7f982d200700 1 -- 172.21.15.27:0/627607113 shutdown_connections 2024-04-26T22:56:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.061+0000 7f982d200700 1 -- 172.21.15.27:0/627607113 wait complete. 2024-04-26T22:56:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.062+0000 7f982d200700 1 Processor -- start 2024-04-26T22:56:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.062+0000 7f982d200700 1 -- start start 2024-04-26T22:56:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.063+0000 7f982d200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9828103470 0x7f982819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.063+0000 7f982d200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9828103e30 0x7f982819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.063+0000 7f982d200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98281094c0 0x7f98281998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:42.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.063+0000 7f982d200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f98281135c0 con 0x7f98281094c0 2024-04-26T22:56:42.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.063+0000 7f982d200700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9828113440 con 0x7f9828103470 2024-04-26T22:56:42.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.063+0000 7f982d200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9828113740 con 0x7f9828103e30 2024-04-26T22:56:42.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.063+0000 7f9825e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9828103470 0x7f982819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:42.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.063+0000 7f9825e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9828103470 0x7f982819f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:37440/0 (socket says 172.21.15.27:37440) 2024-04-26T22:56:42.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.063+0000 7f9825e00700 1 -- 172.21.15.27:0/94202498 learned_addr learned my addr 172.21.15.27:0/94202498 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:42.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.063+0000 7f9826800700 1 --2- 172.21.15.27:0/94202498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98281094c0 0x7f98281998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:42.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.063+0000 7f9825400700 1 --2- 172.21.15.27:0/94202498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9828103e30 0x7f982819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:42.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.064+0000 7f9825e00700 1 -- 172.21.15.27:0/94202498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9828103e30 msgr2=0x7f982819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:42.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.064+0000 7f9825e00700 1 --2- 172.21.15.27:0/94202498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9828103e30 0x7f982819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:42.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.064+0000 7f9825e00700 1 -- 172.21.15.27:0/94202498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98281094c0 msgr2=0x7f98281998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:42.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.064+0000 7f9825e00700 1 --2- 172.21.15.27:0/94202498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98281094c0 0x7f98281998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:42.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.064+0000 7f9825e00700 1 -- 172.21.15.27:0/94202498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982819a000 con 0x7f9828103470 2024-04-26T22:56:42.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.064+0000 7f9825e00700 1 --2- 172.21.15.27:0/94202498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9828103470 0x7f982819f6e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f981800e9a0 tx=0x7f981800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:42.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.064+0000 7f981ea00700 1 -- 172.21.15.27:0/94202498 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981800ca10 con 0x7f9828103470 2024-04-26T22:56:42.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.064+0000 7f982d200700 1 -- 172.21.15.27:0/94202498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982819a2f0 con 0x7f9828103470 2024-04-26T22:56:42.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.064+0000 7f981ea00700 1 -- 172.21.15.27:0/94202498 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f981800cbb0 con 0x7f9828103470 2024-04-26T22:56:42.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.064+0000 7f981ea00700 1 -- 172.21.15.27:0/94202498 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9818018680 con 0x7f9828103470 2024-04-26T22:56:42.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.065+0000 7f982d200700 1 -- 172.21.15.27:0/94202498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98281a65b0 con 0x7f9828103470 2024-04-26T22:56:42.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.065+0000 7f982d200700 1 -- 172.21.15.27:0/94202498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9828063bb0 con 0x7f9828103470 2024-04-26T22:56:42.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.067+0000 7f981ea00700 1 -- 172.21.15.27:0/94202498 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9818010430 con 0x7f9828103470 2024-04-26T22:56:42.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.069+0000 7f981ea00700 1 --2- 172.21.15.27:0/94202498 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f980c0a3cd0 0x7f980c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:42.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.069+0000 7f981ea00700 1 -- 172.21.15.27:0/94202498 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f98180d0a10 con 0x7f9828103470 2024-04-26T22:56:42.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.069+0000 7f9825400700 1 --2- 172.21.15.27:0/94202498 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f980c0a3cd0 0x7f980c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:42.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.070+0000 7f981ea00700 1 --2- 172.21.15.27:0/94202498 >> [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849] conn(0x7f980c0aedb0 0x7f980c0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:42.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.070+0000 7f981ea00700 1 -- 172.21.15.27:0/94202498 --> [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f980c0b1920 con 0x7f980c0aedb0 2024-04-26T22:56:42.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.070+0000 7f981ea00700 1 -- 172.21.15.27:0/94202498 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f98180d52a0 con 0x7f9828103470 2024-04-26T22:56:42.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.070+0000 7f9826800700 1 --2- 172.21.15.27:0/94202498 >> [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849] conn(0x7f980c0aedb0 0x7f980c0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:42.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.070+0000 7f9825400700 1 --2- 172.21.15.27:0/94202498 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f980c0a3cd0 0x7f980c0a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9814009680 tx=0x7f9814009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:42.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.071+0000 7f9826800700 1 --2- 172.21.15.27:0/94202498 >> [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849] conn(0x7f980c0aedb0 0x7f980c0b1250 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:42.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.072+0000 7f981ea00700 1 -- 172.21.15.27:0/94202498 <== osd.3 v2:172.21.15.27:6810/506143849 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f980c0b1920 con 0x7f980c0aedb0 2024-04-26T22:56:42.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.093+0000 7f982d200700 1 -- 172.21.15.27:0/94202498 --> [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9828063fe0 con 0x7f980c0aedb0 2024-04-26T22:56:42.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.094+0000 7f981ea00700 1 -- 172.21.15.27:0/94202498 <== osd.3 v2:172.21.15.27:6810/506143849 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9828063fe0 con 0x7f980c0aedb0 2024-04-26T22:56:42.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.094+0000 7f982d200700 1 -- 172.21.15.27:0/94202498 >> [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849] conn(0x7f980c0aedb0 msgr2=0x7f980c0b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:42.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.094+0000 7f982d200700 1 --2- 172.21.15.27:0/94202498 >> [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849] conn(0x7f980c0aedb0 0x7f980c0b1250 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:42.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.095+0000 7f982d200700 1 -- 172.21.15.27:0/94202498 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f980c0a3cd0 msgr2=0x7f980c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:42.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.095+0000 7f982d200700 1 --2- 172.21.15.27:0/94202498 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f980c0a3cd0 0x7f980c0a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9814009680 tx=0x7f9814009260 comp rx=0 tx=0).stop 2024-04-26T22:56:42.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.095+0000 7f982d200700 1 -- 172.21.15.27:0/94202498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9828103470 msgr2=0x7f982819f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:42.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.095+0000 7f982d200700 1 --2- 172.21.15.27:0/94202498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9828103470 0x7f982819f6e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f981800e9a0 tx=0x7f981800ecb0 comp rx=0 tx=0).stop 2024-04-26T22:56:42.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.095+0000 7f9826800700 1 -- 172.21.15.27:0/94202498 reap_dead start 2024-04-26T22:56:42.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.095+0000 7f982d200700 1 -- 172.21.15.27:0/94202498 shutdown_connections 2024-04-26T22:56:42.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.095+0000 7f982d200700 1 -- 172.21.15.27:0/94202498 >> 172.21.15.27:0/94202498 conn(0x7f982807a580 msgr2=0x7f9828102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:42.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.095+0000 7f982d200700 1 -- 172.21.15.27:0/94202498 shutdown_connections 2024-04-26T22:56:42.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:42.095+0000 7f982d200700 1 -- 172.21.15.27:0/94202498 wait complete. 2024-04-26T22:56:43.850 INFO:teuthology.orchestra.run.smithi027.stdout:343597383710 2024-04-26T22:56:43.851 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-26T22:56:43.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.445+0000 7f0962600700 1 -- 172.21.15.27:0/1868338044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c06bad0 msgr2=0x7f095c1102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:43.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.445+0000 7f0962600700 1 --2- 172.21.15.27:0/1868338044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c06bad0 0x7f095c1102e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0954009990 tx=0x7f0954009ca0 comp rx=0 tx=0).stop 2024-04-26T22:56:43.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.446+0000 7f0962600700 1 -- 172.21.15.27:0/1868338044 shutdown_connections 2024-04-26T22:56:43.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.446+0000 7f0962600700 1 --2- 172.21.15.27:0/1868338044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c06bad0 0x7f095c1102e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:43.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.446+0000 7f0962600700 1 --2- 172.21.15.27:0/1868338044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f095c1094a0 0x7f095c109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:43.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.446+0000 7f0962600700 1 --2- 172.21.15.27:0/1868338044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c06b130 0x7f095c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:43.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.446+0000 7f0962600700 1 -- 172.21.15.27:0/1868338044 >> 172.21.15.27:0/1868338044 conn(0x7f095c0792d0 msgr2=0x7f095c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:43.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.446+0000 7f0962600700 1 -- 172.21.15.27:0/1868338044 shutdown_connections 2024-04-26T22:56:43.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.446+0000 7f0962600700 1 -- 172.21.15.27:0/1868338044 wait complete. 2024-04-26T22:56:43.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.447+0000 7f0962600700 1 Processor -- start 2024-04-26T22:56:43.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.447+0000 7f0962600700 1 -- start start 2024-04-26T22:56:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.447+0000 7f0962600700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f095c06b130 0x7f095c10f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.448+0000 7f0962600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c06bad0 0x7f095c10fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.448+0000 7f0962600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c1094a0 0x7f095c1a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.448+0000 7f0962600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1135a0 con 0x7f095c06bad0 2024-04-26T22:56:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.448+0000 7f0962600700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f095c113420 con 0x7f095c06b130 2024-04-26T22:56:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.448+0000 7f0962600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f095c113720 con 0x7f095c1094a0 2024-04-26T22:56:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.448+0000 7f0952a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c06bad0 0x7f095c10fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.448+0000 7f095b400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f095c06b130 0x7f095c10f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.448+0000 7f0952a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c06bad0 0x7f095c10fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36486/0 (socket says 172.21.15.27:36486) 2024-04-26T22:56:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.448+0000 7f0952a00700 1 -- 172.21.15.27:0/2421037979 learned_addr learned my addr 172.21.15.27:0/2421037979 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.448+0000 7f095be00700 1 --2- 172.21.15.27:0/2421037979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c1094a0 0x7f095c1a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.448+0000 7f095b400700 1 -- 172.21.15.27:0/2421037979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c1094a0 msgr2=0x7f095c1a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.448+0000 7f095b400700 1 --2- 172.21.15.27:0/2421037979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c1094a0 0x7f095c1a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.448+0000 7f095b400700 1 -- 172.21.15.27:0/2421037979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c06bad0 msgr2=0x7f095c10fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.448+0000 7f095b400700 1 --2- 172.21.15.27:0/2421037979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c06bad0 0x7f095c10fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.448+0000 7f095b400700 1 -- 172.21.15.27:0/2421037979 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c1a5060 con 0x7f095c06b130 2024-04-26T22:56:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.448+0000 7f0952a00700 1 --2- 172.21.15.27:0/2421037979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c06bad0 0x7f095c10fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:56:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.449+0000 7f095b400700 1 --2- 172.21.15.27:0/2421037979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f095c06b130 0x7f095c10f640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f094c00ef10 tx=0x7f094c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.449+0000 7f0958c00700 1 -- 172.21.15.27:0/2421037979 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c00c9c0 con 0x7f095c06b130 2024-04-26T22:56:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.449+0000 7f0958c00700 1 -- 172.21.15.27:0/2421037979 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f094c019070 con 0x7f095c06b130 2024-04-26T22:56:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.449+0000 7f0962600700 1 -- 172.21.15.27:0/2421037979 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095c1a52f0 con 0x7f095c06b130 2024-04-26T22:56:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.449+0000 7f0958c00700 1 -- 172.21.15.27:0/2421037979 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c014480 con 0x7f095c06b130 2024-04-26T22:56:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.449+0000 7f0962600700 1 -- 172.21.15.27:0/2421037979 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095c1acbd0 con 0x7f095c06b130 2024-04-26T22:56:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.450+0000 7f0962600700 1 -- 172.21.15.27:0/2421037979 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f095c100c10 con 0x7f095c06b130 2024-04-26T22:56:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.453+0000 7f0958c00700 1 -- 172.21.15.27:0/2421037979 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f094c012050 con 0x7f095c06b130 2024-04-26T22:56:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.454+0000 7f0958c00700 1 --2- 172.21.15.27:0/2421037979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f09400a3cd0 0x7f09400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.454+0000 7f0958c00700 1 -- 172.21.15.27:0/2421037979 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f094c0d12b0 con 0x7f095c06b130 2024-04-26T22:56:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.454+0000 7f0952a00700 1 --2- 172.21.15.27:0/2421037979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f09400a3cd0 0x7f09400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.455+0000 7f0958c00700 1 --2- 172.21.15.27:0/2421037979 >> [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655] conn(0x7f09400aedb0 0x7f09400b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.455+0000 7f0958c00700 1 -- 172.21.15.27:0/2421037979 --> [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f09400b1920 con 0x7f09400aedb0 2024-04-26T22:56:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.455+0000 7f0958c00700 1 -- 172.21.15.27:0/2421037979 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f094c0d1660 con 0x7f095c06b130 2024-04-26T22:56:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.455+0000 7f095be00700 1 --2- 172.21.15.27:0/2421037979 >> [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655] conn(0x7f09400aedb0 0x7f09400b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.455+0000 7f0952a00700 1 --2- 172.21.15.27:0/2421037979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f09400a3cd0 0x7f09400a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f095c1042c0 tx=0x7f0944009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.456+0000 7f095be00700 1 --2- 172.21.15.27:0/2421037979 >> [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655] conn(0x7f09400aedb0 0x7f09400b1250 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.457+0000 7f0958c00700 1 -- 172.21.15.27:0/2421037979 <== osd.6 v2:172.21.15.27:6818/1730660655 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09400b1920 con 0x7f09400aedb0 2024-04-26T22:56:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.475+0000 7f0962600700 1 -- 172.21.15.27:0/2421037979 --> [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f095c104b50 con 0x7f09400aedb0 2024-04-26T22:56:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.476+0000 7f0958c00700 1 -- 172.21.15.27:0/2421037979 <== osd.6 v2:172.21.15.27:6818/1730660655 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f095c104b50 con 0x7f09400aedb0 2024-04-26T22:56:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.476+0000 7f0962600700 1 -- 172.21.15.27:0/2421037979 >> [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655] conn(0x7f09400aedb0 msgr2=0x7f09400b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.476+0000 7f0962600700 1 --2- 172.21.15.27:0/2421037979 >> [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655] conn(0x7f09400aedb0 0x7f09400b1250 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.476+0000 7f0962600700 1 -- 172.21.15.27:0/2421037979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f09400a3cd0 msgr2=0x7f09400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.476+0000 7f0962600700 1 --2- 172.21.15.27:0/2421037979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f09400a3cd0 0x7f09400a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f095c1042c0 tx=0x7f0944009260 comp rx=0 tx=0).stop 2024-04-26T22:56:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.476+0000 7f0962600700 1 -- 172.21.15.27:0/2421037979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f095c06b130 msgr2=0x7f095c10f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.476+0000 7f0962600700 1 --2- 172.21.15.27:0/2421037979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f095c06b130 0x7f095c10f640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f094c00ef10 tx=0x7f094c00c3a0 comp rx=0 tx=0).stop 2024-04-26T22:56:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.476+0000 7f095be00700 1 -- 172.21.15.27:0/2421037979 reap_dead start 2024-04-26T22:56:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.476+0000 7f0962600700 1 -- 172.21.15.27:0/2421037979 shutdown_connections 2024-04-26T22:56:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.476+0000 7f0962600700 1 -- 172.21.15.27:0/2421037979 >> 172.21.15.27:0/2421037979 conn(0x7f095c0792d0 msgr2=0x7f095c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.477+0000 7f0962600700 1 -- 172.21.15.27:0/2421037979 shutdown_connections 2024-04-26T22:56:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:43.477+0000 7f0962600700 1 -- 172.21.15.27:0/2421037979 wait complete. 2024-04-26T22:56:44.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:43 smithi027 ceph-mon[28034]: pgmap v371: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:56:44.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:43 smithi080 ceph-mon[32324]: pgmap v371: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:56:44.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:43 smithi094 ceph-mon[32889]: pgmap v371: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:56:46.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:45 smithi027 ceph-mon[28034]: pgmap v372: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T22:56:46.227 INFO:teuthology.orchestra.run.smithi027.stdout:107374182476 2024-04-26T22:56:46.227 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T22:56:46.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:45 smithi080 ceph-mon[32324]: pgmap v372: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T22:56:46.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:45 smithi094 ceph-mon[32889]: pgmap v372: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T22:56:47.249 INFO:teuthology.orchestra.run.smithi027.stdout:219043332149 2024-04-26T22:56:47.249 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-26T22:56:47.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:47 smithi027 ceph-mon[28034]: pgmap v373: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:56:47.935 INFO:teuthology.orchestra.run.smithi027.stdout:34359738460 2024-04-26T22:56:47.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T22:56:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.652+0000 7f9503e00700 1 -- 172.21.15.27:0/1206482789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f950410b620 msgr2=0x7f950410ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.652+0000 7f9503e00700 1 --2- 172.21.15.27:0/1206482789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f950410b620 0x7f950410ba00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f94f40099c0 tx=0x7f94f4009cd0 comp rx=0 tx=0).stop 2024-04-26T22:56:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.653+0000 7f9503e00700 1 -- 172.21.15.27:0/1206482789 shutdown_connections 2024-04-26T22:56:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.653+0000 7f9503e00700 1 --2- 172.21.15.27:0/1206482789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9504105f30 0x7f95041123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.653+0000 7f9503e00700 1 --2- 172.21.15.27:0/1206482789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f950410b620 0x7f950410ba00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.653+0000 7f9503e00700 1 --2- 172.21.15.27:0/1206482789 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9504105570 0x7f95041059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.653+0000 7f9503e00700 1 -- 172.21.15.27:0/1206482789 >> 172.21.15.27:0/1206482789 conn(0x7f9504101040 msgr2=0x7f9504103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.653+0000 7f9503e00700 1 -- 172.21.15.27:0/1206482789 shutdown_connections 2024-04-26T22:56:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.653+0000 7f9503e00700 1 -- 172.21.15.27:0/1206482789 wait complete. 2024-04-26T22:56:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.654+0000 7f9503e00700 1 Processor -- start 2024-04-26T22:56:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.654+0000 7f9503e00700 1 -- start start 2024-04-26T22:56:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.654+0000 7f9503e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9504105570 0x7f95041a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.654+0000 7f9503e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9504105f30 0x7f95041a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.654+0000 7f9503e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f950410b620 0x7f950419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.654+0000 7f9503e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9504115790 con 0x7f9504105f30 2024-04-26T22:56:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.654+0000 7f9503e00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9504115610 con 0x7f950410b620 2024-04-26T22:56:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.654+0000 7f9503e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9504115910 con 0x7f9504105570 2024-04-26T22:56:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.655+0000 7f94fbe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9504105f30 0x7f95041a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.655+0000 7f9503400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f950410b620 0x7f950419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.655+0000 7f9502a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9504105570 0x7f95041a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.655+0000 7f9503400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f950410b620 0x7f950419b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:37482/0 (socket says 172.21.15.27:37482) 2024-04-26T22:56:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.655+0000 7f9503400700 1 -- 172.21.15.27:0/2086227448 learned_addr learned my addr 172.21.15.27:0/2086227448 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.655+0000 7f9502a00700 1 --2- 172.21.15.27:0/2086227448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9504105570 0x7f95041a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:37012/0 (socket says 172.21.15.27:37012) 2024-04-26T22:56:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.655+0000 7f9503400700 1 -- 172.21.15.27:0/2086227448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9504105570 msgr2=0x7f95041a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.655+0000 7f9503400700 1 --2- 172.21.15.27:0/2086227448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9504105570 0x7f95041a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.655+0000 7f9503400700 1 -- 172.21.15.27:0/2086227448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9504105f30 msgr2=0x7f95041a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:48.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.655+0000 7f9503400700 1 --2- 172.21.15.27:0/2086227448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9504105f30 0x7f95041a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:48.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.655+0000 7f9503400700 1 -- 172.21.15.27:0/2086227448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f950419c110 con 0x7f950410b620 2024-04-26T22:56:48.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.655+0000 7f94fbe00700 1 --2- 172.21.15.27:0/2086227448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9504105f30 0x7f95041a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:56:48.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.655+0000 7f9503400700 1 --2- 172.21.15.27:0/2086227448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f950410b620 0x7f950419b9b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f94fc00ef30 tx=0x7f94fc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:48.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.656+0000 7f94fb400700 1 -- 172.21.15.27:0/2086227448 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc0092e0 con 0x7f950410b620 2024-04-26T22:56:48.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.656+0000 7f9503e00700 1 -- 172.21.15.27:0/2086227448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f950419c3a0 con 0x7f950410b620 2024-04-26T22:56:48.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.656+0000 7f94fb400700 1 -- 172.21.15.27:0/2086227448 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94fc004bb0 con 0x7f950410b620 2024-04-26T22:56:48.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.656+0000 7f94fb400700 1 -- 172.21.15.27:0/2086227448 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc004020 con 0x7f950410b620 2024-04-26T22:56:48.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.656+0000 7f9503e00700 1 -- 172.21.15.27:0/2086227448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95041a8670 con 0x7f950410b620 2024-04-26T22:56:48.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.657+0000 7f9503e00700 1 -- 172.21.15.27:0/2086227448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9504063bb0 con 0x7f950410b620 2024-04-26T22:56:48.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.660+0000 7f94fb400700 1 -- 172.21.15.27:0/2086227448 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f94fc01b030 con 0x7f950410b620 2024-04-26T22:56:48.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.661+0000 7f94fb400700 1 --2- 172.21.15.27:0/2086227448 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f94e80a3cd0 0x7f94e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:48.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.661+0000 7f94fb400700 1 -- 172.21.15.27:0/2086227448 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f94fc0d0d70 con 0x7f950410b620 2024-04-26T22:56:48.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.661+0000 7f9502a00700 1 --2- 172.21.15.27:0/2086227448 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f94e80a3cd0 0x7f94e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:48.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.661+0000 7f94fb400700 1 --2- 172.21.15.27:0/2086227448 >> [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588] conn(0x7f94e80aedb0 0x7f94e80b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:48.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.661+0000 7f94fb400700 1 -- 172.21.15.27:0/2086227448 --> [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94e80b1920 con 0x7f94e80aedb0 2024-04-26T22:56:48.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.661+0000 7f94fb400700 1 -- 172.21.15.27:0/2086227448 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f94fc0d1120 con 0x7f950410b620 2024-04-26T22:56:48.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.662+0000 7f94fbe00700 1 --2- 172.21.15.27:0/2086227448 >> [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588] conn(0x7f94e80aedb0 0x7f94e80b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:48.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.662+0000 7f9502a00700 1 --2- 172.21.15.27:0/2086227448 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f94e80a3cd0 0x7f94e80a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f94f4000c00 tx=0x7f94f4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:48.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.662+0000 7f94fbe00700 1 --2- 172.21.15.27:0/2086227448 >> [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588] conn(0x7f94e80aedb0 0x7f94e80b1250 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:48.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.663+0000 7f94fb400700 1 -- 172.21.15.27:0/2086227448 <== osd.4 v2:172.21.15.80:6808/1483108588 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94e80b1920 con 0x7f94e80aedb0 2024-04-26T22:56:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.682+0000 7f9503e00700 1 -- 172.21.15.27:0/2086227448 --> [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9504063fb0 con 0x7f94e80aedb0 2024-04-26T22:56:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.682+0000 7f94fb400700 1 -- 172.21.15.27:0/2086227448 <== osd.4 v2:172.21.15.80:6808/1483108588 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9504063fb0 con 0x7f94e80aedb0 2024-04-26T22:56:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.683+0000 7f9503e00700 1 -- 172.21.15.27:0/2086227448 >> [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588] conn(0x7f94e80aedb0 msgr2=0x7f94e80b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.683+0000 7f9503e00700 1 --2- 172.21.15.27:0/2086227448 >> [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588] conn(0x7f94e80aedb0 0x7f94e80b1250 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.683+0000 7f9503e00700 1 -- 172.21.15.27:0/2086227448 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f94e80a3cd0 msgr2=0x7f94e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.683+0000 7f9503e00700 1 --2- 172.21.15.27:0/2086227448 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f94e80a3cd0 0x7f94e80a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f94f4000c00 tx=0x7f94f4005fb0 comp rx=0 tx=0).stop 2024-04-26T22:56:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.683+0000 7f9503e00700 1 -- 172.21.15.27:0/2086227448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f950410b620 msgr2=0x7f950419b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.683+0000 7f9503e00700 1 --2- 172.21.15.27:0/2086227448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f950410b620 0x7f950419b9b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f94fc00ef30 tx=0x7f94fc00c3a0 comp rx=0 tx=0).stop 2024-04-26T22:56:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.683+0000 7f9503400700 1 -- 172.21.15.27:0/2086227448 reap_dead start 2024-04-26T22:56:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.683+0000 7f9503e00700 1 -- 172.21.15.27:0/2086227448 shutdown_connections 2024-04-26T22:56:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.683+0000 7f9503e00700 1 -- 172.21.15.27:0/2086227448 >> 172.21.15.27:0/2086227448 conn(0x7f9504101040 msgr2=0x7f95041026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.683+0000 7f9503e00700 1 -- 172.21.15.27:0/2086227448 shutdown_connections 2024-04-26T22:56:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:47.684+0000 7f9503e00700 1 -- 172.21.15.27:0/2086227448 wait complete. 2024-04-26T22:56:48.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:47 smithi080 ceph-mon[32324]: pgmap v373: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:56:48.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:47 smithi094 ceph-mon[32889]: pgmap v373: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:56:48.835 INFO:teuthology.orchestra.run.smithi027.stdout:287762808873 2024-04-26T22:56:48.835 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-26T22:56:49.290 INFO:teuthology.orchestra.run.smithi027.stdout:141733920840 2024-04-26T22:56:49.290 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T22:56:49.461 INFO:teuthology.orchestra.run.smithi027.stdout:197568495676 2024-04-26T22:56:49.461 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-26T22:56:49.733 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:49.736 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:49.736 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:50.101 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:49 smithi027 ceph-mon[28034]: pgmap v374: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T22:56:50.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:49 smithi080 ceph-mon[32324]: pgmap v374: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T22:56:50.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:49 smithi094 ceph-mon[32889]: pgmap v374: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T22:56:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.568+0000 7fda38e00700 1 -- 172.21.15.27:0/2668390044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda341094c0 msgr2=0x7fda341098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.568+0000 7fda38e00700 1 --2- 172.21.15.27:0/2668390044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda341094c0 0x7fda341098a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fda240098f0 tx=0x7fda24009c00 comp rx=0 tx=0).stop 2024-04-26T22:56:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.568+0000 7fda38e00700 1 -- 172.21.15.27:0/2668390044 shutdown_connections 2024-04-26T22:56:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.568+0000 7fda38e00700 1 --2- 172.21.15.27:0/2668390044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda34103e30 0x7fda34110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.568+0000 7fda38e00700 1 --2- 172.21.15.27:0/2668390044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda341094c0 0x7fda341098a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.568+0000 7fda38e00700 1 --2- 172.21.15.27:0/2668390044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda34103470 0x7fda341038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.568+0000 7fda38e00700 1 -- 172.21.15.27:0/2668390044 >> 172.21.15.27:0/2668390044 conn(0x7fda3407a580 msgr2=0x7fda3407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.568+0000 7fda38e00700 1 -- 172.21.15.27:0/2668390044 shutdown_connections 2024-04-26T22:56:50.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.568+0000 7fda38e00700 1 -- 172.21.15.27:0/2668390044 wait complete. 2024-04-26T22:56:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.569+0000 7fda38e00700 1 Processor -- start 2024-04-26T22:56:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.570+0000 7fda38e00700 1 -- start start 2024-04-26T22:56:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.570+0000 7fda38e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda34103470 0x7fda3419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.570+0000 7fda38e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda34103e30 0x7fda3419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.570+0000 7fda38e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda341094c0 0x7fda341a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.570+0000 7fda38e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda34113610 con 0x7fda34103470 2024-04-26T22:56:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.570+0000 7fda38e00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fda34113490 con 0x7fda341094c0 2024-04-26T22:56:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.570+0000 7fda38e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fda34113790 con 0x7fda34103e30 2024-04-26T22:56:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.570+0000 7fda31e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda34103470 0x7fda3419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.570+0000 7fda31e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda34103470 0x7fda3419bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39902/0 (socket says 172.21.15.27:39902) 2024-04-26T22:56:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.570+0000 7fda31e00700 1 -- 172.21.15.27:0/3542063048 learned_addr learned my addr 172.21.15.27:0/3542063048 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.570+0000 7fda32800700 1 --2- 172.21.15.27:0/3542063048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda341094c0 0x7fda341a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.571+0000 7fda31400700 1 --2- 172.21.15.27:0/3542063048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda34103e30 0x7fda3419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.571+0000 7fda32800700 1 -- 172.21.15.27:0/3542063048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda34103e30 msgr2=0x7fda3419c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.571+0000 7fda32800700 1 --2- 172.21.15.27:0/3542063048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda34103e30 0x7fda3419c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.571+0000 7fda32800700 1 -- 172.21.15.27:0/3542063048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda34103470 msgr2=0x7fda3419bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.571+0000 7fda32800700 1 --2- 172.21.15.27:0/3542063048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda34103470 0x7fda3419bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.571+0000 7fda32800700 1 -- 172.21.15.27:0/3542063048 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda341a0d60 con 0x7fda341094c0 2024-04-26T22:56:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.571+0000 7fda31e00700 1 --2- 172.21.15.27:0/3542063048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda34103470 0x7fda3419bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:56:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.571+0000 7fda32800700 1 --2- 172.21.15.27:0/3542063048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda341094c0 0x7fda341a0620 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00efc0 tx=0x7fda2c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.572+0000 7fda2aa00700 1 -- 172.21.15.27:0/3542063048 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0092e0 con 0x7fda341094c0 2024-04-26T22:56:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.572+0000 7fda2aa00700 1 -- 172.21.15.27:0/3542063048 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda2c004bb0 con 0x7fda341094c0 2024-04-26T22:56:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.572+0000 7fda38e00700 1 -- 172.21.15.27:0/3542063048 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda341a0ff0 con 0x7fda341094c0 2024-04-26T22:56:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.572+0000 7fda2aa00700 1 -- 172.21.15.27:0/3542063048 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c004020 con 0x7fda341094c0 2024-04-26T22:56:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.572+0000 7fda38e00700 1 -- 172.21.15.27:0/3542063048 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda341a88d0 con 0x7fda341094c0 2024-04-26T22:56:50.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.573+0000 7fda38e00700 1 -- 172.21.15.27:0/3542063048 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fda34102aa0 con 0x7fda341094c0 2024-04-26T22:56:50.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.575+0000 7fda2aa00700 1 -- 172.21.15.27:0/3542063048 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fda2c01b030 con 0x7fda341094c0 2024-04-26T22:56:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.577+0000 7fda2aa00700 1 --2- 172.21.15.27:0/3542063048 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fda180a3cd0 0x7fda180a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.577+0000 7fda2aa00700 1 -- 172.21.15.27:0/3542063048 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fda2c0d0cb0 con 0x7fda341094c0 2024-04-26T22:56:50.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.577+0000 7fda31e00700 1 --2- 172.21.15.27:0/3542063048 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fda180a3cd0 0x7fda180a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:50.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.577+0000 7fda2aa00700 1 --2- 172.21.15.27:0/3542063048 >> [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245] conn(0x7fda180aedb0 0x7fda180b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:50.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.577+0000 7fda2aa00700 1 -- 172.21.15.27:0/3542063048 --> [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda2c010ec0 con 0x7fda180aedb0 2024-04-26T22:56:50.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.577+0000 7fda2aa00700 1 -- 172.21.15.27:0/3542063048 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fda2c0d1060 con 0x7fda341094c0 2024-04-26T22:56:50.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.578+0000 7fda31400700 1 --2- 172.21.15.27:0/3542063048 >> [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245] conn(0x7fda180aedb0 0x7fda180b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:50.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.578+0000 7fda31e00700 1 --2- 172.21.15.27:0/3542063048 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fda180a3cd0 0x7fda180a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fda24009fd0 tx=0x7fda24019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.578+0000 7fda31400700 1 --2- 172.21.15.27:0/3542063048 >> [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245] conn(0x7fda180aedb0 0x7fda180b1250 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.579+0000 7fda2aa00700 1 -- 172.21.15.27:0/3542063048 <== osd.1 v2:172.21.15.80:6800/691528245 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fda2c010ec0 con 0x7fda180aedb0 2024-04-26T22:56:50.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.598+0000 7fda38e00700 1 -- 172.21.15.27:0/3542063048 --> [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fda34104b60 con 0x7fda180aedb0 2024-04-26T22:56:50.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.598+0000 7fda2aa00700 1 -- 172.21.15.27:0/3542063048 <== osd.1 v2:172.21.15.80:6800/691528245 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fda34104b60 con 0x7fda180aedb0 2024-04-26T22:56:50.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.599+0000 7fda38e00700 1 -- 172.21.15.27:0/3542063048 >> [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245] conn(0x7fda180aedb0 msgr2=0x7fda180b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:50.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.599+0000 7fda38e00700 1 --2- 172.21.15.27:0/3542063048 >> [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245] conn(0x7fda180aedb0 0x7fda180b1250 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:50.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.599+0000 7fda38e00700 1 -- 172.21.15.27:0/3542063048 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fda180a3cd0 msgr2=0x7fda180a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:50.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.599+0000 7fda38e00700 1 --2- 172.21.15.27:0/3542063048 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fda180a3cd0 0x7fda180a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fda24009fd0 tx=0x7fda24019040 comp rx=0 tx=0).stop 2024-04-26T22:56:50.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.599+0000 7fda38e00700 1 -- 172.21.15.27:0/3542063048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda341094c0 msgr2=0x7fda341a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:50.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.599+0000 7fda38e00700 1 --2- 172.21.15.27:0/3542063048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda341094c0 0x7fda341a0620 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00efc0 tx=0x7fda2c00c3a0 comp rx=0 tx=0).stop 2024-04-26T22:56:50.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.599+0000 7fda32800700 1 -- 172.21.15.27:0/3542063048 reap_dead start 2024-04-26T22:56:50.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.599+0000 7fda38e00700 1 -- 172.21.15.27:0/3542063048 shutdown_connections 2024-04-26T22:56:50.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.599+0000 7fda38e00700 1 -- 172.21.15.27:0/3542063048 >> 172.21.15.27:0/3542063048 conn(0x7fda3407a580 msgr2=0x7fda34102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:50.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.599+0000 7fda38e00700 1 -- 172.21.15.27:0/3542063048 shutdown_connections 2024-04-26T22:56:50.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.600+0000 7fda38e00700 1 -- 172.21.15.27:0/3542063048 wait complete. 2024-04-26T22:56:51.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.427+0000 7f76c0c00700 1 -- 172.21.15.27:0/1034669468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76bc1094a0 msgr2=0x7f76bc109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:51.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.427+0000 7f76c0c00700 1 --2- 172.21.15.27:0/1034669468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76bc1094a0 0x7f76bc109880 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009990 tx=0x7f76ac009ca0 comp rx=0 tx=0).stop 2024-04-26T22:56:51.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.427+0000 7f76c0c00700 1 -- 172.21.15.27:0/1034669468 shutdown_connections 2024-04-26T22:56:51.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.427+0000 7f76c0c00700 1 --2- 172.21.15.27:0/1034669468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76bc06bad0 0x7f76bc1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:51.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.427+0000 7f76c0c00700 1 --2- 172.21.15.27:0/1034669468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76bc1094a0 0x7f76bc109880 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:51.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.427+0000 7f76c0c00700 1 --2- 172.21.15.27:0/1034669468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76bc06b130 0x7f76bc06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:51.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.427+0000 7f76c0c00700 1 -- 172.21.15.27:0/1034669468 >> 172.21.15.27:0/1034669468 conn(0x7f76bc0792d0 msgr2=0x7f76bc0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:51.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.427+0000 7f76c0c00700 1 -- 172.21.15.27:0/1034669468 shutdown_connections 2024-04-26T22:56:51.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.427+0000 7f76c0c00700 1 -- 172.21.15.27:0/1034669468 wait complete. 2024-04-26T22:56:51.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.428+0000 7f76c0c00700 1 Processor -- start 2024-04-26T22:56:51.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.429+0000 7f76c0c00700 1 -- start start 2024-04-26T22:56:51.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.429+0000 7f76c0c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76bc06b130 0x7f76bc19be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:51.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.429+0000 7f76c0c00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76bc06bad0 0x7f76bc19c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:51.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.429+0000 7f76c0c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76bc1094a0 0x7f76bc1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:51.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.429+0000 7f76c0c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc1135e0 con 0x7f76bc1094a0 2024-04-26T22:56:51.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.429+0000 7f76c0c00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc113460 con 0x7f76bc06bad0 2024-04-26T22:56:51.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.429+0000 7f76c0c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc113760 con 0x7f76bc06b130 2024-04-26T22:56:51.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.429+0000 7f76b9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76bc06b130 0x7f76bc19be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:51.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.429+0000 7f76b9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76bc06b130 0x7f76bc19be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:46278/0 (socket says 172.21.15.27:46278) 2024-04-26T22:56:51.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.429+0000 7f76b9e00700 1 -- 172.21.15.27:0/2382784005 learned_addr learned my addr 172.21.15.27:0/2382784005 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:51.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.429+0000 7f76ba800700 1 --2- 172.21.15.27:0/2382784005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76bc1094a0 0x7f76bc1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:51.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.430+0000 7f76b9400700 1 --2- 172.21.15.27:0/2382784005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76bc06bad0 0x7f76bc19c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:51.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.430+0000 7f76b9e00700 1 -- 172.21.15.27:0/2382784005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76bc06bad0 msgr2=0x7f76bc19c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:51.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.430+0000 7f76b9e00700 1 --2- 172.21.15.27:0/2382784005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76bc06bad0 0x7f76bc19c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:51.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.430+0000 7f76b9e00700 1 -- 172.21.15.27:0/2382784005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76bc1094a0 msgr2=0x7f76bc1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:51.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.430+0000 7f76b9e00700 1 --2- 172.21.15.27:0/2382784005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76bc1094a0 0x7f76bc1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:51.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.430+0000 7f76b9e00700 1 -- 172.21.15.27:0/2382784005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76bc1a0da0 con 0x7f76bc06b130 2024-04-26T22:56:51.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.430+0000 7f76b9e00700 1 --2- 172.21.15.27:0/2382784005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76bc06b130 0x7f76bc19be30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0047b0 tx=0x7f76ac00bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:51.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.431+0000 7f76b2a00700 1 -- 172.21.15.27:0/2382784005 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac027070 con 0x7f76bc06b130 2024-04-26T22:56:51.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.431+0000 7f76b2a00700 1 -- 172.21.15.27:0/2382784005 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76ac005da0 con 0x7f76bc06b130 2024-04-26T22:56:51.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.431+0000 7f76b2a00700 1 -- 172.21.15.27:0/2382784005 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac0173f0 con 0x7f76bc06b130 2024-04-26T22:56:51.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.431+0000 7f76c0c00700 1 -- 172.21.15.27:0/2382784005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76bc1a1030 con 0x7f76bc06b130 2024-04-26T22:56:51.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.431+0000 7f76c0c00700 1 -- 172.21.15.27:0/2382784005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76bc1a8910 con 0x7f76bc06b130 2024-04-26T22:56:51.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.432+0000 7f76c0c00700 1 -- 172.21.15.27:0/2382784005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f76bc100b70 con 0x7f76bc06b130 2024-04-26T22:56:51.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.435+0000 7f76b2a00700 1 -- 172.21.15.27:0/2382784005 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f76ac02a070 con 0x7f76bc06b130 2024-04-26T22:56:51.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.437+0000 7f76b2a00700 1 --2- 172.21.15.27:0/2382784005 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f76a00a3d50 0x7f76a00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:51.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.437+0000 7f76b2a00700 1 -- 172.21.15.27:0/2382784005 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f76ac0d1bc0 con 0x7f76bc06b130 2024-04-26T22:56:51.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.437+0000 7f76b9400700 1 --2- 172.21.15.27:0/2382784005 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f76a00a3d50 0x7f76a00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:51.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.437+0000 7f76b2a00700 1 --2- 172.21.15.27:0/2382784005 >> [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187] conn(0x7f76a00aee30 0x7f76a00b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:51.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.437+0000 7f76b2a00700 1 -- 172.21.15.27:0/2382784005 --> [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76ac02ee50 con 0x7f76a00aee30 2024-04-26T22:56:51.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.437+0000 7f76b2a00700 1 -- 172.21.15.27:0/2382784005 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f76ac0d1f70 con 0x7f76bc06b130 2024-04-26T22:56:51.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.438+0000 7f76ba800700 1 --2- 172.21.15.27:0/2382784005 >> [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187] conn(0x7f76a00aee30 0x7f76a00b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:51.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.438+0000 7f76b9400700 1 --2- 172.21.15.27:0/2382784005 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f76a00a3d50 0x7f76a00a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f76bc110720 tx=0x7f76a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:51.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.438+0000 7f76ba800700 1 --2- 172.21.15.27:0/2382784005 >> [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187] conn(0x7f76a00aee30 0x7f76a00b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:51.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.439+0000 7f76b2a00700 1 -- 172.21.15.27:0/2382784005 <== osd.2 v2:172.21.15.94:6800/3315198187 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76ac02ee50 con 0x7f76a00aee30 2024-04-26T22:56:51.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.458+0000 7f76c0c00700 1 -- 172.21.15.27:0/2382784005 --> [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f76bc104b50 con 0x7f76a00aee30 2024-04-26T22:56:51.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.459+0000 7f76b2a00700 1 -- 172.21.15.27:0/2382784005 <== osd.2 v2:172.21.15.94:6800/3315198187 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f76bc104b50 con 0x7f76a00aee30 2024-04-26T22:56:51.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.459+0000 7f76c0c00700 1 -- 172.21.15.27:0/2382784005 >> [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187] conn(0x7f76a00aee30 msgr2=0x7f76a00b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:51.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.459+0000 7f76c0c00700 1 --2- 172.21.15.27:0/2382784005 >> [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187] conn(0x7f76a00aee30 0x7f76a00b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:51.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.460+0000 7f76c0c00700 1 -- 172.21.15.27:0/2382784005 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f76a00a3d50 msgr2=0x7f76a00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:51.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.460+0000 7f76c0c00700 1 --2- 172.21.15.27:0/2382784005 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f76a00a3d50 0x7f76a00a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f76bc110720 tx=0x7f76a8009260 comp rx=0 tx=0).stop 2024-04-26T22:56:51.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.460+0000 7f76c0c00700 1 -- 172.21.15.27:0/2382784005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76bc06b130 msgr2=0x7f76bc19be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:51.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.460+0000 7f76c0c00700 1 --2- 172.21.15.27:0/2382784005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76bc06b130 0x7f76bc19be30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0047b0 tx=0x7f76ac00bf90 comp rx=0 tx=0).stop 2024-04-26T22:56:51.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.460+0000 7f76ba800700 1 -- 172.21.15.27:0/2382784005 reap_dead start 2024-04-26T22:56:51.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.460+0000 7f76c0c00700 1 -- 172.21.15.27:0/2382784005 shutdown_connections 2024-04-26T22:56:51.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.460+0000 7f76c0c00700 1 -- 172.21.15.27:0/2382784005 >> 172.21.15.27:0/2382784005 conn(0x7f76bc0792d0 msgr2=0x7f76bc100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:51.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.460+0000 7f76c0c00700 1 -- 172.21.15.27:0/2382784005 shutdown_connections 2024-04-26T22:56:51.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:50.460+0000 7f76c0c00700 1 -- 172.21.15.27:0/2382784005 wait complete. 2024-04-26T22:56:52.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:51 smithi027 ceph-mon[28034]: pgmap v375: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:56:52.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:51 smithi080 ceph-mon[32324]: pgmap v375: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:56:52.281 INFO:teuthology.orchestra.run.smithi027.stdout:77309411413 2024-04-26T22:56:52.282 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T22:56:52.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:51 smithi094 ceph-mon[32889]: pgmap v375: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:56:54.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:53 smithi027 ceph-mon[28034]: pgmap v376: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:56:54.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:53 smithi080 ceph-mon[32324]: pgmap v376: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:56:54.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:53 smithi094 ceph-mon[32889]: pgmap v376: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:56:54.446 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:54.447 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:54.475 INFO:teuthology.orchestra.run.smithi027.stdout:55834574939 2024-04-26T22:56:54.475 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T22:56:54.790 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:56:56.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:55 smithi027 ceph-mon[28034]: pgmap v377: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:56:56.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:55 smithi080 ceph-mon[32324]: pgmap v377: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:56:56.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:55 smithi094 ceph-mon[32889]: pgmap v377: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:56:58.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:57 smithi027 ceph-mon[28034]: pgmap v378: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-04-26T22:56:58.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:57 smithi080 ceph-mon[32324]: pgmap v378: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-04-26T22:56:58.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:57 smithi094 ceph-mon[32889]: pgmap v378: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-04-26T22:56:59.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.493+0000 7f27c5e00700 1 -- 172.21.15.27:0/798827099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27c010a410 msgr2=0x7f27c0116920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:59.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.493+0000 7f27c5e00700 1 --2- 172.21.15.27:0/798827099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27c010a410 0x7f27c0116920 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009f90 tx=0x7f27b800b330 comp rx=0 tx=0).stop 2024-04-26T22:56:59.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.493+0000 7f27c5e00700 1 -- 172.21.15.27:0/798827099 shutdown_connections 2024-04-26T22:56:59.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.493+0000 7f27c5e00700 1 --2- 172.21.15.27:0/798827099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27c010a410 0x7f27c0116920 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:59.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.493+0000 7f27c5e00700 1 --2- 172.21.15.27:0/798827099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27c010fa50 0x7f27c010fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:59.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.493+0000 7f27c5e00700 1 --2- 172.21.15.27:0/798827099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c0109a50 0x7f27c0109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:59.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.493+0000 7f27c5e00700 1 -- 172.21.15.27:0/798827099 >> 172.21.15.27:0/798827099 conn(0x7f27c0076bf0 msgr2=0x7f27c0108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:59.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.494+0000 7f27c5e00700 1 -- 172.21.15.27:0/798827099 shutdown_connections 2024-04-26T22:56:59.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.494+0000 7f27c5e00700 1 -- 172.21.15.27:0/798827099 wait complete. 2024-04-26T22:56:59.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.495+0000 7f27c5e00700 1 Processor -- start 2024-04-26T22:56:59.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.495+0000 7f27c5e00700 1 -- start start 2024-04-26T22:56:59.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.495+0000 7f27c5e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27c0109a50 0x7f27c01a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:59.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.495+0000 7f27c5e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27c010a410 0x7f27c01a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:59.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.495+0000 7f27c5e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c010fa50 0x7f27c01a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:59.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.495+0000 7f27c5e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f27c0119c90 con 0x7f27c010fa50 2024-04-26T22:56:59.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.495+0000 7f27c5e00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f27c0119b10 con 0x7f27c010a410 2024-04-26T22:56:59.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.495+0000 7f27c5e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f27c0119e10 con 0x7f27c0109a50 2024-04-26T22:56:59.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.496+0000 7f27bea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27c0109a50 0x7f27c01a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:59.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.496+0000 7f27bf400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c010fa50 0x7f27c01a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:59.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.496+0000 7f27bea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27c0109a50 0x7f27c01a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:49354/0 (socket says 172.21.15.27:49354) 2024-04-26T22:56:59.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.496+0000 7f27be000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27c010a410 0x7f27c01a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:59.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.496+0000 7f27bea00700 1 -- 172.21.15.27:0/491304427 learned_addr learned my addr 172.21.15.27:0/491304427 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:56:59.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.496+0000 7f27be000700 1 -- 172.21.15.27:0/491304427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27c0109a50 msgr2=0x7f27c01a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:59.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.496+0000 7f27be000700 1 --2- 172.21.15.27:0/491304427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27c0109a50 0x7f27c01a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:59.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.496+0000 7f27be000700 1 -- 172.21.15.27:0/491304427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c010fa50 msgr2=0x7f27c01a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:59.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.496+0000 7f27be000700 1 --2- 172.21.15.27:0/491304427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c010fa50 0x7f27c01a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:59.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.496+0000 7f27be000700 1 -- 172.21.15.27:0/491304427 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c01a50b0 con 0x7f27c010a410 2024-04-26T22:56:59.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.496+0000 7f27bf400700 1 --2- 172.21.15.27:0/491304427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c010fa50 0x7f27c01a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:56:59.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.497+0000 7f27be000700 1 --2- 172.21.15.27:0/491304427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27c010a410 0x7f27c01a05e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00e9c0 tx=0x7f27ac00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:59.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.497+0000 7f27c4a00700 1 -- 172.21.15.27:0/491304427 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac00c9c0 con 0x7f27c010a410 2024-04-26T22:56:59.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.497+0000 7f27c5e00700 1 -- 172.21.15.27:0/491304427 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c01a53a0 con 0x7f27c010a410 2024-04-26T22:56:59.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.497+0000 7f27c4a00700 1 -- 172.21.15.27:0/491304427 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27ac00cb60 con 0x7f27c010a410 2024-04-26T22:56:59.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.497+0000 7f27c4a00700 1 -- 172.21.15.27:0/491304427 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac018420 con 0x7f27c010a410 2024-04-26T22:56:59.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.497+0000 7f27c5e00700 1 -- 172.21.15.27:0/491304427 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c01acc20 con 0x7f27c010a410 2024-04-26T22:56:59.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.498+0000 7f27c5e00700 1 -- 172.21.15.27:0/491304427 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c006a110 con 0x7f27c010a410 2024-04-26T22:56:59.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.500+0000 7f27c4a00700 1 -- 172.21.15.27:0/491304427 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f27ac010430 con 0x7f27c010a410 2024-04-26T22:56:59.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.502+0000 7f27c4a00700 1 --2- 172.21.15.27:0/491304427 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f27a40a3cd0 0x7f27a40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:56:59.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.502+0000 7f27c4a00700 1 -- 172.21.15.27:0/491304427 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f27ac0d0a20 con 0x7f27c010a410 2024-04-26T22:56:59.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.502+0000 7f27bea00700 1 --2- 172.21.15.27:0/491304427 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f27a40a3cd0 0x7f27a40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:56:59.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.503+0000 7f27bea00700 1 --2- 172.21.15.27:0/491304427 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f27a40a3cd0 0x7f27a40a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f27b0005950 tx=0x7f27b000b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:56:59.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.504+0000 7f27c4a00700 1 -- 172.21.15.27:0/491304427 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f27ac09cf00 con 0x7f27c010a410 2024-04-26T22:56:59.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.742+0000 7f27c5e00700 1 -- 172.21.15.27:0/491304427 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f27c01a1400 con 0x7f27c010a410 2024-04-26T22:56:59.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.743+0000 7f27c4a00700 1 -- 172.21.15.27:0/491304427 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f27ac09c800 con 0x7f27c010a410 2024-04-26T22:56:59.744 INFO:teuthology.orchestra.run.smithi027.stdout:163208757317 2024-04-26T22:56:59.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.746+0000 7f27c5e00700 1 -- 172.21.15.27:0/491304427 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f27a40a3cd0 msgr2=0x7f27a40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.746+0000 7f27c5e00700 1 --2- 172.21.15.27:0/491304427 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f27a40a3cd0 0x7f27a40a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f27b0005950 tx=0x7f27b000b500 comp rx=0 tx=0).stop 2024-04-26T22:56:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.746+0000 7f27c5e00700 1 -- 172.21.15.27:0/491304427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27c010a410 msgr2=0x7f27c01a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:56:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.746+0000 7f27c5e00700 1 --2- 172.21.15.27:0/491304427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27c010a410 0x7f27c01a05e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00e9c0 tx=0x7f27ac00ecd0 comp rx=0 tx=0).stop 2024-04-26T22:56:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.746+0000 7f27c5e00700 1 -- 172.21.15.27:0/491304427 shutdown_connections 2024-04-26T22:56:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.746+0000 7f27c5e00700 1 --2- 172.21.15.27:0/491304427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c010fa50 0x7f27c01a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.746+0000 7f27c5e00700 1 --2- 172.21.15.27:0/491304427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27c0109a50 0x7f27c01a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.746+0000 7f27c5e00700 1 --2- 172.21.15.27:0/491304427 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f27a40a3cd0 0x7f27a40a6190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:59.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.746+0000 7f27c5e00700 1 --2- 172.21.15.27:0/491304427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27c010a410 0x7f27c01a05e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:56:59.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.746+0000 7f27c5e00700 1 -- 172.21.15.27:0/491304427 >> 172.21.15.27:0/491304427 conn(0x7f27c0076bf0 msgr2=0x7f27c0077130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:56:59.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.747+0000 7f27c5e00700 1 -- 172.21.15.27:0/491304427 shutdown_connections 2024-04-26T22:56:59.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:56:59.747+0000 7f27c5e00700 1 -- 172.21.15.27:0/491304427 wait complete. 2024-04-26T22:57:00.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:59 smithi027 ceph-mon[28034]: pgmap v379: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:00.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:56:59 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/491304427' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T22:57:00.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:59 smithi080 ceph-mon[32324]: pgmap v379: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:00.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:56:59 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/491304427' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T22:57:00.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:59 smithi094 ceph-mon[32889]: pgmap v379: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:00.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:56:59 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/491304427' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T22:57:02.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:01 smithi027 ceph-mon[28034]: pgmap v380: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:57:02.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:01 smithi080 ceph-mon[32324]: pgmap v380: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:57:02.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:01 smithi094 ceph-mon[32889]: pgmap v380: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:57:03.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:02 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/182823865' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T22:57:03.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:02 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/182823865' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T22:57:03.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:02 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/182823865' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T22:57:03.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.542+0000 7f0c67c00700 1 -- 172.21.15.27:0/3223336812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6010a410 msgr2=0x7f0c60116920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:03.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.542+0000 7f0c67c00700 1 --2- 172.21.15.27:0/3223336812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6010a410 0x7f0c60116920 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009f90 tx=0x7f0c5800b330 comp rx=0 tx=0).stop 2024-04-26T22:57:03.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.542+0000 7f0c67c00700 1 -- 172.21.15.27:0/3223336812 shutdown_connections 2024-04-26T22:57:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.542+0000 7f0c67c00700 1 --2- 172.21.15.27:0/3223336812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6010a410 0x7f0c60116920 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.542+0000 7f0c67c00700 1 --2- 172.21.15.27:0/3223336812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c6010fa50 0x7f0c6010fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.542+0000 7f0c67c00700 1 --2- 172.21.15.27:0/3223336812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c60109a50 0x7f0c60109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.542+0000 7f0c67c00700 1 -- 172.21.15.27:0/3223336812 >> 172.21.15.27:0/3223336812 conn(0x7f0c60076bf0 msgr2=0x7f0c60108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.542+0000 7f0c67c00700 1 -- 172.21.15.27:0/3223336812 shutdown_connections 2024-04-26T22:57:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.542+0000 7f0c67c00700 1 -- 172.21.15.27:0/3223336812 wait complete. 2024-04-26T22:57:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.544+0000 7f0c67c00700 1 Processor -- start 2024-04-26T22:57:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.544+0000 7f0c67c00700 1 -- start start 2024-04-26T22:57:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.544+0000 7f0c67c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c60109a50 0x7f0c601a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.544+0000 7f0c67c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6010a410 0x7f0c601a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.544+0000 7f0c67c00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c6010fa50 0x7f0c601a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.544+0000 7f0c67c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0c60119c90 con 0x7f0c60109a50 2024-04-26T22:57:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.544+0000 7f0c67c00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0c60119b10 con 0x7f0c6010fa50 2024-04-26T22:57:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.544+0000 7f0c67c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c60119e10 con 0x7f0c6010a410 2024-04-26T22:57:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.545+0000 7f0c65200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c60109a50 0x7f0c601a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.545+0000 7f0c5fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6010a410 0x7f0c601a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.545+0000 7f0c5fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6010a410 0x7f0c601a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:49370/0 (socket says 172.21.15.27:49370) 2024-04-26T22:57:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.545+0000 7f0c65200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c60109a50 0x7f0c601a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35722/0 (socket says 172.21.15.27:35722) 2024-04-26T22:57:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.545+0000 7f0c5fe00700 1 -- 172.21.15.27:0/182823865 learned_addr learned my addr 172.21.15.27:0/182823865 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.545+0000 7f0c65c00700 1 --2- 172.21.15.27:0/182823865 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c6010fa50 0x7f0c601a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.545+0000 7f0c5fe00700 1 -- 172.21.15.27:0/182823865 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c6010fa50 msgr2=0x7f0c601a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.545+0000 7f0c5fe00700 1 --2- 172.21.15.27:0/182823865 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c6010fa50 0x7f0c601a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.545+0000 7f0c5fe00700 1 -- 172.21.15.27:0/182823865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c60109a50 msgr2=0x7f0c601a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.545+0000 7f0c5fe00700 1 --2- 172.21.15.27:0/182823865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c60109a50 0x7f0c601a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.545+0000 7f0c5fe00700 1 -- 172.21.15.27:0/182823865 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c601a50b0 con 0x7f0c6010a410 2024-04-26T22:57:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.545+0000 7f0c65200700 1 --2- 172.21.15.27:0/182823865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c60109a50 0x7f0c601a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:03.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.545+0000 7f0c5fe00700 1 --2- 172.21.15.27:0/182823865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6010a410 0x7f0c601a05e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00e9a0 tx=0x7f0c4c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:03.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.546+0000 7f0c5d600700 1 -- 172.21.15.27:0/182823865 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c00c9c0 con 0x7f0c6010a410 2024-04-26T22:57:03.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.546+0000 7f0c67c00700 1 -- 172.21.15.27:0/182823865 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c601a53a0 con 0x7f0c6010a410 2024-04-26T22:57:03.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.546+0000 7f0c5d600700 1 -- 172.21.15.27:0/182823865 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c4c00cb60 con 0x7f0c6010a410 2024-04-26T22:57:03.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.546+0000 7f0c5d600700 1 -- 172.21.15.27:0/182823865 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c018680 con 0x7f0c6010a410 2024-04-26T22:57:03.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.546+0000 7f0c67c00700 1 -- 172.21.15.27:0/182823865 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c601acc20 con 0x7f0c6010a410 2024-04-26T22:57:03.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.547+0000 7f0c67c00700 1 -- 172.21.15.27:0/182823865 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c6006a110 con 0x7f0c6010a410 2024-04-26T22:57:03.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.549+0000 7f0c5d600700 1 -- 172.21.15.27:0/182823865 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0c4c010430 con 0x7f0c6010a410 2024-04-26T22:57:03.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.550+0000 7f0c5d600700 1 --2- 172.21.15.27:0/182823865 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0c480a3cd0 0x7f0c480a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:03.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.551+0000 7f0c5d600700 1 -- 172.21.15.27:0/182823865 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f0c4c0d0be0 con 0x7f0c6010a410 2024-04-26T22:57:03.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.551+0000 7f0c65200700 1 --2- 172.21.15.27:0/182823865 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0c480a3cd0 0x7f0c480a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:03.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.552+0000 7f0c65200700 1 --2- 172.21.15.27:0/182823865 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0c480a3cd0 0x7f0c480a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0c54005fd0 tx=0x7f0c54009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:03.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.555+0000 7f0c5d600700 1 -- 172.21.15.27:0/182823865 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0c4c09d0c0 con 0x7f0c6010a410 2024-04-26T22:57:03.510 INFO:teuthology.orchestra.run.smithi027.stdout:253403070516 2024-04-26T22:57:03.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.794+0000 7f0c67c00700 1 -- 172.21.15.27:0/182823865 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f0c6002e490 con 0x7f0c6010a410 2024-04-26T22:57:03.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.794+0000 7f0c5d600700 1 -- 172.21.15.27:0/182823865 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0c4c09c9c0 con 0x7f0c6010a410 2024-04-26T22:57:03.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.797+0000 7f0c67c00700 1 -- 172.21.15.27:0/182823865 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0c480a3cd0 msgr2=0x7f0c480a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:03.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.797+0000 7f0c67c00700 1 --2- 172.21.15.27:0/182823865 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0c480a3cd0 0x7f0c480a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0c54005fd0 tx=0x7f0c54009310 comp rx=0 tx=0).stop 2024-04-26T22:57:03.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.797+0000 7f0c67c00700 1 -- 172.21.15.27:0/182823865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6010a410 msgr2=0x7f0c601a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:03.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.797+0000 7f0c67c00700 1 --2- 172.21.15.27:0/182823865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6010a410 0x7f0c601a05e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00e9a0 tx=0x7f0c4c00ecb0 comp rx=0 tx=0).stop 2024-04-26T22:57:03.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.798+0000 7f0c67c00700 1 -- 172.21.15.27:0/182823865 shutdown_connections 2024-04-26T22:57:03.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.798+0000 7f0c67c00700 1 --2- 172.21.15.27:0/182823865 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0c480a3cd0 0x7f0c480a6190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:03.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.798+0000 7f0c67c00700 1 --2- 172.21.15.27:0/182823865 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c6010fa50 0x7f0c601a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:03.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.798+0000 7f0c67c00700 1 --2- 172.21.15.27:0/182823865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c60109a50 0x7f0c601a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:03.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.798+0000 7f0c67c00700 1 --2- 172.21.15.27:0/182823865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6010a410 0x7f0c601a05e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:03.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.798+0000 7f0c67c00700 1 -- 172.21.15.27:0/182823865 >> 172.21.15.27:0/182823865 conn(0x7f0c60076bf0 msgr2=0x7f0c60077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:03.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.798+0000 7f0c67c00700 1 -- 172.21.15.27:0/182823865 shutdown_connections 2024-04-26T22:57:03.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:02.798+0000 7f0c67c00700 1 -- 172.21.15.27:0/182823865 wait complete. 2024-04-26T22:57:03.711 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:57:03.713 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:57:03.714 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:57:04.027 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:03 smithi027 ceph-mon[28034]: pgmap v381: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T22:57:04.027 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:03 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2644273691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T22:57:04.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:03 smithi080 ceph-mon[32324]: pgmap v381: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T22:57:04.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:03 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2644273691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T22:57:04.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:03 smithi094 ceph-mon[32889]: pgmap v381: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T22:57:04.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:03 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2644273691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T22:57:04.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.689+0000 7fddbc000700 1 -- 172.21.15.27:0/1590117647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb4109a50 msgr2=0x7fddb4109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:04.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.689+0000 7fddbc000700 1 --2- 172.21.15.27:0/1590117647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb4109a50 0x7fddb4109ed0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdda4009990 tx=0x7fdda4009ca0 comp rx=0 tx=0).stop 2024-04-26T22:57:04.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.689+0000 7fddbc000700 1 -- 172.21.15.27:0/1590117647 shutdown_connections 2024-04-26T22:57:04.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.689+0000 7fddbc000700 1 --2- 172.21.15.27:0/1590117647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddb410a410 0x7fddb4116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:04.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.689+0000 7fddbc000700 1 --2- 172.21.15.27:0/1590117647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fddb410fa50 0x7fddb410fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:04.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.689+0000 7fddbc000700 1 --2- 172.21.15.27:0/1590117647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb4109a50 0x7fddb4109ed0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:04.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.689+0000 7fddbc000700 1 -- 172.21.15.27:0/1590117647 >> 172.21.15.27:0/1590117647 conn(0x7fddb4076bf0 msgr2=0x7fddb4108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:04.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.689+0000 7fddbc000700 1 -- 172.21.15.27:0/1590117647 shutdown_connections 2024-04-26T22:57:04.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.689+0000 7fddbc000700 1 -- 172.21.15.27:0/1590117647 wait complete. 2024-04-26T22:57:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.691+0000 7fddbc000700 1 Processor -- start 2024-04-26T22:57:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.691+0000 7fddbc000700 1 -- start start 2024-04-26T22:57:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.691+0000 7fddbc000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddb4109a50 0x7fddb41a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.692+0000 7fddbc000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fddb410a410 0x7fddb41a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.692+0000 7fddbc000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb410fa50 0x7fddb41a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.692+0000 7fddbc000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fddb4119ce0 con 0x7fddb410fa50 2024-04-26T22:57:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.692+0000 7fddbc000700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fddb4119b60 con 0x7fddb410a410 2024-04-26T22:57:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.692+0000 7fddbc000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fddb4119e60 con 0x7fddb4109a50 2024-04-26T22:57:04.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.692+0000 7fddb8c00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fddb410a410 0x7fddb41a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:04.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.692+0000 7fddb8c00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fddb410a410 0x7fddb41a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:53092/0 (socket says 172.21.15.27:53092) 2024-04-26T22:57:04.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.692+0000 7fddb8c00700 1 -- 172.21.15.27:0/3126330704 learned_addr learned my addr 172.21.15.27:0/3126330704 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:04.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.692+0000 7fddb9600700 1 --2- 172.21.15.27:0/3126330704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddb4109a50 0x7fddb41a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:04.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.692+0000 7fddba000700 1 --2- 172.21.15.27:0/3126330704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb410fa50 0x7fddb41a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:04.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.693+0000 7fddb8c00700 1 -- 172.21.15.27:0/3126330704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddb4109a50 msgr2=0x7fddb41a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:04.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.693+0000 7fddb8c00700 1 --2- 172.21.15.27:0/3126330704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddb4109a50 0x7fddb41a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:04.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.693+0000 7fddb8c00700 1 -- 172.21.15.27:0/3126330704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb410fa50 msgr2=0x7fddb41a4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:04.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.693+0000 7fddb8c00700 1 --2- 172.21.15.27:0/3126330704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb410fa50 0x7fddb41a4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:04.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.693+0000 7fddb8c00700 1 -- 172.21.15.27:0/3126330704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb41a5140 con 0x7fddb410a410 2024-04-26T22:57:04.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.693+0000 7fddb8c00700 1 --2- 172.21.15.27:0/3126330704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fddb410a410 0x7fddb41a0670 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdda4000c00 tx=0x7fdda40057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:04.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.693+0000 7fddb2000700 1 -- 172.21.15.27:0/3126330704 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4027070 con 0x7fddb410a410 2024-04-26T22:57:04.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.693+0000 7fddbc000700 1 -- 172.21.15.27:0/3126330704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb41a53d0 con 0x7fddb410a410 2024-04-26T22:57:04.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.693+0000 7fddb2000700 1 -- 172.21.15.27:0/3126330704 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdda4022470 con 0x7fddb410a410 2024-04-26T22:57:04.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.693+0000 7fddb2000700 1 -- 172.21.15.27:0/3126330704 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4017650 con 0x7fddb410a410 2024-04-26T22:57:04.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.694+0000 7fddbc000700 1 -- 172.21.15.27:0/3126330704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb41accb0 con 0x7fddb410a410 2024-04-26T22:57:04.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.695+0000 7fddbc000700 1 -- 172.21.15.27:0/3126330704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb410b190 con 0x7fddb410a410 2024-04-26T22:57:04.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.697+0000 7fddb2000700 1 -- 172.21.15.27:0/3126330704 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdda402a070 con 0x7fddb410a410 2024-04-26T22:57:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.698+0000 7fddb2000700 1 --2- 172.21.15.27:0/3126330704 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdd9c0a3cd0 0x7fdd9c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.699+0000 7fddb2000700 1 -- 172.21.15.27:0/3126330704 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fdda40d1600 con 0x7fddb410a410 2024-04-26T22:57:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.699+0000 7fddb9600700 1 --2- 172.21.15.27:0/3126330704 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdd9c0a3cd0 0x7fdd9c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.699+0000 7fddb9600700 1 --2- 172.21.15.27:0/3126330704 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdd9c0a3cd0 0x7fdd9c0a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009680 tx=0x7fdda8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:04.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.700+0000 7fddb2000700 1 -- 172.21.15.27:0/3126330704 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdda4013070 con 0x7fddb410a410 2024-04-26T22:57:04.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.950+0000 7fddbc000700 1 -- 172.21.15.27:0/3126330704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fddb402e490 con 0x7fddb410a410 2024-04-26T22:57:04.951 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-04-26T22:57:04.951 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [1, 8]}]: dispatch 2024-04-26T22:57:04.951 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]: dispatch 2024-04-26T22:57:04.951 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-26T22:57:04.951 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6, 5, 9]}]: dispatch 2024-04-26T22:57:04.951 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-04-26T22:57:04.951 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-26T22:57:04.951 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:04 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-26T22:57:04.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.950+0000 7fddb2000700 1 -- 172.21.15.27:0/3126330704 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdda409db70 con 0x7fddb410a410 2024-04-26T22:57:04.952 INFO:teuthology.orchestra.run.smithi027.stdout:287762808878 2024-04-26T22:57:04.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.954+0000 7fddbc000700 1 -- 172.21.15.27:0/3126330704 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdd9c0a3cd0 msgr2=0x7fdd9c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:04.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.954+0000 7fddbc000700 1 --2- 172.21.15.27:0/3126330704 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdd9c0a3cd0 0x7fdd9c0a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009680 tx=0x7fdda8009260 comp rx=0 tx=0).stop 2024-04-26T22:57:04.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.954+0000 7fddbc000700 1 -- 172.21.15.27:0/3126330704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fddb410a410 msgr2=0x7fddb41a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:04.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.954+0000 7fddbc000700 1 --2- 172.21.15.27:0/3126330704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fddb410a410 0x7fddb41a0670 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdda4000c00 tx=0x7fdda40057c0 comp rx=0 tx=0).stop 2024-04-26T22:57:04.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.954+0000 7fddbc000700 1 -- 172.21.15.27:0/3126330704 shutdown_connections 2024-04-26T22:57:04.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.954+0000 7fddbc000700 1 --2- 172.21.15.27:0/3126330704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb410fa50 0x7fddb41a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:04.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.954+0000 7fddbc000700 1 --2- 172.21.15.27:0/3126330704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddb4109a50 0x7fddb41a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:04.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.954+0000 7fddbc000700 1 --2- 172.21.15.27:0/3126330704 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdd9c0a3cd0 0x7fdd9c0a6190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:04.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.954+0000 7fddbc000700 1 --2- 172.21.15.27:0/3126330704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fddb410a410 0x7fddb41a0670 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:04.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.954+0000 7fddbc000700 1 -- 172.21.15.27:0/3126330704 >> 172.21.15.27:0/3126330704 conn(0x7fddb4076bf0 msgr2=0x7fddb4077210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:04.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.954+0000 7fddbc000700 1 -- 172.21.15.27:0/3126330704 shutdown_connections 2024-04-26T22:57:04.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:04.954+0000 7fddbc000700 1 -- 172.21.15.27:0/3126330704 wait complete. 2024-04-26T22:57:05.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-04-26T22:57:05.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [1, 8]}]: dispatch 2024-04-26T22:57:05.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]: dispatch 2024-04-26T22:57:05.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-26T22:57:05.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6, 5, 9]}]: dispatch 2024-04-26T22:57:05.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-04-26T22:57:05.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-26T22:57:05.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:04 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-26T22:57:05.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-04-26T22:57:05.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [1, 8]}]: dispatch 2024-04-26T22:57:05.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]: dispatch 2024-04-26T22:57:05.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-26T22:57:05.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6, 5, 9]}]: dispatch 2024-04-26T22:57:05.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-04-26T22:57:05.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-26T22:57:05.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:04 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-26T22:57:05.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.467+0000 7f25b6000700 1 -- 172.21.15.27:0/530986991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b01094c0 msgr2=0x7f25b01098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:05.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.467+0000 7f25b6000700 1 --2- 172.21.15.27:0/530986991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b01094c0 0x7f25b01098a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f25a00098f0 tx=0x7f25a0009c00 comp rx=0 tx=0).stop 2024-04-26T22:57:05.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.467+0000 7f25b6000700 1 -- 172.21.15.27:0/530986991 shutdown_connections 2024-04-26T22:57:05.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.467+0000 7f25b6000700 1 --2- 172.21.15.27:0/530986991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25b0103e30 0x7f25b0110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:05.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.467+0000 7f25b6000700 1 --2- 172.21.15.27:0/530986991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b01094c0 0x7f25b01098a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:05.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.467+0000 7f25b6000700 1 --2- 172.21.15.27:0/530986991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25b0103470 0x7f25b01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:05.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.467+0000 7f25b6000700 1 -- 172.21.15.27:0/530986991 >> 172.21.15.27:0/530986991 conn(0x7f25b007a580 msgr2=0x7f25b007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:05.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.467+0000 7f25b6000700 1 -- 172.21.15.27:0/530986991 shutdown_connections 2024-04-26T22:57:05.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.467+0000 7f25b6000700 1 -- 172.21.15.27:0/530986991 wait complete. 2024-04-26T22:57:05.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.469+0000 7f25b6000700 1 Processor -- start 2024-04-26T22:57:05.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.469+0000 7f25b6000700 1 -- start start 2024-04-26T22:57:05.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.469+0000 7f25b6000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25b0103470 0x7f25b010f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:05.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.469+0000 7f25b6000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b0103e30 0x7f25b010fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:05.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.469+0000 7f25b6000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25b01094c0 0x7f25b01a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:05.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.469+0000 7f25b6000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f25b0113610 con 0x7f25b01094c0 2024-04-26T22:57:05.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.469+0000 7f25b6000700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f25b0113490 con 0x7f25b0103470 2024-04-26T22:57:05.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.469+0000 7f25b6000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f25b0113790 con 0x7f25b0103e30 2024-04-26T22:57:05.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.470+0000 7f25afe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25b01094c0 0x7f25b01a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:05.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.470+0000 7f25aea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b0103e30 0x7f25b010fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:05.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.470+0000 7f25af400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25b0103470 0x7f25b010f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:05.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.470+0000 7f25aea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b0103e30 0x7f25b010fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:49378/0 (socket says 172.21.15.27:49378) 2024-04-26T22:57:05.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.470+0000 7f25afe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25b01094c0 0x7f25b01a4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35748/0 (socket says 172.21.15.27:35748) 2024-04-26T22:57:05.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.470+0000 7f25aea00700 1 -- 172.21.15.27:0/2644273691 learned_addr learned my addr 172.21.15.27:0/2644273691 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:05.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.470+0000 7f25af400700 1 -- 172.21.15.27:0/2644273691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b0103e30 msgr2=0x7f25b010fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:05.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.470+0000 7f25af400700 1 --2- 172.21.15.27:0/2644273691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b0103e30 0x7f25b010fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:05.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.470+0000 7f25af400700 1 -- 172.21.15.27:0/2644273691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25b01094c0 msgr2=0x7f25b01a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:05.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.470+0000 7f25af400700 1 --2- 172.21.15.27:0/2644273691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25b01094c0 0x7f25b01a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:05.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.470+0000 7f25af400700 1 -- 172.21.15.27:0/2644273691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b01a51f0 con 0x7f25b0103470 2024-04-26T22:57:05.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.470+0000 7f25afe00700 1 --2- 172.21.15.27:0/2644273691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25b01094c0 0x7f25b01a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:05.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.471+0000 7f25af400700 1 --2- 172.21.15.27:0/2644273691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25b0103470 0x7f25b010f6e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f25a000c010 tx=0x7f25a000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:05.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.471+0000 7f25a7e00700 1 -- 172.21.15.27:0/2644273691 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a00051c0 con 0x7f25b0103470 2024-04-26T22:57:05.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.471+0000 7f25b6000700 1 -- 172.21.15.27:0/2644273691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b01a5480 con 0x7f25b0103470 2024-04-26T22:57:05.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.471+0000 7f25a7e00700 1 -- 172.21.15.27:0/2644273691 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25a000b560 con 0x7f25b0103470 2024-04-26T22:57:05.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.471+0000 7f25a7e00700 1 -- 172.21.15.27:0/2644273691 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0003970 con 0x7f25b0103470 2024-04-26T22:57:05.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.471+0000 7f25b6000700 1 -- 172.21.15.27:0/2644273691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b01acd60 con 0x7f25b0103470 2024-04-26T22:57:05.372 INFO:teuthology.orchestra.run.smithi027.stdout:219043332153 2024-04-26T22:57:05.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.472+0000 7f25b6000700 1 -- 172.21.15.27:0/2644273691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b006a110 con 0x7f25b0103470 2024-04-26T22:57:05.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.474+0000 7f25a7e00700 1 -- 172.21.15.27:0/2644273691 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f25a000b930 con 0x7f25b0103470 2024-04-26T22:57:05.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.476+0000 7f25a7e00700 1 --2- 172.21.15.27:0/2644273691 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f25940a3cd0 0x7f25940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:05.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.476+0000 7f25a7e00700 1 -- 172.21.15.27:0/2644273691 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f25a00d18e0 con 0x7f25b0103470 2024-04-26T22:57:05.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.476+0000 7f25aea00700 1 --2- 172.21.15.27:0/2644273691 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f25940a3cd0 0x7f25940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:05.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.477+0000 7f25aea00700 1 --2- 172.21.15.27:0/2644273691 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f25940a3cd0 0x7f25940a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f25b0077740 tx=0x7f259c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:05.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.478+0000 7f25a7e00700 1 -- 172.21.15.27:0/2644273691 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f25a005f060 con 0x7f25b0103470 2024-04-26T22:57:05.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.724+0000 7f25b6000700 1 -- 172.21.15.27:0/2644273691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f25b002e490 con 0x7f25b0103470 2024-04-26T22:57:05.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.725+0000 7f25a7e00700 1 -- 172.21.15.27:0/2644273691 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f25a0013030 con 0x7f25b0103470 2024-04-26T22:57:05.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.728+0000 7f25b6000700 1 -- 172.21.15.27:0/2644273691 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f25940a3cd0 msgr2=0x7f25940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:05.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.728+0000 7f25b6000700 1 --2- 172.21.15.27:0/2644273691 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f25940a3cd0 0x7f25940a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f25b0077740 tx=0x7f259c009310 comp rx=0 tx=0).stop 2024-04-26T22:57:05.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.728+0000 7f25b6000700 1 -- 172.21.15.27:0/2644273691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25b0103470 msgr2=0x7f25b010f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:05.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.728+0000 7f25b6000700 1 --2- 172.21.15.27:0/2644273691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25b0103470 0x7f25b010f6e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f25a000c010 tx=0x7f25a000fa00 comp rx=0 tx=0).stop 2024-04-26T22:57:05.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.728+0000 7f25b6000700 1 -- 172.21.15.27:0/2644273691 shutdown_connections 2024-04-26T22:57:05.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.728+0000 7f25b6000700 1 --2- 172.21.15.27:0/2644273691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25b01094c0 0x7f25b01a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:05.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.728+0000 7f25b6000700 1 --2- 172.21.15.27:0/2644273691 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f25940a3cd0 0x7f25940a6190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:05.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.728+0000 7f25b6000700 1 --2- 172.21.15.27:0/2644273691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25b0103470 0x7f25b010f6e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:05.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.728+0000 7f25b6000700 1 --2- 172.21.15.27:0/2644273691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b0103e30 0x7f25b010fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:05.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.728+0000 7f25b6000700 1 -- 172.21.15.27:0/2644273691 >> 172.21.15.27:0/2644273691 conn(0x7f25b007a580 msgr2=0x7f25b0102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:05.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.728+0000 7f25b6000700 1 -- 172.21.15.27:0/2644273691 shutdown_connections 2024-04-26T22:57:05.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:03.728+0000 7f25b6000700 1 -- 172.21.15.27:0/2644273691 wait complete. 2024-04-26T22:57:06.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:05 smithi027 ceph-mon[28034]: pgmap v382: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:06.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-04-26T22:57:06.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [1, 8]}]': finished 2024-04-26T22:57:06.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]': finished 2024-04-26T22:57:06.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-26T22:57:06.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6, 5, 9]}]': finished 2024-04-26T22:57:06.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-04-26T22:57:06.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-26T22:57:06.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:05 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-26T22:57:06.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:05 smithi027 ceph-mon[28034]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T22:57:06.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:05 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3126330704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T22:57:06.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:05 smithi080 ceph-mon[32324]: pgmap v382: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:06.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-04-26T22:57:06.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [1, 8]}]': finished 2024-04-26T22:57:06.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]': finished 2024-04-26T22:57:06.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-26T22:57:06.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6, 5, 9]}]': finished 2024-04-26T22:57:06.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-04-26T22:57:06.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-26T22:57:06.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:05 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-26T22:57:06.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:05 smithi080 ceph-mon[32324]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T22:57:06.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:05 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3126330704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T22:57:06.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:05 smithi094 ceph-mon[32889]: pgmap v382: 129 pgs: 129 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:06.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-04-26T22:57:06.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [1, 8]}]': finished 2024-04-26T22:57:06.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]': finished 2024-04-26T22:57:06.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-26T22:57:06.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6, 5, 9]}]': finished 2024-04-26T22:57:06.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-04-26T22:57:06.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-26T22:57:06.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:05 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-26T22:57:06.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:05 smithi094 ceph-mon[32889]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T22:57:06.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:05 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3126330704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T22:57:06.306 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757310 got 163208757317 for osd.5 2024-04-26T22:57:06.306 DEBUG:teuthology.parallel:result is None 2024-04-26T22:57:07.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:06 smithi080 ceph-mon[32324]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T22:57:07.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:06 smithi094 ceph-mon[32889]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T22:57:07.337 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:06 smithi027 ceph-mon[28034]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T22:57:07.999 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:07 smithi027 ceph-mon[28034]: pgmap v385: 129 pgs: 4 peering, 125 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:57:07.999 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:07 smithi027 ceph-mon[28034]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-04-26T22:57:08.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.167+0000 7fcb06800700 1 -- 172.21.15.27:0/341403635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb00075f70 msgr2=0x7fcb0010f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:08.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.167+0000 7fcb06800700 1 --2- 172.21.15.27:0/341403635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb00075f70 0x7fcb0010f8e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009990 tx=0x7fcaec009ca0 comp rx=0 tx=0).stop 2024-04-26T22:57:08.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.167+0000 7fcb06800700 1 -- 172.21.15.27:0/341403635 shutdown_connections 2024-04-26T22:57:08.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.167+0000 7fcb06800700 1 --2- 172.21.15.27:0/341403635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb0010fe20 0x7fcb00112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:08.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.167+0000 7fcb06800700 1 --2- 172.21.15.27:0/341403635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb00075650 0x7fcb00075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:08.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.167+0000 7fcb06800700 1 --2- 172.21.15.27:0/341403635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb00075f70 0x7fcb0010f8e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:08.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.167+0000 7fcb06800700 1 -- 172.21.15.27:0/341403635 >> 172.21.15.27:0/341403635 conn(0x7fcb0007a580 msgr2=0x7fcb0007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:08.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.167+0000 7fcb06800700 1 -- 172.21.15.27:0/341403635 shutdown_connections 2024-04-26T22:57:08.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.167+0000 7fcb06800700 1 -- 172.21.15.27:0/341403635 wait complete. 2024-04-26T22:57:08.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.169+0000 7fcb06800700 1 Processor -- start 2024-04-26T22:57:08.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.169+0000 7fcb06800700 1 -- start start 2024-04-26T22:57:08.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.169+0000 7fcb06800700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb00075650 0x7fcb0019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:08.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.169+0000 7fcb06800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb00075f70 0x7fcb0019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:08.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.170+0000 7fcb06800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb0010fe20 0x7fcb001a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:08.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.170+0000 7fcb06800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcb00115850 con 0x7fcb0010fe20 2024-04-26T22:57:08.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.170+0000 7fcb06800700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcb001156d0 con 0x7fcb00075650 2024-04-26T22:57:08.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.170+0000 7fcb06800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcb001159d0 con 0x7fcb00075f70 2024-04-26T22:57:08.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.170+0000 7fcafea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb00075f70 0x7fcb0019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:08.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.170+0000 7fcaff400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb00075650 0x7fcb0019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:08.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.170+0000 7fcaffe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb0010fe20 0x7fcb001a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:08.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.170+0000 7fcafea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb00075f70 0x7fcb0019c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:40008/0 (socket says 172.21.15.27:40008) 2024-04-26T22:57:08.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.170+0000 7fcafea00700 1 -- 172.21.15.27:0/2242522711 learned_addr learned my addr 172.21.15.27:0/2242522711 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:08.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.170+0000 7fcaff400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb00075650 0x7fcb0019bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:43706/0 (socket says 172.21.15.27:43706) 2024-04-26T22:57:08.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.170+0000 7fcafea00700 1 -- 172.21.15.27:0/2242522711 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb00075650 msgr2=0x7fcb0019bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:08.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.170+0000 7fcafea00700 1 --2- 172.21.15.27:0/2242522711 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb00075650 0x7fcb0019bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:08.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.170+0000 7fcafea00700 1 -- 172.21.15.27:0/2242522711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb0010fe20 msgr2=0x7fcb001a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:08.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.170+0000 7fcafea00700 1 --2- 172.21.15.27:0/2242522711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb0010fe20 0x7fcb001a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:08.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.170+0000 7fcafea00700 1 -- 172.21.15.27:0/2242522711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb001a0d70 con 0x7fcb00075f70 2024-04-26T22:57:08.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.171+0000 7fcafea00700 1 --2- 172.21.15.27:0/2242522711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb00075f70 0x7fcb0019c2a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcaec005910 tx=0x7fcaec00b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:08.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.171+0000 7fcaf7e00700 1 -- 172.21.15.27:0/2242522711 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec027070 con 0x7fcb00075f70 2024-04-26T22:57:08.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.171+0000 7fcaf7e00700 1 -- 172.21.15.27:0/2242522711 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaec022470 con 0x7fcb00075f70 2024-04-26T22:57:08.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.171+0000 7fcb06800700 1 -- 172.21.15.27:0/2242522711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb001a1000 con 0x7fcb00075f70 2024-04-26T22:57:08.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.171+0000 7fcaf7e00700 1 -- 172.21.15.27:0/2242522711 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec017630 con 0x7fcb00075f70 2024-04-26T22:57:08.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.171+0000 7fcb06800700 1 -- 172.21.15.27:0/2242522711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb001a88e0 con 0x7fcb00075f70 2024-04-26T22:57:08.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.172+0000 7fcb06800700 1 -- 172.21.15.27:0/2242522711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb0006a110 con 0x7fcb00075f70 2024-04-26T22:57:08.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.174+0000 7fcaf7e00700 1 -- 172.21.15.27:0/2242522711 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcaec02a070 con 0x7fcb00075f70 2024-04-26T22:57:08.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.176+0000 7fcaf7e00700 1 --2- 172.21.15.27:0/2242522711 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcae40a3d50 0x7fcae40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:08.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.176+0000 7fcaf7e00700 1 -- 172.21.15.27:0/2242522711 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fcaec0d1610 con 0x7fcb00075f70 2024-04-26T22:57:08.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.176+0000 7fcaff400700 1 --2- 172.21.15.27:0/2242522711 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcae40a3d50 0x7fcae40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:08.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.177+0000 7fcaff400700 1 --2- 172.21.15.27:0/2242522711 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcae40a3d50 0x7fcae40a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009680 tx=0x7fcaf0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:08.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.180+0000 7fcaf7e00700 1 -- 172.21.15.27:0/2242522711 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcaec013070 con 0x7fcb00075f70 2024-04-26T22:57:08.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:07 smithi080 ceph-mon[32324]: pgmap v385: 129 pgs: 4 peering, 125 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:57:08.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:07 smithi080 ceph-mon[32324]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-04-26T22:57:08.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:07 smithi094 ceph-mon[32889]: pgmap v385: 129 pgs: 4 peering, 125 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:57:08.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:07 smithi094 ceph-mon[32889]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-04-26T22:57:08.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.418+0000 7fcb06800700 1 -- 172.21.15.27:0/2242522711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fcb0002e490 con 0x7fcb00075f70 2024-04-26T22:57:08.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.419+0000 7fcaf7e00700 1 -- 172.21.15.27:0/2242522711 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fcaec09da90 con 0x7fcb00075f70 2024-04-26T22:57:08.420 INFO:teuthology.orchestra.run.smithi027.stdout:343597383715 2024-04-26T22:57:08.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.422+0000 7fcb06800700 1 -- 172.21.15.27:0/2242522711 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcae40a3d50 msgr2=0x7fcae40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.422+0000 7fcb06800700 1 --2- 172.21.15.27:0/2242522711 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcae40a3d50 0x7fcae40a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009680 tx=0x7fcaf0009260 comp rx=0 tx=0).stop 2024-04-26T22:57:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.422+0000 7fcb06800700 1 -- 172.21.15.27:0/2242522711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb00075f70 msgr2=0x7fcb0019c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.422+0000 7fcb06800700 1 --2- 172.21.15.27:0/2242522711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb00075f70 0x7fcb0019c2a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcaec005910 tx=0x7fcaec00b670 comp rx=0 tx=0).stop 2024-04-26T22:57:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.422+0000 7fcb06800700 1 -- 172.21.15.27:0/2242522711 shutdown_connections 2024-04-26T22:57:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.422+0000 7fcb06800700 1 --2- 172.21.15.27:0/2242522711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb0010fe20 0x7fcb001a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.422+0000 7fcb06800700 1 --2- 172.21.15.27:0/2242522711 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcae40a3d50 0x7fcae40a6210 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.422+0000 7fcb06800700 1 --2- 172.21.15.27:0/2242522711 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb00075650 0x7fcb0019bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.422+0000 7fcb06800700 1 --2- 172.21.15.27:0/2242522711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb00075f70 0x7fcb0019c2a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.422+0000 7fcb06800700 1 -- 172.21.15.27:0/2242522711 >> 172.21.15.27:0/2242522711 conn(0x7fcb0007a580 msgr2=0x7fcb001006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.422+0000 7fcb06800700 1 -- 172.21.15.27:0/2242522711 shutdown_connections 2024-04-26T22:57:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:08.422+0000 7fcb06800700 1 -- 172.21.15.27:0/2242522711 wait complete. 2024-04-26T22:57:08.613 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808873 got 287762808878 for osd.9 2024-04-26T22:57:08.613 DEBUG:teuthology.parallel:result is None 2024-04-26T22:57:09.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:08 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2242522711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T22:57:09.283 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:08 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2242522711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T22:57:09.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:08 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2242522711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T22:57:09.836 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332149 got 219043332153 for osd.7 2024-04-26T22:57:09.836 DEBUG:teuthology.parallel:result is None 2024-04-26T22:57:10.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:09 smithi027 ceph-mon[28034]: pgmap v386: 129 pgs: 4 peering, 125 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T22:57:10.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:09 smithi080 ceph-mon[32324]: pgmap v386: 129 pgs: 4 peering, 125 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T22:57:10.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:09 smithi094 ceph-mon[32889]: pgmap v386: 129 pgs: 4 peering, 125 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T22:57:10.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.532+0000 7fd8aee00700 1 -- 172.21.15.27:0/345645946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8a8103e30 msgr2=0x7fd8a8110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:10.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.532+0000 7fd8aee00700 1 --2- 172.21.15.27:0/345645946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8a8103e30 0x7fd8a8110290 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000b0a0 tx=0x7fd8a000b3b0 comp rx=0 tx=0).stop 2024-04-26T22:57:10.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.532+0000 7fd8aee00700 1 -- 172.21.15.27:0/345645946 shutdown_connections 2024-04-26T22:57:10.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.532+0000 7fd8aee00700 1 --2- 172.21.15.27:0/345645946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8a8103e30 0x7fd8a8110290 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:10.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.532+0000 7fd8aee00700 1 --2- 172.21.15.27:0/345645946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8a81094c0 0x7fd8a81098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:10.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.532+0000 7fd8aee00700 1 --2- 172.21.15.27:0/345645946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a8103470 0x7fd8a81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:10.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.532+0000 7fd8aee00700 1 -- 172.21.15.27:0/345645946 >> 172.21.15.27:0/345645946 conn(0x7fd8a807a580 msgr2=0x7fd8a807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:10.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.532+0000 7fd8aee00700 1 -- 172.21.15.27:0/345645946 shutdown_connections 2024-04-26T22:57:10.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.532+0000 7fd8aee00700 1 -- 172.21.15.27:0/345645946 wait complete. 2024-04-26T22:57:10.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.533+0000 7fd8aee00700 1 Processor -- start 2024-04-26T22:57:10.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.534+0000 7fd8aee00700 1 -- start start 2024-04-26T22:57:10.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.534+0000 7fd8aee00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a8103470 0x7fd8a81a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:10.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.534+0000 7fd8aee00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8a8103e30 0x7fd8a81a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:10.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.534+0000 7fd8aee00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8a81094c0 0x7fd8a819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:10.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.534+0000 7fd8aee00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a8113630 con 0x7fd8a8103470 2024-04-26T22:57:10.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.534+0000 7fd8aee00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81134b0 con 0x7fd8a81094c0 2024-04-26T22:57:10.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.534+0000 7fd8aee00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81137b0 con 0x7fd8a8103e30 2024-04-26T22:57:10.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.534+0000 7fd8a7e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a8103470 0x7fd8a81a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:10.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.535+0000 7fd8a7e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a8103470 0x7fd8a81a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35780/0 (socket says 172.21.15.27:35780) 2024-04-26T22:57:10.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.535+0000 7fd8a7e00700 1 -- 172.21.15.27:0/1796764854 learned_addr learned my addr 172.21.15.27:0/1796764854 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:10.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.535+0000 7fd8a7400700 1 --2- 172.21.15.27:0/1796764854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8a8103e30 0x7fd8a81a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:10.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.535+0000 7fd8ace00700 1 --2- 172.21.15.27:0/1796764854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8a81094c0 0x7fd8a819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:10.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.535+0000 7fd8a7e00700 1 -- 172.21.15.27:0/1796764854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8a8103e30 msgr2=0x7fd8a81a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:10.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.535+0000 7fd8a7e00700 1 --2- 172.21.15.27:0/1796764854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8a8103e30 0x7fd8a81a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:10.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.535+0000 7fd8a7e00700 1 -- 172.21.15.27:0/1796764854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8a81094c0 msgr2=0x7fd8a819bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:10.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.535+0000 7fd8a7e00700 1 --2- 172.21.15.27:0/1796764854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8a81094c0 0x7fd8a819bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:10.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.535+0000 7fd8a7e00700 1 -- 172.21.15.27:0/1796764854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a819c400 con 0x7fd8a8103470 2024-04-26T22:57:10.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.535+0000 7fd8a7400700 1 --2- 172.21.15.27:0/1796764854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8a8103e30 0x7fd8a81a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:10.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.535+0000 7fd8ace00700 1 --2- 172.21.15.27:0/1796764854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8a81094c0 0x7fd8a819bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:10.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.536+0000 7fd8a7e00700 1 --2- 172.21.15.27:0/1796764854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a8103470 0x7fd8a81a1a50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd89800e9c0 tx=0x7fd89800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:10.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.536+0000 7fd8a4c00700 1 -- 172.21.15.27:0/1796764854 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89800cbd0 con 0x7fd8a8103470 2024-04-26T22:57:10.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.536+0000 7fd8a4c00700 1 -- 172.21.15.27:0/1796764854 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd89800ee60 con 0x7fd8a8103470 2024-04-26T22:57:10.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.536+0000 7fd8aee00700 1 -- 172.21.15.27:0/1796764854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a819c6f0 con 0x7fd8a8103470 2024-04-26T22:57:10.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.536+0000 7fd8a4c00700 1 -- 172.21.15.27:0/1796764854 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8980055c0 con 0x7fd8a8103470 2024-04-26T22:57:10.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.536+0000 7fd8aee00700 1 -- 172.21.15.27:0/1796764854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a81a89d0 con 0x7fd8a8103470 2024-04-26T22:57:10.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.537+0000 7fd8aee00700 1 -- 172.21.15.27:0/1796764854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a8104b60 con 0x7fd8a8103470 2024-04-26T22:57:10.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.539+0000 7fd8a4c00700 1 -- 172.21.15.27:0/1796764854 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd898004750 con 0x7fd8a8103470 2024-04-26T22:57:10.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.541+0000 7fd8a4c00700 1 --2- 172.21.15.27:0/1796764854 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fd88c0a3d50 0x7fd88c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:10.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.541+0000 7fd8a4c00700 1 -- 172.21.15.27:0/1796764854 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fd8980d0dd0 con 0x7fd8a8103470 2024-04-26T22:57:10.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.541+0000 7fd8a7400700 1 --2- 172.21.15.27:0/1796764854 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fd88c0a3d50 0x7fd88c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:10.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.542+0000 7fd8a7400700 1 --2- 172.21.15.27:0/1796764854 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fd88c0a3d50 0x7fd88c0a6210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd8a819d270 tx=0x7fd894009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:10.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.544+0000 7fd8a4c00700 1 -- 172.21.15.27:0/1796764854 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd898054060 con 0x7fd8a8103470 2024-04-26T22:57:10.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.777+0000 7fd8aee00700 1 -- 172.21.15.27:0/1796764854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fd8a81a8ba0 con 0x7fd8a8103470 2024-04-26T22:57:10.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.778+0000 7fd8a4c00700 1 -- 172.21.15.27:0/1796764854 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd89809d1c0 con 0x7fd8a8103470 2024-04-26T22:57:10.778 INFO:teuthology.orchestra.run.smithi027.stdout:309237645353 2024-04-26T22:57:10.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.780+0000 7fd8aee00700 1 -- 172.21.15.27:0/1796764854 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fd88c0a3d50 msgr2=0x7fd88c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:10.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.780+0000 7fd8aee00700 1 --2- 172.21.15.27:0/1796764854 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fd88c0a3d50 0x7fd88c0a6210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd8a819d270 tx=0x7fd894009260 comp rx=0 tx=0).stop 2024-04-26T22:57:10.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.780+0000 7fd8aee00700 1 -- 172.21.15.27:0/1796764854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a8103470 msgr2=0x7fd8a81a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:10.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.780+0000 7fd8aee00700 1 --2- 172.21.15.27:0/1796764854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a8103470 0x7fd8a81a1a50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd89800e9c0 tx=0x7fd89800ecd0 comp rx=0 tx=0).stop 2024-04-26T22:57:10.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.781+0000 7fd8aee00700 1 -- 172.21.15.27:0/1796764854 shutdown_connections 2024-04-26T22:57:10.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.781+0000 7fd8aee00700 1 --2- 172.21.15.27:0/1796764854 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fd88c0a3d50 0x7fd88c0a6210 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:10.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.781+0000 7fd8aee00700 1 --2- 172.21.15.27:0/1796764854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8a81094c0 0x7fd8a819bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:10.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.781+0000 7fd8aee00700 1 --2- 172.21.15.27:0/1796764854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a8103470 0x7fd8a81a1a50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:10.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.781+0000 7fd8aee00700 1 --2- 172.21.15.27:0/1796764854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8a8103e30 0x7fd8a81a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:10.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.781+0000 7fd8aee00700 1 -- 172.21.15.27:0/1796764854 >> 172.21.15.27:0/1796764854 conn(0x7fd8a807a580 msgr2=0x7fd8a8101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:10.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.781+0000 7fd8aee00700 1 -- 172.21.15.27:0/1796764854 shutdown_connections 2024-04-26T22:57:10.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:10.781+0000 7fd8aee00700 1 -- 172.21.15.27:0/1796764854 wait complete. 2024-04-26T22:57:11.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:10 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1796764854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T22:57:11.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:10 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1796764854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T22:57:11.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:10 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1796764854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T22:57:11.878 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383710 got 343597383715 for osd.11 2024-04-26T22:57:11.878 DEBUG:teuthology.parallel:result is None 2024-04-26T22:57:12.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:11 smithi080 ceph-mon[32324]: pgmap v387: 129 pgs: 3 peering, 126 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-26T22:57:12.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:11 smithi094 ceph-mon[32889]: pgmap v387: 129 pgs: 3 peering, 126 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-26T22:57:12.308 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:11 smithi027 ceph-mon[28034]: pgmap v387: 129 pgs: 3 peering, 126 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-26T22:57:12.938 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070510 got 253403070516 for osd.8 2024-04-26T22:57:12.939 DEBUG:teuthology.parallel:result is None 2024-04-26T22:57:13.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:13 smithi080 ceph-mon[32324]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-26T22:57:13.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:13 smithi080 ceph-mon[32324]: Cluster is now healthy 2024-04-26T22:57:13.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:13 smithi094 ceph-mon[32889]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-26T22:57:13.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:13 smithi094 ceph-mon[32889]: Cluster is now healthy 2024-04-26T22:57:13.335 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:13 smithi027 ceph-mon[28034]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-26T22:57:13.335 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:13 smithi027 ceph-mon[28034]: Cluster is now healthy 2024-04-26T22:57:13.900 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645345 got 309237645353 for osd.10 2024-04-26T22:57:13.900 DEBUG:teuthology.parallel:result is None 2024-04-26T22:57:14.118 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:14 smithi027 ceph-mon[28034]: pgmap v388: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-26T22:57:14.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:14 smithi080 ceph-mon[32324]: pgmap v388: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-26T22:57:14.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:14 smithi094 ceph-mon[32889]: pgmap v388: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-26T22:57:14.364 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:57:14.368 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:57:14.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.164+0000 7fe8abe00700 1 -- 172.21.15.27:0/3218774983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8a4105640 msgr2=0x7fe8a4105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.164+0000 7fe8abe00700 1 --2- 172.21.15.27:0/3218774983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8a4105640 0x7fe8a4105ac0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe890009990 tx=0x7fe890009ca0 comp rx=0 tx=0).stop 2024-04-26T22:57:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.164+0000 7fe8abe00700 1 -- 172.21.15.27:0/3218774983 shutdown_connections 2024-04-26T22:57:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.164+0000 7fe8abe00700 1 --2- 172.21.15.27:0/3218774983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8a4106000 0x7fe8a4112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.164+0000 7fe8abe00700 1 --2- 172.21.15.27:0/3218774983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8a410b640 0x7fe8a410ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.164+0000 7fe8abe00700 1 --2- 172.21.15.27:0/3218774983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8a4105640 0x7fe8a4105ac0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.164+0000 7fe8abe00700 1 -- 172.21.15.27:0/3218774983 >> 172.21.15.27:0/3218774983 conn(0x7fe8a4101130 msgr2=0x7fe8a4103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.165+0000 7fe8abe00700 1 -- 172.21.15.27:0/3218774983 shutdown_connections 2024-04-26T22:57:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.165+0000 7fe8abe00700 1 -- 172.21.15.27:0/3218774983 wait complete. 2024-04-26T22:57:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.166+0000 7fe8abe00700 1 Processor -- start 2024-04-26T22:57:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.166+0000 7fe8abe00700 1 -- start start 2024-04-26T22:57:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.166+0000 7fe8abe00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8a4105640 0x7fe8a41a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.166+0000 7fe8abe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8a4106000 0x7fe8a41a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.166+0000 7fe8abe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8a410b640 0x7fe8a419bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.166+0000 7fe8abe00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a4115890 con 0x7fe8a410b640 2024-04-26T22:57:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.166+0000 7fe8abe00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a4115710 con 0x7fe8a4105640 2024-04-26T22:57:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.166+0000 7fe8abe00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a4115a10 con 0x7fe8a4106000 2024-04-26T22:57:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.167+0000 7fe8a9400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8a4105640 0x7fe8a41a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.167+0000 7fe8a8a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8a4106000 0x7fe8a41a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.167+0000 7fe8a9400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8a4105640 0x7fe8a41a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:43744/0 (socket says 172.21.15.27:43744) 2024-04-26T22:57:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.167+0000 7fe8a9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8a410b640 0x7fe8a419bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.167+0000 7fe8a8a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8a4106000 0x7fe8a41a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:40024/0 (socket says 172.21.15.27:40024) 2024-04-26T22:57:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.167+0000 7fe8a9400700 1 -- 172.21.15.27:0/28652684 learned_addr learned my addr 172.21.15.27:0/28652684 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.167+0000 7fe8a8a00700 1 -- 172.21.15.27:0/28652684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8a4105640 msgr2=0x7fe8a41a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.167+0000 7fe8a8a00700 1 --2- 172.21.15.27:0/28652684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8a4105640 0x7fe8a41a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.167+0000 7fe8a8a00700 1 -- 172.21.15.27:0/28652684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8a410b640 msgr2=0x7fe8a419bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.167+0000 7fe8a8a00700 1 --2- 172.21.15.27:0/28652684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8a410b640 0x7fe8a419bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.167+0000 7fe8a8a00700 1 -- 172.21.15.27:0/28652684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a419c410 con 0x7fe8a4106000 2024-04-26T22:57:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.167+0000 7fe8a9400700 1 --2- 172.21.15.27:0/28652684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8a4105640 0x7fe8a41a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.167+0000 7fe8a8a00700 1 --2- 172.21.15.27:0/28652684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8a4106000 0x7fe8a41a1fa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe890000c00 tx=0x7fe8900047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.168+0000 7fe8a2000700 1 -- 172.21.15.27:0/28652684 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890026070 con 0x7fe8a4106000 2024-04-26T22:57:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.168+0000 7fe8abe00700 1 -- 172.21.15.27:0/28652684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a419c6a0 con 0x7fe8a4106000 2024-04-26T22:57:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.168+0000 7fe8a2000700 1 -- 172.21.15.27:0/28652684 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe890021410 con 0x7fe8a4106000 2024-04-26T22:57:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.168+0000 7fe8a2000700 1 -- 172.21.15.27:0/28652684 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890017650 con 0x7fe8a4106000 2024-04-26T22:57:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.168+0000 7fe8abe00700 1 -- 172.21.15.27:0/28652684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a41a89e0 con 0x7fe8a4106000 2024-04-26T22:57:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.169+0000 7fe8abe00700 1 -- 172.21.15.27:0/28652684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a419ca00 con 0x7fe8a4106000 2024-04-26T22:57:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.171+0000 7fe8a2000700 1 -- 172.21.15.27:0/28652684 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe890029070 con 0x7fe8a4106000 2024-04-26T22:57:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.172+0000 7fe8a2000700 1 --2- 172.21.15.27:0/28652684 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe88c0a3d50 0x7fe88c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.173+0000 7fe8a2000700 1 -- 172.21.15.27:0/28652684 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fe8900d16e0 con 0x7fe8a4106000 2024-04-26T22:57:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.173+0000 7fe8a9400700 1 --2- 172.21.15.27:0/28652684 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe88c0a3d50 0x7fe88c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:14.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.173+0000 7fe8a9400700 1 --2- 172.21.15.27:0/28652684 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe88c0a3d50 0x7fe88c0a6210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe898011550 tx=0x7fe898009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:14.732 INFO:teuthology.orchestra.run.smithi027.stdout:34359738467 2024-04-26T22:57:14.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.177+0000 7fe8a2000700 1 -- 172.21.15.27:0/28652684 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe89005e060 con 0x7fe8a4106000 2024-04-26T22:57:14.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.422+0000 7fe8abe00700 1 -- 172.21.15.27:0/28652684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe8a419cff0 con 0x7fe8a4106000 2024-04-26T22:57:14.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.423+0000 7fe8a2000700 1 -- 172.21.15.27:0/28652684 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe890013070 con 0x7fe8a4106000 2024-04-26T22:57:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.427+0000 7fe8abe00700 1 -- 172.21.15.27:0/28652684 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe88c0a3d50 msgr2=0x7fe88c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.427+0000 7fe8abe00700 1 --2- 172.21.15.27:0/28652684 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe88c0a3d50 0x7fe88c0a6210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe898011550 tx=0x7fe898009260 comp rx=0 tx=0).stop 2024-04-26T22:57:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.427+0000 7fe8abe00700 1 -- 172.21.15.27:0/28652684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8a4106000 msgr2=0x7fe8a41a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.427+0000 7fe8abe00700 1 --2- 172.21.15.27:0/28652684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8a4106000 0x7fe8a41a1fa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe890000c00 tx=0x7fe8900047b0 comp rx=0 tx=0).stop 2024-04-26T22:57:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.427+0000 7fe8abe00700 1 -- 172.21.15.27:0/28652684 shutdown_connections 2024-04-26T22:57:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.427+0000 7fe8abe00700 1 --2- 172.21.15.27:0/28652684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8a410b640 0x7fe8a419bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.427+0000 7fe8abe00700 1 --2- 172.21.15.27:0/28652684 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe88c0a3d50 0x7fe88c0a6210 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.427+0000 7fe8abe00700 1 --2- 172.21.15.27:0/28652684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8a4105640 0x7fe8a41a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.427+0000 7fe8abe00700 1 --2- 172.21.15.27:0/28652684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8a4106000 0x7fe8a41a1fa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.427+0000 7fe8abe00700 1 -- 172.21.15.27:0/28652684 >> 172.21.15.27:0/28652684 conn(0x7fe8a4101130 msgr2=0x7fe8a4101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.427+0000 7fe8abe00700 1 -- 172.21.15.27:0/28652684 shutdown_connections 2024-04-26T22:57:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:14.427+0000 7fe8abe00700 1 -- 172.21.15.27:0/28652684 wait complete. 2024-04-26T22:57:15.011 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:57:15.034 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:15 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/28652684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T22:57:15.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:15 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/28652684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T22:57:15.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:15 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/28652684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T22:57:15.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.655+0000 7fe383200700 1 -- 172.21.15.27:0/94732282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe37c108670 msgr2=0x7fe37c110240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:15.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.655+0000 7fe383200700 1 --2- 172.21.15.27:0/94732282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe37c108670 0x7fe37c110240 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe37800b0a0 tx=0x7fe37800b3b0 comp rx=0 tx=0).stop 2024-04-26T22:57:15.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.655+0000 7fe383200700 1 -- 172.21.15.27:0/94732282 shutdown_connections 2024-04-26T22:57:15.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.655+0000 7fe383200700 1 --2- 172.21.15.27:0/94732282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe37c108670 0x7fe37c110240 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:15.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.655+0000 7fe383200700 1 --2- 172.21.15.27:0/94732282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe37c103590 0x7fe37c103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:15.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.655+0000 7fe383200700 1 --2- 172.21.15.27:0/94732282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe37c103f40 0x7fe37c107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:15.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.655+0000 7fe383200700 1 -- 172.21.15.27:0/94732282 >> 172.21.15.27:0/94732282 conn(0x7fe37c0fee40 msgr2=0x7fe37c101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:15.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.656+0000 7fe383200700 1 -- 172.21.15.27:0/94732282 shutdown_connections 2024-04-26T22:57:15.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.656+0000 7fe383200700 1 -- 172.21.15.27:0/94732282 wait complete. 2024-04-26T22:57:15.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.657+0000 7fe383200700 1 Processor -- start 2024-04-26T22:57:15.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.657+0000 7fe383200700 1 -- start start 2024-04-26T22:57:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.657+0000 7fe383200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe37c103590 0x7fe37c075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.657+0000 7fe383200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe37c103f40 0x7fe37c075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.657+0000 7fe383200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe37c108670 0x7fe37c0761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.657+0000 7fe383200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c113580 con 0x7fe37c103590 2024-04-26T22:57:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.657+0000 7fe383200700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c113400 con 0x7fe37c108670 2024-04-26T22:57:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.657+0000 7fe383200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c113700 con 0x7fe37c103f40 2024-04-26T22:57:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.658+0000 7fe381400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe37c103f40 0x7fe37c075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.658+0000 7fe381400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe37c103f40 0x7fe37c075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:40054/0 (socket says 172.21.15.27:40054) 2024-04-26T22:57:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.658+0000 7fe381400700 1 -- 172.21.15.27:0/2382324076 learned_addr learned my addr 172.21.15.27:0/2382324076 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.658+0000 7fe381e00700 1 --2- 172.21.15.27:0/2382324076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe37c103590 0x7fe37c075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.658+0000 7fe382800700 1 --2- 172.21.15.27:0/2382324076 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe37c108670 0x7fe37c0761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.658+0000 7fe381400700 1 -- 172.21.15.27:0/2382324076 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe37c108670 msgr2=0x7fe37c0761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.658+0000 7fe381400700 1 --2- 172.21.15.27:0/2382324076 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe37c108670 0x7fe37c0761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.658+0000 7fe381400700 1 -- 172.21.15.27:0/2382324076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe37c103590 msgr2=0x7fe37c075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.658+0000 7fe381400700 1 --2- 172.21.15.27:0/2382324076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe37c103590 0x7fe37c075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.658+0000 7fe381400700 1 -- 172.21.15.27:0/2382324076 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c1a7f50 con 0x7fe37c103f40 2024-04-26T22:57:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.658+0000 7fe382800700 1 --2- 172.21.15.27:0/2382324076 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe37c108670 0x7fe37c0761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:57:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.658+0000 7fe381e00700 1 --2- 172.21.15.27:0/2382324076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe37c103590 0x7fe37c075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.659+0000 7fe381400700 1 --2- 172.21.15.27:0/2382324076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe37c103f40 0x7fe37c075c60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00e9f0 tx=0x7fe36c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.659+0000 7fe376a00700 1 -- 172.21.15.27:0/2382324076 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c00cbd0 con 0x7fe37c103f40 2024-04-26T22:57:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.659+0000 7fe383200700 1 -- 172.21.15.27:0/2382324076 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c1a8240 con 0x7fe37c103f40 2024-04-26T22:57:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.659+0000 7fe376a00700 1 -- 172.21.15.27:0/2382324076 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe36c00cd70 con 0x7fe37c103f40 2024-04-26T22:57:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.659+0000 7fe376a00700 1 -- 172.21.15.27:0/2382324076 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c005650 con 0x7fe37c103f40 2024-04-26T22:57:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.659+0000 7fe383200700 1 -- 172.21.15.27:0/2382324076 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c1a8730 con 0x7fe37c103f40 2024-04-26T22:57:15.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.660+0000 7fe383200700 1 -- 172.21.15.27:0/2382324076 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe37c06a110 con 0x7fe37c103f40 2024-04-26T22:57:15.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.663+0000 7fe376a00700 1 -- 172.21.15.27:0/2382324076 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe36c004750 con 0x7fe37c103f40 2024-04-26T22:57:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.666+0000 7fe376a00700 1 --2- 172.21.15.27:0/2382324076 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe3640a3cd0 0x7fe3640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.666+0000 7fe376a00700 1 -- 172.21.15.27:0/2382324076 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fe36c0d0ca0 con 0x7fe37c103f40 2024-04-26T22:57:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.666+0000 7fe381e00700 1 --2- 172.21.15.27:0/2382324076 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe3640a3cd0 0x7fe3640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:15.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.667+0000 7fe381e00700 1 --2- 172.21.15.27:0/2382324076 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe3640a3cd0 0x7fe3640a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe370011490 tx=0x7fe370009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:15.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.669+0000 7fe376a00700 1 -- 172.21.15.27:0/2382324076 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe36c09d090 con 0x7fe37c103f40 2024-04-26T22:57:15.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.963+0000 7fe383200700 1 -- 172.21.15.27:0/2382324076 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe37c076c00 con 0x7fe37c103f40 2024-04-26T22:57:15.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.964+0000 7fe376a00700 1 -- 172.21.15.27:0/2382324076 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe36c09c990 con 0x7fe37c103f40 2024-04-26T22:57:15.964 INFO:teuthology.orchestra.run.smithi027.stdout:107374182483 2024-04-26T22:57:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.966+0000 7fe383200700 1 -- 172.21.15.27:0/2382324076 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe3640a3cd0 msgr2=0x7fe3640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.966+0000 7fe383200700 1 --2- 172.21.15.27:0/2382324076 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe3640a3cd0 0x7fe3640a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe370011490 tx=0x7fe370009260 comp rx=0 tx=0).stop 2024-04-26T22:57:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.967+0000 7fe383200700 1 -- 172.21.15.27:0/2382324076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe37c103f40 msgr2=0x7fe37c075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.967+0000 7fe383200700 1 --2- 172.21.15.27:0/2382324076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe37c103f40 0x7fe37c075c60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00e9f0 tx=0x7fe36c00ed00 comp rx=0 tx=0).stop 2024-04-26T22:57:15.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.967+0000 7fe383200700 1 -- 172.21.15.27:0/2382324076 shutdown_connections 2024-04-26T22:57:15.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.967+0000 7fe383200700 1 --2- 172.21.15.27:0/2382324076 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe3640a3cd0 0x7fe3640a6190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:15.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.967+0000 7fe383200700 1 --2- 172.21.15.27:0/2382324076 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe37c108670 0x7fe37c0761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:15.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.967+0000 7fe383200700 1 --2- 172.21.15.27:0/2382324076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe37c103590 0x7fe37c075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:15.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.967+0000 7fe383200700 1 --2- 172.21.15.27:0/2382324076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe37c103f40 0x7fe37c075c60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:15.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.967+0000 7fe383200700 1 -- 172.21.15.27:0/2382324076 >> 172.21.15.27:0/2382324076 conn(0x7fe37c0fee40 msgr2=0x7fe37c101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:15.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.967+0000 7fe383200700 1 -- 172.21.15.27:0/2382324076 shutdown_connections 2024-04-26T22:57:15.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:15.967+0000 7fe383200700 1 -- 172.21.15.27:0/2382324076 wait complete. 2024-04-26T22:57:16.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:16 smithi027 ceph-mon[28034]: pgmap v389: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 106 B/s wr, 6 op/s 2024-04-26T22:57:16.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:16 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2382324076' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T22:57:16.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:16 smithi094 ceph-mon[32889]: pgmap v389: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 106 B/s wr, 6 op/s 2024-04-26T22:57:16.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:16 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2382324076' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T22:57:16.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:16 smithi080 ceph-mon[32324]: pgmap v389: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 106 B/s wr, 6 op/s 2024-04-26T22:57:16.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:16 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2382324076' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T22:57:17.677 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738460 got 34359738467 for osd.0 2024-04-26T22:57:17.678 DEBUG:teuthology.parallel:result is None 2024-04-26T22:57:18.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:18 smithi027 ceph-mon[28034]: pgmap v390: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 96 B/s wr, 6 op/s 2024-04-26T22:57:18.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:18 smithi080 ceph-mon[32324]: pgmap v390: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 96 B/s wr, 6 op/s 2024-04-26T22:57:18.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:18 smithi094 ceph-mon[32889]: pgmap v390: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 96 B/s wr, 6 op/s 2024-04-26T22:57:18.897 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182476 got 107374182483 for osd.3 2024-04-26T22:57:18.897 DEBUG:teuthology.parallel:result is None 2024-04-26T22:57:20.294 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:20 smithi027 ceph-mon[28034]: pgmap v391: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T22:57:20.294 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:20 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:57:20.294 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:20 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:57:20.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:20 smithi080 ceph-mon[32324]: pgmap v391: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T22:57:20.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:20 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:57:20.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:20 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:57:20.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:20 smithi094 ceph-mon[32889]: pgmap v391: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T22:57:20.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:20 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:57:20.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:20 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:57:21.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:57:21.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:57:21.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:57:21.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:21 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/4249633253' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T22:57:21.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:57:21.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:57:21.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:57:21.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:21 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/4249633253' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T22:57:21.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:57:21.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:57:21.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:57:21.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:21 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/4249633253' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T22:57:22.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:22 smithi027 ceph-mon[28034]: pgmap v392: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T22:57:22.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.736+0000 7f62b5400700 1 -- 172.21.15.27:0/500642851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b0103e30 msgr2=0x7f62b0110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:22.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.736+0000 7f62b5400700 1 --2- 172.21.15.27:0/500642851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b0103e30 0x7f62b0110290 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f62a800b0a0 tx=0x7f62a800b3b0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.736+0000 7f62b5400700 1 -- 172.21.15.27:0/500642851 shutdown_connections 2024-04-26T22:57:22.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.736+0000 7f62b5400700 1 --2- 172.21.15.27:0/500642851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b0103e30 0x7f62b0110290 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.736+0000 7f62b5400700 1 --2- 172.21.15.27:0/500642851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62b01094c0 0x7f62b01098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.736+0000 7f62b5400700 1 --2- 172.21.15.27:0/500642851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62b0103470 0x7f62b01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.736+0000 7f62b5400700 1 -- 172.21.15.27:0/500642851 >> 172.21.15.27:0/500642851 conn(0x7f62b007a580 msgr2=0x7f62b007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:22.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.736+0000 7f62b5400700 1 -- 172.21.15.27:0/500642851 shutdown_connections 2024-04-26T22:57:22.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.736+0000 7f62b5400700 1 -- 172.21.15.27:0/500642851 wait complete. 2024-04-26T22:57:22.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.737+0000 7f62b5400700 1 Processor -- start 2024-04-26T22:57:22.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.738+0000 7f62b5400700 1 -- start start 2024-04-26T22:57:22.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.738+0000 7f62b5400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b0103470 0x7f62b01a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:22.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.738+0000 7f62b5400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62b0103e30 0x7f62b01a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:22.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.738+0000 7f62b5400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62b01094c0 0x7f62b019bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:22.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.738+0000 7f62b5400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0113610 con 0x7f62b0103e30 2024-04-26T22:57:22.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.738+0000 7f62b5400700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0113490 con 0x7f62b0103470 2024-04-26T22:57:22.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.738+0000 7f62b5400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0113790 con 0x7f62b01094c0 2024-04-26T22:57:22.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.738+0000 7f62ae800700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b0103470 0x7f62b01a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:22.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.738+0000 7f62ade00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62b0103e30 0x7f62b01a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:22.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.738+0000 7f62ae800700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b0103470 0x7f62b01a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:49426/0 (socket says 172.21.15.27:49426) 2024-04-26T22:57:22.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.738+0000 7f62ae800700 1 -- 172.21.15.27:0/4249633253 learned_addr learned my addr 172.21.15.27:0/4249633253 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:22.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.739+0000 7f62af200700 1 --2- 172.21.15.27:0/4249633253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62b01094c0 0x7f62b019bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:22.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.739+0000 7f62ae800700 1 -- 172.21.15.27:0/4249633253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62b01094c0 msgr2=0x7f62b019bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:22.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.739+0000 7f62ae800700 1 --2- 172.21.15.27:0/4249633253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62b01094c0 0x7f62b019bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.739+0000 7f62ae800700 1 -- 172.21.15.27:0/4249633253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62b0103e30 msgr2=0x7f62b01a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:22.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.739+0000 7f62ae800700 1 --2- 172.21.15.27:0/4249633253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62b0103e30 0x7f62b01a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.739+0000 7f62ae800700 1 -- 172.21.15.27:0/4249633253 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b019c410 con 0x7f62b0103470 2024-04-26T22:57:22.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.739+0000 7f62ade00700 1 --2- 172.21.15.27:0/4249633253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62b0103e30 0x7f62b01a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:22.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.739+0000 7f62ae800700 1 --2- 172.21.15.27:0/4249633253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b0103470 0x7f62b01a1a60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f62a000eb30 tx=0x7f62a000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:22.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.740+0000 7f62a7400700 1 -- 172.21.15.27:0/4249633253 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a000cbd0 con 0x7f62b0103470 2024-04-26T22:57:22.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.740+0000 7f62a7400700 1 -- 172.21.15.27:0/4249633253 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62a00044d0 con 0x7f62b0103470 2024-04-26T22:57:22.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.740+0000 7f62a7400700 1 -- 172.21.15.27:0/4249633253 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a0003e30 con 0x7f62b0103470 2024-04-26T22:57:22.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.740+0000 7f62b5400700 1 -- 172.21.15.27:0/4249633253 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b019c700 con 0x7f62b0103470 2024-04-26T22:57:22.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.740+0000 7f62b5400700 1 -- 172.21.15.27:0/4249633253 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b01a89e0 con 0x7f62b0103470 2024-04-26T22:57:22.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.741+0000 7f62b5400700 1 -- 172.21.15.27:0/4249633253 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b0104b60 con 0x7f62b0103470 2024-04-26T22:57:22.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.743+0000 7f62a7400700 1 -- 172.21.15.27:0/4249633253 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f62a0004000 con 0x7f62b0103470 2024-04-26T22:57:22.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.745+0000 7f62a7400700 1 --2- 172.21.15.27:0/4249633253 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f62940a3d50 0x7f62940a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:22.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.745+0000 7f62a7400700 1 -- 172.21.15.27:0/4249633253 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f62a00d0c70 con 0x7f62b0103470 2024-04-26T22:57:22.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.745+0000 7f62ade00700 1 --2- 172.21.15.27:0/4249633253 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f62940a3d50 0x7f62940a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:22.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.746+0000 7f62ade00700 1 --2- 172.21.15.27:0/4249633253 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f62940a3d50 0x7f62940a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f62b019d280 tx=0x7f629c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:22.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.749+0000 7f62a7400700 1 -- 172.21.15.27:0/4249633253 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f62a00042f0 con 0x7f62b0103470 2024-04-26T22:57:22.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.996+0000 7f62b5400700 1 -- 172.21.15.27:0/4249633253 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f62b01a8bb0 con 0x7f62b0103470 2024-04-26T22:57:22.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:20.997+0000 7f62a7400700 1 -- 172.21.15.27:0/4249633253 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f62a009d060 con 0x7f62b0103470 2024-04-26T22:57:22.505 INFO:teuthology.orchestra.run.smithi027.stdout:141733920846 2024-04-26T22:57:22.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:21.000+0000 7f62b5400700 1 -- 172.21.15.27:0/4249633253 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f62940a3d50 msgr2=0x7f62940a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:22.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:21.000+0000 7f62b5400700 1 --2- 172.21.15.27:0/4249633253 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f62940a3d50 0x7f62940a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f62b019d280 tx=0x7f629c009260 comp rx=0 tx=0).stop 2024-04-26T22:57:22.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:21.000+0000 7f62b5400700 1 -- 172.21.15.27:0/4249633253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b0103470 msgr2=0x7f62b01a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:22.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:21.000+0000 7f62b5400700 1 --2- 172.21.15.27:0/4249633253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b0103470 0x7f62b01a1a60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f62a000eb30 tx=0x7f62a000ee40 comp rx=0 tx=0).stop 2024-04-26T22:57:22.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:21.000+0000 7f62b5400700 1 -- 172.21.15.27:0/4249633253 shutdown_connections 2024-04-26T22:57:22.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:21.000+0000 7f62b5400700 1 --2- 172.21.15.27:0/4249633253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62b01094c0 0x7f62b019bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:21.000+0000 7f62b5400700 1 --2- 172.21.15.27:0/4249633253 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f62940a3d50 0x7f62940a6210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:21.000+0000 7f62b5400700 1 --2- 172.21.15.27:0/4249633253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b0103470 0x7f62b01a1a60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:21.000+0000 7f62b5400700 1 --2- 172.21.15.27:0/4249633253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62b0103e30 0x7f62b01a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:21.000+0000 7f62b5400700 1 -- 172.21.15.27:0/4249633253 >> 172.21.15.27:0/4249633253 conn(0x7f62b007a580 msgr2=0x7f62b0101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:22.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:21.000+0000 7f62b5400700 1 -- 172.21.15.27:0/4249633253 shutdown_connections 2024-04-26T22:57:22.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:21.001+0000 7f62b5400700 1 -- 172.21.15.27:0/4249633253 wait complete. 2024-04-26T22:57:22.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:22 smithi080 ceph-mon[32324]: pgmap v392: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T22:57:22.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:22 smithi094 ceph-mon[32889]: pgmap v392: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T22:57:22.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.566+0000 7f96e4400700 1 -- 172.21.15.27:0/1791023719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96dc10a410 msgr2=0x7f96dc116920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:22.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.566+0000 7f96e4400700 1 --2- 172.21.15.27:0/1791023719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96dc10a410 0x7f96dc116920 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009f90 tx=0x7f96d400b330 comp rx=0 tx=0).stop 2024-04-26T22:57:22.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.566+0000 7f96e4400700 1 -- 172.21.15.27:0/1791023719 shutdown_connections 2024-04-26T22:57:22.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.566+0000 7f96e4400700 1 --2- 172.21.15.27:0/1791023719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96dc10a410 0x7f96dc116920 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.566+0000 7f96e4400700 1 --2- 172.21.15.27:0/1791023719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96dc10fa50 0x7f96dc10fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.566+0000 7f96e4400700 1 --2- 172.21.15.27:0/1791023719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc109a50 0x7f96dc109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.566+0000 7f96e4400700 1 -- 172.21.15.27:0/1791023719 >> 172.21.15.27:0/1791023719 conn(0x7f96dc076bf0 msgr2=0x7f96dc108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:22.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.566+0000 7f96e4400700 1 -- 172.21.15.27:0/1791023719 shutdown_connections 2024-04-26T22:57:22.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.566+0000 7f96e4400700 1 -- 172.21.15.27:0/1791023719 wait complete. 2024-04-26T22:57:22.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.567+0000 7f96e4400700 1 Processor -- start 2024-04-26T22:57:22.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.567+0000 7f96e4400700 1 -- start start 2024-04-26T22:57:22.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.568+0000 7f96e4400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96dc109a50 0x7f96dc1a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:22.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.568+0000 7f96e4400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96dc10a410 0x7f96dc1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:22.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.568+0000 7f96e4400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc10fa50 0x7f96dc1a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:22.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.568+0000 7f96e4400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc119c90 con 0x7f96dc109a50 2024-04-26T22:57:22.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.568+0000 7f96e4400700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc119b10 con 0x7f96dc10a410 2024-04-26T22:57:22.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.568+0000 7f96e4400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc119e10 con 0x7f96dc10fa50 2024-04-26T22:57:22.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.568+0000 7f96e1a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96dc109a50 0x7f96dc1a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:22.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.568+0000 7f96e1a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96dc109a50 0x7f96dc1a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52400/0 (socket says 172.21.15.27:52400) 2024-04-26T22:57:22.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.568+0000 7f96e1a00700 1 -- 172.21.15.27:0/1816266046 learned_addr learned my addr 172.21.15.27:0/1816266046 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:22.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.568+0000 7f96e1000700 1 --2- 172.21.15.27:0/1816266046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96dc10a410 0x7f96dc1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:22.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.568+0000 7f96e2400700 1 --2- 172.21.15.27:0/1816266046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc10fa50 0x7f96dc1a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:22.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.569+0000 7f96e1a00700 1 -- 172.21.15.27:0/1816266046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc10fa50 msgr2=0x7f96dc1a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.569+0000 7f96e1a00700 1 --2- 172.21.15.27:0/1816266046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc10fa50 0x7f96dc1a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.569+0000 7f96e1a00700 1 -- 172.21.15.27:0/1816266046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96dc10a410 msgr2=0x7f96dc1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.569+0000 7f96e1a00700 1 --2- 172.21.15.27:0/1816266046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96dc10a410 0x7f96dc1a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.569+0000 7f96e1a00700 1 -- 172.21.15.27:0/1816266046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc1a50b0 con 0x7f96dc109a50 2024-04-26T22:57:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.569+0000 7f96e2400700 1 --2- 172.21.15.27:0/1816266046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc10fa50 0x7f96dc1a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.569+0000 7f96e1a00700 1 --2- 172.21.15.27:0/1816266046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96dc109a50 0x7f96dc1a00a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f96d000ec10 tx=0x7f96d000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.569+0000 7f96da000700 1 -- 172.21.15.27:0/1816266046 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d000cc90 con 0x7f96dc109a50 2024-04-26T22:57:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.569+0000 7f96da000700 1 -- 172.21.15.27:0/1816266046 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96d000ce30 con 0x7f96dc109a50 2024-04-26T22:57:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.569+0000 7f96da000700 1 -- 172.21.15.27:0/1816266046 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0010430 con 0x7f96dc109a50 2024-04-26T22:57:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.570+0000 7f96e4400700 1 -- 172.21.15.27:0/1816266046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96dc1a53a0 con 0x7f96dc109a50 2024-04-26T22:57:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.570+0000 7f96e4400700 1 -- 172.21.15.27:0/1816266046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96dc1acc20 con 0x7f96dc109a50 2024-04-26T22:57:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.571+0000 7f96e4400700 1 -- 172.21.15.27:0/1816266046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96dc06a110 con 0x7f96dc109a50 2024-04-26T22:57:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.573+0000 7f96da000700 1 -- 172.21.15.27:0/1816266046 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f96d0004000 con 0x7f96dc109a50 2024-04-26T22:57:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.575+0000 7f96da000700 1 --2- 172.21.15.27:0/1816266046 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f96c40a3d50 0x7f96c40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.575+0000 7f96da000700 1 -- 172.21.15.27:0/1816266046 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f96d00d0c90 con 0x7f96dc109a50 2024-04-26T22:57:22.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.575+0000 7f96e1000700 1 --2- 172.21.15.27:0/1816266046 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f96c40a3d50 0x7f96c40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:22.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.576+0000 7f96e1000700 1 --2- 172.21.15.27:0/1816266046 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f96c40a3d50 0x7f96c40a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f96dc078a70 tx=0x7f96c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:22.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.577+0000 7f96da000700 1 -- 172.21.15.27:0/1816266046 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f96d009d080 con 0x7f96dc109a50 2024-04-26T22:57:22.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.815+0000 7f96e4400700 1 -- 172.21.15.27:0/1816266046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f96dc02e490 con 0x7f96dc109a50 2024-04-26T22:57:22.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.816+0000 7f96da000700 1 -- 172.21.15.27:0/1816266046 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f96d009c980 con 0x7f96dc109a50 2024-04-26T22:57:22.940 INFO:teuthology.orchestra.run.smithi027.stdout:77309411419 2024-04-26T22:57:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.819+0000 7f96e4400700 1 -- 172.21.15.27:0/1816266046 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f96c40a3d50 msgr2=0x7f96c40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.819+0000 7f96e4400700 1 --2- 172.21.15.27:0/1816266046 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f96c40a3d50 0x7f96c40a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f96dc078a70 tx=0x7f96c8009260 comp rx=0 tx=0).stop 2024-04-26T22:57:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.819+0000 7f96e4400700 1 -- 172.21.15.27:0/1816266046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96dc109a50 msgr2=0x7f96dc1a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.819+0000 7f96e4400700 1 --2- 172.21.15.27:0/1816266046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96dc109a50 0x7f96dc1a00a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f96d000ec10 tx=0x7f96d000ef20 comp rx=0 tx=0).stop 2024-04-26T22:57:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.820+0000 7f96e4400700 1 -- 172.21.15.27:0/1816266046 shutdown_connections 2024-04-26T22:57:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.820+0000 7f96e4400700 1 --2- 172.21.15.27:0/1816266046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc10fa50 0x7f96dc1a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.820+0000 7f96e4400700 1 --2- 172.21.15.27:0/1816266046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96dc109a50 0x7f96dc1a00a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.820+0000 7f96e4400700 1 --2- 172.21.15.27:0/1816266046 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f96c40a3d50 0x7f96c40a6210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.820+0000 7f96e4400700 1 --2- 172.21.15.27:0/1816266046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96dc10a410 0x7f96dc1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.820+0000 7f96e4400700 1 -- 172.21.15.27:0/1816266046 >> 172.21.15.27:0/1816266046 conn(0x7f96dc076bf0 msgr2=0x7f96dc077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.820+0000 7f96e4400700 1 -- 172.21.15.27:0/1816266046 shutdown_connections 2024-04-26T22:57:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:22.822+0000 7f96e4400700 1 -- 172.21.15.27:0/1816266046 wait complete. 2024-04-26T22:57:23.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:23 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1816266046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T22:57:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:23 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1816266046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T22:57:23.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:23 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1816266046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T22:57:23.964 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411413 got 77309411419 for osd.2 2024-04-26T22:57:23.964 DEBUG:teuthology.parallel:result is None 2024-04-26T22:57:24.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:24 smithi027 ceph-mon[28034]: pgmap v393: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:57:24.435 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:24 smithi094 ceph-mon[32889]: pgmap v393: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:57:24.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:24 smithi080 ceph-mon[32324]: pgmap v393: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:57:24.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.945+0000 7f785dc00700 1 -- 172.21.15.27:0/2023904362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78581094a0 msgr2=0x7f7858109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:24.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.945+0000 7f785dc00700 1 --2- 172.21.15.27:0/2023904362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78581094a0 0x7f7858109880 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7848009990 tx=0x7f7848009ca0 comp rx=0 tx=0).stop 2024-04-26T22:57:24.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.946+0000 7f785dc00700 1 -- 172.21.15.27:0/2023904362 shutdown_connections 2024-04-26T22:57:24.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.946+0000 7f785dc00700 1 --2- 172.21.15.27:0/2023904362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f785806bad0 0x7f78581102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:24.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.946+0000 7f785dc00700 1 --2- 172.21.15.27:0/2023904362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78581094a0 0x7f7858109880 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:24.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.946+0000 7f785dc00700 1 --2- 172.21.15.27:0/2023904362 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f785806b130 0x7f785806b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:24.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.946+0000 7f785dc00700 1 -- 172.21.15.27:0/2023904362 >> 172.21.15.27:0/2023904362 conn(0x7f78580792d0 msgr2=0x7f78580796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:24.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.946+0000 7f785dc00700 1 -- 172.21.15.27:0/2023904362 shutdown_connections 2024-04-26T22:57:24.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.946+0000 7f785dc00700 1 -- 172.21.15.27:0/2023904362 wait complete. 2024-04-26T22:57:24.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.947+0000 7f785dc00700 1 Processor -- start 2024-04-26T22:57:24.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.948+0000 7f785dc00700 1 -- start start 2024-04-26T22:57:24.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.948+0000 7f785dc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f785806b130 0x7f785819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:24.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.948+0000 7f785dc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f785806bad0 0x7f785819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:24.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.948+0000 7f785dc00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78581094a0 0x7f78581a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:24.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.948+0000 7f785dc00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7858113600 con 0x7f785806bad0 2024-04-26T22:57:24.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.948+0000 7f785dc00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7858113480 con 0x7f78581094a0 2024-04-26T22:57:24.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.948+0000 7f785dc00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7858113780 con 0x7f785806b130 2024-04-26T22:57:24.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.949+0000 7f7856a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f785806b130 0x7f785819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:24.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.949+0000 7f7857400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78581094a0 0x7f78581a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:24.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.949+0000 7f7856a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f785806b130 0x7f785819bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:45346/0 (socket says 172.21.15.27:45346) 2024-04-26T22:57:24.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.949+0000 7f7856a00700 1 -- 172.21.15.27:0/4262518705 learned_addr learned my addr 172.21.15.27:0/4262518705 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:24.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.949+0000 7f7856000700 1 --2- 172.21.15.27:0/4262518705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f785806bad0 0x7f785819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:24.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.949+0000 7f7856a00700 1 -- 172.21.15.27:0/4262518705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78581094a0 msgr2=0x7f78581a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:24.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.949+0000 7f7856a00700 1 --2- 172.21.15.27:0/4262518705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78581094a0 0x7f78581a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:24.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.949+0000 7f7856a00700 1 -- 172.21.15.27:0/4262518705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f785806bad0 msgr2=0x7f785819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:24.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.949+0000 7f7856a00700 1 --2- 172.21.15.27:0/4262518705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f785806bad0 0x7f785819c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:24.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.949+0000 7f7856a00700 1 -- 172.21.15.27:0/4262518705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78581a0d60 con 0x7f785806b130 2024-04-26T22:57:24.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.950+0000 7f7856a00700 1 --2- 172.21.15.27:0/4262518705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f785806b130 0x7f785819bd50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7848000c00 tx=0x7f78480057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:24.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.950+0000 7f784f400700 1 -- 172.21.15.27:0/4262518705 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848027070 con 0x7f785806b130 2024-04-26T22:57:24.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.950+0000 7f785dc00700 1 -- 172.21.15.27:0/4262518705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78581a0ff0 con 0x7f785806b130 2024-04-26T22:57:24.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.950+0000 7f784f400700 1 -- 172.21.15.27:0/4262518705 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7848022470 con 0x7f785806b130 2024-04-26T22:57:24.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.950+0000 7f784f400700 1 -- 172.21.15.27:0/4262518705 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848017650 con 0x7f785806b130 2024-04-26T22:57:24.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.950+0000 7f785dc00700 1 -- 172.21.15.27:0/4262518705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78581a88d0 con 0x7f785806b130 2024-04-26T22:57:24.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.951+0000 7f785dc00700 1 -- 172.21.15.27:0/4262518705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f785806a110 con 0x7f785806b130 2024-04-26T22:57:24.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.954+0000 7f784f400700 1 -- 172.21.15.27:0/4262518705 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f784802a070 con 0x7f785806b130 2024-04-26T22:57:24.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.956+0000 7f784f400700 1 --2- 172.21.15.27:0/4262518705 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f783c0a3d50 0x7f783c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:24.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.956+0000 7f784f400700 1 -- 172.21.15.27:0/4262518705 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f78480d1860 con 0x7f785806b130 2024-04-26T22:57:24.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.956+0000 7f7856000700 1 --2- 172.21.15.27:0/4262518705 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f783c0a3d50 0x7f783c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:24.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.957+0000 7f784f400700 1 -- 172.21.15.27:0/4262518705 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7848013070 con 0x7f785806b130 2024-04-26T22:57:24.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.957+0000 7f7856000700 1 --2- 172.21.15.27:0/4262518705 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f783c0a3d50 0x7f783c0a6210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f78580799d0 tx=0x7f78440094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:24.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.790+0000 7f3d86e00700 1 -- 172.21.15.27:0/2183847920 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d80108080 msgr2=0x7f3d80108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:24.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.790+0000 7f3d86e00700 1 --2- 172.21.15.27:0/2183847920 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d80108080 0x7f3d80108500 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0098f0 tx=0x7f3d6c009c00 comp rx=0 tx=0).stop 2024-04-26T22:57:24.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.791+0000 7f3d86e00700 1 -- 172.21.15.27:0/2183847920 shutdown_connections 2024-04-26T22:57:24.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.791+0000 7f3d86e00700 1 --2- 172.21.15.27:0/2183847920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d80108a40 0x7f3d80110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:24.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.791+0000 7f3d86e00700 1 --2- 172.21.15.27:0/2183847920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d80103a00 0x7f3d80103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:24.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.791+0000 7f3d86e00700 1 --2- 172.21.15.27:0/2183847920 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d80108080 0x7f3d80108500 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:24.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.791+0000 7f3d86e00700 1 -- 172.21.15.27:0/2183847920 >> 172.21.15.27:0/2183847920 conn(0x7f3d8007b3b0 msgr2=0x7f3d8007b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:24.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.791+0000 7f3d86e00700 1 -- 172.21.15.27:0/2183847920 shutdown_connections 2024-04-26T22:57:24.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.792+0000 7f3d86e00700 1 -- 172.21.15.27:0/2183847920 wait complete. 2024-04-26T22:57:24.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.793+0000 7f3d86e00700 1 Processor -- start 2024-04-26T22:57:24.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.793+0000 7f3d86e00700 1 -- start start 2024-04-26T22:57:24.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.794+0000 7f3d86e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d80103a00 0x7f3d8019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:24.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.794+0000 7f3d86e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d80108080 0x7f3d8019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:24.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.794+0000 7f3d86e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d80108a40 0x7f3d801a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:24.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.794+0000 7f3d86e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80113660 con 0x7f3d80108a40 2024-04-26T22:57:24.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.794+0000 7f3d86e00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3d801134e0 con 0x7f3d80108080 2024-04-26T22:57:24.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.794+0000 7f3d86e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d801137e0 con 0x7f3d80103a00 2024-04-26T22:57:24.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.794+0000 7f3d7f400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d80108080 0x7f3d8019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:24.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.794+0000 7f3d7f400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d80108080 0x7f3d8019c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:49462/0 (socket says 172.21.15.27:49462) 2024-04-26T22:57:24.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.794+0000 7f3d7f400700 1 -- 172.21.15.27:0/3565896438 learned_addr learned my addr 172.21.15.27:0/3565896438 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:24.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.794+0000 7f3d7fe00700 1 --2- 172.21.15.27:0/3565896438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d80103a00 0x7f3d8019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:24.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.794+0000 7f3d84e00700 1 --2- 172.21.15.27:0/3565896438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d80108a40 0x7f3d801a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:24.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.795+0000 7f3d7f400700 1 -- 172.21.15.27:0/3565896438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d80103a00 msgr2=0x7f3d8019bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:24.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.795+0000 7f3d7f400700 1 --2- 172.21.15.27:0/3565896438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d80103a00 0x7f3d8019bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:24.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.795+0000 7f3d7f400700 1 -- 172.21.15.27:0/3565896438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d80108a40 msgr2=0x7f3d801a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:24.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.795+0000 7f3d7f400700 1 --2- 172.21.15.27:0/3565896438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d80108a40 0x7f3d801a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:24.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.795+0000 7f3d7f400700 1 -- 172.21.15.27:0/3565896438 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d801a0d60 con 0x7f3d80108080 2024-04-26T22:57:24.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.795+0000 7f3d7fe00700 1 --2- 172.21.15.27:0/3565896438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d80103a00 0x7f3d8019bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:24.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.795+0000 7f3d7f400700 1 --2- 172.21.15.27:0/3565896438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d80108080 0x7f3d8019c290 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009fd0 tx=0x7f3d6c00fa50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:24.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.795+0000 7f3d7cc00700 1 -- 172.21.15.27:0/3565896438 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c027070 con 0x7f3d80108080 2024-04-26T22:57:24.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.796+0000 7f3d7cc00700 1 -- 172.21.15.27:0/3565896438 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d6c005570 con 0x7f3d80108080 2024-04-26T22:57:24.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.796+0000 7f3d86e00700 1 -- 172.21.15.27:0/3565896438 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d801a0ff0 con 0x7f3d80108080 2024-04-26T22:57:24.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.796+0000 7f3d7cc00700 1 -- 172.21.15.27:0/3565896438 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c004930 con 0x7f3d80108080 2024-04-26T22:57:24.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.796+0000 7f3d86e00700 1 -- 172.21.15.27:0/3565896438 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d801a88d0 con 0x7f3d80108080 2024-04-26T22:57:24.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.797+0000 7f3d86e00700 1 -- 172.21.15.27:0/3565896438 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d8006a110 con 0x7f3d80108080 2024-04-26T22:57:24.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.800+0000 7f3d7cc00700 1 -- 172.21.15.27:0/3565896438 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3d6c02a070 con 0x7f3d80108080 2024-04-26T22:57:24.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.801+0000 7f3d7cc00700 1 --2- 172.21.15.27:0/3565896438 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3d640a3d50 0x7f3d640a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:24.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.801+0000 7f3d7cc00700 1 -- 172.21.15.27:0/3565896438 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f3d6c0d13a0 con 0x7f3d80108080 2024-04-26T22:57:24.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.801+0000 7f3d7fe00700 1 --2- 172.21.15.27:0/3565896438 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3d640a3d50 0x7f3d640a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:24.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.802+0000 7f3d7fe00700 1 --2- 172.21.15.27:0/3565896438 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3d640a3d50 0x7f3d640a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009750 tx=0x7f3d70009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:24.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:24.803+0000 7f3d7cc00700 1 -- 172.21.15.27:0/3565896438 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3d6c013030 con 0x7f3d80108080 2024-04-26T22:57:25.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.090+0000 7f3d86e00700 1 -- 172.21.15.27:0/3565896438 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3d801a8aa0 con 0x7f3d80108080 2024-04-26T22:57:25.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.090+0000 7f3d7cc00700 1 -- 172.21.15.27:0/3565896438 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3d6c09d820 con 0x7f3d80108080 2024-04-26T22:57:25.091 INFO:teuthology.orchestra.run.smithi027.stdout:55834574945 2024-04-26T22:57:25.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.093+0000 7f3d86e00700 1 -- 172.21.15.27:0/3565896438 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3d640a3d50 msgr2=0x7f3d640a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:25.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.093+0000 7f3d86e00700 1 --2- 172.21.15.27:0/3565896438 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3d640a3d50 0x7f3d640a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009750 tx=0x7f3d70009310 comp rx=0 tx=0).stop 2024-04-26T22:57:25.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.093+0000 7f3d86e00700 1 -- 172.21.15.27:0/3565896438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d80108080 msgr2=0x7f3d8019c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:25.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.093+0000 7f3d86e00700 1 --2- 172.21.15.27:0/3565896438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d80108080 0x7f3d8019c290 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009fd0 tx=0x7f3d6c00fa50 comp rx=0 tx=0).stop 2024-04-26T22:57:25.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.093+0000 7f3d86e00700 1 -- 172.21.15.27:0/3565896438 shutdown_connections 2024-04-26T22:57:25.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.093+0000 7f3d86e00700 1 --2- 172.21.15.27:0/3565896438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d80108a40 0x7f3d801a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:25.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.093+0000 7f3d86e00700 1 --2- 172.21.15.27:0/3565896438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d80103a00 0x7f3d8019bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:25.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.093+0000 7f3d86e00700 1 --2- 172.21.15.27:0/3565896438 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3d640a3d50 0x7f3d640a6210 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:25.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.094+0000 7f3d86e00700 1 --2- 172.21.15.27:0/3565896438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d80108080 0x7f3d8019c290 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:25.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.094+0000 7f3d86e00700 1 -- 172.21.15.27:0/3565896438 >> 172.21.15.27:0/3565896438 conn(0x7f3d8007b3b0 msgr2=0x7f3d801015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:25.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.094+0000 7f3d86e00700 1 -- 172.21.15.27:0/3565896438 shutdown_connections 2024-04-26T22:57:25.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.094+0000 7f3d86e00700 1 -- 172.21.15.27:0/3565896438 wait complete. 2024-04-26T22:57:25.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.196+0000 7f785dc00700 1 -- 172.21.15.27:0/4262518705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f785802e490 con 0x7f785806b130 2024-04-26T22:57:25.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.197+0000 7f784f400700 1 -- 172.21.15.27:0/4262518705 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f784809dce0 con 0x7f785806b130 2024-04-26T22:57:25.198 INFO:teuthology.orchestra.run.smithi027.stdout:197568495684 2024-04-26T22:57:25.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.200+0000 7f785dc00700 1 -- 172.21.15.27:0/4262518705 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f783c0a3d50 msgr2=0x7f783c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:25.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.200+0000 7f785dc00700 1 --2- 172.21.15.27:0/4262518705 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f783c0a3d50 0x7f783c0a6210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f78580799d0 tx=0x7f78440094f0 comp rx=0 tx=0).stop 2024-04-26T22:57:25.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.200+0000 7f785dc00700 1 -- 172.21.15.27:0/4262518705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f785806b130 msgr2=0x7f785819bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:25.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.200+0000 7f785dc00700 1 --2- 172.21.15.27:0/4262518705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f785806b130 0x7f785819bd50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7848000c00 tx=0x7f78480057c0 comp rx=0 tx=0).stop 2024-04-26T22:57:25.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.200+0000 7f785dc00700 1 -- 172.21.15.27:0/4262518705 shutdown_connections 2024-04-26T22:57:25.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.200+0000 7f785dc00700 1 --2- 172.21.15.27:0/4262518705 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f783c0a3d50 0x7f783c0a6210 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:25.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.200+0000 7f785dc00700 1 --2- 172.21.15.27:0/4262518705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78581094a0 0x7f78581a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:25.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.200+0000 7f785dc00700 1 --2- 172.21.15.27:0/4262518705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f785806b130 0x7f785819bd50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:25.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.200+0000 7f785dc00700 1 --2- 172.21.15.27:0/4262518705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f785806bad0 0x7f785819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:25.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.200+0000 7f785dc00700 1 -- 172.21.15.27:0/4262518705 >> 172.21.15.27:0/4262518705 conn(0x7f78580792d0 msgr2=0x7f785810e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:25.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.200+0000 7f785dc00700 1 -- 172.21.15.27:0/4262518705 shutdown_connections 2024-04-26T22:57:25.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:25.201+0000 7f785dc00700 1 -- 172.21.15.27:0/4262518705 wait complete. 2024-04-26T22:57:25.978 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920840 got 141733920846 for osd.4 2024-04-26T22:57:25.978 DEBUG:teuthology.parallel:result is None 2024-04-26T22:57:26.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:26 smithi027 ceph-mon[28034]: pgmap v394: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:26.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:26 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3565896438' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T22:57:26.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:26 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/4262518705' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T22:57:26.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:26 smithi080 ceph-mon[32324]: pgmap v394: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:26.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:26 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3565896438' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T22:57:26.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:26 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/4262518705' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T22:57:26.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:26 smithi094 ceph-mon[32889]: pgmap v394: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:26.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:26 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3565896438' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T22:57:26.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:26 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/4262518705' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T22:57:26.720 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574939 got 55834574945 for osd.1 2024-04-26T22:57:26.720 DEBUG:teuthology.parallel:result is None 2024-04-26T22:57:27.191 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495676 got 197568495684 for osd.6 2024-04-26T22:57:27.191 DEBUG:teuthology.parallel:result is None 2024-04-26T22:57:27.191 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T22:57:27.191 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T22:57:27.663 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:57:28.323 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:28 smithi080 ceph-mon[32324]: pgmap v395: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:57:28.351 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:28 smithi027 ceph-mon[28034]: pgmap v395: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:57:28.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:28 smithi094 ceph-mon[32889]: pgmap v395: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:57:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.124+0000 7fe474a00700 1 -- 172.21.15.27:0/2232271052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe470114360 msgr2=0x7fe470116750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.124+0000 7fe474a00700 1 --2- 172.21.15.27:0/2232271052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe470114360 0x7fe470116750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe46800b0a0 tx=0x7fe46800b3b0 comp rx=0 tx=0).stop 2024-04-26T22:57:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.124+0000 7fe474a00700 1 -- 172.21.15.27:0/2232271052 shutdown_connections 2024-04-26T22:57:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.124+0000 7fe474a00700 1 --2- 172.21.15.27:0/2232271052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe470114360 0x7fe470116750 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.124+0000 7fe474a00700 1 --2- 172.21.15.27:0/2232271052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4700757c0 0x7fe470075ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.124+0000 7fe474a00700 1 --2- 172.21.15.27:0/2232271052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4700760e0 0x7fe470113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:30.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.124+0000 7fe474a00700 1 -- 172.21.15.27:0/2232271052 >> 172.21.15.27:0/2232271052 conn(0x7fe4700feea0 msgr2=0x7fe4701012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:30.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.124+0000 7fe474a00700 1 -- 172.21.15.27:0/2232271052 shutdown_connections 2024-04-26T22:57:30.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.124+0000 7fe474a00700 1 -- 172.21.15.27:0/2232271052 wait complete. 2024-04-26T22:57:30.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.125+0000 7fe474a00700 1 Processor -- start 2024-04-26T22:57:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.126+0000 7fe474a00700 1 -- start start 2024-04-26T22:57:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.126+0000 7fe474a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4700757c0 0x7fe4701a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.126+0000 7fe474a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4700760e0 0x7fe4701a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.126+0000 7fe474a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe470114360 0x7fe4701a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.126+0000 7fe474a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe470119b60 con 0x7fe470114360 2024-04-26T22:57:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.126+0000 7fe474a00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe4701199e0 con 0x7fe4700757c0 2024-04-26T22:57:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.126+0000 7fe474a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe470119ce0 con 0x7fe4700760e0 2024-04-26T22:57:30.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.127+0000 7fe46ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4700760e0 0x7fe4701a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:30.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.127+0000 7fe46f400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4700757c0 0x7fe4701a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:30.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.127+0000 7fe46ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4700760e0 0x7fe4701a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:50152/0 (socket says 172.21.15.27:50152) 2024-04-26T22:57:30.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.127+0000 7fe46ea00700 1 -- 172.21.15.27:0/3865097094 learned_addr learned my addr 172.21.15.27:0/3865097094 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:30.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.127+0000 7fe46f400700 1 --2- 172.21.15.27:0/3865097094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4700757c0 0x7fe4701a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:42938/0 (socket says 172.21.15.27:42938) 2024-04-26T22:57:30.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.127+0000 7fe46fe00700 1 --2- 172.21.15.27:0/3865097094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe470114360 0x7fe4701a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:30.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.127+0000 7fe46ea00700 1 -- 172.21.15.27:0/3865097094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4700757c0 msgr2=0x7fe4701a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:30.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.127+0000 7fe46ea00700 1 --2- 172.21.15.27:0/3865097094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4700757c0 0x7fe4701a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:30.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.127+0000 7fe46ea00700 1 -- 172.21.15.27:0/3865097094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe470114360 msgr2=0x7fe4701a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:30.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.127+0000 7fe46ea00700 1 --2- 172.21.15.27:0/3865097094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe470114360 0x7fe4701a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:30.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.127+0000 7fe46ea00700 1 -- 172.21.15.27:0/3865097094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4701a5020 con 0x7fe4700760e0 2024-04-26T22:57:30.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.127+0000 7fe46f400700 1 --2- 172.21.15.27:0/3865097094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4700757c0 0x7fe4701a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:57:30.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.128+0000 7fe46ea00700 1 --2- 172.21.15.27:0/3865097094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4700760e0 0x7fe4701a0550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00e9a0 tx=0x7fe45c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:30.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.128+0000 7fe467e00700 1 -- 172.21.15.27:0/3865097094 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c00cbd0 con 0x7fe4700760e0 2024-04-26T22:57:30.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.128+0000 7fe467e00700 1 -- 172.21.15.27:0/3865097094 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe45c00cd70 con 0x7fe4700760e0 2024-04-26T22:57:30.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.128+0000 7fe467e00700 1 -- 172.21.15.27:0/3865097094 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c018680 con 0x7fe4700760e0 2024-04-26T22:57:30.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.128+0000 7fe474a00700 1 -- 172.21.15.27:0/3865097094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4701a5310 con 0x7fe4700760e0 2024-04-26T22:57:30.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.128+0000 7fe474a00700 1 -- 172.21.15.27:0/3865097094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4701acb90 con 0x7fe4700760e0 2024-04-26T22:57:30.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.129+0000 7fe474a00700 1 -- 172.21.15.27:0/3865097094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe47006a110 con 0x7fe4700760e0 2024-04-26T22:57:30.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.132+0000 7fe467e00700 1 -- 172.21.15.27:0/3865097094 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe45c010430 con 0x7fe4700760e0 2024-04-26T22:57:30.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.134+0000 7fe467e00700 1 --2- 172.21.15.27:0/3865097094 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe4540a3d50 0x7fe4540a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:30.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.134+0000 7fe467e00700 1 -- 172.21.15.27:0/3865097094 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fe45c0d0b50 con 0x7fe4700760e0 2024-04-26T22:57:30.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.134+0000 7fe46f400700 1 --2- 172.21.15.27:0/3865097094 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe4540a3d50 0x7fe4540a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:30.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.135+0000 7fe46f400700 1 --2- 172.21.15.27:0/3865097094 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe4540a3d50 0x7fe4540a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe460009cb0 tx=0x7fe460009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:30.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.138+0000 7fe467e00700 1 -- 172.21.15.27:0/3865097094 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe45c09cf40 con 0x7fe4700760e0 2024-04-26T22:57:30.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:30 smithi027 ceph-mon[28034]: pgmap v396: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.369+0000 7fe474a00700 1 -- 172.21.15.27:0/3865097094 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe470063860 con 0x7fe4540a3d50 2024-04-26T22:57:30.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.400+0000 7fe467e00700 1 -- 172.21.15.27:0/3865097094 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340678 (secure 0 0 0) 0x7fe470063860 con 0x7fe4540a3d50 2024-04-26T22:57:30.408 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:57:30.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.408+0000 7fe474a00700 1 -- 172.21.15.27:0/3865097094 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe4540a3d50 msgr2=0x7fe4540a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:30.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.408+0000 7fe474a00700 1 --2- 172.21.15.27:0/3865097094 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe4540a3d50 0x7fe4540a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe460009cb0 tx=0x7fe460009260 comp rx=0 tx=0).stop 2024-04-26T22:57:30.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.408+0000 7fe474a00700 1 -- 172.21.15.27:0/3865097094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4700760e0 msgr2=0x7fe4701a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:30.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.408+0000 7fe474a00700 1 --2- 172.21.15.27:0/3865097094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4700760e0 0x7fe4701a0550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00e9a0 tx=0x7fe45c00ecb0 comp rx=0 tx=0).stop 2024-04-26T22:57:30.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.408+0000 7fe474a00700 1 -- 172.21.15.27:0/3865097094 shutdown_connections 2024-04-26T22:57:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.408+0000 7fe474a00700 1 --2- 172.21.15.27:0/3865097094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe470114360 0x7fe4701a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.408+0000 7fe474a00700 1 --2- 172.21.15.27:0/3865097094 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe4540a3d50 0x7fe4540a6210 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.408+0000 7fe474a00700 1 --2- 172.21.15.27:0/3865097094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4700757c0 0x7fe4701a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.408+0000 7fe474a00700 1 --2- 172.21.15.27:0/3865097094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4700760e0 0x7fe4701a0550 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.408+0000 7fe474a00700 1 -- 172.21.15.27:0/3865097094 >> 172.21.15.27:0/3865097094 conn(0x7fe4700feea0 msgr2=0x7fe470100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.408+0000 7fe474a00700 1 -- 172.21.15.27:0/3865097094 shutdown_connections 2024-04-26T22:57:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:30.408+0000 7fe474a00700 1 -- 172.21.15.27:0/3865097094 wait complete. 2024-04-26T22:57:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-26T22:57:30.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:30 smithi080 ceph-mon[32324]: pgmap v396: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:30.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:30 smithi094 ceph-mon[32889]: pgmap v396: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:31.374 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-04-26T22:57:28.501359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650279,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":713,"num_read_kb":471,"num_write":348,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382568,"kb_used_data":12560,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559256,"statfs":{"total":1151940427776,"available":1151548678144,"internally_reserved":0,"allocated":12861440,"data_stored":8265659,"data_compressed":434286,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005427"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328702+0000","last_change":"2024-04-26T22:55:25.404468+0000","last_active":"2024-04-26T22:57:06.328702+0000","last_peered":"2024-04-26T22:57:06.328702+0000","last_clean":"2024-04-26T22:57:06.328702+0000","last_became_active":"2024-04-26T22:55:25.404178+0000","last_became_peered":"2024-04-26T22:55:25.404178+0000","last_unstale":"2024-04-26T22:57:06.328702+0000","last_undegraded":"2024-04-26T22:57:06.328702+0000","last_fullsized":"2024-04-26T22:57:06.328702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:46.249569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.332507+0000","last_change":"2024-04-26T22:55:25.401044+0000","last_active":"2024-04-26T22:57:06.332507+0000","last_peered":"2024-04-26T22:57:06.332507+0000","last_clean":"2024-04-26T22:57:06.332507+0000","last_became_active":"2024-04-26T22:55:25.397925+0000","last_became_peered":"2024-04-26T22:55:25.397925+0000","last_unstale":"2024-04-26T22:57:06.332507+0000","last_undegraded":"2024-04-26T22:57:06.332507+0000","last_fullsized":"2024-04-26T22:57:06.332507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:28.503480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.331821+0000","last_change":"2024-04-26T22:55:25.396780+0000","last_active":"2024-04-26T22:57:06.331821+0000","last_peered":"2024-04-26T22:57:06.331821+0000","last_clean":"2024-04-26T22:57:06.331821+0000","last_became_active":"2024-04-26T22:55:25.395401+0000","last_became_peered":"2024-04-26T22:55:25.395401+0000","last_unstale":"2024-04-26T22:57:06.331821+0000","last_undegraded":"2024-04-26T22:57:06.331821+0000","last_fullsized":"2024-04-26T22:57:06.331821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:32.295042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.924904+0000","last_change":"2024-04-26T22:57:05.922677+0000","last_active":"2024-04-26T22:57:05.924904+0000","last_peered":"2024-04-26T22:57:05.924904+0000","last_clean":"2024-04-26T22:57:05.924904+0000","last_became_active":"2024-04-26T22:57:05.921292+0000","last_became_peered":"2024-04-26T22:57:05.921292+0000","last_unstale":"2024-04-26T22:57:05.924904+0000","last_undegraded":"2024-04-26T22:57:05.924904+0000","last_fullsized":"2024-04-26T22:57:05.924904+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:57.108735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912809+0000","last_change":"2024-04-26T22:55:25.391341+0000","last_active":"2024-04-26T22:57:05.912809+0000","last_peered":"2024-04-26T22:57:05.912809+0000","last_clean":"2024-04-26T22:57:05.912809+0000","last_became_active":"2024-04-26T22:55:25.391061+0000","last_became_peered":"2024-04-26T22:55:25.391061+0000","last_unstale":"2024-04-26T22:57:05.912809+0000","last_undegraded":"2024-04-26T22:57:05.912809+0000","last_fullsized":"2024-04-26T22:57:05.912809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:08.391492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.327082+0000","last_change":"2024-04-26T22:55:25.388412+0000","last_active":"2024-04-26T22:57:06.327082+0000","last_peered":"2024-04-26T22:57:06.327082+0000","last_clean":"2024-04-26T22:57:06.327082+0000","last_became_active":"2024-04-26T22:55:25.388010+0000","last_became_peered":"2024-04-26T22:55:25.388010+0000","last_unstale":"2024-04-26T22:57:06.327082+0000","last_undegraded":"2024-04-26T22:57:06.327082+0000","last_fullsized":"2024-04-26T22:57:06.327082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:45.851109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.924654+0000","last_change":"2024-04-26T22:57:05.924654+0000","last_active":"2024-04-26T22:57:05.924654+0000","last_peered":"2024-04-26T22:57:05.924654+0000","last_clean":"2024-04-26T22:57:05.924654+0000","last_became_active":"2024-04-26T22:57:05.924231+0000","last_became_peered":"2024-04-26T22:57:05.924231+0000","last_unstale":"2024-04-26T22:57:05.924654+0000","last_undegraded":"2024-04-26T22:57:05.924654+0000","last_fullsized":"2024-04-26T22:57:05.924654+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:44.145297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.330003+0000","last_change":"2024-04-26T22:55:25.408952+0000","last_active":"2024-04-26T22:57:06.330003+0000","last_peered":"2024-04-26T22:57:06.330003+0000","last_clean":"2024-04-26T22:57:06.330003+0000","last_became_active":"2024-04-26T22:55:25.408485+0000","last_became_peered":"2024-04-26T22:55:25.408485+0000","last_unstale":"2024-04-26T22:57:06.330003+0000","last_undegraded":"2024-04-26T22:57:06.330003+0000","last_fullsized":"2024-04-26T22:57:06.330003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.043114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.222603+0000","last_change":"2024-04-26T22:55:25.377377+0000","last_active":"2024-04-26T22:57:06.222603+0000","last_peered":"2024-04-26T22:57:06.222603+0000","last_clean":"2024-04-26T22:57:06.222603+0000","last_became_active":"2024-04-26T22:55:25.377015+0000","last_became_peered":"2024-04-26T22:55:25.377015+0000","last_unstale":"2024-04-26T22:57:06.222603+0000","last_undegraded":"2024-04-26T22:57:06.222603+0000","last_fullsized":"2024-04-26T22:57:06.222603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:25.676469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.327710+0000","last_change":"2024-04-26T22:55:25.375549+0000","last_active":"2024-04-26T22:57:06.327710+0000","last_peered":"2024-04-26T22:57:06.327710+0000","last_clean":"2024-04-26T22:57:06.327710+0000","last_became_active":"2024-04-26T22:55:25.375232+0000","last_became_peered":"2024-04-26T22:55:25.375232+0000","last_unstale":"2024-04-26T22:57:06.327710+0000","last_undegraded":"2024-04-26T22:57:06.327710+0000","last_fullsized":"2024-04-26T22:57:06.327710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:22.561623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913442+0000","last_change":"2024-04-26T22:55:25.374224+0000","last_active":"2024-04-26T22:57:05.913442+0000","last_peered":"2024-04-26T22:57:05.913442+0000","last_clean":"2024-04-26T22:57:05.913442+0000","last_became_active":"2024-04-26T22:55:25.373813+0000","last_became_peered":"2024-04-26T22:55:25.373813+0000","last_unstale":"2024-04-26T22:57:05.913442+0000","last_undegraded":"2024-04-26T22:57:05.913442+0000","last_fullsized":"2024-04-26T22:57:05.913442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:34.874888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.909140+0000","last_change":"2024-04-26T22:55:25.376666+0000","last_active":"2024-04-26T22:57:05.909140+0000","last_peered":"2024-04-26T22:57:05.909140+0000","last_clean":"2024-04-26T22:57:05.909140+0000","last_became_active":"2024-04-26T22:55:25.376328+0000","last_became_peered":"2024-04-26T22:55:25.376328+0000","last_unstale":"2024-04-26T22:57:05.909140+0000","last_undegraded":"2024-04-26T22:57:05.909140+0000","last_fullsized":"2024-04-26T22:57:05.909140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:32:34.287984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.911680+0000","last_change":"2024-04-26T22:55:25.377220+0000","last_active":"2024-04-26T22:57:05.911680+0000","last_peered":"2024-04-26T22:57:05.911680+0000","last_clean":"2024-04-26T22:57:05.911680+0000","last_became_active":"2024-04-26T22:55:25.376637+0000","last_became_peered":"2024-04-26T22:55:25.376637+0000","last_unstale":"2024-04-26T22:57:05.911680+0000","last_undegraded":"2024-04-26T22:57:05.911680+0000","last_fullsized":"2024-04-26T22:57:05.911680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:57.934912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.330416+0000","last_change":"2024-04-26T22:55:25.404669+0000","last_active":"2024-04-26T22:57:06.330416+0000","last_peered":"2024-04-26T22:57:06.330416+0000","last_clean":"2024-04-26T22:57:06.330416+0000","last_became_active":"2024-04-26T22:55:25.402804+0000","last_became_peered":"2024-04-26T22:55:25.402804+0000","last_unstale":"2024-04-26T22:57:06.330416+0000","last_undegraded":"2024-04-26T22:57:06.330416+0000","last_fullsized":"2024-04-26T22:57:06.330416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:37.239667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.330769+0000","last_change":"2024-04-26T22:55:25.399877+0000","last_active":"2024-04-26T22:57:06.330769+0000","last_peered":"2024-04-26T22:57:06.330769+0000","last_clean":"2024-04-26T22:57:06.330769+0000","last_became_active":"2024-04-26T22:55:25.397476+0000","last_became_peered":"2024-04-26T22:55:25.397476+0000","last_unstale":"2024-04-26T22:57:06.330769+0000","last_undegraded":"2024-04-26T22:57:06.330769+0000","last_fullsized":"2024-04-26T22:57:06.330769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:26.732300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.221950+0000","last_change":"2024-04-26T22:55:25.393682+0000","last_active":"2024-04-26T22:57:06.221950+0000","last_peered":"2024-04-26T22:57:06.221950+0000","last_clean":"2024-04-26T22:57:06.221950+0000","last_became_active":"2024-04-26T22:55:25.393349+0000","last_became_peered":"2024-04-26T22:55:25.393349+0000","last_unstale":"2024-04-26T22:57:06.221950+0000","last_undegraded":"2024-04-26T22:57:06.221950+0000","last_fullsized":"2024-04-26T22:57:06.221950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:48.065904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328473+0000","last_change":"2024-04-26T22:55:25.404449+0000","last_active":"2024-04-26T22:57:06.328473+0000","last_peered":"2024-04-26T22:57:06.328473+0000","last_clean":"2024-04-26T22:57:06.328473+0000","last_became_active":"2024-04-26T22:55:25.404170+0000","last_became_peered":"2024-04-26T22:55:25.404170+0000","last_unstale":"2024-04-26T22:57:06.328473+0000","last_undegraded":"2024-04-26T22:57:06.328473+0000","last_fullsized":"2024-04-26T22:57:06.328473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:04.731208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.917737+0000","last_change":"2024-04-26T22:55:25.390368+0000","last_active":"2024-04-26T22:57:05.917737+0000","last_peered":"2024-04-26T22:57:05.917737+0000","last_clean":"2024-04-26T22:57:05.917737+0000","last_became_active":"2024-04-26T22:55:25.389201+0000","last_became_peered":"2024-04-26T22:55:25.389201+0000","last_unstale":"2024-04-26T22:57:05.917737+0000","last_undegraded":"2024-04-26T22:57:05.917737+0000","last_fullsized":"2024-04-26T22:57:05.917737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:35.985419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908616+0000","last_change":"2024-04-26T22:55:25.387616+0000","last_active":"2024-04-26T22:57:05.908616+0000","last_peered":"2024-04-26T22:57:05.908616+0000","last_clean":"2024-04-26T22:57:05.908616+0000","last_became_active":"2024-04-26T22:55:25.387360+0000","last_became_peered":"2024-04-26T22:55:25.387360+0000","last_unstale":"2024-04-26T22:57:05.908616+0000","last_undegraded":"2024-04-26T22:57:05.908616+0000","last_fullsized":"2024-04-26T22:57:05.908616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:49.378287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908302+0000","last_change":"2024-04-26T22:55:30.425407+0000","last_active":"2024-04-26T22:57:05.908302+0000","last_peered":"2024-04-26T22:57:05.908302+0000","last_clean":"2024-04-26T22:57:05.908302+0000","last_became_active":"2024-04-26T22:55:30.424796+0000","last_became_peered":"2024-04-26T22:55:30.424796+0000","last_unstale":"2024-04-26T22:57:05.908302+0000","last_undegraded":"2024-04-26T22:57:05.908302+0000","last_fullsized":"2024-04-26T22:57:05.908302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:42.539415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.917495+0000","last_change":"2024-04-26T22:55:24.776253+0000","last_active":"2024-04-26T22:57:05.917495+0000","last_peered":"2024-04-26T22:57:05.917495+0000","last_clean":"2024-04-26T22:57:05.917495+0000","last_became_active":"2024-04-26T22:55:24.775700+0000","last_became_peered":"2024-04-26T22:55:24.775700+0000","last_unstale":"2024-04-26T22:57:05.917495+0000","last_undegraded":"2024-04-26T22:57:05.917495+0000","last_fullsized":"2024-04-26T22:57:05.917495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:45.608363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.333737+0000","last_change":"2024-04-26T22:55:25.390690+0000","last_active":"2024-04-26T22:57:06.333737+0000","last_peered":"2024-04-26T22:57:06.333737+0000","last_clean":"2024-04-26T22:57:06.333737+0000","last_became_active":"2024-04-26T22:55:25.390289+0000","last_became_peered":"2024-04-26T22:55:25.390289+0000","last_unstale":"2024-04-26T22:57:06.333737+0000","last_undegraded":"2024-04-26T22:57:06.333737+0000","last_fullsized":"2024-04-26T22:57:06.333737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:54.637992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908485+0000","last_change":"2024-04-26T22:55:30.433555+0000","last_active":"2024-04-26T22:57:05.908485+0000","last_peered":"2024-04-26T22:57:05.908485+0000","last_clean":"2024-04-26T22:57:05.908485+0000","last_became_active":"2024-04-26T22:55:30.433240+0000","last_became_peered":"2024-04-26T22:55:30.433240+0000","last_unstale":"2024-04-26T22:57:05.908485+0000","last_undegraded":"2024-04-26T22:57:05.908485+0000","last_fullsized":"2024-04-26T22:57:05.908485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:55:21.215073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.333901+0000","last_change":"2024-04-26T22:55:24.788218+0000","last_active":"2024-04-26T22:57:06.333901+0000","last_peered":"2024-04-26T22:57:06.333901+0000","last_clean":"2024-04-26T22:57:06.333901+0000","last_became_active":"2024-04-26T22:55:24.787762+0000","last_became_peered":"2024-04-26T22:55:24.787762+0000","last_unstale":"2024-04-26T22:57:06.333901+0000","last_undegraded":"2024-04-26T22:57:06.333901+0000","last_fullsized":"2024-04-26T22:57:06.333901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:42.264617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.331669+0000","last_change":"2024-04-26T22:55:25.400356+0000","last_active":"2024-04-26T22:57:06.331669+0000","last_peered":"2024-04-26T22:57:06.331669+0000","last_clean":"2024-04-26T22:57:06.331669+0000","last_became_active":"2024-04-26T22:55:25.399907+0000","last_became_peered":"2024-04-26T22:55:25.399907+0000","last_unstale":"2024-04-26T22:57:06.331669+0000","last_undegraded":"2024-04-26T22:57:06.331669+0000","last_fullsized":"2024-04-26T22:57:06.331669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:50:50.804112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908697+0000","last_change":"2024-04-26T22:56:05.816337+0000","last_active":"2024-04-26T22:57:05.908697+0000","last_peered":"2024-04-26T22:57:05.908697+0000","last_clean":"2024-04-26T22:57:05.908697+0000","last_became_active":"2024-04-26T22:56:05.815846+0000","last_became_peered":"2024-04-26T22:56:05.815846+0000","last_unstale":"2024-04-26T22:57:05.908697+0000","last_undegraded":"2024-04-26T22:57:05.908697+0000","last_fullsized":"2024-04-26T22:57:05.908697+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:41.793454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328372+0000","last_change":"2024-04-26T22:56:05.732306+0000","last_active":"2024-04-26T22:57:06.328372+0000","last_peered":"2024-04-26T22:57:06.328372+0000","last_clean":"2024-04-26T22:57:06.328372+0000","last_became_active":"2024-04-26T22:56:05.722564+0000","last_became_peered":"2024-04-26T22:56:05.722564+0000","last_unstale":"2024-04-26T22:57:06.328372+0000","last_undegraded":"2024-04-26T22:57:06.328372+0000","last_fullsized":"2024-04-26T22:57:06.328372+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:28.784664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.222308+0000","last_change":"2024-04-26T22:55:25.381896+0000","last_active":"2024-04-26T22:57:06.222308+0000","last_peered":"2024-04-26T22:57:06.222308+0000","last_clean":"2024-04-26T22:57:06.222308+0000","last_became_active":"2024-04-26T22:55:25.381411+0000","last_became_peered":"2024-04-26T22:55:25.381411+0000","last_unstale":"2024-04-26T22:57:06.222308+0000","last_undegraded":"2024-04-26T22:57:06.222308+0000","last_fullsized":"2024-04-26T22:57:06.222308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:54.731407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908578+0000","last_change":"2024-04-26T22:56:05.816353+0000","last_active":"2024-04-26T22:57:05.908578+0000","last_peered":"2024-04-26T22:57:05.908578+0000","last_clean":"2024-04-26T22:57:05.908578+0000","last_became_active":"2024-04-26T22:56:05.815871+0000","last_became_peered":"2024-04-26T22:56:05.815871+0000","last_unstale":"2024-04-26T22:57:05.908578+0000","last_undegraded":"2024-04-26T22:57:05.908578+0000","last_fullsized":"2024-04-26T22:57:05.908578+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:24.383516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912819+0000","last_change":"2024-04-26T22:55:24.756017+0000","last_active":"2024-04-26T22:57:05.912819+0000","last_peered":"2024-04-26T22:57:05.912819+0000","last_clean":"2024-04-26T22:57:05.912819+0000","last_became_active":"2024-04-26T22:55:24.755661+0000","last_became_peered":"2024-04-26T22:55:24.755661+0000","last_unstale":"2024-04-26T22:57:05.912819+0000","last_undegraded":"2024-04-26T22:57:05.912819+0000","last_fullsized":"2024-04-26T22:57:05.912819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:53.909567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.910896+0000","last_change":"2024-04-26T22:55:25.375929+0000","last_active":"2024-04-26T22:57:05.910896+0000","last_peered":"2024-04-26T22:57:05.910896+0000","last_clean":"2024-04-26T22:57:05.910896+0000","last_became_active":"2024-04-26T22:55:25.375435+0000","last_became_peered":"2024-04-26T22:55:25.375435+0000","last_unstale":"2024-04-26T22:57:05.910896+0000","last_undegraded":"2024-04-26T22:57:05.910896+0000","last_fullsized":"2024-04-26T22:57:05.910896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:43.336889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908743+0000","last_change":"2024-04-26T22:55:30.409074+0000","last_active":"2024-04-26T22:57:05.908743+0000","last_peered":"2024-04-26T22:57:05.908743+0000","last_clean":"2024-04-26T22:57:05.908743+0000","last_became_active":"2024-04-26T22:55:30.408613+0000","last_became_peered":"2024-04-26T22:55:30.408613+0000","last_unstale":"2024-04-26T22:57:05.908743+0000","last_undegraded":"2024-04-26T22:57:05.908743+0000","last_fullsized":"2024-04-26T22:57:05.908743+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:24.312971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":130,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:23.794824+0000","last_change":"2024-04-26T22:55:24.763787+0000","last_active":"2024-04-26T22:57:23.794824+0000","last_peered":"2024-04-26T22:57:23.794824+0000","last_clean":"2024-04-26T22:57:23.794824+0000","last_became_active":"2024-04-26T22:55:24.763480+0000","last_became_peered":"2024-04-26T22:55:24.763480+0000","last_unstale":"2024-04-26T22:57:23.794824+0000","last_undegraded":"2024-04-26T22:57:23.794824+0000","last_fullsized":"2024-04-26T22:57:23.794824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:18.591009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.924534+0000","last_change":"2024-04-26T22:57:05.924534+0000","last_active":"2024-04-26T22:57:05.924534+0000","last_peered":"2024-04-26T22:57:05.924534+0000","last_clean":"2024-04-26T22:57:05.924534+0000","last_became_active":"2024-04-26T22:57:05.924142+0000","last_became_peered":"2024-04-26T22:57:05.924142+0000","last_unstale":"2024-04-26T22:57:05.924534+0000","last_undegraded":"2024-04-26T22:57:05.924534+0000","last_fullsized":"2024-04-26T22:57:05.924534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:31:00.380807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908805+0000","last_change":"2024-04-26T22:55:30.425891+0000","last_active":"2024-04-26T22:57:05.908805+0000","last_peered":"2024-04-26T22:57:05.908805+0000","last_clean":"2024-04-26T22:57:05.908805+0000","last_became_active":"2024-04-26T22:55:30.424742+0000","last_became_peered":"2024-04-26T22:55:30.424742+0000","last_unstale":"2024-04-26T22:57:05.908805+0000","last_undegraded":"2024-04-26T22:57:05.908805+0000","last_fullsized":"2024-04-26T22:57:05.908805+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:04.876670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.327018+0000","last_change":"2024-04-26T22:56:05.736174+0000","last_active":"2024-04-26T22:57:06.327018+0000","last_peered":"2024-04-26T22:57:06.327018+0000","last_clean":"2024-04-26T22:57:06.327018+0000","last_became_active":"2024-04-26T22:56:05.732271+0000","last_became_peered":"2024-04-26T22:56:05.732271+0000","last_unstale":"2024-04-26T22:57:06.327018+0000","last_undegraded":"2024-04-26T22:57:06.327018+0000","last_fullsized":"2024-04-26T22:57:06.327018+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:41.980613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.910982+0000","last_change":"2024-04-26T22:55:25.399892+0000","last_active":"2024-04-26T22:57:05.910982+0000","last_peered":"2024-04-26T22:57:05.910982+0000","last_clean":"2024-04-26T22:57:05.910982+0000","last_became_active":"2024-04-26T22:55:25.399515+0000","last_became_peered":"2024-04-26T22:55:25.399515+0000","last_unstale":"2024-04-26T22:57:05.910982+0000","last_undegraded":"2024-04-26T22:57:05.910982+0000","last_fullsized":"2024-04-26T22:57:05.910982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.472103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328465+0000","last_change":"2024-04-26T22:55:30.422932+0000","last_active":"2024-04-26T22:57:06.328465+0000","last_peered":"2024-04-26T22:57:06.328465+0000","last_clean":"2024-04-26T22:57:06.328465+0000","last_became_active":"2024-04-26T22:55:30.422481+0000","last_became_peered":"2024-04-26T22:55:30.422481+0000","last_unstale":"2024-04-26T22:57:06.328465+0000","last_undegraded":"2024-04-26T22:57:06.328465+0000","last_fullsized":"2024-04-26T22:57:06.328465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:11.469515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.917309+0000","last_change":"2024-04-26T22:55:24.780973+0000","last_active":"2024-04-26T22:57:05.917309+0000","last_peered":"2024-04-26T22:57:05.917309+0000","last_clean":"2024-04-26T22:57:05.917309+0000","last_became_active":"2024-04-26T22:55:24.779810+0000","last_became_peered":"2024-04-26T22:55:24.779810+0000","last_unstale":"2024-04-26T22:57:05.917309+0000","last_undegraded":"2024-04-26T22:57:05.917309+0000","last_fullsized":"2024-04-26T22:57:05.917309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:33.708588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.331182+0000","last_change":"2024-04-26T22:55:25.402067+0000","last_active":"2024-04-26T22:57:06.331182+0000","last_peered":"2024-04-26T22:57:06.331182+0000","last_clean":"2024-04-26T22:57:06.331182+0000","last_became_active":"2024-04-26T22:55:25.401717+0000","last_became_peered":"2024-04-26T22:55:25.401717+0000","last_unstale":"2024-04-26T22:57:06.331182+0000","last_undegraded":"2024-04-26T22:57:06.331182+0000","last_fullsized":"2024-04-26T22:57:06.331182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:16.260620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.332402+0000","last_change":"2024-04-26T22:55:30.429430+0000","last_active":"2024-04-26T22:57:06.332402+0000","last_peered":"2024-04-26T22:57:06.332402+0000","last_clean":"2024-04-26T22:57:06.332402+0000","last_became_active":"2024-04-26T22:55:30.428800+0000","last_became_peered":"2024-04-26T22:55:30.428800+0000","last_unstale":"2024-04-26T22:57:06.332402+0000","last_undegraded":"2024-04-26T22:57:06.332402+0000","last_fullsized":"2024-04-26T22:57:06.332402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:05.501004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.910522+0000","last_change":"2024-04-26T22:55:24.765181+0000","last_active":"2024-04-26T22:57:05.910522+0000","last_peered":"2024-04-26T22:57:05.910522+0000","last_clean":"2024-04-26T22:57:05.910522+0000","last_became_active":"2024-04-26T22:55:24.764847+0000","last_became_peered":"2024-04-26T22:55:24.764847+0000","last_unstale":"2024-04-26T22:57:05.910522+0000","last_undegraded":"2024-04-26T22:57:05.910522+0000","last_fullsized":"2024-04-26T22:57:05.910522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:25.949304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.329040+0000","last_change":"2024-04-26T22:57:06.329040+0000","last_active":"2024-04-26T22:57:06.329040+0000","last_peered":"2024-04-26T22:57:06.329040+0000","last_clean":"2024-04-26T22:57:06.329040+0000","last_became_active":"2024-04-26T22:57:06.328430+0000","last_became_peered":"2024-04-26T22:57:06.328430+0000","last_unstale":"2024-04-26T22:57:06.329040+0000","last_undegraded":"2024-04-26T22:57:06.329040+0000","last_fullsized":"2024-04-26T22:57:06.329040+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:25.464945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.331052+0000","last_change":"2024-04-26T22:55:30.424537+0000","last_active":"2024-04-26T22:57:06.331052+0000","last_peered":"2024-04-26T22:57:06.331052+0000","last_clean":"2024-04-26T22:57:06.331052+0000","last_became_active":"2024-04-26T22:55:30.423262+0000","last_became_peered":"2024-04-26T22:55:30.423262+0000","last_unstale":"2024-04-26T22:57:06.331052+0000","last_undegraded":"2024-04-26T22:57:06.331052+0000","last_fullsized":"2024-04-26T22:57:06.331052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:48.666453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":128,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:24.795428+0000","last_change":"2024-04-26T22:55:24.776580+0000","last_active":"2024-04-26T22:57:24.795428+0000","last_peered":"2024-04-26T22:57:24.795428+0000","last_clean":"2024-04-26T22:57:24.795428+0000","last_became_active":"2024-04-26T22:55:24.776143+0000","last_became_peered":"2024-04-26T22:55:24.776143+0000","last_unstale":"2024-04-26T22:57:24.795428+0000","last_undegraded":"2024-04-26T22:57:24.795428+0000","last_fullsized":"2024-04-26T22:57:24.795428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:02.291237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328855+0000","last_change":"2024-04-26T22:57:06.328855+0000","last_active":"2024-04-26T22:57:06.328855+0000","last_peered":"2024-04-26T22:57:06.328855+0000","last_clean":"2024-04-26T22:57:06.328855+0000","last_became_active":"2024-04-26T22:57:06.328500+0000","last_became_peered":"2024-04-26T22:57:06.328500+0000","last_unstale":"2024-04-26T22:57:06.328855+0000","last_undegraded":"2024-04-26T22:57:06.328855+0000","last_fullsized":"2024-04-26T22:57:06.328855+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:34.583831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,9],"acting":[8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913215+0000","last_change":"2024-04-26T22:55:30.437915+0000","last_active":"2024-04-26T22:57:05.913215+0000","last_peered":"2024-04-26T22:57:05.913215+0000","last_clean":"2024-04-26T22:57:05.913215+0000","last_became_active":"2024-04-26T22:55:30.437148+0000","last_became_peered":"2024-04-26T22:55:30.437148+0000","last_unstale":"2024-04-26T22:57:05.913215+0000","last_undegraded":"2024-04-26T22:57:05.913215+0000","last_fullsized":"2024-04-26T22:57:05.913215+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:11.647498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328587+0000","last_change":"2024-04-26T22:56:05.728108+0000","last_active":"2024-04-26T22:57:06.328587+0000","last_peered":"2024-04-26T22:57:06.328587+0000","last_clean":"2024-04-26T22:57:06.328587+0000","last_became_active":"2024-04-26T22:56:05.721930+0000","last_became_peered":"2024-04-26T22:56:05.721930+0000","last_unstale":"2024-04-26T22:57:06.328587+0000","last_undegraded":"2024-04-26T22:57:06.328587+0000","last_fullsized":"2024-04-26T22:57:06.328587+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:10.079562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,4],"acting":[11,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.331674+0000","last_change":"2024-04-26T22:55:25.401057+0000","last_active":"2024-04-26T22:57:06.331674+0000","last_peered":"2024-04-26T22:57:06.331674+0000","last_clean":"2024-04-26T22:57:06.331674+0000","last_became_active":"2024-04-26T22:55:25.398265+0000","last_became_peered":"2024-04-26T22:55:25.398265+0000","last_unstale":"2024-04-26T22:57:06.331674+0000","last_undegraded":"2024-04-26T22:57:06.331674+0000","last_fullsized":"2024-04-26T22:57:06.331674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:49.967012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908843+0000","last_change":"2024-04-26T22:55:30.426682+0000","last_active":"2024-04-26T22:57:05.908843+0000","last_peered":"2024-04-26T22:57:05.908843+0000","last_clean":"2024-04-26T22:57:05.908843+0000","last_became_active":"2024-04-26T22:55:30.425918+0000","last_became_peered":"2024-04-26T22:55:30.425918+0000","last_unstale":"2024-04-26T22:57:05.908843+0000","last_undegraded":"2024-04-26T22:57:05.908843+0000","last_fullsized":"2024-04-26T22:57:05.908843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:44.230083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.910815+0000","last_change":"2024-04-26T22:55:24.773497+0000","last_active":"2024-04-26T22:57:05.910815+0000","last_peered":"2024-04-26T22:57:05.910815+0000","last_clean":"2024-04-26T22:57:05.910815+0000","last_became_active":"2024-04-26T22:55:24.773183+0000","last_became_peered":"2024-04-26T22:55:24.773183+0000","last_unstale":"2024-04-26T22:57:05.910815+0000","last_undegraded":"2024-04-26T22:57:05.910815+0000","last_fullsized":"2024-04-26T22:57:05.910815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:58:07.157785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.222503+0000","last_change":"2024-04-26T22:55:25.378420+0000","last_active":"2024-04-26T22:57:06.222503+0000","last_peered":"2024-04-26T22:57:06.222503+0000","last_clean":"2024-04-26T22:57:06.222503+0000","last_became_active":"2024-04-26T22:55:25.378068+0000","last_became_peered":"2024-04-26T22:55:25.378068+0000","last_unstale":"2024-04-26T22:57:06.222503+0000","last_undegraded":"2024-04-26T22:57:06.222503+0000","last_fullsized":"2024-04-26T22:57:06.222503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:10.910541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.915267+0000","last_change":"2024-04-26T22:55:30.436973+0000","last_active":"2024-04-26T22:57:05.915267+0000","last_peered":"2024-04-26T22:57:05.915267+0000","last_clean":"2024-04-26T22:57:05.915267+0000","last_became_active":"2024-04-26T22:55:30.436327+0000","last_became_peered":"2024-04-26T22:55:30.436327+0000","last_unstale":"2024-04-26T22:57:05.915267+0000","last_undegraded":"2024-04-26T22:57:05.915267+0000","last_fullsized":"2024-04-26T22:57:05.915267+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:36.616328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'32","reported_seq":257,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:25.796972+0000","last_change":"2024-04-26T22:55:24.760508+0000","last_active":"2024-04-26T22:57:25.796972+0000","last_peered":"2024-04-26T22:57:25.796972+0000","last_clean":"2024-04-26T22:57:25.796972+0000","last_became_active":"2024-04-26T22:55:24.760180+0000","last_became_peered":"2024-04-26T22:55:24.760180+0000","last_unstale":"2024-04-26T22:57:25.796972+0000","last_undegraded":"2024-04-26T22:57:25.796972+0000","last_fullsized":"2024-04-26T22:57:25.796972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:50.094352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55676,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":192,"num_read_kb":150,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913041+0000","last_change":"2024-04-26T22:55:25.376091+0000","last_active":"2024-04-26T22:57:05.913041+0000","last_peered":"2024-04-26T22:57:05.913041+0000","last_clean":"2024-04-26T22:57:05.913041+0000","last_became_active":"2024-04-26T22:55:25.375623+0000","last_became_peered":"2024-04-26T22:55:25.375623+0000","last_unstale":"2024-04-26T22:57:05.913041+0000","last_undegraded":"2024-04-26T22:57:05.913041+0000","last_fullsized":"2024-04-26T22:57:05.913041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:09.466719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.911432+0000","last_change":"2024-04-26T22:55:30.413534+0000","last_active":"2024-04-26T22:57:05.911432+0000","last_peered":"2024-04-26T22:57:05.911432+0000","last_clean":"2024-04-26T22:57:05.911432+0000","last_became_active":"2024-04-26T22:55:30.413120+0000","last_became_peered":"2024-04-26T22:55:30.413120+0000","last_unstale":"2024-04-26T22:57:05.911432+0000","last_undegraded":"2024-04-26T22:57:05.911432+0000","last_fullsized":"2024-04-26T22:57:05.911432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:13.829733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908553+0000","last_change":"2024-04-26T22:55:24.781446+0000","last_active":"2024-04-26T22:57:05.908553+0000","last_peered":"2024-04-26T22:57:05.908553+0000","last_clean":"2024-04-26T22:57:05.908553+0000","last_became_active":"2024-04-26T22:55:24.781098+0000","last_became_peered":"2024-04-26T22:55:24.781098+0000","last_unstale":"2024-04-26T22:57:05.908553+0000","last_undegraded":"2024-04-26T22:57:05.908553+0000","last_fullsized":"2024-04-26T22:57:05.908553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:40.939755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.911425+0000","last_change":"2024-04-26T22:55:25.394147+0000","last_active":"2024-04-26T22:57:05.911425+0000","last_peered":"2024-04-26T22:57:05.911425+0000","last_clean":"2024-04-26T22:57:05.911425+0000","last_became_active":"2024-04-26T22:55:25.393654+0000","last_became_peered":"2024-04-26T22:55:25.393654+0000","last_unstale":"2024-04-26T22:57:05.911425+0000","last_undegraded":"2024-04-26T22:57:05.911425+0000","last_fullsized":"2024-04-26T22:57:05.911425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:16.335027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328118+0000","last_change":"2024-04-26T22:55:30.431692+0000","last_active":"2024-04-26T22:57:06.328118+0000","last_peered":"2024-04-26T22:57:06.328118+0000","last_clean":"2024-04-26T22:57:06.328118+0000","last_became_active":"2024-04-26T22:55:30.431380+0000","last_became_peered":"2024-04-26T22:55:30.431380+0000","last_unstale":"2024-04-26T22:57:06.328118+0000","last_undegraded":"2024-04-26T22:57:06.328118+0000","last_fullsized":"2024-04-26T22:57:06.328118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:14.556048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.910767+0000","last_change":"2024-04-26T22:55:24.788121+0000","last_active":"2024-04-26T22:57:05.910767+0000","last_peered":"2024-04-26T22:57:05.910767+0000","last_clean":"2024-04-26T22:57:05.910767+0000","last_became_active":"2024-04-26T22:55:24.787759+0000","last_became_peered":"2024-04-26T22:55:24.787759+0000","last_unstale":"2024-04-26T22:57:05.910767+0000","last_undegraded":"2024-04-26T22:57:05.910767+0000","last_fullsized":"2024-04-26T22:57:05.910767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:49.734675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913194+0000","last_change":"2024-04-26T22:55:25.377576+0000","last_active":"2024-04-26T22:57:05.913194+0000","last_peered":"2024-04-26T22:57:05.913194+0000","last_clean":"2024-04-26T22:57:05.913194+0000","last_became_active":"2024-04-26T22:55:25.377203+0000","last_became_peered":"2024-04-26T22:55:25.377203+0000","last_unstale":"2024-04-26T22:57:05.913194+0000","last_undegraded":"2024-04-26T22:57:05.913194+0000","last_fullsized":"2024-04-26T22:57:05.913194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:39.361118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.918980+0000","last_change":"2024-04-26T22:55:30.436672+0000","last_active":"2024-04-26T22:57:05.918980+0000","last_peered":"2024-04-26T22:57:05.918980+0000","last_clean":"2024-04-26T22:57:05.918980+0000","last_became_active":"2024-04-26T22:55:30.435178+0000","last_became_peered":"2024-04-26T22:55:30.435178+0000","last_unstale":"2024-04-26T22:57:05.918980+0000","last_undegraded":"2024-04-26T22:57:05.918980+0000","last_fullsized":"2024-04-26T22:57:05.918980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:34.435290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":131,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:24.826424+0000","last_change":"2024-04-26T22:55:24.760965+0000","last_active":"2024-04-26T22:57:24.826424+0000","last_peered":"2024-04-26T22:57:24.826424+0000","last_clean":"2024-04-26T22:57:24.826424+0000","last_became_active":"2024-04-26T22:55:24.760589+0000","last_became_peered":"2024-04-26T22:55:24.760589+0000","last_unstale":"2024-04-26T22:57:24.826424+0000","last_undegraded":"2024-04-26T22:57:24.826424+0000","last_fullsized":"2024-04-26T22:57:24.826424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:25.493209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912960+0000","last_change":"2024-04-26T22:55:25.375433+0000","last_active":"2024-04-26T22:57:05.912960+0000","last_peered":"2024-04-26T22:57:05.912960+0000","last_clean":"2024-04-26T22:57:05.912960+0000","last_became_active":"2024-04-26T22:55:25.375062+0000","last_became_peered":"2024-04-26T22:55:25.375062+0000","last_unstale":"2024-04-26T22:57:05.912960+0000","last_undegraded":"2024-04-26T22:57:05.912960+0000","last_fullsized":"2024-04-26T22:57:05.912960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:55.423666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.914428+0000","last_change":"2024-04-26T22:56:05.818921+0000","last_active":"2024-04-26T22:57:05.914428+0000","last_peered":"2024-04-26T22:57:05.914428+0000","last_clean":"2024-04-26T22:57:05.914428+0000","last_became_active":"2024-04-26T22:56:05.816873+0000","last_became_peered":"2024-04-26T22:56:05.816873+0000","last_unstale":"2024-04-26T22:57:05.914428+0000","last_undegraded":"2024-04-26T22:57:05.914428+0000","last_fullsized":"2024-04-26T22:57:05.914428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:46:57.475468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912603+0000","last_change":"2024-04-26T22:55:24.773551+0000","last_active":"2024-04-26T22:57:05.912603+0000","last_peered":"2024-04-26T22:57:05.912603+0000","last_clean":"2024-04-26T22:57:05.912603+0000","last_became_active":"2024-04-26T22:55:24.773202+0000","last_became_peered":"2024-04-26T22:55:24.773202+0000","last_unstale":"2024-04-26T22:57:05.912603+0000","last_undegraded":"2024-04-26T22:57:05.912603+0000","last_fullsized":"2024-04-26T22:57:05.912603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:08.271682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.223634+0000","last_change":"2024-04-26T22:57:06.223634+0000","last_active":"2024-04-26T22:57:06.223634+0000","last_peered":"2024-04-26T22:57:06.223634+0000","last_clean":"2024-04-26T22:57:06.223634+0000","last_became_active":"2024-04-26T22:57:06.223154+0000","last_became_peered":"2024-04-26T22:57:06.223154+0000","last_unstale":"2024-04-26T22:57:06.223634+0000","last_undegraded":"2024-04-26T22:57:06.223634+0000","last_fullsized":"2024-04-26T22:57:06.223634+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:03:38.391874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.911480+0000","last_change":"2024-04-26T22:55:30.422835+0000","last_active":"2024-04-26T22:57:05.911480+0000","last_peered":"2024-04-26T22:57:05.911480+0000","last_clean":"2024-04-26T22:57:05.911480+0000","last_became_active":"2024-04-26T22:55:30.422349+0000","last_became_peered":"2024-04-26T22:55:30.422349+0000","last_unstale":"2024-04-26T22:57:05.911480+0000","last_undegraded":"2024-04-26T22:57:05.911480+0000","last_fullsized":"2024-04-26T22:57:05.911480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:26:33.613861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913328+0000","last_change":"2024-04-26T22:55:24.756236+0000","last_active":"2024-04-26T22:57:05.913328+0000","last_peered":"2024-04-26T22:57:05.913328+0000","last_clean":"2024-04-26T22:57:05.913328+0000","last_became_active":"2024-04-26T22:55:24.755858+0000","last_became_peered":"2024-04-26T22:55:24.755858+0000","last_unstale":"2024-04-26T22:57:05.913328+0000","last_undegraded":"2024-04-26T22:57:05.913328+0000","last_fullsized":"2024-04-26T22:57:05.913328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:55.257152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328349+0000","last_change":"2024-04-26T22:55:25.374367+0000","last_active":"2024-04-26T22:57:06.328349+0000","last_peered":"2024-04-26T22:57:06.328349+0000","last_clean":"2024-04-26T22:57:06.328349+0000","last_became_active":"2024-04-26T22:55:25.373186+0000","last_became_peered":"2024-04-26T22:55:25.373186+0000","last_unstale":"2024-04-26T22:57:06.328349+0000","last_undegraded":"2024-04-26T22:57:06.328349+0000","last_fullsized":"2024-04-26T22:57:06.328349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:31:55.780924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913108+0000","last_change":"2024-04-26T22:55:30.425818+0000","last_active":"2024-04-26T22:57:05.913108+0000","last_peered":"2024-04-26T22:57:05.913108+0000","last_clean":"2024-04-26T22:57:05.913108+0000","last_became_active":"2024-04-26T22:55:30.425400+0000","last_became_peered":"2024-04-26T22:55:30.425400+0000","last_unstale":"2024-04-26T22:57:05.913108+0000","last_undegraded":"2024-04-26T22:57:05.913108+0000","last_fullsized":"2024-04-26T22:57:05.913108+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:36.856545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.917144+0000","last_change":"2024-04-26T22:55:24.780607+0000","last_active":"2024-04-26T22:57:05.917144+0000","last_peered":"2024-04-26T22:57:05.917144+0000","last_clean":"2024-04-26T22:57:05.917144+0000","last_became_active":"2024-04-26T22:55:24.780271+0000","last_became_peered":"2024-04-26T22:55:24.780271+0000","last_unstale":"2024-04-26T22:57:05.917144+0000","last_undegraded":"2024-04-26T22:57:05.917144+0000","last_fullsized":"2024-04-26T22:57:05.917144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:25.057816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.327513+0000","last_change":"2024-04-26T22:55:25.399709+0000","last_active":"2024-04-26T22:57:06.327513+0000","last_peered":"2024-04-26T22:57:06.327513+0000","last_clean":"2024-04-26T22:57:06.327513+0000","last_became_active":"2024-04-26T22:55:25.399392+0000","last_became_peered":"2024-04-26T22:55:25.399392+0000","last_unstale":"2024-04-26T22:57:06.327513+0000","last_undegraded":"2024-04-26T22:57:06.327513+0000","last_fullsized":"2024-04-26T22:57:06.327513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:22.821809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912413+0000","last_change":"2024-04-26T22:55:30.424070+0000","last_active":"2024-04-26T22:57:05.912413+0000","last_peered":"2024-04-26T22:57:05.912413+0000","last_clean":"2024-04-26T22:57:05.912413+0000","last_became_active":"2024-04-26T22:55:30.423637+0000","last_became_peered":"2024-04-26T22:55:30.423637+0000","last_unstale":"2024-04-26T22:57:05.912413+0000","last_undegraded":"2024-04-26T22:57:05.912413+0000","last_fullsized":"2024-04-26T22:57:05.912413+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:48.020079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912956+0000","last_change":"2024-04-26T22:55:24.771881+0000","last_active":"2024-04-26T22:57:05.912956+0000","last_peered":"2024-04-26T22:57:05.912956+0000","last_clean":"2024-04-26T22:57:05.912956+0000","last_became_active":"2024-04-26T22:55:24.771563+0000","last_became_peered":"2024-04-26T22:55:24.771563+0000","last_unstale":"2024-04-26T22:57:05.912956+0000","last_undegraded":"2024-04-26T22:57:05.912956+0000","last_fullsized":"2024-04-26T22:57:05.912956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:14.306677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.327122+0000","last_change":"2024-04-26T22:55:25.399789+0000","last_active":"2024-04-26T22:57:06.327122+0000","last_peered":"2024-04-26T22:57:06.327122+0000","last_clean":"2024-04-26T22:57:06.327122+0000","last_became_active":"2024-04-26T22:55:25.399493+0000","last_became_peered":"2024-04-26T22:55:25.399493+0000","last_unstale":"2024-04-26T22:57:06.327122+0000","last_undegraded":"2024-04-26T22:57:06.327122+0000","last_fullsized":"2024-04-26T22:57:06.327122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:45.096556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.914439+0000","last_change":"2024-04-26T22:55:30.428107+0000","last_active":"2024-04-26T22:57:05.914439+0000","last_peered":"2024-04-26T22:57:05.914439+0000","last_clean":"2024-04-26T22:57:05.914439+0000","last_became_active":"2024-04-26T22:55:30.421955+0000","last_became_peered":"2024-04-26T22:55:30.421955+0000","last_unstale":"2024-04-26T22:57:05.914439+0000","last_undegraded":"2024-04-26T22:57:05.914439+0000","last_fullsized":"2024-04-26T22:57:05.914439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:52.430809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.911381+0000","last_change":"2024-04-26T22:56:05.720181+0000","last_active":"2024-04-26T22:57:05.911381+0000","last_peered":"2024-04-26T22:57:05.911381+0000","last_clean":"2024-04-26T22:57:05.911381+0000","last_became_active":"2024-04-26T22:56:05.719790+0000","last_became_peered":"2024-04-26T22:56:05.719790+0000","last_unstale":"2024-04-26T22:57:05.911381+0000","last_undegraded":"2024-04-26T22:57:05.911381+0000","last_fullsized":"2024-04-26T22:57:05.911381+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:43.760736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":128,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:23.823624+0000","last_change":"2024-04-26T22:55:24.763998+0000","last_active":"2024-04-26T22:57:23.823624+0000","last_peered":"2024-04-26T22:57:23.823624+0000","last_clean":"2024-04-26T22:57:23.823624+0000","last_became_active":"2024-04-26T22:55:24.763561+0000","last_became_peered":"2024-04-26T22:55:24.763561+0000","last_unstale":"2024-04-26T22:57:23.823624+0000","last_undegraded":"2024-04-26T22:57:23.823624+0000","last_fullsized":"2024-04-26T22:57:23.823624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:25.270310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913323+0000","last_change":"2024-04-26T22:55:30.424930+0000","last_active":"2024-04-26T22:57:05.913323+0000","last_peered":"2024-04-26T22:57:05.913323+0000","last_clean":"2024-04-26T22:57:05.913323+0000","last_became_active":"2024-04-26T22:55:30.424660+0000","last_became_peered":"2024-04-26T22:55:30.424660+0000","last_unstale":"2024-04-26T22:57:05.913323+0000","last_undegraded":"2024-04-26T22:57:05.913323+0000","last_fullsized":"2024-04-26T22:57:05.913323+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:29.483956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.333305+0000","last_change":"2024-04-26T22:55:25.390547+0000","last_active":"2024-04-26T22:57:06.333305+0000","last_peered":"2024-04-26T22:57:06.333305+0000","last_clean":"2024-04-26T22:57:06.333305+0000","last_became_active":"2024-04-26T22:55:25.386566+0000","last_became_peered":"2024-04-26T22:55:25.386566+0000","last_unstale":"2024-04-26T22:57:06.333305+0000","last_undegraded":"2024-04-26T22:57:06.333305+0000","last_fullsized":"2024-04-26T22:57:06.333305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:43.809125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908728+0000","last_change":"2024-04-26T22:55:24.778254+0000","last_active":"2024-04-26T22:57:05.908728+0000","last_peered":"2024-04-26T22:57:05.908728+0000","last_clean":"2024-04-26T22:57:05.908728+0000","last_became_active":"2024-04-26T22:55:24.777850+0000","last_became_peered":"2024-04-26T22:55:24.777850+0000","last_unstale":"2024-04-26T22:57:05.908728+0000","last_undegraded":"2024-04-26T22:57:05.908728+0000","last_fullsized":"2024-04-26T22:57:05.908728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:48.053712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912771+0000","last_change":"2024-04-26T22:53:59.233875+0000","last_active":"2024-04-26T22:57:05.912771+0000","last_peered":"2024-04-26T22:57:05.912771+0000","last_clean":"2024-04-26T22:57:05.912771+0000","last_became_active":"2024-04-26T22:53:59.233488+0000","last_became_peered":"2024-04-26T22:53:59.233488+0000","last_unstale":"2024-04-26T22:57:05.912771+0000","last_undegraded":"2024-04-26T22:57:05.912771+0000","last_fullsized":"2024-04-26T22:57:05.912771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:49:55.319714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:49:55.319714+0000","last_clean_scrub_stamp":"2024-04-26T22:49:55.319714+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:37:32.885519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.910939+0000","last_change":"2024-04-26T22:55:30.427936+0000","last_active":"2024-04-26T22:57:05.910939+0000","last_peered":"2024-04-26T22:57:05.910939+0000","last_clean":"2024-04-26T22:57:05.910939+0000","last_became_active":"2024-04-26T22:55:30.427483+0000","last_became_peered":"2024-04-26T22:55:30.427483+0000","last_unstale":"2024-04-26T22:57:05.910939+0000","last_undegraded":"2024-04-26T22:57:05.910939+0000","last_fullsized":"2024-04-26T22:57:05.910939+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:00.856153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.914101+0000","last_change":"2024-04-26T22:55:25.378277+0000","last_active":"2024-04-26T22:57:05.914101+0000","last_peered":"2024-04-26T22:57:05.914101+0000","last_clean":"2024-04-26T22:57:05.914101+0000","last_became_active":"2024-04-26T22:55:25.377650+0000","last_became_peered":"2024-04-26T22:55:25.377650+0000","last_unstale":"2024-04-26T22:57:05.914101+0000","last_undegraded":"2024-04-26T22:57:05.914101+0000","last_fullsized":"2024-04-26T22:57:05.914101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:47.537306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912961+0000","last_change":"2024-04-26T22:55:24.762439+0000","last_active":"2024-04-26T22:57:05.912961+0000","last_peered":"2024-04-26T22:57:05.912961+0000","last_clean":"2024-04-26T22:57:05.912961+0000","last_became_active":"2024-04-26T22:55:24.760732+0000","last_became_peered":"2024-04-26T22:55:24.760732+0000","last_unstale":"2024-04-26T22:57:05.912961+0000","last_undegraded":"2024-04-26T22:57:05.912961+0000","last_fullsized":"2024-04-26T22:57:05.912961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:33.857025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328134+0000","last_change":"2024-04-26T22:55:30.411883+0000","last_active":"2024-04-26T22:57:06.328134+0000","last_peered":"2024-04-26T22:57:06.328134+0000","last_clean":"2024-04-26T22:57:06.328134+0000","last_became_active":"2024-04-26T22:55:30.411419+0000","last_became_peered":"2024-04-26T22:55:30.411419+0000","last_unstale":"2024-04-26T22:57:06.328134+0000","last_undegraded":"2024-04-26T22:57:06.328134+0000","last_fullsized":"2024-04-26T22:57:06.328134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:14.977947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913297+0000","last_change":"2024-04-26T22:55:24.779481+0000","last_active":"2024-04-26T22:57:05.913297+0000","last_peered":"2024-04-26T22:57:05.913297+0000","last_clean":"2024-04-26T22:57:05.913297+0000","last_became_active":"2024-04-26T22:55:24.779182+0000","last_became_peered":"2024-04-26T22:55:24.779182+0000","last_unstale":"2024-04-26T22:57:05.913297+0000","last_undegraded":"2024-04-26T22:57:05.913297+0000","last_fullsized":"2024-04-26T22:57:05.913297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:52.076060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913178+0000","last_change":"2024-04-26T22:55:25.390938+0000","last_active":"2024-04-26T22:57:05.913178+0000","last_peered":"2024-04-26T22:57:05.913178+0000","last_clean":"2024-04-26T22:57:05.913178+0000","last_became_active":"2024-04-26T22:55:25.390566+0000","last_became_peered":"2024-04-26T22:55:25.390566+0000","last_unstale":"2024-04-26T22:57:05.913178+0000","last_undegraded":"2024-04-26T22:57:05.913178+0000","last_fullsized":"2024-04-26T22:57:05.913178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:48.079024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.330999+0000","last_change":"2024-04-26T22:55:30.430926+0000","last_active":"2024-04-26T22:57:06.330999+0000","last_peered":"2024-04-26T22:57:06.330999+0000","last_clean":"2024-04-26T22:57:06.330999+0000","last_became_active":"2024-04-26T22:55:30.428286+0000","last_became_peered":"2024-04-26T22:55:30.428286+0000","last_unstale":"2024-04-26T22:57:06.330999+0000","last_undegraded":"2024-04-26T22:57:06.330999+0000","last_fullsized":"2024-04-26T22:57:06.330999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:26.095658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.916369+0000","last_change":"2024-04-26T22:55:24.784960+0000","last_active":"2024-04-26T22:57:05.916369+0000","last_peered":"2024-04-26T22:57:05.916369+0000","last_clean":"2024-04-26T22:57:05.916369+0000","last_became_active":"2024-04-26T22:55:24.784550+0000","last_became_peered":"2024-04-26T22:55:24.784550+0000","last_unstale":"2024-04-26T22:57:05.916369+0000","last_undegraded":"2024-04-26T22:57:05.916369+0000","last_fullsized":"2024-04-26T22:57:05.916369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:05.058172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.334186+0000","last_change":"2024-04-26T22:55:25.392352+0000","last_active":"2024-04-26T22:57:06.334186+0000","last_peered":"2024-04-26T22:57:06.334186+0000","last_clean":"2024-04-26T22:57:06.334186+0000","last_became_active":"2024-04-26T22:55:25.391983+0000","last_became_peered":"2024-04-26T22:55:25.391983+0000","last_unstale":"2024-04-26T22:57:06.334186+0000","last_undegraded":"2024-04-26T22:57:06.334186+0000","last_fullsized":"2024-04-26T22:57:06.334186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:37.192419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908098+0000","last_change":"2024-04-26T22:55:30.409951+0000","last_active":"2024-04-26T22:57:05.908098+0000","last_peered":"2024-04-26T22:57:05.908098+0000","last_clean":"2024-04-26T22:57:05.908098+0000","last_became_active":"2024-04-26T22:55:30.409502+0000","last_became_peered":"2024-04-26T22:55:30.409502+0000","last_unstale":"2024-04-26T22:57:05.908098+0000","last_undegraded":"2024-04-26T22:57:05.908098+0000","last_fullsized":"2024-04-26T22:57:05.908098+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:40:48.416697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.909550+0000","last_change":"2024-04-26T22:55:24.770999+0000","last_active":"2024-04-26T22:57:05.909550+0000","last_peered":"2024-04-26T22:57:05.909550+0000","last_clean":"2024-04-26T22:57:05.909550+0000","last_became_active":"2024-04-26T22:55:24.770630+0000","last_became_peered":"2024-04-26T22:55:24.770630+0000","last_unstale":"2024-04-26T22:57:05.909550+0000","last_undegraded":"2024-04-26T22:57:05.909550+0000","last_fullsized":"2024-04-26T22:57:05.909550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:01.651705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.332559+0000","last_change":"2024-04-26T22:55:25.401845+0000","last_active":"2024-04-26T22:57:06.332559+0000","last_peered":"2024-04-26T22:57:06.332559+0000","last_clean":"2024-04-26T22:57:06.332559+0000","last_became_active":"2024-04-26T22:55:25.398759+0000","last_became_peered":"2024-04-26T22:55:25.398759+0000","last_unstale":"2024-04-26T22:57:06.332559+0000","last_undegraded":"2024-04-26T22:57:06.332559+0000","last_fullsized":"2024-04-26T22:57:06.332559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:14.158710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.909088+0000","last_change":"2024-04-26T22:55:30.426206+0000","last_active":"2024-04-26T22:57:05.909088+0000","last_peered":"2024-04-26T22:57:05.909088+0000","last_clean":"2024-04-26T22:57:05.909088+0000","last_became_active":"2024-04-26T22:55:30.425678+0000","last_became_peered":"2024-04-26T22:55:30.425678+0000","last_unstale":"2024-04-26T22:57:05.909088+0000","last_undegraded":"2024-04-26T22:57:05.909088+0000","last_fullsized":"2024-04-26T22:57:05.909088+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:48.913899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.221954+0000","last_change":"2024-04-26T22:55:24.772775+0000","last_active":"2024-04-26T22:57:06.221954+0000","last_peered":"2024-04-26T22:57:06.221954+0000","last_clean":"2024-04-26T22:57:06.221954+0000","last_became_active":"2024-04-26T22:55:24.772434+0000","last_became_peered":"2024-04-26T22:55:24.772434+0000","last_unstale":"2024-04-26T22:57:06.221954+0000","last_undegraded":"2024-04-26T22:57:06.221954+0000","last_fullsized":"2024-04-26T22:57:06.221954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:23.309545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912634+0000","last_change":"2024-04-26T22:55:25.372051+0000","last_active":"2024-04-26T22:57:05.912634+0000","last_peered":"2024-04-26T22:57:05.912634+0000","last_clean":"2024-04-26T22:57:05.912634+0000","last_became_active":"2024-04-26T22:55:25.371681+0000","last_became_peered":"2024-04-26T22:55:25.371681+0000","last_unstale":"2024-04-26T22:57:05.912634+0000","last_undegraded":"2024-04-26T22:57:05.912634+0000","last_fullsized":"2024-04-26T22:57:05.912634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:34:39.412974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.918768+0000","last_change":"2024-04-26T22:55:30.427160+0000","last_active":"2024-04-26T22:57:05.918768+0000","last_peered":"2024-04-26T22:57:05.918768+0000","last_clean":"2024-04-26T22:57:05.918768+0000","last_became_active":"2024-04-26T22:55:30.426555+0000","last_became_peered":"2024-04-26T22:55:30.426555+0000","last_unstale":"2024-04-26T22:57:05.918768+0000","last_undegraded":"2024-04-26T22:57:05.918768+0000","last_fullsized":"2024-04-26T22:57:05.918768+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:31.688532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.332372+0000","last_change":"2024-04-26T22:55:24.785265+0000","last_active":"2024-04-26T22:57:06.332372+0000","last_peered":"2024-04-26T22:57:06.332372+0000","last_clean":"2024-04-26T22:57:06.332372+0000","last_became_active":"2024-04-26T22:55:24.784896+0000","last_became_peered":"2024-04-26T22:55:24.784896+0000","last_unstale":"2024-04-26T22:57:06.332372+0000","last_undegraded":"2024-04-26T22:57:06.332372+0000","last_fullsized":"2024-04-26T22:57:06.332372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:28.815337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.327964+0000","last_change":"2024-04-26T22:55:25.399606+0000","last_active":"2024-04-26T22:57:06.327964+0000","last_peered":"2024-04-26T22:57:06.327964+0000","last_clean":"2024-04-26T22:57:06.327964+0000","last_became_active":"2024-04-26T22:55:25.399195+0000","last_became_peered":"2024-04-26T22:55:25.399195+0000","last_unstale":"2024-04-26T22:57:06.327964+0000","last_undegraded":"2024-04-26T22:57:06.327964+0000","last_fullsized":"2024-04-26T22:57:06.327964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:53:53.618431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908804+0000","last_change":"2024-04-26T22:55:30.427381+0000","last_active":"2024-04-26T22:57:05.908804+0000","last_peered":"2024-04-26T22:57:05.908804+0000","last_clean":"2024-04-26T22:57:05.908804+0000","last_became_active":"2024-04-26T22:55:30.426736+0000","last_became_peered":"2024-04-26T22:55:30.426736+0000","last_unstale":"2024-04-26T22:57:05.908804+0000","last_undegraded":"2024-04-26T22:57:05.908804+0000","last_fullsized":"2024-04-26T22:57:05.908804+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:07.106991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908228+0000","last_change":"2024-04-26T22:55:24.781483+0000","last_active":"2024-04-26T22:57:05.908228+0000","last_peered":"2024-04-26T22:57:05.908228+0000","last_clean":"2024-04-26T22:57:05.908228+0000","last_became_active":"2024-04-26T22:55:24.781174+0000","last_became_peered":"2024-04-26T22:55:24.781174+0000","last_unstale":"2024-04-26T22:57:05.908228+0000","last_undegraded":"2024-04-26T22:57:05.908228+0000","last_fullsized":"2024-04-26T22:57:05.908228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:58:05.550242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.332650+0000","last_change":"2024-04-26T22:55:25.406764+0000","last_active":"2024-04-26T22:57:06.332650+0000","last_peered":"2024-04-26T22:57:06.332650+0000","last_clean":"2024-04-26T22:57:06.332650+0000","last_became_active":"2024-04-26T22:55:25.405948+0000","last_became_peered":"2024-04-26T22:55:25.405948+0000","last_unstale":"2024-04-26T22:57:06.332650+0000","last_undegraded":"2024-04-26T22:57:06.332650+0000","last_fullsized":"2024-04-26T22:57:06.332650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:58.414530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908955+0000","last_change":"2024-04-26T22:55:30.422989+0000","last_active":"2024-04-26T22:57:05.908955+0000","last_peered":"2024-04-26T22:57:05.908955+0000","last_clean":"2024-04-26T22:57:05.908955+0000","last_became_active":"2024-04-26T22:55:30.422529+0000","last_became_peered":"2024-04-26T22:55:30.422529+0000","last_unstale":"2024-04-26T22:57:05.908955+0000","last_undegraded":"2024-04-26T22:57:05.908955+0000","last_fullsized":"2024-04-26T22:57:05.908955+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:08.312667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912755+0000","last_change":"2024-04-26T22:56:05.725131+0000","last_active":"2024-04-26T22:57:05.912755+0000","last_peered":"2024-04-26T22:57:05.912755+0000","last_clean":"2024-04-26T22:57:05.912755+0000","last_became_active":"2024-04-26T22:56:05.719614+0000","last_became_peered":"2024-04-26T22:56:05.719614+0000","last_unstale":"2024-04-26T22:57:05.912755+0000","last_undegraded":"2024-04-26T22:57:05.912755+0000","last_fullsized":"2024-04-26T22:57:05.912755+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:57.965791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912792+0000","last_change":"2024-04-26T22:55:25.371428+0000","last_active":"2024-04-26T22:57:05.912792+0000","last_peered":"2024-04-26T22:57:05.912792+0000","last_clean":"2024-04-26T22:57:05.912792+0000","last_became_active":"2024-04-26T22:55:25.371088+0000","last_became_peered":"2024-04-26T22:55:25.371088+0000","last_unstale":"2024-04-26T22:57:05.912792+0000","last_undegraded":"2024-04-26T22:57:05.912792+0000","last_fullsized":"2024-04-26T22:57:05.912792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:55:15.066665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.221961+0000","last_change":"2024-04-26T22:55:30.411966+0000","last_active":"2024-04-26T22:57:06.221961+0000","last_peered":"2024-04-26T22:57:06.221961+0000","last_clean":"2024-04-26T22:57:06.221961+0000","last_became_active":"2024-04-26T22:55:30.411486+0000","last_became_peered":"2024-04-26T22:55:30.411486+0000","last_unstale":"2024-04-26T22:57:06.221961+0000","last_undegraded":"2024-04-26T22:57:06.221961+0000","last_fullsized":"2024-04-26T22:57:06.221961+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:08.207340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908981+0000","last_change":"2024-04-26T22:55:24.771435+0000","last_active":"2024-04-26T22:57:05.908981+0000","last_peered":"2024-04-26T22:57:05.908981+0000","last_clean":"2024-04-26T22:57:05.908981+0000","last_became_active":"2024-04-26T22:55:24.771156+0000","last_became_peered":"2024-04-26T22:55:24.771156+0000","last_unstale":"2024-04-26T22:57:05.908981+0000","last_undegraded":"2024-04-26T22:57:05.908981+0000","last_fullsized":"2024-04-26T22:57:05.908981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:15.130786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913515+0000","last_change":"2024-04-26T22:55:25.374051+0000","last_active":"2024-04-26T22:57:05.913515+0000","last_peered":"2024-04-26T22:57:05.913515+0000","last_clean":"2024-04-26T22:57:05.913515+0000","last_became_active":"2024-04-26T22:55:25.373348+0000","last_became_peered":"2024-04-26T22:55:25.373348+0000","last_unstale":"2024-04-26T22:57:05.913515+0000","last_undegraded":"2024-04-26T22:57:05.913515+0000","last_fullsized":"2024-04-26T22:57:05.913515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:30:31.116829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.330895+0000","last_change":"2024-04-26T22:56:05.728492+0000","last_active":"2024-04-26T22:57:06.330895+0000","last_peered":"2024-04-26T22:57:06.330895+0000","last_clean":"2024-04-26T22:57:06.330895+0000","last_became_active":"2024-04-26T22:56:05.727863+0000","last_became_peered":"2024-04-26T22:56:05.727863+0000","last_unstale":"2024-04-26T22:57:06.330895+0000","last_undegraded":"2024-04-26T22:57:06.330895+0000","last_fullsized":"2024-04-26T22:57:06.330895+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:39.596089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10,1,4],"acting":[0,10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.911080+0000","last_change":"2024-04-26T22:55:24.788057+0000","last_active":"2024-04-26T22:57:05.911080+0000","last_peered":"2024-04-26T22:57:05.911080+0000","last_clean":"2024-04-26T22:57:05.911080+0000","last_became_active":"2024-04-26T22:55:24.787753+0000","last_became_peered":"2024-04-26T22:55:24.787753+0000","last_unstale":"2024-04-26T22:57:05.911080+0000","last_undegraded":"2024-04-26T22:57:05.911080+0000","last_fullsized":"2024-04-26T22:57:05.911080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:53.054665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913257+0000","last_change":"2024-04-26T22:55:25.400227+0000","last_active":"2024-04-26T22:57:05.913257+0000","last_peered":"2024-04-26T22:57:05.913257+0000","last_clean":"2024-04-26T22:57:05.913257+0000","last_became_active":"2024-04-26T22:55:25.399738+0000","last_became_peered":"2024-04-26T22:55:25.399738+0000","last_unstale":"2024-04-26T22:57:05.913257+0000","last_undegraded":"2024-04-26T22:57:05.913257+0000","last_fullsized":"2024-04-26T22:57:05.913257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:22.224654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.910383+0000","last_change":"2024-04-26T22:55:30.410479+0000","last_active":"2024-04-26T22:57:05.910383+0000","last_peered":"2024-04-26T22:57:05.910383+0000","last_clean":"2024-04-26T22:57:05.910383+0000","last_became_active":"2024-04-26T22:55:30.410049+0000","last_became_peered":"2024-04-26T22:55:30.410049+0000","last_unstale":"2024-04-26T22:57:05.910383+0000","last_undegraded":"2024-04-26T22:57:05.910383+0000","last_fullsized":"2024-04-26T22:57:05.910383+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:04.412005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.914722+0000","last_change":"2024-04-26T22:55:24.780741+0000","last_active":"2024-04-26T22:57:05.914722+0000","last_peered":"2024-04-26T22:57:05.914722+0000","last_clean":"2024-04-26T22:57:05.914722+0000","last_became_active":"2024-04-26T22:55:24.780400+0000","last_became_peered":"2024-04-26T22:55:24.780400+0000","last_unstale":"2024-04-26T22:57:05.914722+0000","last_undegraded":"2024-04-26T22:57:05.914722+0000","last_fullsized":"2024-04-26T22:57:05.914722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:52.274456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913500+0000","last_change":"2024-04-26T22:55:25.369449+0000","last_active":"2024-04-26T22:57:05.913500+0000","last_peered":"2024-04-26T22:57:05.913500+0000","last_clean":"2024-04-26T22:57:05.913500+0000","last_became_active":"2024-04-26T22:55:25.369106+0000","last_became_peered":"2024-04-26T22:55:25.369106+0000","last_unstale":"2024-04-26T22:57:05.913500+0000","last_undegraded":"2024-04-26T22:57:05.913500+0000","last_fullsized":"2024-04-26T22:57:05.913500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:21.347385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328039+0000","last_change":"2024-04-26T22:55:25.371959+0000","last_active":"2024-04-26T22:57:06.328039+0000","last_peered":"2024-04-26T22:57:06.328039+0000","last_clean":"2024-04-26T22:57:06.328039+0000","last_became_active":"2024-04-26T22:55:25.371580+0000","last_became_peered":"2024-04-26T22:55:25.371580+0000","last_unstale":"2024-04-26T22:57:06.328039+0000","last_undegraded":"2024-04-26T22:57:06.328039+0000","last_fullsized":"2024-04-26T22:57:06.328039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:11.001269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.922645+0000","last_change":"2024-04-26T22:57:05.922645+0000","last_active":"2024-04-26T22:57:05.922645+0000","last_peered":"2024-04-26T22:57:05.922645+0000","last_clean":"2024-04-26T22:57:05.922645+0000","last_became_active":"2024-04-26T22:57:05.922233+0000","last_became_peered":"2024-04-26T22:57:05.922233+0000","last_unstale":"2024-04-26T22:57:05.922645+0000","last_undegraded":"2024-04-26T22:57:05.922645+0000","last_fullsized":"2024-04-26T22:57:05.922645+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:26.067214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908198+0000","last_change":"2024-04-26T22:55:25.398778+0000","last_active":"2024-04-26T22:57:05.908198+0000","last_peered":"2024-04-26T22:57:05.908198+0000","last_clean":"2024-04-26T22:57:05.908198+0000","last_became_active":"2024-04-26T22:55:25.398460+0000","last_became_peered":"2024-04-26T22:55:25.398460+0000","last_unstale":"2024-04-26T22:57:05.908198+0000","last_undegraded":"2024-04-26T22:57:05.908198+0000","last_fullsized":"2024-04-26T22:57:05.908198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.873418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.331405+0000","last_change":"2024-04-26T22:55:25.396548+0000","last_active":"2024-04-26T22:57:06.331405+0000","last_peered":"2024-04-26T22:57:06.331405+0000","last_clean":"2024-04-26T22:57:06.331405+0000","last_became_active":"2024-04-26T22:55:25.395983+0000","last_became_peered":"2024-04-26T22:55:25.395983+0000","last_unstale":"2024-04-26T22:57:06.331405+0000","last_undegraded":"2024-04-26T22:57:06.331405+0000","last_fullsized":"2024-04-26T22:57:06.331405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:21:34.690636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913012+0000","last_change":"2024-04-26T22:55:25.396742+0000","last_active":"2024-04-26T22:57:05.913012+0000","last_peered":"2024-04-26T22:57:05.913012+0000","last_clean":"2024-04-26T22:57:05.913012+0000","last_became_active":"2024-04-26T22:55:25.396372+0000","last_became_peered":"2024-04-26T22:55:25.396372+0000","last_unstale":"2024-04-26T22:57:05.913012+0000","last_undegraded":"2024-04-26T22:57:05.913012+0000","last_fullsized":"2024-04-26T22:57:05.913012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:32:12.068358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328401+0000","last_change":"2024-04-26T22:55:25.372059+0000","last_active":"2024-04-26T22:57:06.328401+0000","last_peered":"2024-04-26T22:57:06.328401+0000","last_clean":"2024-04-26T22:57:06.328401+0000","last_became_active":"2024-04-26T22:55:25.371750+0000","last_became_peered":"2024-04-26T22:55:25.371750+0000","last_unstale":"2024-04-26T22:57:06.328401+0000","last_undegraded":"2024-04-26T22:57:06.328401+0000","last_fullsized":"2024-04-26T22:57:06.328401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:10.230585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908647+0000","last_change":"2024-04-26T22:55:25.374904+0000","last_active":"2024-04-26T22:57:05.908647+0000","last_peered":"2024-04-26T22:57:05.908647+0000","last_clean":"2024-04-26T22:57:05.908647+0000","last_became_active":"2024-04-26T22:55:25.374600+0000","last_became_peered":"2024-04-26T22:55:25.374600+0000","last_unstale":"2024-04-26T22:57:05.908647+0000","last_undegraded":"2024-04-26T22:57:05.908647+0000","last_fullsized":"2024-04-26T22:57:05.908647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:49.124981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.925653+0000","last_change":"2024-04-26T22:57:05.922936+0000","last_active":"2024-04-26T22:57:05.925653+0000","last_peered":"2024-04-26T22:57:05.925653+0000","last_clean":"2024-04-26T22:57:05.925653+0000","last_became_active":"2024-04-26T22:57:05.922061+0000","last_became_peered":"2024-04-26T22:57:05.922061+0000","last_unstale":"2024-04-26T22:57:05.925653+0000","last_undegraded":"2024-04-26T22:57:05.925653+0000","last_fullsized":"2024-04-26T22:57:05.925653+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:01.416256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.910627+0000","last_change":"2024-04-26T22:55:25.406584+0000","last_active":"2024-04-26T22:57:05.910627+0000","last_peered":"2024-04-26T22:57:05.910627+0000","last_clean":"2024-04-26T22:57:05.910627+0000","last_became_active":"2024-04-26T22:55:25.406352+0000","last_became_peered":"2024-04-26T22:55:25.406352+0000","last_unstale":"2024-04-26T22:57:05.910627+0000","last_undegraded":"2024-04-26T22:57:05.910627+0000","last_fullsized":"2024-04-26T22:57:05.910627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:43.594712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.332494+0000","last_change":"2024-04-26T22:55:25.399578+0000","last_active":"2024-04-26T22:57:06.332494+0000","last_peered":"2024-04-26T22:57:06.332494+0000","last_clean":"2024-04-26T22:57:06.332494+0000","last_became_active":"2024-04-26T22:55:25.396555+0000","last_became_peered":"2024-04-26T22:55:25.396555+0000","last_unstale":"2024-04-26T22:57:06.332494+0000","last_undegraded":"2024-04-26T22:57:06.332494+0000","last_fullsized":"2024-04-26T22:57:06.332494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:07:50.082978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.222158+0000","last_change":"2024-04-26T22:55:25.393034+0000","last_active":"2024-04-26T22:57:06.222158+0000","last_peered":"2024-04-26T22:57:06.222158+0000","last_clean":"2024-04-26T22:57:06.222158+0000","last_became_active":"2024-04-26T22:55:25.392730+0000","last_became_peered":"2024-04-26T22:55:25.392730+0000","last_unstale":"2024-04-26T22:57:06.222158+0000","last_undegraded":"2024-04-26T22:57:06.222158+0000","last_fullsized":"2024-04-26T22:57:06.222158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:11.040311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.911969+0000","last_change":"2024-04-26T22:55:25.398224+0000","last_active":"2024-04-26T22:57:05.911969+0000","last_peered":"2024-04-26T22:57:05.911969+0000","last_clean":"2024-04-26T22:57:05.911969+0000","last_became_active":"2024-04-26T22:55:25.397744+0000","last_became_peered":"2024-04-26T22:55:25.397744+0000","last_unstale":"2024-04-26T22:57:05.911969+0000","last_undegraded":"2024-04-26T22:57:05.911969+0000","last_fullsized":"2024-04-26T22:57:05.911969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:35.107942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328225+0000","last_change":"2024-04-26T22:55:25.374076+0000","last_active":"2024-04-26T22:57:06.328225+0000","last_peered":"2024-04-26T22:57:06.328225+0000","last_clean":"2024-04-26T22:57:06.328225+0000","last_became_active":"2024-04-26T22:55:25.373736+0000","last_became_peered":"2024-04-26T22:55:25.373736+0000","last_unstale":"2024-04-26T22:57:06.328225+0000","last_undegraded":"2024-04-26T22:57:06.328225+0000","last_fullsized":"2024-04-26T22:57:06.328225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:01:56.096004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282639,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59677,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":577,"num_read_kb":352,"num_write":165,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":179031,"data_compressed":5520,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383720,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":528241,"data_compressed":26744,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645357,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1167360,"data_stored":1171576,"data_compressed":64530,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808883,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":532573,"data_compressed":26744,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070522,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":528397,"data_compressed":26744,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":1124306,"data_compressed":62690,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":523653,"data_compressed":26744,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":579239,"data_compressed":28584,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":525041,"data_compressed":26744,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":580589,"data_compressed":28584,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182486,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":523847,"data_compressed":26744,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":528694,"data_compressed":26744,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757323,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1105920,"data_stored":1119503,"data_compressed":62690,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34303,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34303,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34303,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":55676,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":57026,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":57645,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:57:31.380 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T22:57:31.894 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:57:32.326 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:32 smithi027 ceph-mon[28034]: from='client.25072 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:57:32.326 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:32 smithi027 ceph-mon[28034]: pgmap v397: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:57:32.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:32 smithi080 ceph-mon[32324]: from='client.25072 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:57:32.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:32 smithi080 ceph-mon[32324]: pgmap v397: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:57:32.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:32 smithi094 ceph-mon[32889]: from='client.25072 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:57:32.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:32 smithi094 ceph-mon[32889]: pgmap v397: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:57:34.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:34 smithi027 ceph-mon[28034]: pgmap v398: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:57:34.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.476+0000 7f43e7800700 1 -- 172.21.15.27:0/2139859008 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43e0101800 msgr2=0x7f43e0101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:34.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.476+0000 7f43e7800700 1 --2- 172.21.15.27:0/2139859008 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43e0101800 0x7f43e0101be0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009990 tx=0x7f43d4009ca0 comp rx=0 tx=0).stop 2024-04-26T22:57:34.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.477+0000 7f43e7800700 1 -- 172.21.15.27:0/2139859008 shutdown_connections 2024-04-26T22:57:34.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.477+0000 7f43e7800700 1 --2- 172.21.15.27:0/2139859008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43e01098e0 0x7f43e0110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:34.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.477+0000 7f43e7800700 1 --2- 172.21.15.27:0/2139859008 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43e0101800 0x7f43e0101be0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:34.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.477+0000 7f43e7800700 1 --2- 172.21.15.27:0/2139859008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43e01021b0 0x7f43e0109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:34.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.477+0000 7f43e7800700 1 -- 172.21.15.27:0/2139859008 >> 172.21.15.27:0/2139859008 conn(0x7f43e0078520 msgr2=0x7f43e00ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:34.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.477+0000 7f43e7800700 1 -- 172.21.15.27:0/2139859008 shutdown_connections 2024-04-26T22:57:34.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.477+0000 7f43e7800700 1 -- 172.21.15.27:0/2139859008 wait complete. 2024-04-26T22:57:34.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.478+0000 7f43e7800700 1 Processor -- start 2024-04-26T22:57:34.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.478+0000 7f43e7800700 1 -- start start 2024-04-26T22:57:34.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.478+0000 7f43e7800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43e0101800 0x7f43e019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:34.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.478+0000 7f43e7800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43e01021b0 0x7f43e019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:34.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.478+0000 7f43e7800700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43e01098e0 0x7f43e0199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:34.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.478+0000 7f43e7800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43e01135d0 con 0x7f43e01021b0 2024-04-26T22:57:34.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.478+0000 7f43e7800700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f43e0113450 con 0x7f43e01098e0 2024-04-26T22:57:34.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.478+0000 7f43e7800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f43e0113750 con 0x7f43e0101800 2024-04-26T22:57:34.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.479+0000 7f43dfe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43e01021b0 0x7f43e019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:34.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.479+0000 7f43dfe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43e01021b0 0x7f43e019fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39326/0 (socket says 172.21.15.27:39326) 2024-04-26T22:57:34.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.479+0000 7f43dfe00700 1 -- 172.21.15.27:0/3989389730 learned_addr learned my addr 172.21.15.27:0/3989389730 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:34.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.479+0000 7f43e4e00700 1 --2- 172.21.15.27:0/3989389730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43e0101800 0x7f43e019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:34.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.479+0000 7f43e5800700 1 --2- 172.21.15.27:0/3989389730 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43e01098e0 0x7f43e0199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:34.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.479+0000 7f43dfe00700 1 -- 172.21.15.27:0/3989389730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43e0101800 msgr2=0x7f43e019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:34.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.479+0000 7f43dfe00700 1 --2- 172.21.15.27:0/3989389730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43e0101800 0x7f43e019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:34.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.479+0000 7f43dfe00700 1 -- 172.21.15.27:0/3989389730 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43e01098e0 msgr2=0x7f43e0199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:34.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.479+0000 7f43dfe00700 1 --2- 172.21.15.27:0/3989389730 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43e01098e0 0x7f43e0199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:34.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.479+0000 7f43dfe00700 1 -- 172.21.15.27:0/3989389730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e019a0f0 con 0x7f43e01021b0 2024-04-26T22:57:34.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.479+0000 7f43e4e00700 1 --2- 172.21.15.27:0/3989389730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43e0101800 0x7f43e019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:34.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.479+0000 7f43e5800700 1 --2- 172.21.15.27:0/3989389730 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43e01098e0 0x7f43e0199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:34.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.480+0000 7f43dfe00700 1 --2- 172.21.15.27:0/3989389730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43e01021b0 0x7f43e019fc70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00ea30 tx=0x7f43cc00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:34.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.480+0000 7f43dd600700 1 -- 172.21.15.27:0/3989389730 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc00cbd0 con 0x7f43e01021b0 2024-04-26T22:57:34.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.480+0000 7f43e7800700 1 -- 172.21.15.27:0/3989389730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e019a3e0 con 0x7f43e01021b0 2024-04-26T22:57:34.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.480+0000 7f43dd600700 1 -- 172.21.15.27:0/3989389730 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43cc0044d0 con 0x7f43e01021b0 2024-04-26T22:57:34.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.480+0000 7f43dd600700 1 -- 172.21.15.27:0/3989389730 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc003dc0 con 0x7f43e01021b0 2024-04-26T22:57:34.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.480+0000 7f43e7800700 1 -- 172.21.15.27:0/3989389730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e01a6600 con 0x7f43e01021b0 2024-04-26T22:57:34.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.481+0000 7f43e7800700 1 -- 172.21.15.27:0/3989389730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e0102930 con 0x7f43e01021b0 2024-04-26T22:57:34.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.484+0000 7f43dd600700 1 -- 172.21.15.27:0/3989389730 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f43cc003fa0 con 0x7f43e01021b0 2024-04-26T22:57:34.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.485+0000 7f43dd600700 1 --2- 172.21.15.27:0/3989389730 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f43c80a3cd0 0x7f43c80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:34.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.485+0000 7f43dd600700 1 -- 172.21.15.27:0/3989389730 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f43cc0d0b70 con 0x7f43e01021b0 2024-04-26T22:57:34.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.485+0000 7f43e4e00700 1 --2- 172.21.15.27:0/3989389730 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f43c80a3cd0 0x7f43c80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:34.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.486+0000 7f43e4e00700 1 --2- 172.21.15.27:0/3989389730 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f43c80a3cd0 0x7f43c80a6190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f43e0109420 tx=0x7f43d4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:34.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.492+0000 7f43dd600700 1 -- 172.21.15.27:0/3989389730 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f43cc09cf60 con 0x7f43e01021b0 2024-04-26T22:57:34.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:34 smithi080 ceph-mon[32324]: pgmap v398: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:57:34.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:34 smithi094 ceph-mon[32889]: pgmap v398: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:57:34.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.718+0000 7f43e7800700 1 -- 172.21.15.27:0/3989389730 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f43e0063860 con 0x7f43c80a3cd0 2024-04-26T22:57:34.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.748+0000 7f43dd600700 1 -- 172.21.15.27:0/3989389730 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340679 (secure 0 0 0) 0x7f43e0063860 con 0x7f43c80a3cd0 2024-04-26T22:57:34.756 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:57:34.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.757+0000 7f43e7800700 1 -- 172.21.15.27:0/3989389730 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f43c80a3cd0 msgr2=0x7f43c80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:34.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.757+0000 7f43e7800700 1 --2- 172.21.15.27:0/3989389730 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f43c80a3cd0 0x7f43c80a6190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f43e0109420 tx=0x7f43d4005960 comp rx=0 tx=0).stop 2024-04-26T22:57:34.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.757+0000 7f43e7800700 1 -- 172.21.15.27:0/3989389730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43e01021b0 msgr2=0x7f43e019fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:34.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.757+0000 7f43e7800700 1 --2- 172.21.15.27:0/3989389730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43e01021b0 0x7f43e019fc70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00ea30 tx=0x7f43cc00ed40 comp rx=0 tx=0).stop 2024-04-26T22:57:34.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.758+0000 7f43e7800700 1 -- 172.21.15.27:0/3989389730 shutdown_connections 2024-04-26T22:57:34.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.758+0000 7f43e7800700 1 --2- 172.21.15.27:0/3989389730 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f43c80a3cd0 0x7f43c80a6190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:34.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.758+0000 7f43e7800700 1 --2- 172.21.15.27:0/3989389730 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43e01098e0 0x7f43e0199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:34.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.758+0000 7f43e7800700 1 --2- 172.21.15.27:0/3989389730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43e0101800 0x7f43e019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:34.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.758+0000 7f43e7800700 1 --2- 172.21.15.27:0/3989389730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43e01021b0 0x7f43e019fc70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:34.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.758+0000 7f43e7800700 1 -- 172.21.15.27:0/3989389730 >> 172.21.15.27:0/3989389730 conn(0x7f43e0078520 msgr2=0x7f43e0107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:34.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.758+0000 7f43e7800700 1 -- 172.21.15.27:0/3989389730 shutdown_connections 2024-04-26T22:57:34.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:34.758+0000 7f43e7800700 1 -- 172.21.15.27:0/3989389730 wait complete. 2024-04-26T22:57:34.760 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-26T22:57:35.740 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-04-26T22:57:34.504199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650279,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":743,"num_read_kb":486,"num_write":348,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382568,"kb_used_data":12560,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559256,"statfs":{"total":1151940427776,"available":1151548678144,"internally_reserved":0,"allocated":12861440,"data_stored":8265659,"data_compressed":434286,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005466"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328754+0000","last_change":"2024-04-26T22:55:25.404468+0000","last_active":"2024-04-26T22:57:06.328754+0000","last_peered":"2024-04-26T22:57:06.328754+0000","last_clean":"2024-04-26T22:57:06.328754+0000","last_became_active":"2024-04-26T22:55:25.404178+0000","last_became_peered":"2024-04-26T22:55:25.404178+0000","last_unstale":"2024-04-26T22:57:06.328754+0000","last_undegraded":"2024-04-26T22:57:06.328754+0000","last_fullsized":"2024-04-26T22:57:06.328754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:46.249569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.332572+0000","last_change":"2024-04-26T22:55:25.401044+0000","last_active":"2024-04-26T22:57:06.332572+0000","last_peered":"2024-04-26T22:57:06.332572+0000","last_clean":"2024-04-26T22:57:06.332572+0000","last_became_active":"2024-04-26T22:55:25.397925+0000","last_became_peered":"2024-04-26T22:55:25.397925+0000","last_unstale":"2024-04-26T22:57:06.332572+0000","last_undegraded":"2024-04-26T22:57:06.332572+0000","last_fullsized":"2024-04-26T22:57:06.332572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:28.503480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.331899+0000","last_change":"2024-04-26T22:55:25.396780+0000","last_active":"2024-04-26T22:57:06.331899+0000","last_peered":"2024-04-26T22:57:06.331899+0000","last_clean":"2024-04-26T22:57:06.331899+0000","last_became_active":"2024-04-26T22:55:25.395401+0000","last_became_peered":"2024-04-26T22:55:25.395401+0000","last_unstale":"2024-04-26T22:57:06.331899+0000","last_undegraded":"2024-04-26T22:57:06.331899+0000","last_fullsized":"2024-04-26T22:57:06.331899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:32.295042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.924904+0000","last_change":"2024-04-26T22:57:05.922677+0000","last_active":"2024-04-26T22:57:05.924904+0000","last_peered":"2024-04-26T22:57:05.924904+0000","last_clean":"2024-04-26T22:57:05.924904+0000","last_became_active":"2024-04-26T22:57:05.921292+0000","last_became_peered":"2024-04-26T22:57:05.921292+0000","last_unstale":"2024-04-26T22:57:05.924904+0000","last_undegraded":"2024-04-26T22:57:05.924904+0000","last_fullsized":"2024-04-26T22:57:05.924904+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:57.108735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912809+0000","last_change":"2024-04-26T22:55:25.391341+0000","last_active":"2024-04-26T22:57:05.912809+0000","last_peered":"2024-04-26T22:57:05.912809+0000","last_clean":"2024-04-26T22:57:05.912809+0000","last_became_active":"2024-04-26T22:55:25.391061+0000","last_became_peered":"2024-04-26T22:55:25.391061+0000","last_unstale":"2024-04-26T22:57:05.912809+0000","last_undegraded":"2024-04-26T22:57:05.912809+0000","last_fullsized":"2024-04-26T22:57:05.912809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:08.391492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.327082+0000","last_change":"2024-04-26T22:55:25.388412+0000","last_active":"2024-04-26T22:57:06.327082+0000","last_peered":"2024-04-26T22:57:06.327082+0000","last_clean":"2024-04-26T22:57:06.327082+0000","last_became_active":"2024-04-26T22:55:25.388010+0000","last_became_peered":"2024-04-26T22:55:25.388010+0000","last_unstale":"2024-04-26T22:57:06.327082+0000","last_undegraded":"2024-04-26T22:57:06.327082+0000","last_fullsized":"2024-04-26T22:57:06.327082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:45.851109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.924654+0000","last_change":"2024-04-26T22:57:05.924654+0000","last_active":"2024-04-26T22:57:05.924654+0000","last_peered":"2024-04-26T22:57:05.924654+0000","last_clean":"2024-04-26T22:57:05.924654+0000","last_became_active":"2024-04-26T22:57:05.924231+0000","last_became_peered":"2024-04-26T22:57:05.924231+0000","last_unstale":"2024-04-26T22:57:05.924654+0000","last_undegraded":"2024-04-26T22:57:05.924654+0000","last_fullsized":"2024-04-26T22:57:05.924654+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:44.145297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.330003+0000","last_change":"2024-04-26T22:55:25.408952+0000","last_active":"2024-04-26T22:57:06.330003+0000","last_peered":"2024-04-26T22:57:06.330003+0000","last_clean":"2024-04-26T22:57:06.330003+0000","last_became_active":"2024-04-26T22:55:25.408485+0000","last_became_peered":"2024-04-26T22:55:25.408485+0000","last_unstale":"2024-04-26T22:57:06.330003+0000","last_undegraded":"2024-04-26T22:57:06.330003+0000","last_fullsized":"2024-04-26T22:57:06.330003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.043114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.222603+0000","last_change":"2024-04-26T22:55:25.377377+0000","last_active":"2024-04-26T22:57:06.222603+0000","last_peered":"2024-04-26T22:57:06.222603+0000","last_clean":"2024-04-26T22:57:06.222603+0000","last_became_active":"2024-04-26T22:55:25.377015+0000","last_became_peered":"2024-04-26T22:55:25.377015+0000","last_unstale":"2024-04-26T22:57:06.222603+0000","last_undegraded":"2024-04-26T22:57:06.222603+0000","last_fullsized":"2024-04-26T22:57:06.222603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:25.676469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.327710+0000","last_change":"2024-04-26T22:55:25.375549+0000","last_active":"2024-04-26T22:57:06.327710+0000","last_peered":"2024-04-26T22:57:06.327710+0000","last_clean":"2024-04-26T22:57:06.327710+0000","last_became_active":"2024-04-26T22:55:25.375232+0000","last_became_peered":"2024-04-26T22:55:25.375232+0000","last_unstale":"2024-04-26T22:57:06.327710+0000","last_undegraded":"2024-04-26T22:57:06.327710+0000","last_fullsized":"2024-04-26T22:57:06.327710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:22.561623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913442+0000","last_change":"2024-04-26T22:55:25.374224+0000","last_active":"2024-04-26T22:57:05.913442+0000","last_peered":"2024-04-26T22:57:05.913442+0000","last_clean":"2024-04-26T22:57:05.913442+0000","last_became_active":"2024-04-26T22:55:25.373813+0000","last_became_peered":"2024-04-26T22:55:25.373813+0000","last_unstale":"2024-04-26T22:57:05.913442+0000","last_undegraded":"2024-04-26T22:57:05.913442+0000","last_fullsized":"2024-04-26T22:57:05.913442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:34.874888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.909140+0000","last_change":"2024-04-26T22:55:25.376666+0000","last_active":"2024-04-26T22:57:05.909140+0000","last_peered":"2024-04-26T22:57:05.909140+0000","last_clean":"2024-04-26T22:57:05.909140+0000","last_became_active":"2024-04-26T22:55:25.376328+0000","last_became_peered":"2024-04-26T22:55:25.376328+0000","last_unstale":"2024-04-26T22:57:05.909140+0000","last_undegraded":"2024-04-26T22:57:05.909140+0000","last_fullsized":"2024-04-26T22:57:05.909140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:32:34.287984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.911680+0000","last_change":"2024-04-26T22:55:25.377220+0000","last_active":"2024-04-26T22:57:05.911680+0000","last_peered":"2024-04-26T22:57:05.911680+0000","last_clean":"2024-04-26T22:57:05.911680+0000","last_became_active":"2024-04-26T22:55:25.376637+0000","last_became_peered":"2024-04-26T22:55:25.376637+0000","last_unstale":"2024-04-26T22:57:05.911680+0000","last_undegraded":"2024-04-26T22:57:05.911680+0000","last_fullsized":"2024-04-26T22:57:05.911680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:57.934912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.330416+0000","last_change":"2024-04-26T22:55:25.404669+0000","last_active":"2024-04-26T22:57:06.330416+0000","last_peered":"2024-04-26T22:57:06.330416+0000","last_clean":"2024-04-26T22:57:06.330416+0000","last_became_active":"2024-04-26T22:55:25.402804+0000","last_became_peered":"2024-04-26T22:55:25.402804+0000","last_unstale":"2024-04-26T22:57:06.330416+0000","last_undegraded":"2024-04-26T22:57:06.330416+0000","last_fullsized":"2024-04-26T22:57:06.330416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:37.239667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.330769+0000","last_change":"2024-04-26T22:55:25.399877+0000","last_active":"2024-04-26T22:57:06.330769+0000","last_peered":"2024-04-26T22:57:06.330769+0000","last_clean":"2024-04-26T22:57:06.330769+0000","last_became_active":"2024-04-26T22:55:25.397476+0000","last_became_peered":"2024-04-26T22:55:25.397476+0000","last_unstale":"2024-04-26T22:57:06.330769+0000","last_undegraded":"2024-04-26T22:57:06.330769+0000","last_fullsized":"2024-04-26T22:57:06.330769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:26.732300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.221950+0000","last_change":"2024-04-26T22:55:25.393682+0000","last_active":"2024-04-26T22:57:06.221950+0000","last_peered":"2024-04-26T22:57:06.221950+0000","last_clean":"2024-04-26T22:57:06.221950+0000","last_became_active":"2024-04-26T22:55:25.393349+0000","last_became_peered":"2024-04-26T22:55:25.393349+0000","last_unstale":"2024-04-26T22:57:06.221950+0000","last_undegraded":"2024-04-26T22:57:06.221950+0000","last_fullsized":"2024-04-26T22:57:06.221950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:48.065904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328473+0000","last_change":"2024-04-26T22:55:25.404449+0000","last_active":"2024-04-26T22:57:06.328473+0000","last_peered":"2024-04-26T22:57:06.328473+0000","last_clean":"2024-04-26T22:57:06.328473+0000","last_became_active":"2024-04-26T22:55:25.404170+0000","last_became_peered":"2024-04-26T22:55:25.404170+0000","last_unstale":"2024-04-26T22:57:06.328473+0000","last_undegraded":"2024-04-26T22:57:06.328473+0000","last_fullsized":"2024-04-26T22:57:06.328473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:04.731208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.917737+0000","last_change":"2024-04-26T22:55:25.390368+0000","last_active":"2024-04-26T22:57:05.917737+0000","last_peered":"2024-04-26T22:57:05.917737+0000","last_clean":"2024-04-26T22:57:05.917737+0000","last_became_active":"2024-04-26T22:55:25.389201+0000","last_became_peered":"2024-04-26T22:55:25.389201+0000","last_unstale":"2024-04-26T22:57:05.917737+0000","last_undegraded":"2024-04-26T22:57:05.917737+0000","last_fullsized":"2024-04-26T22:57:05.917737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:35.985419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908616+0000","last_change":"2024-04-26T22:55:25.387616+0000","last_active":"2024-04-26T22:57:05.908616+0000","last_peered":"2024-04-26T22:57:05.908616+0000","last_clean":"2024-04-26T22:57:05.908616+0000","last_became_active":"2024-04-26T22:55:25.387360+0000","last_became_peered":"2024-04-26T22:55:25.387360+0000","last_unstale":"2024-04-26T22:57:05.908616+0000","last_undegraded":"2024-04-26T22:57:05.908616+0000","last_fullsized":"2024-04-26T22:57:05.908616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:49.378287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908302+0000","last_change":"2024-04-26T22:55:30.425407+0000","last_active":"2024-04-26T22:57:05.908302+0000","last_peered":"2024-04-26T22:57:05.908302+0000","last_clean":"2024-04-26T22:57:05.908302+0000","last_became_active":"2024-04-26T22:55:30.424796+0000","last_became_peered":"2024-04-26T22:55:30.424796+0000","last_unstale":"2024-04-26T22:57:05.908302+0000","last_undegraded":"2024-04-26T22:57:05.908302+0000","last_fullsized":"2024-04-26T22:57:05.908302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:42.539415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.917648+0000","last_change":"2024-04-26T22:55:24.776253+0000","last_active":"2024-04-26T22:57:05.917648+0000","last_peered":"2024-04-26T22:57:05.917648+0000","last_clean":"2024-04-26T22:57:05.917648+0000","last_became_active":"2024-04-26T22:55:24.775700+0000","last_became_peered":"2024-04-26T22:55:24.775700+0000","last_unstale":"2024-04-26T22:57:05.917648+0000","last_undegraded":"2024-04-26T22:57:05.917648+0000","last_fullsized":"2024-04-26T22:57:05.917648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:45.608363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.333737+0000","last_change":"2024-04-26T22:55:25.390690+0000","last_active":"2024-04-26T22:57:06.333737+0000","last_peered":"2024-04-26T22:57:06.333737+0000","last_clean":"2024-04-26T22:57:06.333737+0000","last_became_active":"2024-04-26T22:55:25.390289+0000","last_became_peered":"2024-04-26T22:55:25.390289+0000","last_unstale":"2024-04-26T22:57:06.333737+0000","last_undegraded":"2024-04-26T22:57:06.333737+0000","last_fullsized":"2024-04-26T22:57:06.333737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:54.637992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908485+0000","last_change":"2024-04-26T22:55:30.433555+0000","last_active":"2024-04-26T22:57:05.908485+0000","last_peered":"2024-04-26T22:57:05.908485+0000","last_clean":"2024-04-26T22:57:05.908485+0000","last_became_active":"2024-04-26T22:55:30.433240+0000","last_became_peered":"2024-04-26T22:55:30.433240+0000","last_unstale":"2024-04-26T22:57:05.908485+0000","last_undegraded":"2024-04-26T22:57:05.908485+0000","last_fullsized":"2024-04-26T22:57:05.908485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:55:21.215073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.333901+0000","last_change":"2024-04-26T22:55:24.788218+0000","last_active":"2024-04-26T22:57:06.333901+0000","last_peered":"2024-04-26T22:57:06.333901+0000","last_clean":"2024-04-26T22:57:06.333901+0000","last_became_active":"2024-04-26T22:55:24.787762+0000","last_became_peered":"2024-04-26T22:55:24.787762+0000","last_unstale":"2024-04-26T22:57:06.333901+0000","last_undegraded":"2024-04-26T22:57:06.333901+0000","last_fullsized":"2024-04-26T22:57:06.333901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:42.264617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.331820+0000","last_change":"2024-04-26T22:55:25.400356+0000","last_active":"2024-04-26T22:57:06.331820+0000","last_peered":"2024-04-26T22:57:06.331820+0000","last_clean":"2024-04-26T22:57:06.331820+0000","last_became_active":"2024-04-26T22:55:25.399907+0000","last_became_peered":"2024-04-26T22:55:25.399907+0000","last_unstale":"2024-04-26T22:57:06.331820+0000","last_undegraded":"2024-04-26T22:57:06.331820+0000","last_fullsized":"2024-04-26T22:57:06.331820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:50:50.804112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908697+0000","last_change":"2024-04-26T22:56:05.816337+0000","last_active":"2024-04-26T22:57:05.908697+0000","last_peered":"2024-04-26T22:57:05.908697+0000","last_clean":"2024-04-26T22:57:05.908697+0000","last_became_active":"2024-04-26T22:56:05.815846+0000","last_became_peered":"2024-04-26T22:56:05.815846+0000","last_unstale":"2024-04-26T22:57:05.908697+0000","last_undegraded":"2024-04-26T22:57:05.908697+0000","last_fullsized":"2024-04-26T22:57:05.908697+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:41.793454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328372+0000","last_change":"2024-04-26T22:56:05.732306+0000","last_active":"2024-04-26T22:57:06.328372+0000","last_peered":"2024-04-26T22:57:06.328372+0000","last_clean":"2024-04-26T22:57:06.328372+0000","last_became_active":"2024-04-26T22:56:05.722564+0000","last_became_peered":"2024-04-26T22:56:05.722564+0000","last_unstale":"2024-04-26T22:57:06.328372+0000","last_undegraded":"2024-04-26T22:57:06.328372+0000","last_fullsized":"2024-04-26T22:57:06.328372+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:28.784664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.222308+0000","last_change":"2024-04-26T22:55:25.381896+0000","last_active":"2024-04-26T22:57:06.222308+0000","last_peered":"2024-04-26T22:57:06.222308+0000","last_clean":"2024-04-26T22:57:06.222308+0000","last_became_active":"2024-04-26T22:55:25.381411+0000","last_became_peered":"2024-04-26T22:55:25.381411+0000","last_unstale":"2024-04-26T22:57:06.222308+0000","last_undegraded":"2024-04-26T22:57:06.222308+0000","last_fullsized":"2024-04-26T22:57:06.222308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:54.731407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908578+0000","last_change":"2024-04-26T22:56:05.816353+0000","last_active":"2024-04-26T22:57:05.908578+0000","last_peered":"2024-04-26T22:57:05.908578+0000","last_clean":"2024-04-26T22:57:05.908578+0000","last_became_active":"2024-04-26T22:56:05.815871+0000","last_became_peered":"2024-04-26T22:56:05.815871+0000","last_unstale":"2024-04-26T22:57:05.908578+0000","last_undegraded":"2024-04-26T22:57:05.908578+0000","last_fullsized":"2024-04-26T22:57:05.908578+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:24.383516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912819+0000","last_change":"2024-04-26T22:55:24.756017+0000","last_active":"2024-04-26T22:57:05.912819+0000","last_peered":"2024-04-26T22:57:05.912819+0000","last_clean":"2024-04-26T22:57:05.912819+0000","last_became_active":"2024-04-26T22:55:24.755661+0000","last_became_peered":"2024-04-26T22:55:24.755661+0000","last_unstale":"2024-04-26T22:57:05.912819+0000","last_undegraded":"2024-04-26T22:57:05.912819+0000","last_fullsized":"2024-04-26T22:57:05.912819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:53.909567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.910896+0000","last_change":"2024-04-26T22:55:25.375929+0000","last_active":"2024-04-26T22:57:05.910896+0000","last_peered":"2024-04-26T22:57:05.910896+0000","last_clean":"2024-04-26T22:57:05.910896+0000","last_became_active":"2024-04-26T22:55:25.375435+0000","last_became_peered":"2024-04-26T22:55:25.375435+0000","last_unstale":"2024-04-26T22:57:05.910896+0000","last_undegraded":"2024-04-26T22:57:05.910896+0000","last_fullsized":"2024-04-26T22:57:05.910896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:43.336889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908743+0000","last_change":"2024-04-26T22:55:30.409074+0000","last_active":"2024-04-26T22:57:05.908743+0000","last_peered":"2024-04-26T22:57:05.908743+0000","last_clean":"2024-04-26T22:57:05.908743+0000","last_became_active":"2024-04-26T22:55:30.408613+0000","last_became_peered":"2024-04-26T22:55:30.408613+0000","last_unstale":"2024-04-26T22:57:05.908743+0000","last_undegraded":"2024-04-26T22:57:05.908743+0000","last_fullsized":"2024-04-26T22:57:05.908743+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:24.312971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":135,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:28.802533+0000","last_change":"2024-04-26T22:55:24.763787+0000","last_active":"2024-04-26T22:57:28.802533+0000","last_peered":"2024-04-26T22:57:28.802533+0000","last_clean":"2024-04-26T22:57:28.802533+0000","last_became_active":"2024-04-26T22:55:24.763480+0000","last_became_peered":"2024-04-26T22:55:24.763480+0000","last_unstale":"2024-04-26T22:57:28.802533+0000","last_undegraded":"2024-04-26T22:57:28.802533+0000","last_fullsized":"2024-04-26T22:57:28.802533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:18.591009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.924534+0000","last_change":"2024-04-26T22:57:05.924534+0000","last_active":"2024-04-26T22:57:05.924534+0000","last_peered":"2024-04-26T22:57:05.924534+0000","last_clean":"2024-04-26T22:57:05.924534+0000","last_became_active":"2024-04-26T22:57:05.924142+0000","last_became_peered":"2024-04-26T22:57:05.924142+0000","last_unstale":"2024-04-26T22:57:05.924534+0000","last_undegraded":"2024-04-26T22:57:05.924534+0000","last_fullsized":"2024-04-26T22:57:05.924534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:31:00.380807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908805+0000","last_change":"2024-04-26T22:55:30.425891+0000","last_active":"2024-04-26T22:57:05.908805+0000","last_peered":"2024-04-26T22:57:05.908805+0000","last_clean":"2024-04-26T22:57:05.908805+0000","last_became_active":"2024-04-26T22:55:30.424742+0000","last_became_peered":"2024-04-26T22:55:30.424742+0000","last_unstale":"2024-04-26T22:57:05.908805+0000","last_undegraded":"2024-04-26T22:57:05.908805+0000","last_fullsized":"2024-04-26T22:57:05.908805+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:04.876670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.327018+0000","last_change":"2024-04-26T22:56:05.736174+0000","last_active":"2024-04-26T22:57:06.327018+0000","last_peered":"2024-04-26T22:57:06.327018+0000","last_clean":"2024-04-26T22:57:06.327018+0000","last_became_active":"2024-04-26T22:56:05.732271+0000","last_became_peered":"2024-04-26T22:56:05.732271+0000","last_unstale":"2024-04-26T22:57:06.327018+0000","last_undegraded":"2024-04-26T22:57:06.327018+0000","last_fullsized":"2024-04-26T22:57:06.327018+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:41.980613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.910982+0000","last_change":"2024-04-26T22:55:25.399892+0000","last_active":"2024-04-26T22:57:05.910982+0000","last_peered":"2024-04-26T22:57:05.910982+0000","last_clean":"2024-04-26T22:57:05.910982+0000","last_became_active":"2024-04-26T22:55:25.399515+0000","last_became_peered":"2024-04-26T22:55:25.399515+0000","last_unstale":"2024-04-26T22:57:05.910982+0000","last_undegraded":"2024-04-26T22:57:05.910982+0000","last_fullsized":"2024-04-26T22:57:05.910982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.472103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328465+0000","last_change":"2024-04-26T22:55:30.422932+0000","last_active":"2024-04-26T22:57:06.328465+0000","last_peered":"2024-04-26T22:57:06.328465+0000","last_clean":"2024-04-26T22:57:06.328465+0000","last_became_active":"2024-04-26T22:55:30.422481+0000","last_became_peered":"2024-04-26T22:55:30.422481+0000","last_unstale":"2024-04-26T22:57:06.328465+0000","last_undegraded":"2024-04-26T22:57:06.328465+0000","last_fullsized":"2024-04-26T22:57:06.328465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:11.469515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.917309+0000","last_change":"2024-04-26T22:55:24.780973+0000","last_active":"2024-04-26T22:57:05.917309+0000","last_peered":"2024-04-26T22:57:05.917309+0000","last_clean":"2024-04-26T22:57:05.917309+0000","last_became_active":"2024-04-26T22:55:24.779810+0000","last_became_peered":"2024-04-26T22:55:24.779810+0000","last_unstale":"2024-04-26T22:57:05.917309+0000","last_undegraded":"2024-04-26T22:57:05.917309+0000","last_fullsized":"2024-04-26T22:57:05.917309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:33.708588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.331182+0000","last_change":"2024-04-26T22:55:25.402067+0000","last_active":"2024-04-26T22:57:06.331182+0000","last_peered":"2024-04-26T22:57:06.331182+0000","last_clean":"2024-04-26T22:57:06.331182+0000","last_became_active":"2024-04-26T22:55:25.401717+0000","last_became_peered":"2024-04-26T22:55:25.401717+0000","last_unstale":"2024-04-26T22:57:06.331182+0000","last_undegraded":"2024-04-26T22:57:06.331182+0000","last_fullsized":"2024-04-26T22:57:06.331182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:16.260620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.332478+0000","last_change":"2024-04-26T22:55:30.429430+0000","last_active":"2024-04-26T22:57:06.332478+0000","last_peered":"2024-04-26T22:57:06.332478+0000","last_clean":"2024-04-26T22:57:06.332478+0000","last_became_active":"2024-04-26T22:55:30.428800+0000","last_became_peered":"2024-04-26T22:55:30.428800+0000","last_unstale":"2024-04-26T22:57:06.332478+0000","last_undegraded":"2024-04-26T22:57:06.332478+0000","last_fullsized":"2024-04-26T22:57:06.332478+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:05.501004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.910522+0000","last_change":"2024-04-26T22:55:24.765181+0000","last_active":"2024-04-26T22:57:05.910522+0000","last_peered":"2024-04-26T22:57:05.910522+0000","last_clean":"2024-04-26T22:57:05.910522+0000","last_became_active":"2024-04-26T22:55:24.764847+0000","last_became_peered":"2024-04-26T22:55:24.764847+0000","last_unstale":"2024-04-26T22:57:05.910522+0000","last_undegraded":"2024-04-26T22:57:05.910522+0000","last_fullsized":"2024-04-26T22:57:05.910522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:25.949304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.329040+0000","last_change":"2024-04-26T22:57:06.329040+0000","last_active":"2024-04-26T22:57:06.329040+0000","last_peered":"2024-04-26T22:57:06.329040+0000","last_clean":"2024-04-26T22:57:06.329040+0000","last_became_active":"2024-04-26T22:57:06.328430+0000","last_became_peered":"2024-04-26T22:57:06.328430+0000","last_unstale":"2024-04-26T22:57:06.329040+0000","last_undegraded":"2024-04-26T22:57:06.329040+0000","last_fullsized":"2024-04-26T22:57:06.329040+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:25.464945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.331052+0000","last_change":"2024-04-26T22:55:30.424537+0000","last_active":"2024-04-26T22:57:06.331052+0000","last_peered":"2024-04-26T22:57:06.331052+0000","last_clean":"2024-04-26T22:57:06.331052+0000","last_became_active":"2024-04-26T22:55:30.423262+0000","last_became_peered":"2024-04-26T22:55:30.423262+0000","last_unstale":"2024-04-26T22:57:06.331052+0000","last_undegraded":"2024-04-26T22:57:06.331052+0000","last_fullsized":"2024-04-26T22:57:06.331052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:48.666453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":133,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:29.803318+0000","last_change":"2024-04-26T22:55:24.776580+0000","last_active":"2024-04-26T22:57:29.803318+0000","last_peered":"2024-04-26T22:57:29.803318+0000","last_clean":"2024-04-26T22:57:29.803318+0000","last_became_active":"2024-04-26T22:55:24.776143+0000","last_became_peered":"2024-04-26T22:55:24.776143+0000","last_unstale":"2024-04-26T22:57:29.803318+0000","last_undegraded":"2024-04-26T22:57:29.803318+0000","last_fullsized":"2024-04-26T22:57:29.803318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:02.291237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328855+0000","last_change":"2024-04-26T22:57:06.328855+0000","last_active":"2024-04-26T22:57:06.328855+0000","last_peered":"2024-04-26T22:57:06.328855+0000","last_clean":"2024-04-26T22:57:06.328855+0000","last_became_active":"2024-04-26T22:57:06.328500+0000","last_became_peered":"2024-04-26T22:57:06.328500+0000","last_unstale":"2024-04-26T22:57:06.328855+0000","last_undegraded":"2024-04-26T22:57:06.328855+0000","last_fullsized":"2024-04-26T22:57:06.328855+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:34.583831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,9],"acting":[8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913215+0000","last_change":"2024-04-26T22:55:30.437915+0000","last_active":"2024-04-26T22:57:05.913215+0000","last_peered":"2024-04-26T22:57:05.913215+0000","last_clean":"2024-04-26T22:57:05.913215+0000","last_became_active":"2024-04-26T22:55:30.437148+0000","last_became_peered":"2024-04-26T22:55:30.437148+0000","last_unstale":"2024-04-26T22:57:05.913215+0000","last_undegraded":"2024-04-26T22:57:05.913215+0000","last_fullsized":"2024-04-26T22:57:05.913215+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:11.647498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328587+0000","last_change":"2024-04-26T22:56:05.728108+0000","last_active":"2024-04-26T22:57:06.328587+0000","last_peered":"2024-04-26T22:57:06.328587+0000","last_clean":"2024-04-26T22:57:06.328587+0000","last_became_active":"2024-04-26T22:56:05.721930+0000","last_became_peered":"2024-04-26T22:56:05.721930+0000","last_unstale":"2024-04-26T22:57:06.328587+0000","last_undegraded":"2024-04-26T22:57:06.328587+0000","last_fullsized":"2024-04-26T22:57:06.328587+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:10.079562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,4],"acting":[11,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.331674+0000","last_change":"2024-04-26T22:55:25.401057+0000","last_active":"2024-04-26T22:57:06.331674+0000","last_peered":"2024-04-26T22:57:06.331674+0000","last_clean":"2024-04-26T22:57:06.331674+0000","last_became_active":"2024-04-26T22:55:25.398265+0000","last_became_peered":"2024-04-26T22:55:25.398265+0000","last_unstale":"2024-04-26T22:57:06.331674+0000","last_undegraded":"2024-04-26T22:57:06.331674+0000","last_fullsized":"2024-04-26T22:57:06.331674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:49.967012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908843+0000","last_change":"2024-04-26T22:55:30.426682+0000","last_active":"2024-04-26T22:57:05.908843+0000","last_peered":"2024-04-26T22:57:05.908843+0000","last_clean":"2024-04-26T22:57:05.908843+0000","last_became_active":"2024-04-26T22:55:30.425918+0000","last_became_peered":"2024-04-26T22:55:30.425918+0000","last_unstale":"2024-04-26T22:57:05.908843+0000","last_undegraded":"2024-04-26T22:57:05.908843+0000","last_fullsized":"2024-04-26T22:57:05.908843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:44.230083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.910815+0000","last_change":"2024-04-26T22:55:24.773497+0000","last_active":"2024-04-26T22:57:05.910815+0000","last_peered":"2024-04-26T22:57:05.910815+0000","last_clean":"2024-04-26T22:57:05.910815+0000","last_became_active":"2024-04-26T22:55:24.773183+0000","last_became_peered":"2024-04-26T22:55:24.773183+0000","last_unstale":"2024-04-26T22:57:05.910815+0000","last_undegraded":"2024-04-26T22:57:05.910815+0000","last_fullsized":"2024-04-26T22:57:05.910815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:58:07.157785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.222503+0000","last_change":"2024-04-26T22:55:25.378420+0000","last_active":"2024-04-26T22:57:06.222503+0000","last_peered":"2024-04-26T22:57:06.222503+0000","last_clean":"2024-04-26T22:57:06.222503+0000","last_became_active":"2024-04-26T22:55:25.378068+0000","last_became_peered":"2024-04-26T22:55:25.378068+0000","last_unstale":"2024-04-26T22:57:06.222503+0000","last_undegraded":"2024-04-26T22:57:06.222503+0000","last_fullsized":"2024-04-26T22:57:06.222503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:10.910541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.915267+0000","last_change":"2024-04-26T22:55:30.436973+0000","last_active":"2024-04-26T22:57:05.915267+0000","last_peered":"2024-04-26T22:57:05.915267+0000","last_clean":"2024-04-26T22:57:05.915267+0000","last_became_active":"2024-04-26T22:55:30.436327+0000","last_became_peered":"2024-04-26T22:55:30.436327+0000","last_unstale":"2024-04-26T22:57:05.915267+0000","last_undegraded":"2024-04-26T22:57:05.915267+0000","last_fullsized":"2024-04-26T22:57:05.915267+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:36.616328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'32","reported_seq":267,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:30.804718+0000","last_change":"2024-04-26T22:55:24.760508+0000","last_active":"2024-04-26T22:57:30.804718+0000","last_peered":"2024-04-26T22:57:30.804718+0000","last_clean":"2024-04-26T22:57:30.804718+0000","last_became_active":"2024-04-26T22:55:24.760180+0000","last_became_peered":"2024-04-26T22:55:24.760180+0000","last_unstale":"2024-04-26T22:57:30.804718+0000","last_undegraded":"2024-04-26T22:57:30.804718+0000","last_fullsized":"2024-04-26T22:57:30.804718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:50.094352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55676,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":202,"num_read_kb":155,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913041+0000","last_change":"2024-04-26T22:55:25.376091+0000","last_active":"2024-04-26T22:57:05.913041+0000","last_peered":"2024-04-26T22:57:05.913041+0000","last_clean":"2024-04-26T22:57:05.913041+0000","last_became_active":"2024-04-26T22:55:25.375623+0000","last_became_peered":"2024-04-26T22:55:25.375623+0000","last_unstale":"2024-04-26T22:57:05.913041+0000","last_undegraded":"2024-04-26T22:57:05.913041+0000","last_fullsized":"2024-04-26T22:57:05.913041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:09.466719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.911432+0000","last_change":"2024-04-26T22:55:30.413534+0000","last_active":"2024-04-26T22:57:05.911432+0000","last_peered":"2024-04-26T22:57:05.911432+0000","last_clean":"2024-04-26T22:57:05.911432+0000","last_became_active":"2024-04-26T22:55:30.413120+0000","last_became_peered":"2024-04-26T22:55:30.413120+0000","last_unstale":"2024-04-26T22:57:05.911432+0000","last_undegraded":"2024-04-26T22:57:05.911432+0000","last_fullsized":"2024-04-26T22:57:05.911432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:13.829733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908553+0000","last_change":"2024-04-26T22:55:24.781446+0000","last_active":"2024-04-26T22:57:05.908553+0000","last_peered":"2024-04-26T22:57:05.908553+0000","last_clean":"2024-04-26T22:57:05.908553+0000","last_became_active":"2024-04-26T22:55:24.781098+0000","last_became_peered":"2024-04-26T22:55:24.781098+0000","last_unstale":"2024-04-26T22:57:05.908553+0000","last_undegraded":"2024-04-26T22:57:05.908553+0000","last_fullsized":"2024-04-26T22:57:05.908553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:40.939755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.911425+0000","last_change":"2024-04-26T22:55:25.394147+0000","last_active":"2024-04-26T22:57:05.911425+0000","last_peered":"2024-04-26T22:57:05.911425+0000","last_clean":"2024-04-26T22:57:05.911425+0000","last_became_active":"2024-04-26T22:55:25.393654+0000","last_became_peered":"2024-04-26T22:55:25.393654+0000","last_unstale":"2024-04-26T22:57:05.911425+0000","last_undegraded":"2024-04-26T22:57:05.911425+0000","last_fullsized":"2024-04-26T22:57:05.911425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:16.335027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328118+0000","last_change":"2024-04-26T22:55:30.431692+0000","last_active":"2024-04-26T22:57:06.328118+0000","last_peered":"2024-04-26T22:57:06.328118+0000","last_clean":"2024-04-26T22:57:06.328118+0000","last_became_active":"2024-04-26T22:55:30.431380+0000","last_became_peered":"2024-04-26T22:55:30.431380+0000","last_unstale":"2024-04-26T22:57:06.328118+0000","last_undegraded":"2024-04-26T22:57:06.328118+0000","last_fullsized":"2024-04-26T22:57:06.328118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:14.556048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.910767+0000","last_change":"2024-04-26T22:55:24.788121+0000","last_active":"2024-04-26T22:57:05.910767+0000","last_peered":"2024-04-26T22:57:05.910767+0000","last_clean":"2024-04-26T22:57:05.910767+0000","last_became_active":"2024-04-26T22:55:24.787759+0000","last_became_peered":"2024-04-26T22:55:24.787759+0000","last_unstale":"2024-04-26T22:57:05.910767+0000","last_undegraded":"2024-04-26T22:57:05.910767+0000","last_fullsized":"2024-04-26T22:57:05.910767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:49.734675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913194+0000","last_change":"2024-04-26T22:55:25.377576+0000","last_active":"2024-04-26T22:57:05.913194+0000","last_peered":"2024-04-26T22:57:05.913194+0000","last_clean":"2024-04-26T22:57:05.913194+0000","last_became_active":"2024-04-26T22:55:25.377203+0000","last_became_peered":"2024-04-26T22:55:25.377203+0000","last_unstale":"2024-04-26T22:57:05.913194+0000","last_undegraded":"2024-04-26T22:57:05.913194+0000","last_fullsized":"2024-04-26T22:57:05.913194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:39.361118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.918980+0000","last_change":"2024-04-26T22:55:30.436672+0000","last_active":"2024-04-26T22:57:05.918980+0000","last_peered":"2024-04-26T22:57:05.918980+0000","last_clean":"2024-04-26T22:57:05.918980+0000","last_became_active":"2024-04-26T22:55:30.435178+0000","last_became_peered":"2024-04-26T22:55:30.435178+0000","last_unstale":"2024-04-26T22:57:05.918980+0000","last_undegraded":"2024-04-26T22:57:05.918980+0000","last_fullsized":"2024-04-26T22:57:05.918980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:34.435290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":136,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:29.836058+0000","last_change":"2024-04-26T22:55:24.760965+0000","last_active":"2024-04-26T22:57:29.836058+0000","last_peered":"2024-04-26T22:57:29.836058+0000","last_clean":"2024-04-26T22:57:29.836058+0000","last_became_active":"2024-04-26T22:55:24.760589+0000","last_became_peered":"2024-04-26T22:55:24.760589+0000","last_unstale":"2024-04-26T22:57:29.836058+0000","last_undegraded":"2024-04-26T22:57:29.836058+0000","last_fullsized":"2024-04-26T22:57:29.836058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:25.493209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912960+0000","last_change":"2024-04-26T22:55:25.375433+0000","last_active":"2024-04-26T22:57:05.912960+0000","last_peered":"2024-04-26T22:57:05.912960+0000","last_clean":"2024-04-26T22:57:05.912960+0000","last_became_active":"2024-04-26T22:55:25.375062+0000","last_became_peered":"2024-04-26T22:55:25.375062+0000","last_unstale":"2024-04-26T22:57:05.912960+0000","last_undegraded":"2024-04-26T22:57:05.912960+0000","last_fullsized":"2024-04-26T22:57:05.912960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:55.423666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.914428+0000","last_change":"2024-04-26T22:56:05.818921+0000","last_active":"2024-04-26T22:57:05.914428+0000","last_peered":"2024-04-26T22:57:05.914428+0000","last_clean":"2024-04-26T22:57:05.914428+0000","last_became_active":"2024-04-26T22:56:05.816873+0000","last_became_peered":"2024-04-26T22:56:05.816873+0000","last_unstale":"2024-04-26T22:57:05.914428+0000","last_undegraded":"2024-04-26T22:57:05.914428+0000","last_fullsized":"2024-04-26T22:57:05.914428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:46:57.475468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912603+0000","last_change":"2024-04-26T22:55:24.773551+0000","last_active":"2024-04-26T22:57:05.912603+0000","last_peered":"2024-04-26T22:57:05.912603+0000","last_clean":"2024-04-26T22:57:05.912603+0000","last_became_active":"2024-04-26T22:55:24.773202+0000","last_became_peered":"2024-04-26T22:55:24.773202+0000","last_unstale":"2024-04-26T22:57:05.912603+0000","last_undegraded":"2024-04-26T22:57:05.912603+0000","last_fullsized":"2024-04-26T22:57:05.912603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:08.271682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.223634+0000","last_change":"2024-04-26T22:57:06.223634+0000","last_active":"2024-04-26T22:57:06.223634+0000","last_peered":"2024-04-26T22:57:06.223634+0000","last_clean":"2024-04-26T22:57:06.223634+0000","last_became_active":"2024-04-26T22:57:06.223154+0000","last_became_peered":"2024-04-26T22:57:06.223154+0000","last_unstale":"2024-04-26T22:57:06.223634+0000","last_undegraded":"2024-04-26T22:57:06.223634+0000","last_fullsized":"2024-04-26T22:57:06.223634+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:03:38.391874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.911480+0000","last_change":"2024-04-26T22:55:30.422835+0000","last_active":"2024-04-26T22:57:05.911480+0000","last_peered":"2024-04-26T22:57:05.911480+0000","last_clean":"2024-04-26T22:57:05.911480+0000","last_became_active":"2024-04-26T22:55:30.422349+0000","last_became_peered":"2024-04-26T22:55:30.422349+0000","last_unstale":"2024-04-26T22:57:05.911480+0000","last_undegraded":"2024-04-26T22:57:05.911480+0000","last_fullsized":"2024-04-26T22:57:05.911480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:26:33.613861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913328+0000","last_change":"2024-04-26T22:55:24.756236+0000","last_active":"2024-04-26T22:57:05.913328+0000","last_peered":"2024-04-26T22:57:05.913328+0000","last_clean":"2024-04-26T22:57:05.913328+0000","last_became_active":"2024-04-26T22:55:24.755858+0000","last_became_peered":"2024-04-26T22:55:24.755858+0000","last_unstale":"2024-04-26T22:57:05.913328+0000","last_undegraded":"2024-04-26T22:57:05.913328+0000","last_fullsized":"2024-04-26T22:57:05.913328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:55.257152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328349+0000","last_change":"2024-04-26T22:55:25.374367+0000","last_active":"2024-04-26T22:57:06.328349+0000","last_peered":"2024-04-26T22:57:06.328349+0000","last_clean":"2024-04-26T22:57:06.328349+0000","last_became_active":"2024-04-26T22:55:25.373186+0000","last_became_peered":"2024-04-26T22:55:25.373186+0000","last_unstale":"2024-04-26T22:57:06.328349+0000","last_undegraded":"2024-04-26T22:57:06.328349+0000","last_fullsized":"2024-04-26T22:57:06.328349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:31:55.780924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913108+0000","last_change":"2024-04-26T22:55:30.425818+0000","last_active":"2024-04-26T22:57:05.913108+0000","last_peered":"2024-04-26T22:57:05.913108+0000","last_clean":"2024-04-26T22:57:05.913108+0000","last_became_active":"2024-04-26T22:55:30.425400+0000","last_became_peered":"2024-04-26T22:55:30.425400+0000","last_unstale":"2024-04-26T22:57:05.913108+0000","last_undegraded":"2024-04-26T22:57:05.913108+0000","last_fullsized":"2024-04-26T22:57:05.913108+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:36.856545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.917291+0000","last_change":"2024-04-26T22:55:24.780607+0000","last_active":"2024-04-26T22:57:05.917291+0000","last_peered":"2024-04-26T22:57:05.917291+0000","last_clean":"2024-04-26T22:57:05.917291+0000","last_became_active":"2024-04-26T22:55:24.780271+0000","last_became_peered":"2024-04-26T22:55:24.780271+0000","last_unstale":"2024-04-26T22:57:05.917291+0000","last_undegraded":"2024-04-26T22:57:05.917291+0000","last_fullsized":"2024-04-26T22:57:05.917291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:25.057816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.327595+0000","last_change":"2024-04-26T22:55:25.399709+0000","last_active":"2024-04-26T22:57:06.327595+0000","last_peered":"2024-04-26T22:57:06.327595+0000","last_clean":"2024-04-26T22:57:06.327595+0000","last_became_active":"2024-04-26T22:55:25.399392+0000","last_became_peered":"2024-04-26T22:55:25.399392+0000","last_unstale":"2024-04-26T22:57:06.327595+0000","last_undegraded":"2024-04-26T22:57:06.327595+0000","last_fullsized":"2024-04-26T22:57:06.327595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:22.821809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912413+0000","last_change":"2024-04-26T22:55:30.424070+0000","last_active":"2024-04-26T22:57:05.912413+0000","last_peered":"2024-04-26T22:57:05.912413+0000","last_clean":"2024-04-26T22:57:05.912413+0000","last_became_active":"2024-04-26T22:55:30.423637+0000","last_became_peered":"2024-04-26T22:55:30.423637+0000","last_unstale":"2024-04-26T22:57:05.912413+0000","last_undegraded":"2024-04-26T22:57:05.912413+0000","last_fullsized":"2024-04-26T22:57:05.912413+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:48.020079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913045+0000","last_change":"2024-04-26T22:55:24.771881+0000","last_active":"2024-04-26T22:57:05.913045+0000","last_peered":"2024-04-26T22:57:05.913045+0000","last_clean":"2024-04-26T22:57:05.913045+0000","last_became_active":"2024-04-26T22:55:24.771563+0000","last_became_peered":"2024-04-26T22:55:24.771563+0000","last_unstale":"2024-04-26T22:57:05.913045+0000","last_undegraded":"2024-04-26T22:57:05.913045+0000","last_fullsized":"2024-04-26T22:57:05.913045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:14.306677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.327169+0000","last_change":"2024-04-26T22:55:25.399789+0000","last_active":"2024-04-26T22:57:06.327169+0000","last_peered":"2024-04-26T22:57:06.327169+0000","last_clean":"2024-04-26T22:57:06.327169+0000","last_became_active":"2024-04-26T22:55:25.399493+0000","last_became_peered":"2024-04-26T22:55:25.399493+0000","last_unstale":"2024-04-26T22:57:06.327169+0000","last_undegraded":"2024-04-26T22:57:06.327169+0000","last_fullsized":"2024-04-26T22:57:06.327169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:45.096556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.914439+0000","last_change":"2024-04-26T22:55:30.428107+0000","last_active":"2024-04-26T22:57:05.914439+0000","last_peered":"2024-04-26T22:57:05.914439+0000","last_clean":"2024-04-26T22:57:05.914439+0000","last_became_active":"2024-04-26T22:55:30.421955+0000","last_became_peered":"2024-04-26T22:55:30.421955+0000","last_unstale":"2024-04-26T22:57:05.914439+0000","last_undegraded":"2024-04-26T22:57:05.914439+0000","last_fullsized":"2024-04-26T22:57:05.914439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:52.430809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.911381+0000","last_change":"2024-04-26T22:56:05.720181+0000","last_active":"2024-04-26T22:57:05.911381+0000","last_peered":"2024-04-26T22:57:05.911381+0000","last_clean":"2024-04-26T22:57:05.911381+0000","last_became_active":"2024-04-26T22:56:05.719790+0000","last_became_peered":"2024-04-26T22:56:05.719790+0000","last_unstale":"2024-04-26T22:57:05.911381+0000","last_undegraded":"2024-04-26T22:57:05.911381+0000","last_fullsized":"2024-04-26T22:57:05.911381+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:43.760736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":134,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:29.835276+0000","last_change":"2024-04-26T22:55:24.763998+0000","last_active":"2024-04-26T22:57:29.835276+0000","last_peered":"2024-04-26T22:57:29.835276+0000","last_clean":"2024-04-26T22:57:29.835276+0000","last_became_active":"2024-04-26T22:55:24.763561+0000","last_became_peered":"2024-04-26T22:55:24.763561+0000","last_unstale":"2024-04-26T22:57:29.835276+0000","last_undegraded":"2024-04-26T22:57:29.835276+0000","last_fullsized":"2024-04-26T22:57:29.835276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:25.270310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913323+0000","last_change":"2024-04-26T22:55:30.424930+0000","last_active":"2024-04-26T22:57:05.913323+0000","last_peered":"2024-04-26T22:57:05.913323+0000","last_clean":"2024-04-26T22:57:05.913323+0000","last_became_active":"2024-04-26T22:55:30.424660+0000","last_became_peered":"2024-04-26T22:55:30.424660+0000","last_unstale":"2024-04-26T22:57:05.913323+0000","last_undegraded":"2024-04-26T22:57:05.913323+0000","last_fullsized":"2024-04-26T22:57:05.913323+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:29.483956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.333402+0000","last_change":"2024-04-26T22:55:25.390547+0000","last_active":"2024-04-26T22:57:06.333402+0000","last_peered":"2024-04-26T22:57:06.333402+0000","last_clean":"2024-04-26T22:57:06.333402+0000","last_became_active":"2024-04-26T22:55:25.386566+0000","last_became_peered":"2024-04-26T22:55:25.386566+0000","last_unstale":"2024-04-26T22:57:06.333402+0000","last_undegraded":"2024-04-26T22:57:06.333402+0000","last_fullsized":"2024-04-26T22:57:06.333402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:43.809125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908812+0000","last_change":"2024-04-26T22:55:24.778254+0000","last_active":"2024-04-26T22:57:05.908812+0000","last_peered":"2024-04-26T22:57:05.908812+0000","last_clean":"2024-04-26T22:57:05.908812+0000","last_became_active":"2024-04-26T22:55:24.777850+0000","last_became_peered":"2024-04-26T22:55:24.777850+0000","last_unstale":"2024-04-26T22:57:05.908812+0000","last_undegraded":"2024-04-26T22:57:05.908812+0000","last_fullsized":"2024-04-26T22:57:05.908812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:48.053712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912771+0000","last_change":"2024-04-26T22:53:59.233875+0000","last_active":"2024-04-26T22:57:05.912771+0000","last_peered":"2024-04-26T22:57:05.912771+0000","last_clean":"2024-04-26T22:57:05.912771+0000","last_became_active":"2024-04-26T22:53:59.233488+0000","last_became_peered":"2024-04-26T22:53:59.233488+0000","last_unstale":"2024-04-26T22:57:05.912771+0000","last_undegraded":"2024-04-26T22:57:05.912771+0000","last_fullsized":"2024-04-26T22:57:05.912771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:49:55.319714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:49:55.319714+0000","last_clean_scrub_stamp":"2024-04-26T22:49:55.319714+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:37:32.885519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.910939+0000","last_change":"2024-04-26T22:55:30.427936+0000","last_active":"2024-04-26T22:57:05.910939+0000","last_peered":"2024-04-26T22:57:05.910939+0000","last_clean":"2024-04-26T22:57:05.910939+0000","last_became_active":"2024-04-26T22:55:30.427483+0000","last_became_peered":"2024-04-26T22:55:30.427483+0000","last_unstale":"2024-04-26T22:57:05.910939+0000","last_undegraded":"2024-04-26T22:57:05.910939+0000","last_fullsized":"2024-04-26T22:57:05.910939+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:00.856153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.914272+0000","last_change":"2024-04-26T22:55:25.378277+0000","last_active":"2024-04-26T22:57:05.914272+0000","last_peered":"2024-04-26T22:57:05.914272+0000","last_clean":"2024-04-26T22:57:05.914272+0000","last_became_active":"2024-04-26T22:55:25.377650+0000","last_became_peered":"2024-04-26T22:55:25.377650+0000","last_unstale":"2024-04-26T22:57:05.914272+0000","last_undegraded":"2024-04-26T22:57:05.914272+0000","last_fullsized":"2024-04-26T22:57:05.914272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:47.537306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912961+0000","last_change":"2024-04-26T22:55:24.762439+0000","last_active":"2024-04-26T22:57:05.912961+0000","last_peered":"2024-04-26T22:57:05.912961+0000","last_clean":"2024-04-26T22:57:05.912961+0000","last_became_active":"2024-04-26T22:55:24.760732+0000","last_became_peered":"2024-04-26T22:55:24.760732+0000","last_unstale":"2024-04-26T22:57:05.912961+0000","last_undegraded":"2024-04-26T22:57:05.912961+0000","last_fullsized":"2024-04-26T22:57:05.912961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:33.857025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328134+0000","last_change":"2024-04-26T22:55:30.411883+0000","last_active":"2024-04-26T22:57:06.328134+0000","last_peered":"2024-04-26T22:57:06.328134+0000","last_clean":"2024-04-26T22:57:06.328134+0000","last_became_active":"2024-04-26T22:55:30.411419+0000","last_became_peered":"2024-04-26T22:55:30.411419+0000","last_unstale":"2024-04-26T22:57:06.328134+0000","last_undegraded":"2024-04-26T22:57:06.328134+0000","last_fullsized":"2024-04-26T22:57:06.328134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:14.977947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913355+0000","last_change":"2024-04-26T22:55:24.779481+0000","last_active":"2024-04-26T22:57:05.913355+0000","last_peered":"2024-04-26T22:57:05.913355+0000","last_clean":"2024-04-26T22:57:05.913355+0000","last_became_active":"2024-04-26T22:55:24.779182+0000","last_became_peered":"2024-04-26T22:55:24.779182+0000","last_unstale":"2024-04-26T22:57:05.913355+0000","last_undegraded":"2024-04-26T22:57:05.913355+0000","last_fullsized":"2024-04-26T22:57:05.913355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:52.076060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913272+0000","last_change":"2024-04-26T22:55:25.390938+0000","last_active":"2024-04-26T22:57:05.913272+0000","last_peered":"2024-04-26T22:57:05.913272+0000","last_clean":"2024-04-26T22:57:05.913272+0000","last_became_active":"2024-04-26T22:55:25.390566+0000","last_became_peered":"2024-04-26T22:55:25.390566+0000","last_unstale":"2024-04-26T22:57:05.913272+0000","last_undegraded":"2024-04-26T22:57:05.913272+0000","last_fullsized":"2024-04-26T22:57:05.913272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:48.079024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.331078+0000","last_change":"2024-04-26T22:55:30.430926+0000","last_active":"2024-04-26T22:57:06.331078+0000","last_peered":"2024-04-26T22:57:06.331078+0000","last_clean":"2024-04-26T22:57:06.331078+0000","last_became_active":"2024-04-26T22:55:30.428286+0000","last_became_peered":"2024-04-26T22:55:30.428286+0000","last_unstale":"2024-04-26T22:57:06.331078+0000","last_undegraded":"2024-04-26T22:57:06.331078+0000","last_fullsized":"2024-04-26T22:57:06.331078+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:26.095658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.916369+0000","last_change":"2024-04-26T22:55:24.784960+0000","last_active":"2024-04-26T22:57:05.916369+0000","last_peered":"2024-04-26T22:57:05.916369+0000","last_clean":"2024-04-26T22:57:05.916369+0000","last_became_active":"2024-04-26T22:55:24.784550+0000","last_became_peered":"2024-04-26T22:55:24.784550+0000","last_unstale":"2024-04-26T22:57:05.916369+0000","last_undegraded":"2024-04-26T22:57:05.916369+0000","last_fullsized":"2024-04-26T22:57:05.916369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:05.058172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.334186+0000","last_change":"2024-04-26T22:55:25.392352+0000","last_active":"2024-04-26T22:57:06.334186+0000","last_peered":"2024-04-26T22:57:06.334186+0000","last_clean":"2024-04-26T22:57:06.334186+0000","last_became_active":"2024-04-26T22:55:25.391983+0000","last_became_peered":"2024-04-26T22:55:25.391983+0000","last_unstale":"2024-04-26T22:57:06.334186+0000","last_undegraded":"2024-04-26T22:57:06.334186+0000","last_fullsized":"2024-04-26T22:57:06.334186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:37.192419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908098+0000","last_change":"2024-04-26T22:55:30.409951+0000","last_active":"2024-04-26T22:57:05.908098+0000","last_peered":"2024-04-26T22:57:05.908098+0000","last_clean":"2024-04-26T22:57:05.908098+0000","last_became_active":"2024-04-26T22:55:30.409502+0000","last_became_peered":"2024-04-26T22:55:30.409502+0000","last_unstale":"2024-04-26T22:57:05.908098+0000","last_undegraded":"2024-04-26T22:57:05.908098+0000","last_fullsized":"2024-04-26T22:57:05.908098+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:40:48.416697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.909550+0000","last_change":"2024-04-26T22:55:24.770999+0000","last_active":"2024-04-26T22:57:05.909550+0000","last_peered":"2024-04-26T22:57:05.909550+0000","last_clean":"2024-04-26T22:57:05.909550+0000","last_became_active":"2024-04-26T22:55:24.770630+0000","last_became_peered":"2024-04-26T22:55:24.770630+0000","last_unstale":"2024-04-26T22:57:05.909550+0000","last_undegraded":"2024-04-26T22:57:05.909550+0000","last_fullsized":"2024-04-26T22:57:05.909550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:01.651705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.332559+0000","last_change":"2024-04-26T22:55:25.401845+0000","last_active":"2024-04-26T22:57:06.332559+0000","last_peered":"2024-04-26T22:57:06.332559+0000","last_clean":"2024-04-26T22:57:06.332559+0000","last_became_active":"2024-04-26T22:55:25.398759+0000","last_became_peered":"2024-04-26T22:55:25.398759+0000","last_unstale":"2024-04-26T22:57:06.332559+0000","last_undegraded":"2024-04-26T22:57:06.332559+0000","last_fullsized":"2024-04-26T22:57:06.332559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:14.158710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.909088+0000","last_change":"2024-04-26T22:55:30.426206+0000","last_active":"2024-04-26T22:57:05.909088+0000","last_peered":"2024-04-26T22:57:05.909088+0000","last_clean":"2024-04-26T22:57:05.909088+0000","last_became_active":"2024-04-26T22:55:30.425678+0000","last_became_peered":"2024-04-26T22:55:30.425678+0000","last_unstale":"2024-04-26T22:57:05.909088+0000","last_undegraded":"2024-04-26T22:57:05.909088+0000","last_fullsized":"2024-04-26T22:57:05.909088+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:48.913899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.222051+0000","last_change":"2024-04-26T22:55:24.772775+0000","last_active":"2024-04-26T22:57:06.222051+0000","last_peered":"2024-04-26T22:57:06.222051+0000","last_clean":"2024-04-26T22:57:06.222051+0000","last_became_active":"2024-04-26T22:55:24.772434+0000","last_became_peered":"2024-04-26T22:55:24.772434+0000","last_unstale":"2024-04-26T22:57:06.222051+0000","last_undegraded":"2024-04-26T22:57:06.222051+0000","last_fullsized":"2024-04-26T22:57:06.222051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:23.309545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912634+0000","last_change":"2024-04-26T22:55:25.372051+0000","last_active":"2024-04-26T22:57:05.912634+0000","last_peered":"2024-04-26T22:57:05.912634+0000","last_clean":"2024-04-26T22:57:05.912634+0000","last_became_active":"2024-04-26T22:55:25.371681+0000","last_became_peered":"2024-04-26T22:55:25.371681+0000","last_unstale":"2024-04-26T22:57:05.912634+0000","last_undegraded":"2024-04-26T22:57:05.912634+0000","last_fullsized":"2024-04-26T22:57:05.912634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:34:39.412974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.918768+0000","last_change":"2024-04-26T22:55:30.427160+0000","last_active":"2024-04-26T22:57:05.918768+0000","last_peered":"2024-04-26T22:57:05.918768+0000","last_clean":"2024-04-26T22:57:05.918768+0000","last_became_active":"2024-04-26T22:55:30.426555+0000","last_became_peered":"2024-04-26T22:55:30.426555+0000","last_unstale":"2024-04-26T22:57:05.918768+0000","last_undegraded":"2024-04-26T22:57:05.918768+0000","last_fullsized":"2024-04-26T22:57:05.918768+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:31.688532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.332372+0000","last_change":"2024-04-26T22:55:24.785265+0000","last_active":"2024-04-26T22:57:06.332372+0000","last_peered":"2024-04-26T22:57:06.332372+0000","last_clean":"2024-04-26T22:57:06.332372+0000","last_became_active":"2024-04-26T22:55:24.784896+0000","last_became_peered":"2024-04-26T22:55:24.784896+0000","last_unstale":"2024-04-26T22:57:06.332372+0000","last_undegraded":"2024-04-26T22:57:06.332372+0000","last_fullsized":"2024-04-26T22:57:06.332372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:28.815337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.327964+0000","last_change":"2024-04-26T22:55:25.399606+0000","last_active":"2024-04-26T22:57:06.327964+0000","last_peered":"2024-04-26T22:57:06.327964+0000","last_clean":"2024-04-26T22:57:06.327964+0000","last_became_active":"2024-04-26T22:55:25.399195+0000","last_became_peered":"2024-04-26T22:55:25.399195+0000","last_unstale":"2024-04-26T22:57:06.327964+0000","last_undegraded":"2024-04-26T22:57:06.327964+0000","last_fullsized":"2024-04-26T22:57:06.327964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:53:53.618431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908804+0000","last_change":"2024-04-26T22:55:30.427381+0000","last_active":"2024-04-26T22:57:05.908804+0000","last_peered":"2024-04-26T22:57:05.908804+0000","last_clean":"2024-04-26T22:57:05.908804+0000","last_became_active":"2024-04-26T22:55:30.426736+0000","last_became_peered":"2024-04-26T22:55:30.426736+0000","last_unstale":"2024-04-26T22:57:05.908804+0000","last_undegraded":"2024-04-26T22:57:05.908804+0000","last_fullsized":"2024-04-26T22:57:05.908804+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:07.106991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908478+0000","last_change":"2024-04-26T22:55:24.781483+0000","last_active":"2024-04-26T22:57:05.908478+0000","last_peered":"2024-04-26T22:57:05.908478+0000","last_clean":"2024-04-26T22:57:05.908478+0000","last_became_active":"2024-04-26T22:55:24.781174+0000","last_became_peered":"2024-04-26T22:55:24.781174+0000","last_unstale":"2024-04-26T22:57:05.908478+0000","last_undegraded":"2024-04-26T22:57:05.908478+0000","last_fullsized":"2024-04-26T22:57:05.908478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:58:05.550242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.332650+0000","last_change":"2024-04-26T22:55:25.406764+0000","last_active":"2024-04-26T22:57:06.332650+0000","last_peered":"2024-04-26T22:57:06.332650+0000","last_clean":"2024-04-26T22:57:06.332650+0000","last_became_active":"2024-04-26T22:55:25.405948+0000","last_became_peered":"2024-04-26T22:55:25.405948+0000","last_unstale":"2024-04-26T22:57:06.332650+0000","last_undegraded":"2024-04-26T22:57:06.332650+0000","last_fullsized":"2024-04-26T22:57:06.332650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:58.414530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908955+0000","last_change":"2024-04-26T22:55:30.422989+0000","last_active":"2024-04-26T22:57:05.908955+0000","last_peered":"2024-04-26T22:57:05.908955+0000","last_clean":"2024-04-26T22:57:05.908955+0000","last_became_active":"2024-04-26T22:55:30.422529+0000","last_became_peered":"2024-04-26T22:55:30.422529+0000","last_unstale":"2024-04-26T22:57:05.908955+0000","last_undegraded":"2024-04-26T22:57:05.908955+0000","last_fullsized":"2024-04-26T22:57:05.908955+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:08.312667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912755+0000","last_change":"2024-04-26T22:56:05.725131+0000","last_active":"2024-04-26T22:57:05.912755+0000","last_peered":"2024-04-26T22:57:05.912755+0000","last_clean":"2024-04-26T22:57:05.912755+0000","last_became_active":"2024-04-26T22:56:05.719614+0000","last_became_peered":"2024-04-26T22:56:05.719614+0000","last_unstale":"2024-04-26T22:57:05.912755+0000","last_undegraded":"2024-04-26T22:57:05.912755+0000","last_fullsized":"2024-04-26T22:57:05.912755+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:57.965791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.912792+0000","last_change":"2024-04-26T22:55:25.371428+0000","last_active":"2024-04-26T22:57:05.912792+0000","last_peered":"2024-04-26T22:57:05.912792+0000","last_clean":"2024-04-26T22:57:05.912792+0000","last_became_active":"2024-04-26T22:55:25.371088+0000","last_became_peered":"2024-04-26T22:55:25.371088+0000","last_unstale":"2024-04-26T22:57:05.912792+0000","last_undegraded":"2024-04-26T22:57:05.912792+0000","last_fullsized":"2024-04-26T22:57:05.912792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:55:15.066665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.221961+0000","last_change":"2024-04-26T22:55:30.411966+0000","last_active":"2024-04-26T22:57:06.221961+0000","last_peered":"2024-04-26T22:57:06.221961+0000","last_clean":"2024-04-26T22:57:06.221961+0000","last_became_active":"2024-04-26T22:55:30.411486+0000","last_became_peered":"2024-04-26T22:55:30.411486+0000","last_unstale":"2024-04-26T22:57:06.221961+0000","last_undegraded":"2024-04-26T22:57:06.221961+0000","last_fullsized":"2024-04-26T22:57:06.221961+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:08.207340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908981+0000","last_change":"2024-04-26T22:55:24.771435+0000","last_active":"2024-04-26T22:57:05.908981+0000","last_peered":"2024-04-26T22:57:05.908981+0000","last_clean":"2024-04-26T22:57:05.908981+0000","last_became_active":"2024-04-26T22:55:24.771156+0000","last_became_peered":"2024-04-26T22:55:24.771156+0000","last_unstale":"2024-04-26T22:57:05.908981+0000","last_undegraded":"2024-04-26T22:57:05.908981+0000","last_fullsized":"2024-04-26T22:57:05.908981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:15.130786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913515+0000","last_change":"2024-04-26T22:55:25.374051+0000","last_active":"2024-04-26T22:57:05.913515+0000","last_peered":"2024-04-26T22:57:05.913515+0000","last_clean":"2024-04-26T22:57:05.913515+0000","last_became_active":"2024-04-26T22:55:25.373348+0000","last_became_peered":"2024-04-26T22:55:25.373348+0000","last_unstale":"2024-04-26T22:57:05.913515+0000","last_undegraded":"2024-04-26T22:57:05.913515+0000","last_fullsized":"2024-04-26T22:57:05.913515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:30:31.116829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.330895+0000","last_change":"2024-04-26T22:56:05.728492+0000","last_active":"2024-04-26T22:57:06.330895+0000","last_peered":"2024-04-26T22:57:06.330895+0000","last_clean":"2024-04-26T22:57:06.330895+0000","last_became_active":"2024-04-26T22:56:05.727863+0000","last_became_peered":"2024-04-26T22:56:05.727863+0000","last_unstale":"2024-04-26T22:57:06.330895+0000","last_undegraded":"2024-04-26T22:57:06.330895+0000","last_fullsized":"2024-04-26T22:57:06.330895+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:39.596089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10,1,4],"acting":[0,10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.911080+0000","last_change":"2024-04-26T22:55:24.788057+0000","last_active":"2024-04-26T22:57:05.911080+0000","last_peered":"2024-04-26T22:57:05.911080+0000","last_clean":"2024-04-26T22:57:05.911080+0000","last_became_active":"2024-04-26T22:55:24.787753+0000","last_became_peered":"2024-04-26T22:55:24.787753+0000","last_unstale":"2024-04-26T22:57:05.911080+0000","last_undegraded":"2024-04-26T22:57:05.911080+0000","last_fullsized":"2024-04-26T22:57:05.911080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:53.054665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913257+0000","last_change":"2024-04-26T22:55:25.400227+0000","last_active":"2024-04-26T22:57:05.913257+0000","last_peered":"2024-04-26T22:57:05.913257+0000","last_clean":"2024-04-26T22:57:05.913257+0000","last_became_active":"2024-04-26T22:55:25.399738+0000","last_became_peered":"2024-04-26T22:55:25.399738+0000","last_unstale":"2024-04-26T22:57:05.913257+0000","last_undegraded":"2024-04-26T22:57:05.913257+0000","last_fullsized":"2024-04-26T22:57:05.913257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:22.224654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.910383+0000","last_change":"2024-04-26T22:55:30.410479+0000","last_active":"2024-04-26T22:57:05.910383+0000","last_peered":"2024-04-26T22:57:05.910383+0000","last_clean":"2024-04-26T22:57:05.910383+0000","last_became_active":"2024-04-26T22:55:30.410049+0000","last_became_peered":"2024-04-26T22:55:30.410049+0000","last_unstale":"2024-04-26T22:57:05.910383+0000","last_undegraded":"2024-04-26T22:57:05.910383+0000","last_fullsized":"2024-04-26T22:57:05.910383+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:29.370418+0000","last_clean_scrub_stamp":"2024-04-26T22:55:29.370418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:04.412005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.914722+0000","last_change":"2024-04-26T22:55:24.780741+0000","last_active":"2024-04-26T22:57:05.914722+0000","last_peered":"2024-04-26T22:57:05.914722+0000","last_clean":"2024-04-26T22:57:05.914722+0000","last_became_active":"2024-04-26T22:55:24.780400+0000","last_became_peered":"2024-04-26T22:55:24.780400+0000","last_unstale":"2024-04-26T22:57:05.914722+0000","last_undegraded":"2024-04-26T22:57:05.914722+0000","last_fullsized":"2024-04-26T22:57:05.914722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:23.728968+0000","last_clean_scrub_stamp":"2024-04-26T22:55:23.728968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:52.274456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913500+0000","last_change":"2024-04-26T22:55:25.369449+0000","last_active":"2024-04-26T22:57:05.913500+0000","last_peered":"2024-04-26T22:57:05.913500+0000","last_clean":"2024-04-26T22:57:05.913500+0000","last_became_active":"2024-04-26T22:55:25.369106+0000","last_became_peered":"2024-04-26T22:55:25.369106+0000","last_unstale":"2024-04-26T22:57:05.913500+0000","last_undegraded":"2024-04-26T22:57:05.913500+0000","last_fullsized":"2024-04-26T22:57:05.913500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:21.347385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328121+0000","last_change":"2024-04-26T22:55:25.371959+0000","last_active":"2024-04-26T22:57:06.328121+0000","last_peered":"2024-04-26T22:57:06.328121+0000","last_clean":"2024-04-26T22:57:06.328121+0000","last_became_active":"2024-04-26T22:55:25.371580+0000","last_became_peered":"2024-04-26T22:55:25.371580+0000","last_unstale":"2024-04-26T22:57:06.328121+0000","last_undegraded":"2024-04-26T22:57:06.328121+0000","last_fullsized":"2024-04-26T22:57:06.328121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:11.001269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.923285+0000","last_change":"2024-04-26T22:57:05.922645+0000","last_active":"2024-04-26T22:57:05.923285+0000","last_peered":"2024-04-26T22:57:05.923285+0000","last_clean":"2024-04-26T22:57:05.923285+0000","last_became_active":"2024-04-26T22:57:05.922233+0000","last_became_peered":"2024-04-26T22:57:05.922233+0000","last_unstale":"2024-04-26T22:57:05.923285+0000","last_undegraded":"2024-04-26T22:57:05.923285+0000","last_fullsized":"2024-04-26T22:57:05.923285+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:26.067214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908263+0000","last_change":"2024-04-26T22:55:25.398778+0000","last_active":"2024-04-26T22:57:05.908263+0000","last_peered":"2024-04-26T22:57:05.908263+0000","last_clean":"2024-04-26T22:57:05.908263+0000","last_became_active":"2024-04-26T22:55:25.398460+0000","last_became_peered":"2024-04-26T22:55:25.398460+0000","last_unstale":"2024-04-26T22:57:05.908263+0000","last_undegraded":"2024-04-26T22:57:05.908263+0000","last_fullsized":"2024-04-26T22:57:05.908263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.873418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.331552+0000","last_change":"2024-04-26T22:55:25.396548+0000","last_active":"2024-04-26T22:57:06.331552+0000","last_peered":"2024-04-26T22:57:06.331552+0000","last_clean":"2024-04-26T22:57:06.331552+0000","last_became_active":"2024-04-26T22:55:25.395983+0000","last_became_peered":"2024-04-26T22:55:25.395983+0000","last_unstale":"2024-04-26T22:57:06.331552+0000","last_undegraded":"2024-04-26T22:57:06.331552+0000","last_fullsized":"2024-04-26T22:57:06.331552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:21:34.690636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.913012+0000","last_change":"2024-04-26T22:55:25.396742+0000","last_active":"2024-04-26T22:57:05.913012+0000","last_peered":"2024-04-26T22:57:05.913012+0000","last_clean":"2024-04-26T22:57:05.913012+0000","last_became_active":"2024-04-26T22:55:25.396372+0000","last_became_peered":"2024-04-26T22:55:25.396372+0000","last_unstale":"2024-04-26T22:57:05.913012+0000","last_undegraded":"2024-04-26T22:57:05.913012+0000","last_fullsized":"2024-04-26T22:57:05.913012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:32:12.068358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328401+0000","last_change":"2024-04-26T22:55:25.372059+0000","last_active":"2024-04-26T22:57:06.328401+0000","last_peered":"2024-04-26T22:57:06.328401+0000","last_clean":"2024-04-26T22:57:06.328401+0000","last_became_active":"2024-04-26T22:55:25.371750+0000","last_became_peered":"2024-04-26T22:55:25.371750+0000","last_unstale":"2024-04-26T22:57:06.328401+0000","last_undegraded":"2024-04-26T22:57:06.328401+0000","last_fullsized":"2024-04-26T22:57:06.328401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:10.230585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.908647+0000","last_change":"2024-04-26T22:55:25.374904+0000","last_active":"2024-04-26T22:57:05.908647+0000","last_peered":"2024-04-26T22:57:05.908647+0000","last_clean":"2024-04-26T22:57:05.908647+0000","last_became_active":"2024-04-26T22:55:25.374600+0000","last_became_peered":"2024-04-26T22:55:25.374600+0000","last_unstale":"2024-04-26T22:57:05.908647+0000","last_undegraded":"2024-04-26T22:57:05.908647+0000","last_fullsized":"2024-04-26T22:57:05.908647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:49.124981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.925653+0000","last_change":"2024-04-26T22:57:05.922936+0000","last_active":"2024-04-26T22:57:05.925653+0000","last_peered":"2024-04-26T22:57:05.925653+0000","last_clean":"2024-04-26T22:57:05.925653+0000","last_became_active":"2024-04-26T22:57:05.922061+0000","last_became_peered":"2024-04-26T22:57:05.922061+0000","last_unstale":"2024-04-26T22:57:05.925653+0000","last_undegraded":"2024-04-26T22:57:05.925653+0000","last_fullsized":"2024-04-26T22:57:05.925653+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:01.416256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.910627+0000","last_change":"2024-04-26T22:55:25.406584+0000","last_active":"2024-04-26T22:57:05.910627+0000","last_peered":"2024-04-26T22:57:05.910627+0000","last_clean":"2024-04-26T22:57:05.910627+0000","last_became_active":"2024-04-26T22:55:25.406352+0000","last_became_peered":"2024-04-26T22:55:25.406352+0000","last_unstale":"2024-04-26T22:57:05.910627+0000","last_undegraded":"2024-04-26T22:57:05.910627+0000","last_fullsized":"2024-04-26T22:57:05.910627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:43.594712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.332494+0000","last_change":"2024-04-26T22:55:25.399578+0000","last_active":"2024-04-26T22:57:06.332494+0000","last_peered":"2024-04-26T22:57:06.332494+0000","last_clean":"2024-04-26T22:57:06.332494+0000","last_became_active":"2024-04-26T22:55:25.396555+0000","last_became_peered":"2024-04-26T22:55:25.396555+0000","last_unstale":"2024-04-26T22:57:06.332494+0000","last_undegraded":"2024-04-26T22:57:06.332494+0000","last_fullsized":"2024-04-26T22:57:06.332494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:07:50.082978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.222158+0000","last_change":"2024-04-26T22:55:25.393034+0000","last_active":"2024-04-26T22:57:06.222158+0000","last_peered":"2024-04-26T22:57:06.222158+0000","last_clean":"2024-04-26T22:57:06.222158+0000","last_became_active":"2024-04-26T22:55:25.392730+0000","last_became_peered":"2024-04-26T22:55:25.392730+0000","last_unstale":"2024-04-26T22:57:06.222158+0000","last_undegraded":"2024-04-26T22:57:06.222158+0000","last_fullsized":"2024-04-26T22:57:06.222158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:11.040311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:05.911969+0000","last_change":"2024-04-26T22:55:25.398224+0000","last_active":"2024-04-26T22:57:05.911969+0000","last_peered":"2024-04-26T22:57:05.911969+0000","last_clean":"2024-04-26T22:57:05.911969+0000","last_became_active":"2024-04-26T22:55:25.397744+0000","last_became_peered":"2024-04-26T22:55:25.397744+0000","last_unstale":"2024-04-26T22:57:05.911969+0000","last_undegraded":"2024-04-26T22:57:05.911969+0000","last_fullsized":"2024-04-26T22:57:05.911969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:35.107942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T22:57:06.328225+0000","last_change":"2024-04-26T22:55:25.374076+0000","last_active":"2024-04-26T22:57:06.328225+0000","last_peered":"2024-04-26T22:57:06.328225+0000","last_clean":"2024-04-26T22:57:06.328225+0000","last_became_active":"2024-04-26T22:55:25.373736+0000","last_became_peered":"2024-04-26T22:55:25.373736+0000","last_unstale":"2024-04-26T22:57:06.328225+0000","last_undegraded":"2024-04-26T22:57:06.328225+0000","last_fullsized":"2024-04-26T22:57:06.328225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:55:24.742209+0000","last_clean_scrub_stamp":"2024-04-26T22:55:24.742209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:01:56.096004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282639,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59677,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":607,"num_read_kb":367,"num_write":165,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":179031,"data_compressed":5520,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":528241,"data_compressed":26744,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1167360,"data_stored":1171576,"data_compressed":64530,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":532573,"data_compressed":26744,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070523,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":528397,"data_compressed":26744,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":1124306,"data_compressed":62690,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495686,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":523653,"data_compressed":26744,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":579239,"data_compressed":28584,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":525041,"data_compressed":26744,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411422,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":580589,"data_compressed":28584,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182487,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":523847,"data_compressed":26744,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920849,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":528694,"data_compressed":26744,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757324,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1105920,"data_stored":1119503,"data_compressed":62690,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34303,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34303,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34303,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35946,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":55676,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":57026,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":57645,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:57:35.745 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T22:57:35.745 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T22:57:35.745 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T22:57:35.745 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T22:57:36.267 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:57:36.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:36 smithi027 ceph-mon[28034]: pgmap v399: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:36.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:36 smithi027 ceph-mon[28034]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:57:36.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:36 smithi080 ceph-mon[32324]: pgmap v399: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:36.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:36 smithi080 ceph-mon[32324]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:57:36.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:36 smithi094 ceph-mon[32889]: pgmap v399: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:36.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:36 smithi094 ceph-mon[32889]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:57:38.356 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:38 smithi027 ceph-mon[28034]: pgmap v400: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:57:38.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:38 smithi080 ceph-mon[32324]: pgmap v400: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:57:38.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:38 smithi094 ceph-mon[32889]: pgmap v400: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:57:38.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.709+0000 7f2f5c800700 1 -- 172.21.15.27:0/1125940130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f5406bad0 msgr2=0x7f2f541102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:38.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.709+0000 7f2f5c800700 1 --2- 172.21.15.27:0/1125940130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f5406bad0 0x7f2f541102e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000b0a0 tx=0x7f2f5000b3b0 comp rx=0 tx=0).stop 2024-04-26T22:57:38.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.709+0000 7f2f5c800700 1 -- 172.21.15.27:0/1125940130 shutdown_connections 2024-04-26T22:57:38.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.709+0000 7f2f5c800700 1 --2- 172.21.15.27:0/1125940130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f5406bad0 0x7f2f541102e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:38.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.709+0000 7f2f5c800700 1 --2- 172.21.15.27:0/1125940130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f541094a0 0x7f2f54109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:38.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.709+0000 7f2f5c800700 1 --2- 172.21.15.27:0/1125940130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f5406b130 0x7f2f5406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:38.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.709+0000 7f2f5c800700 1 -- 172.21.15.27:0/1125940130 >> 172.21.15.27:0/1125940130 conn(0x7f2f540792d0 msgr2=0x7f2f540796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:38.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.709+0000 7f2f5c800700 1 -- 172.21.15.27:0/1125940130 shutdown_connections 2024-04-26T22:57:38.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.709+0000 7f2f5c800700 1 -- 172.21.15.27:0/1125940130 wait complete. 2024-04-26T22:57:38.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.710+0000 7f2f5c800700 1 Processor -- start 2024-04-26T22:57:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.710+0000 7f2f5c800700 1 -- start start 2024-04-26T22:57:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.711+0000 7f2f5c800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f5406b130 0x7f2f541023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.711+0000 7f2f5c800700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f5406bad0 0x7f2f54102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.711+0000 7f2f5c800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f541094a0 0x7f2f54102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.711+0000 7f2f5c800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54113610 con 0x7f2f541094a0 2024-04-26T22:57:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.711+0000 7f2f5c800700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54113490 con 0x7f2f5406bad0 2024-04-26T22:57:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.711+0000 7f2f5c800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54113790 con 0x7f2f5406b130 2024-04-26T22:57:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.711+0000 7f2f59400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f5406bad0 0x7f2f54102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.711+0000 7f2f5a800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f541094a0 0x7f2f54102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.711+0000 7f2f59400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f5406bad0 0x7f2f54102930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:58564/0 (socket says 172.21.15.27:58564) 2024-04-26T22:57:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.711+0000 7f2f59400700 1 -- 172.21.15.27:0/359350612 learned_addr learned my addr 172.21.15.27:0/359350612 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.711+0000 7f2f59e00700 1 --2- 172.21.15.27:0/359350612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f5406b130 0x7f2f541023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:38.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.712+0000 7f2f5a800700 1 -- 172.21.15.27:0/359350612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f5406b130 msgr2=0x7f2f541023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:38.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.712+0000 7f2f5a800700 1 --2- 172.21.15.27:0/359350612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f5406b130 0x7f2f541023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:38.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.712+0000 7f2f5a800700 1 -- 172.21.15.27:0/359350612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f5406bad0 msgr2=0x7f2f54102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:38.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.712+0000 7f2f5a800700 1 --2- 172.21.15.27:0/359350612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f5406bad0 0x7f2f54102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:38.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.712+0000 7f2f5a800700 1 -- 172.21.15.27:0/359350612 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f541a8050 con 0x7f2f541094a0 2024-04-26T22:57:38.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.712+0000 7f2f59400700 1 --2- 172.21.15.27:0/359350612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f5406bad0 0x7f2f54102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:38.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.712+0000 7f2f5a800700 1 --2- 172.21.15.27:0/359350612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f541094a0 0x7f2f54102e70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000b070 tx=0x7f2f50004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:38.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.713+0000 7f2f4ea00700 1 -- 172.21.15.27:0/359350612 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f50021070 con 0x7f2f541094a0 2024-04-26T22:57:38.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.713+0000 7f2f5c800700 1 -- 172.21.15.27:0/359350612 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f541a82e0 con 0x7f2f541094a0 2024-04-26T22:57:38.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.713+0000 7f2f4ea00700 1 -- 172.21.15.27:0/359350612 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f50007e30 con 0x7f2f541094a0 2024-04-26T22:57:38.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.713+0000 7f2f4ea00700 1 -- 172.21.15.27:0/359350612 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f500045f0 con 0x7f2f541094a0 2024-04-26T22:57:38.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.713+0000 7f2f5c800700 1 -- 172.21.15.27:0/359350612 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f541a87e0 con 0x7f2f541094a0 2024-04-26T22:57:38.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.714+0000 7f2f5c800700 1 -- 172.21.15.27:0/359350612 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f5406a110 con 0x7f2f541094a0 2024-04-26T22:57:38.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.716+0000 7f2f4ea00700 1 -- 172.21.15.27:0/359350612 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2f50007980 con 0x7f2f541094a0 2024-04-26T22:57:38.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.718+0000 7f2f4ea00700 1 --2- 172.21.15.27:0/359350612 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2f3c0a3d50 0x7f2f3c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:38.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.718+0000 7f2f4ea00700 1 -- 172.21.15.27:0/359350612 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f2f5000f020 con 0x7f2f541094a0 2024-04-26T22:57:38.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.718+0000 7f2f59e00700 1 --2- 172.21.15.27:0/359350612 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2f3c0a3d50 0x7f2f3c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:38.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.719+0000 7f2f59e00700 1 --2- 172.21.15.27:0/359350612 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2f3c0a3d50 0x7f2f3c0a6210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009be0 tx=0x7f2f48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:38.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:38.723+0000 7f2f4ea00700 1 -- 172.21.15.27:0/359350612 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2f5002a020 con 0x7f2f541094a0 2024-04-26T22:57:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:39.008+0000 7f2f5c800700 1 -- 172.21.15.27:0/359350612 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2f5402e490 con 0x7f2f541094a0 2024-04-26T22:57:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:39.009+0000 7f2f4ea00700 1 -- 172.21.15.27:0/359350612 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2f5009e170 con 0x7f2f541094a0 2024-04-26T22:57:39.009 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:57:39.010 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T22:57:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:39.012+0000 7f2f5c800700 1 -- 172.21.15.27:0/359350612 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2f3c0a3d50 msgr2=0x7f2f3c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:39.012+0000 7f2f5c800700 1 --2- 172.21.15.27:0/359350612 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2f3c0a3d50 0x7f2f3c0a6210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009be0 tx=0x7f2f48009260 comp rx=0 tx=0).stop 2024-04-26T22:57:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:39.012+0000 7f2f5c800700 1 -- 172.21.15.27:0/359350612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f541094a0 msgr2=0x7f2f54102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:39.012+0000 7f2f5c800700 1 --2- 172.21.15.27:0/359350612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f541094a0 0x7f2f54102e70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000b070 tx=0x7f2f50004c80 comp rx=0 tx=0).stop 2024-04-26T22:57:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:39.012+0000 7f2f5c800700 1 -- 172.21.15.27:0/359350612 shutdown_connections 2024-04-26T22:57:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:39.012+0000 7f2f5c800700 1 --2- 172.21.15.27:0/359350612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f541094a0 0x7f2f54102e70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:39.012+0000 7f2f5c800700 1 --2- 172.21.15.27:0/359350612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f5406b130 0x7f2f541023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:39.012+0000 7f2f5c800700 1 --2- 172.21.15.27:0/359350612 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2f3c0a3d50 0x7f2f3c0a6210 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:39.012+0000 7f2f5c800700 1 --2- 172.21.15.27:0/359350612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f5406bad0 0x7f2f54102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:39.012+0000 7f2f5c800700 1 -- 172.21.15.27:0/359350612 >> 172.21.15.27:0/359350612 conn(0x7f2f540792d0 msgr2=0x7f2f54100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:39.013+0000 7f2f5c800700 1 -- 172.21.15.27:0/359350612 shutdown_connections 2024-04-26T22:57:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:39.013+0000 7f2f5c800700 1 -- 172.21.15.27:0/359350612 wait complete. 2024-04-26T22:57:39.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:39 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/359350612' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T22:57:39.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:39 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/359350612' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T22:57:39.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:39 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/359350612' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T22:57:39.941 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T22:57:39.941 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:57:40.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.157+0000 7f0746000640 1 -- 172.21.15.27:0/699492371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f074010a940 msgr2=0x7f07401111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:40.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.157+0000 7f0746000640 1 --2- 172.21.15.27:0/699492371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f074010a940 0x7f07401111d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0738009be0 tx=0x7f073802a6c0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.157+0000 7f0746000640 1 -- 172.21.15.27:0/699492371 shutdown_connections 2024-04-26T22:57:40.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.157+0000 7f0746000640 1 --2- 172.21.15.27:0/699492371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f074010a940 0x7f07401111d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.157+0000 7f0746000640 1 --2- 172.21.15.27:0/699492371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0740109f80 0x7f074010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.157+0000 7f0746000640 1 --2- 172.21.15.27:0/699492371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0740108d90 0x7f07401091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.157+0000 7f0746000640 1 -- 172.21.15.27:0/699492371 >> 172.21.15.27:0/699492371 conn(0x7f0740077d30 msgr2=0x7f074007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:40.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.157+0000 7f0746000640 1 -- 172.21.15.27:0/699492371 shutdown_connections 2024-04-26T22:57:40.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.157+0000 7f0746000640 1 -- 172.21.15.27:0/699492371 wait complete. 2024-04-26T22:57:40.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.158+0000 7f0746000640 1 Processor -- start 2024-04-26T22:57:40.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.158+0000 7f0746000640 1 -- start start 2024-04-26T22:57:40.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.158+0000 7f0746000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0740108d90 0x7f074019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:40.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.158+0000 7f0746000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0740109f80 0x7f074019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:40.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.158+0000 7f0746000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f074010a940 0x7f07401a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:40.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.158+0000 7f0746000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f07401148c0 con 0x7f074010a940 2024-04-26T22:57:40.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.158+0000 7f0746000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0740114740 con 0x7f0740108d90 2024-04-26T22:57:40.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.159+0000 7f0746000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0740114a40 con 0x7f0740109f80 2024-04-26T22:57:40.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.159+0000 7f0744c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0740108d90 0x7f074019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:40.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.159+0000 7f0744c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0740108d90 0x7f074019ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:58578/0 (socket says 172.21.15.27:58578) 2024-04-26T22:57:40.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.159+0000 7f0744c00640 1 -- 172.21.15.27:0/2352797593 learned_addr learned my addr 172.21.15.27:0/2352797593 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:40.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.159+0000 7f0745600640 1 --2- 172.21.15.27:0/2352797593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f074010a940 0x7f07401a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:40.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.159+0000 7f073fe00640 1 --2- 172.21.15.27:0/2352797593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0740109f80 0x7f074019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:40.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.159+0000 7f0744c00640 1 -- 172.21.15.27:0/2352797593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0740109f80 msgr2=0x7f074019f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:40.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.159+0000 7f0744c00640 1 --2- 172.21.15.27:0/2352797593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0740109f80 0x7f074019f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.159+0000 7f0744c00640 1 -- 172.21.15.27:0/2352797593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f074010a940 msgr2=0x7f07401a6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:40.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.159+0000 7f0744c00640 1 --2- 172.21.15.27:0/2352797593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f074010a940 0x7f07401a6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.159+0000 7f0744c00640 1 -- 172.21.15.27:0/2352797593 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07401a6a90 con 0x7f0740108d90 2024-04-26T22:57:40.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.159+0000 7f0744c00640 1 --2- 172.21.15.27:0/2352797593 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0740108d90 0x7f074019ee70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f073400e9c0 tx=0x7f073400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:40.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.160+0000 7f073d600640 1 -- 172.21.15.27:0/2352797593 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073400cd80 con 0x7f0740108d90 2024-04-26T22:57:40.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.160+0000 7f073d600640 1 -- 172.21.15.27:0/2352797593 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07340035f0 con 0x7f0740108d90 2024-04-26T22:57:40.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.160+0000 7f0746000640 1 -- 172.21.15.27:0/2352797593 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07401a6d80 con 0x7f0740108d90 2024-04-26T22:57:40.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.160+0000 7f073d600640 1 -- 172.21.15.27:0/2352797593 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734010690 con 0x7f0740108d90 2024-04-26T22:57:40.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.160+0000 7f0746000640 1 -- 172.21.15.27:0/2352797593 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07401a7210 con 0x7f0740108d90 2024-04-26T22:57:40.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.161+0000 7f0746000640 1 -- 172.21.15.27:0/2352797593 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0740110050 con 0x7f0740108d90 2024-04-26T22:57:40.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.163+0000 7f073d600640 1 -- 172.21.15.27:0/2352797593 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0734004000 con 0x7f0740108d90 2024-04-26T22:57:40.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.164+0000 7f073d600640 1 --2- 172.21.15.27:0/2352797593 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f07280a3d70 0x7f07280a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:40.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.164+0000 7f073d600640 1 -- 172.21.15.27:0/2352797593 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f07340d0f70 con 0x7f0740108d90 2024-04-26T22:57:40.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.164+0000 7f073fe00640 1 --2- 172.21.15.27:0/2352797593 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f07280a3d70 0x7f07280a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:40.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.165+0000 7f073fe00640 1 --2- 172.21.15.27:0/2352797593 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f07280a3d70 0x7f07280a6230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f07401a0380 tx=0x7f072c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:40.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.169+0000 7f073d600640 1 -- 172.21.15.27:0/2352797593 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f073409d360 con 0x7f0740108d90 2024-04-26T22:57:40.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:40 smithi027 ceph-mon[28034]: pgmap v401: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:40.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.376+0000 7f0746000640 1 -- 172.21.15.27:0/2352797593 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0740063640 con 0x7f0740108d90 2024-04-26T22:57:40.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.378+0000 7f073d600640 1 -- 172.21.15.27:0/2352797593 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24902 (secure 0 0 0) 0x7f07340bbe20 con 0x7f0740108d90 2024-04-26T22:57:40.379 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:57:40.379 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":94,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:57:05.902371+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4},{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:57:40.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.381+0000 7f0746000640 1 -- 172.21.15.27:0/2352797593 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f07280a3d70 msgr2=0x7f07280a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:40.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.381+0000 7f0746000640 1 --2- 172.21.15.27:0/2352797593 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f07280a3d70 0x7f07280a6230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f07401a0380 tx=0x7f072c009260 comp rx=0 tx=0).stop 2024-04-26T22:57:40.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.381+0000 7f0746000640 1 -- 172.21.15.27:0/2352797593 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0740108d90 msgr2=0x7f074019ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:40.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.381+0000 7f0746000640 1 --2- 172.21.15.27:0/2352797593 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0740108d90 0x7f074019ee70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f073400e9c0 tx=0x7f073400ee80 comp rx=0 tx=0).stop 2024-04-26T22:57:40.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.381+0000 7f0746000640 1 -- 172.21.15.27:0/2352797593 shutdown_connections 2024-04-26T22:57:40.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.381+0000 7f0746000640 1 --2- 172.21.15.27:0/2352797593 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f07280a3d70 0x7f07280a6230 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.381+0000 7f0746000640 1 --2- 172.21.15.27:0/2352797593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f074010a940 0x7f07401a6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.381+0000 7f0746000640 1 --2- 172.21.15.27:0/2352797593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0740109f80 0x7f074019f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.381+0000 7f0746000640 1 --2- 172.21.15.27:0/2352797593 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0740108d90 0x7f074019ee70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.381+0000 7f0746000640 1 -- 172.21.15.27:0/2352797593 >> 172.21.15.27:0/2352797593 conn(0x7f0740077d30 msgr2=0x7f074010d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:40.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.381+0000 7f0746000640 1 -- 172.21.15.27:0/2352797593 shutdown_connections 2024-04-26T22:57:40.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.381+0000 7f0746000640 1 -- 172.21.15.27:0/2352797593 wait complete. 2024-04-26T22:57:40.403 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:49:54.482679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:55:22.899174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:55:24.218295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T22:55:28.394547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T22:57:40.403 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:57:40.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:40 smithi080 ceph-mon[32324]: pgmap v401: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:40.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:40 smithi094 ceph-mon[32889]: pgmap v401: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:40.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.626+0000 7fa2ecc00640 1 -- 172.21.15.27:0/3225785023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2e8102300 msgr2=0x7fa2e8102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:40.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.626+0000 7fa2ecc00640 1 --2- 172.21.15.27:0/3225785023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2e8102300 0x7fa2e8102780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009ca0 tx=0x7fa2e002a8c0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.627+0000 7fa2ecc00640 1 -- 172.21.15.27:0/3225785023 shutdown_connections 2024-04-26T22:57:40.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.627+0000 7fa2ecc00640 1 --2- 172.21.15.27:0/3225785023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2e8102300 0x7fa2e8102780 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.627+0000 7fa2ecc00640 1 --2- 172.21.15.27:0/3225785023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2e8101960 0x7fa2e8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.627+0000 7fa2ecc00640 1 --2- 172.21.15.27:0/3225785023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e8101000 0x7fa2e8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.627+0000 7fa2ecc00640 1 -- 172.21.15.27:0/3225785023 >> 172.21.15.27:0/3225785023 conn(0x7fa2e80fcc60 msgr2=0x7fa2e80ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:40.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.627+0000 7fa2ecc00640 1 -- 172.21.15.27:0/3225785023 shutdown_connections 2024-04-26T22:57:40.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.627+0000 7fa2ecc00640 1 -- 172.21.15.27:0/3225785023 wait complete. 2024-04-26T22:57:40.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.628+0000 7fa2ecc00640 1 Processor -- start 2024-04-26T22:57:40.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.628+0000 7fa2ecc00640 1 -- start start 2024-04-26T22:57:40.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.628+0000 7fa2ecc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2e8101000 0x7fa2e819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:40.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.628+0000 7fa2ecc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e8101960 0x7fa2e819b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:40.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.628+0000 7fa2ecc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2e8102300 0x7fa2e81a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:40.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.628+0000 7fa2ecc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e81104e0 con 0x7fa2e8101960 2024-04-26T22:57:40.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.628+0000 7fa2ecc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e8110360 con 0x7fa2e8101000 2024-04-26T22:57:40.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.628+0000 7fa2ecc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e8110660 con 0x7fa2e8102300 2024-04-26T22:57:40.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.629+0000 7fa2e6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e8101960 0x7fa2e819b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:40.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.629+0000 7fa2e7400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2e8101000 0x7fa2e819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:40.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.629+0000 7fa2e6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e8101960 0x7fa2e819b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41402/0 (socket says 172.21.15.27:41402) 2024-04-26T22:57:40.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.629+0000 7fa2e7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2e8102300 0x7fa2e81a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:40.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.629+0000 7fa2e7400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2e8101000 0x7fa2e819ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:58594/0 (socket says 172.21.15.27:58594) 2024-04-26T22:57:40.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.629+0000 7fa2e6a00640 1 -- 172.21.15.27:0/2843631558 learned_addr learned my addr 172.21.15.27:0/2843631558 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:40.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.629+0000 7fa2e7400640 1 -- 172.21.15.27:0/2843631558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2e8102300 msgr2=0x7fa2e81a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:40.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.629+0000 7fa2e7400640 1 --2- 172.21.15.27:0/2843631558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2e8102300 0x7fa2e81a2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.629+0000 7fa2e7400640 1 -- 172.21.15.27:0/2843631558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e8101960 msgr2=0x7fa2e819b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:40.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.629+0000 7fa2e7400640 1 --2- 172.21.15.27:0/2843631558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e8101960 0x7fa2e819b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.629+0000 7fa2e7400640 1 -- 172.21.15.27:0/2843631558 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e81a2770 con 0x7fa2e8101000 2024-04-26T22:57:40.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.629+0000 7fa2e7e00640 1 --2- 172.21.15.27:0/2843631558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2e8102300 0x7fa2e81a2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:40.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.629+0000 7fa2e6a00640 1 --2- 172.21.15.27:0/2843631558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e8101960 0x7fa2e819b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:40.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.630+0000 7fa2e7400640 1 --2- 172.21.15.27:0/2843631558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2e8101000 0x7fa2e819ab50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800e9f0 tx=0x7fa2d800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:40.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.630+0000 7fa2dfe00640 1 -- 172.21.15.27:0/2843631558 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d800ccc0 con 0x7fa2e8101000 2024-04-26T22:57:40.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.630+0000 7fa2dfe00640 1 -- 172.21.15.27:0/2843631558 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d80108e0 con 0x7fa2e8101000 2024-04-26T22:57:40.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.630+0000 7fa2ecc00640 1 -- 172.21.15.27:0/2843631558 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e81a2a60 con 0x7fa2e8101000 2024-04-26T22:57:40.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.630+0000 7fa2dfe00640 1 -- 172.21.15.27:0/2843631558 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8018880 con 0x7fa2e8101000 2024-04-26T22:57:40.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.630+0000 7fa2ecc00640 1 -- 172.21.15.27:0/2843631558 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e81a2fb0 con 0x7fa2e8101000 2024-04-26T22:57:40.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.631+0000 7fa2ecc00640 1 -- 172.21.15.27:0/2843631558 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e80506c0 con 0x7fa2e8101000 2024-04-26T22:57:40.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.633+0000 7fa2dfe00640 1 -- 172.21.15.27:0/2843631558 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa2d8010430 con 0x7fa2e8101000 2024-04-26T22:57:40.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.635+0000 7fa2dfe00640 1 --2- 172.21.15.27:0/2843631558 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2cc0a3c40 0x7fa2cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:40.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.635+0000 7fa2dfe00640 1 -- 172.21.15.27:0/2843631558 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fa2d80d11a0 con 0x7fa2e8101000 2024-04-26T22:57:40.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.635+0000 7fa2e6a00640 1 --2- 172.21.15.27:0/2843631558 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2cc0a3c40 0x7fa2cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:40.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.636+0000 7fa2e6a00640 1 --2- 172.21.15.27:0/2843631558 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2cc0a3c40 0x7fa2cc0a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa2e819c060 tx=0x7fa2d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:40.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.637+0000 7fa2dfe00640 1 -- 172.21.15.27:0/2843631558 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa2d809cf40 con 0x7fa2e8101000 2024-04-26T22:57:40.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.849+0000 7fa2ecc00640 1 -- 172.21.15.27:0/2843631558 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa2e802db70 con 0x7fa2e8101000 2024-04-26T22:57:40.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.849+0000 7fa2dfe00640 1 -- 172.21.15.27:0/2843631558 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fa2d80a1240 con 0x7fa2e8101000 2024-04-26T22:57:40.850 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T22:57:40.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.852+0000 7fa2ecc00640 1 -- 172.21.15.27:0/2843631558 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2cc0a3c40 msgr2=0x7fa2cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:40.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.852+0000 7fa2ecc00640 1 --2- 172.21.15.27:0/2843631558 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2cc0a3c40 0x7fa2cc0a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa2e819c060 tx=0x7fa2d4005f90 comp rx=0 tx=0).stop 2024-04-26T22:57:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.852+0000 7fa2ecc00640 1 -- 172.21.15.27:0/2843631558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2e8101000 msgr2=0x7fa2e819ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.852+0000 7fa2ecc00640 1 --2- 172.21.15.27:0/2843631558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2e8101000 0x7fa2e819ab50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800e9f0 tx=0x7fa2d800eeb0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.852+0000 7fa2ecc00640 1 -- 172.21.15.27:0/2843631558 shutdown_connections 2024-04-26T22:57:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.852+0000 7fa2ecc00640 1 --2- 172.21.15.27:0/2843631558 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2cc0a3c40 0x7fa2cc0a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.852+0000 7fa2ecc00640 1 --2- 172.21.15.27:0/2843631558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2e8102300 0x7fa2e81a2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.852+0000 7fa2ecc00640 1 --2- 172.21.15.27:0/2843631558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e8101960 0x7fa2e819b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.852+0000 7fa2ecc00640 1 --2- 172.21.15.27:0/2843631558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2e8101000 0x7fa2e819ab50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.852+0000 7fa2ecc00640 1 -- 172.21.15.27:0/2843631558 >> 172.21.15.27:0/2843631558 conn(0x7fa2e80fcc60 msgr2=0x7fa2e80fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.853+0000 7fa2ecc00640 1 -- 172.21.15.27:0/2843631558 shutdown_connections 2024-04-26T22:57:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:40.853+0000 7fa2ecc00640 1 -- 172.21.15.27:0/2843631558 wait complete. 2024-04-26T22:57:40.874 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:57:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.085+0000 7ff4fc800640 1 -- 172.21.15.27:0/2481113517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4106740 msgr2=0x7ff4f410cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.085+0000 7ff4fc800640 1 --2- 172.21.15.27:0/2481113517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4106740 0x7ff4f410cfd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009860 tx=0x7ff4f002a340 comp rx=0 tx=0).stop 2024-04-26T22:57:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.085+0000 7ff4fc800640 1 -- 172.21.15.27:0/2481113517 shutdown_connections 2024-04-26T22:57:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.085+0000 7ff4fc800640 1 --2- 172.21.15.27:0/2481113517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f4105d80 0x7ff4f4106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.085+0000 7ff4fc800640 1 --2- 172.21.15.27:0/2481113517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4106740 0x7ff4f410cfd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.085+0000 7ff4fc800640 1 --2- 172.21.15.27:0/2481113517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4104b90 0x7ff4f4104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.085+0000 7ff4fc800640 1 -- 172.21.15.27:0/2481113517 >> 172.21.15.27:0/2481113517 conn(0x7ff4f4100120 msgr2=0x7ff4f4102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.085+0000 7ff4fc800640 1 -- 172.21.15.27:0/2481113517 shutdown_connections 2024-04-26T22:57:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.085+0000 7ff4fc800640 1 -- 172.21.15.27:0/2481113517 wait complete. 2024-04-26T22:57:41.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.086+0000 7ff4fc800640 1 Processor -- start 2024-04-26T22:57:41.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.086+0000 7ff4fc800640 1 -- start start 2024-04-26T22:57:41.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.087+0000 7ff4fc800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f4104b90 0x7ff4f419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.087+0000 7ff4fc800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4105d80 0x7ff4f419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.087+0000 7ff4fc800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4106740 0x7ff4f41a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.087+0000 7ff4fc800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41106a0 con 0x7ff4f4105d80 2024-04-26T22:57:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.087+0000 7ff4fc800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4110520 con 0x7ff4f4104b90 2024-04-26T22:57:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.087+0000 7ff4fc800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4110820 con 0x7ff4f4106740 2024-04-26T22:57:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.087+0000 7ff4f9e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f4104b90 0x7ff4f419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.087+0000 7ff4f9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4105d80 0x7ff4f419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.087+0000 7ff4f9e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f4104b90 0x7ff4f419ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:58602/0 (socket says 172.21.15.27:58602) 2024-04-26T22:57:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.087+0000 7ff4f9e00640 1 -- 172.21.15.27:0/2321767459 learned_addr learned my addr 172.21.15.27:0/2321767459 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.087+0000 7ff4fa800640 1 --2- 172.21.15.27:0/2321767459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4106740 0x7ff4f41a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.087+0000 7ff4f9e00640 1 -- 172.21.15.27:0/2321767459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4106740 msgr2=0x7ff4f41a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.087+0000 7ff4f9e00640 1 --2- 172.21.15.27:0/2321767459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4106740 0x7ff4f41a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.087+0000 7ff4f9e00640 1 -- 172.21.15.27:0/2321767459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4105d80 msgr2=0x7ff4f419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:41.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.087+0000 7ff4f9e00640 1 --2- 172.21.15.27:0/2321767459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4105d80 0x7ff4f419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.088+0000 7ff4f9e00640 1 -- 172.21.15.27:0/2321767459 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f41a2990 con 0x7ff4f4104b90 2024-04-26T22:57:41.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.088+0000 7ff4f9e00640 1 --2- 172.21.15.27:0/2321767459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f4104b90 0x7ff4f419ad70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00ebc0 tx=0x7ff4ec00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:41.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.088+0000 7ff4eaa00640 1 -- 172.21.15.27:0/2321767459 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec003970 con 0x7ff4f4104b90 2024-04-26T22:57:41.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.088+0000 7ff4eaa00640 1 -- 172.21.15.27:0/2321767459 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4ec004030 con 0x7ff4f4104b90 2024-04-26T22:57:41.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.088+0000 7ff4eaa00640 1 -- 172.21.15.27:0/2321767459 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec0196e0 con 0x7ff4f4104b90 2024-04-26T22:57:41.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.088+0000 7ff4fc800640 1 -- 172.21.15.27:0/2321767459 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f41a2c80 con 0x7ff4f4104b90 2024-04-26T22:57:41.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.088+0000 7ff4fc800640 1 -- 172.21.15.27:0/2321767459 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f41a3180 con 0x7ff4f4104b90 2024-04-26T22:57:41.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.089+0000 7ff4fc800640 1 -- 172.21.15.27:0/2321767459 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f410bf80 con 0x7ff4f4104b90 2024-04-26T22:57:41.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.092+0000 7ff4eaa00640 1 -- 172.21.15.27:0/2321767459 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff4ec004b60 con 0x7ff4f4104b90 2024-04-26T22:57:41.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.093+0000 7ff4eaa00640 1 --2- 172.21.15.27:0/2321767459 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4e00a3cc0 0x7ff4e00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:41.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.093+0000 7ff4eaa00640 1 -- 172.21.15.27:0/2321767459 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7ff4ec0d13f0 con 0x7ff4f4104b90 2024-04-26T22:57:41.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.093+0000 7ff4f9400640 1 --2- 172.21.15.27:0/2321767459 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4e00a3cc0 0x7ff4e00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:41.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.094+0000 7ff4f9400640 1 --2- 172.21.15.27:0/2321767459 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4e00a3cc0 0x7ff4e00a6180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff4f419c280 tx=0x7ff4e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:41.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.095+0000 7ff4eaa00640 1 -- 172.21.15.27:0/2321767459 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff4ec09d7e0 con 0x7ff4f4104b90 2024-04-26T22:57:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.308+0000 7ff4fc800640 1 -- 172.21.15.27:0/2321767459 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff4f402dc80 con 0x7ff4f4104b90 2024-04-26T22:57:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.309+0000 7ff4eaa00640 1 -- 172.21.15.27:0/2321767459 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7ff4ec09d0e0 con 0x7ff4f4104b90 2024-04-26T22:57:41.309 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T22:57:41.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.312+0000 7ff4fc800640 1 -- 172.21.15.27:0/2321767459 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4e00a3cc0 msgr2=0x7ff4e00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:41.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.312+0000 7ff4fc800640 1 --2- 172.21.15.27:0/2321767459 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4e00a3cc0 0x7ff4e00a6180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff4f419c280 tx=0x7ff4e4009260 comp rx=0 tx=0).stop 2024-04-26T22:57:41.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.312+0000 7ff4fc800640 1 -- 172.21.15.27:0/2321767459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f4104b90 msgr2=0x7ff4f419ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:41.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.312+0000 7ff4fc800640 1 --2- 172.21.15.27:0/2321767459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f4104b90 0x7ff4f419ad70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00ebc0 tx=0x7ff4ec00c2d0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.312+0000 7ff4fc800640 1 -- 172.21.15.27:0/2321767459 shutdown_connections 2024-04-26T22:57:41.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.312+0000 7ff4fc800640 1 --2- 172.21.15.27:0/2321767459 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4e00a3cc0 0x7ff4e00a6180 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.312+0000 7ff4fc800640 1 --2- 172.21.15.27:0/2321767459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4106740 0x7ff4f41a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.312+0000 7ff4fc800640 1 --2- 172.21.15.27:0/2321767459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4105d80 0x7ff4f419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.312+0000 7ff4fc800640 1 --2- 172.21.15.27:0/2321767459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f4104b90 0x7ff4f419ad70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.312+0000 7ff4fc800640 1 -- 172.21.15.27:0/2321767459 >> 172.21.15.27:0/2321767459 conn(0x7ff4f4100120 msgr2=0x7ff4f4109150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:41.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.312+0000 7ff4fc800640 1 -- 172.21.15.27:0/2321767459 shutdown_connections 2024-04-26T22:57:41.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.312+0000 7ff4fc800640 1 -- 172.21.15.27:0/2321767459 wait complete. 2024-04-26T22:57:41.330 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:41 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2352797593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:57:41.330 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:41 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2843631558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:57:41.333 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:57:41.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:41 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2352797593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:57:41.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:41 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2843631558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:57:41.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:41 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2352797593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:57:41.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:41 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2843631558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:57:41.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.555+0000 7f00c6a00640 1 -- 172.21.15.27:0/4049557505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00c010a1a0 msgr2=0x7f00c010a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:41.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.555+0000 7f00c6a00640 1 --2- 172.21.15.27:0/4049557505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00c010a1a0 0x7f00c010a620 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009900 tx=0x7f00ac02a400 comp rx=0 tx=0).stop 2024-04-26T22:57:41.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.556+0000 7f00c6a00640 1 -- 172.21.15.27:0/4049557505 shutdown_connections 2024-04-26T22:57:41.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.556+0000 7f00c6a00640 1 --2- 172.21.15.27:0/4049557505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00c010a1a0 0x7f00c010a620 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.556+0000 7f00c6a00640 1 --2- 172.21.15.27:0/4049557505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00c010ab60 0x7f00c01113f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.556+0000 7f00c6a00640 1 --2- 172.21.15.27:0/4049557505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c0108fb0 0x7f00c01093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.556+0000 7f00c6a00640 1 -- 172.21.15.27:0/4049557505 >> 172.21.15.27:0/4049557505 conn(0x7f00c0077f80 msgr2=0x7f00c007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:41.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.556+0000 7f00c6a00640 1 -- 172.21.15.27:0/4049557505 shutdown_connections 2024-04-26T22:57:41.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.556+0000 7f00c6a00640 1 -- 172.21.15.27:0/4049557505 wait complete. 2024-04-26T22:57:41.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.557+0000 7f00c6a00640 1 Processor -- start 2024-04-26T22:57:41.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.557+0000 7f00c6a00640 1 -- start start 2024-04-26T22:57:41.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.557+0000 7f00c6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c0108fb0 0x7f00c019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:41.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.557+0000 7f00c6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00c010a1a0 0x7f00c019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.557+0000 7f00c6a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00c010ab60 0x7f00c01a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.557+0000 7f00c6a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f00c0114b40 con 0x7f00c0108fb0 2024-04-26T22:57:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.557+0000 7f00c6a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f00c01149c0 con 0x7f00c010ab60 2024-04-26T22:57:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.557+0000 7f00c6a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00c0114cc0 con 0x7f00c010a1a0 2024-04-26T22:57:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.558+0000 7f00bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c0108fb0 0x7f00c019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.558+0000 7f00bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c0108fb0 0x7f00c019f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41436/0 (socket says 172.21.15.27:41436) 2024-04-26T22:57:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.558+0000 7f00bfe00640 1 -- 172.21.15.27:0/2150020547 learned_addr learned my addr 172.21.15.27:0/2150020547 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.558+0000 7f00bf400640 1 --2- 172.21.15.27:0/2150020547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00c010a1a0 0x7f00c019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.558+0000 7f00c4a00640 1 --2- 172.21.15.27:0/2150020547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00c010ab60 0x7f00c01a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.558+0000 7f00bfe00640 1 -- 172.21.15.27:0/2150020547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00c010a1a0 msgr2=0x7f00c019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.558+0000 7f00bfe00640 1 --2- 172.21.15.27:0/2150020547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00c010a1a0 0x7f00c019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.558+0000 7f00bfe00640 1 -- 172.21.15.27:0/2150020547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00c010ab60 msgr2=0x7f00c01a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.558+0000 7f00bfe00640 1 --2- 172.21.15.27:0/2150020547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00c010ab60 0x7f00c01a6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.558+0000 7f00bfe00640 1 -- 172.21.15.27:0/2150020547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c01a6d80 con 0x7f00c0108fb0 2024-04-26T22:57:41.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.558+0000 7f00bf400640 1 --2- 172.21.15.27:0/2150020547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00c010a1a0 0x7f00c019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:41.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.558+0000 7f00bfe00640 1 --2- 172.21.15.27:0/2150020547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c0108fb0 0x7f00c019f160 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f00b000e9c0 tx=0x7f00b000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:41.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.559+0000 7f00bcc00640 1 -- 172.21.15.27:0/2150020547 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b000cd80 con 0x7f00c0108fb0 2024-04-26T22:57:41.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.559+0000 7f00bcc00640 1 -- 172.21.15.27:0/2150020547 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00b00035f0 con 0x7f00c0108fb0 2024-04-26T22:57:41.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.559+0000 7f00bcc00640 1 -- 172.21.15.27:0/2150020547 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0010690 con 0x7f00c0108fb0 2024-04-26T22:57:41.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.559+0000 7f00c6a00640 1 -- 172.21.15.27:0/2150020547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c01a7070 con 0x7f00c0108fb0 2024-04-26T22:57:41.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.559+0000 7f00c6a00640 1 -- 172.21.15.27:0/2150020547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c01a7540 con 0x7f00c0108fb0 2024-04-26T22:57:41.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.560+0000 7f00c6a00640 1 -- 172.21.15.27:0/2150020547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c01102d0 con 0x7f00c0108fb0 2024-04-26T22:57:41.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.562+0000 7f00bcc00640 1 -- 172.21.15.27:0/2150020547 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f00b00037a0 con 0x7f00c0108fb0 2024-04-26T22:57:41.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.564+0000 7f00bcc00640 1 --2- 172.21.15.27:0/2150020547 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f00a40a3cc0 0x7f00a40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:41.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.564+0000 7f00bcc00640 1 -- 172.21.15.27:0/2150020547 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f00b00d0d00 con 0x7f00c0108fb0 2024-04-26T22:57:41.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.564+0000 7f00bf400640 1 --2- 172.21.15.27:0/2150020547 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f00a40a3cc0 0x7f00a40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:41.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.565+0000 7f00bf400640 1 --2- 172.21.15.27:0/2150020547 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f00a40a3cc0 0x7f00a40a6180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f00ac031500 tx=0x7f00ac005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:41.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.568+0000 7f00bcc00640 1 -- 172.21.15.27:0/2150020547 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f00b009d0f0 con 0x7f00c0108fb0 2024-04-26T22:57:41.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.777+0000 7f00c6a00640 1 -- 172.21.15.27:0/2150020547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f00c002dd80 con 0x7f00c0108fb0 2024-04-26T22:57:41.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.778+0000 7f00bcc00640 1 -- 172.21.15.27:0/2150020547 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f00b009c9f0 con 0x7f00c0108fb0 2024-04-26T22:57:41.779 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-26T22:57:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.781+0000 7f00c6a00640 1 -- 172.21.15.27:0/2150020547 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f00a40a3cc0 msgr2=0x7f00a40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.781+0000 7f00c6a00640 1 --2- 172.21.15.27:0/2150020547 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f00a40a3cc0 0x7f00a40a6180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f00ac031500 tx=0x7f00ac005a20 comp rx=0 tx=0).stop 2024-04-26T22:57:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.782+0000 7f00c6a00640 1 -- 172.21.15.27:0/2150020547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c0108fb0 msgr2=0x7f00c019f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.782+0000 7f00c6a00640 1 --2- 172.21.15.27:0/2150020547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c0108fb0 0x7f00c019f160 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f00b000e9c0 tx=0x7f00b000ee80 comp rx=0 tx=0).stop 2024-04-26T22:57:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.782+0000 7f00c6a00640 1 -- 172.21.15.27:0/2150020547 shutdown_connections 2024-04-26T22:57:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.782+0000 7f00c6a00640 1 --2- 172.21.15.27:0/2150020547 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f00a40a3cc0 0x7f00a40a6180 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.782+0000 7f00c6a00640 1 --2- 172.21.15.27:0/2150020547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00c010ab60 0x7f00c01a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.782+0000 7f00c6a00640 1 --2- 172.21.15.27:0/2150020547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00c010a1a0 0x7f00c019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.782+0000 7f00c6a00640 1 --2- 172.21.15.27:0/2150020547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c0108fb0 0x7f00c019f160 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:41.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.782+0000 7f00c6a00640 1 -- 172.21.15.27:0/2150020547 >> 172.21.15.27:0/2150020547 conn(0x7f00c0077f80 msgr2=0x7f00c010d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:41.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.782+0000 7f00c6a00640 1 -- 172.21.15.27:0/2150020547 shutdown_connections 2024-04-26T22:57:41.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:41.782+0000 7f00c6a00640 1 -- 172.21.15.27:0/2150020547 wait complete. 2024-04-26T22:57:41.800 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T22:57:42.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.011+0000 7f83c2800640 1 -- 172.21.15.27:0/3886960119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83bc0772f0 msgr2=0x7f83bc077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:42.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.011+0000 7f83c2800640 1 --2- 172.21.15.27:0/3886960119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83bc0772f0 0x7f83bc077710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009ca0 tx=0x7f83b002a8c0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.011+0000 7f83c2800640 1 -- 172.21.15.27:0/3886960119 shutdown_connections 2024-04-26T22:57:42.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.011+0000 7f83c2800640 1 --2- 172.21.15.27:0/3886960119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83bc078530 0x7f83bc078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.011+0000 7f83c2800640 1 --2- 172.21.15.27:0/3886960119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc078ed0 0x7f83bc079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.011+0000 7f83c2800640 1 --2- 172.21.15.27:0/3886960119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83bc0772f0 0x7f83bc077710 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.011+0000 7f83c2800640 1 -- 172.21.15.27:0/3886960119 >> 172.21.15.27:0/3886960119 conn(0x7f83bc0fff40 msgr2=0x7f83bc102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:42.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.011+0000 7f83c2800640 1 -- 172.21.15.27:0/3886960119 shutdown_connections 2024-04-26T22:57:42.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.011+0000 7f83c2800640 1 -- 172.21.15.27:0/3886960119 wait complete. 2024-04-26T22:57:42.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.012+0000 7f83c2800640 1 Processor -- start 2024-04-26T22:57:42.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.012+0000 7f83c2800640 1 -- start start 2024-04-26T22:57:42.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.013+0000 7f83c2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83bc0772f0 0x7f83bc19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:42.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.013+0000 7f83c2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc078530 0x7f83bc19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:42.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.013+0000 7f83c2800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83bc078ed0 0x7f83bc1a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:42.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.013+0000 7f83c2800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1148d0 con 0x7f83bc078530 2024-04-26T22:57:42.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.013+0000 7f83c2800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc114750 con 0x7f83bc078ed0 2024-04-26T22:57:42.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.013+0000 7f83c2800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc114a50 con 0x7f83bc0772f0 2024-04-26T22:57:42.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.013+0000 7f83c0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc078530 0x7f83bc19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:42.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.013+0000 7f83c0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc078530 0x7f83bc19f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41456/0 (socket says 172.21.15.27:41456) 2024-04-26T22:57:42.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.013+0000 7f83c0a00640 1 -- 172.21.15.27:0/2166759105 learned_addr learned my addr 172.21.15.27:0/2166759105 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:42.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.013+0000 7f83c1400640 1 --2- 172.21.15.27:0/2166759105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83bc0772f0 0x7f83bc19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:42.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.013+0000 7f83c1e00640 1 --2- 172.21.15.27:0/2166759105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83bc078ed0 0x7f83bc1a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:42.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.013+0000 7f83c0a00640 1 -- 172.21.15.27:0/2166759105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83bc0772f0 msgr2=0x7f83bc19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:42.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.013+0000 7f83c0a00640 1 --2- 172.21.15.27:0/2166759105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83bc0772f0 0x7f83bc19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.013+0000 7f83c0a00640 1 -- 172.21.15.27:0/2166759105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83bc078ed0 msgr2=0x7f83bc1a65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:42.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.013+0000 7f83c0a00640 1 --2- 172.21.15.27:0/2166759105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83bc078ed0 0x7f83bc1a65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.014+0000 7f83c0a00640 1 -- 172.21.15.27:0/2166759105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc1a6c20 con 0x7f83bc078530 2024-04-26T22:57:42.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.014+0000 7f83c0a00640 1 --2- 172.21.15.27:0/2166759105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc078530 0x7f83bc19f540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f83a800ea60 tx=0x7f83a800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:42.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.014+0000 7f83ba000640 1 -- 172.21.15.27:0/2166759105 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a800cde0 con 0x7f83bc078530 2024-04-26T22:57:42.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.014+0000 7f83c2800640 1 -- 172.21.15.27:0/2166759105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83bc1a6f10 con 0x7f83bc078530 2024-04-26T22:57:42.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.014+0000 7f83ba000640 1 -- 172.21.15.27:0/2166759105 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a80035f0 con 0x7f83bc078530 2024-04-26T22:57:42.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.014+0000 7f83ba000640 1 -- 172.21.15.27:0/2166759105 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8010690 con 0x7f83bc078530 2024-04-26T22:57:42.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.014+0000 7f83c2800640 1 -- 172.21.15.27:0/2166759105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83bc1a73d0 con 0x7f83bc078530 2024-04-26T22:57:42.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.016+0000 7f83c2800640 1 -- 172.21.15.27:0/2166759105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83bc111340 con 0x7f83bc078530 2024-04-26T22:57:42.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.018+0000 7f83ba000640 1 -- 172.21.15.27:0/2166759105 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f83a80037a0 con 0x7f83bc078530 2024-04-26T22:57:42.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.020+0000 7f83ba000640 1 --2- 172.21.15.27:0/2166759105 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f83a40a3c40 0x7f83a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:42.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.020+0000 7f83ba000640 1 -- 172.21.15.27:0/2166759105 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f83a80d0ca0 con 0x7f83bc078530 2024-04-26T22:57:42.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.021+0000 7f83c1400640 1 --2- 172.21.15.27:0/2166759105 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f83a40a3c40 0x7f83a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:42.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.021+0000 7f83c1400640 1 --2- 172.21.15.27:0/2166759105 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f83a40a3c40 0x7f83a40a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f83b0005e30 tx=0x7f83b0005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:42.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.025+0000 7f83ba000640 1 -- 172.21.15.27:0/2166759105 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f83a809d090 con 0x7f83bc078530 2024-04-26T22:57:42.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.236+0000 7f83c2800640 1 -- 172.21.15.27:0/2166759105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f83bc02dd20 con 0x7f83bc078530 2024-04-26T22:57:42.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.237+0000 7f83ba000640 1 -- 172.21.15.27:0/2166759105 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f83a809c990 con 0x7f83bc078530 2024-04-26T22:57:42.237 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T22:57:42.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.240+0000 7f83c2800640 1 -- 172.21.15.27:0/2166759105 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f83a40a3c40 msgr2=0x7f83a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:42.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.240+0000 7f83c2800640 1 --2- 172.21.15.27:0/2166759105 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f83a40a3c40 0x7f83a40a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f83b0005e30 tx=0x7f83b0005dc0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.240+0000 7f83c2800640 1 -- 172.21.15.27:0/2166759105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc078530 msgr2=0x7f83bc19f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:42.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.240+0000 7f83c2800640 1 --2- 172.21.15.27:0/2166759105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc078530 0x7f83bc19f540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f83a800ea60 tx=0x7f83a800ef20 comp rx=0 tx=0).stop 2024-04-26T22:57:42.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.240+0000 7f83c2800640 1 -- 172.21.15.27:0/2166759105 shutdown_connections 2024-04-26T22:57:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.240+0000 7f83c2800640 1 --2- 172.21.15.27:0/2166759105 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f83a40a3c40 0x7f83a40a6100 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.240+0000 7f83c2800640 1 --2- 172.21.15.27:0/2166759105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83bc078ed0 0x7f83bc1a65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.240+0000 7f83c2800640 1 --2- 172.21.15.27:0/2166759105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc078530 0x7f83bc19f540 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.240+0000 7f83c2800640 1 --2- 172.21.15.27:0/2166759105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83bc0772f0 0x7f83bc19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.240+0000 7f83c2800640 1 -- 172.21.15.27:0/2166759105 >> 172.21.15.27:0/2166759105 conn(0x7f83bc0fff40 msgr2=0x7f83bc101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.240+0000 7f83c2800640 1 -- 172.21.15.27:0/2166759105 shutdown_connections 2024-04-26T22:57:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.240+0000 7f83c2800640 1 -- 172.21.15.27:0/2166759105 wait complete. 2024-04-26T22:57:42.261 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:57:42.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.467+0000 7fcb41c00640 1 -- 172.21.15.27:0/2558658061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb3c109f80 msgr2=0x7fcb3c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:42.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.467+0000 7fcb41c00640 1 --2- 172.21.15.27:0/2558658061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb3c109f80 0x7fcb3c10a400 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009900 tx=0x7fcb2802a3f0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.467+0000 7fcb41c00640 1 -- 172.21.15.27:0/2558658061 shutdown_connections 2024-04-26T22:57:42.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.467+0000 7fcb41c00640 1 --2- 172.21.15.27:0/2558658061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb3c109f80 0x7fcb3c10a400 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.467+0000 7fcb41c00640 1 --2- 172.21.15.27:0/2558658061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb3c10a940 0x7fcb3c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.467+0000 7fcb41c00640 1 --2- 172.21.15.27:0/2558658061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb3c108d90 0x7fcb3c1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.467+0000 7fcb41c00640 1 -- 172.21.15.27:0/2558658061 >> 172.21.15.27:0/2558658061 conn(0x7fcb3c077d30 msgr2=0x7fcb3c07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:42.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.468+0000 7fcb41c00640 1 -- 172.21.15.27:0/2558658061 shutdown_connections 2024-04-26T22:57:42.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.468+0000 7fcb41c00640 1 -- 172.21.15.27:0/2558658061 wait complete. 2024-04-26T22:57:42.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.469+0000 7fcb41c00640 1 Processor -- start 2024-04-26T22:57:42.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.469+0000 7fcb41c00640 1 -- start start 2024-04-26T22:57:42.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.469+0000 7fcb41c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb3c108d90 0x7fcb3c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:42.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.469+0000 7fcb41c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb3c109f80 0x7fcb3c19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:42.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.469+0000 7fcb41c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb3c10a940 0x7fcb3c1a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:42.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.469+0000 7fcb41c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c1148f0 con 0x7fcb3c10a940 2024-04-26T22:57:42.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.470+0000 7fcb41c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c114770 con 0x7fcb3c109f80 2024-04-26T22:57:42.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.470+0000 7fcb41c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c114a70 con 0x7fcb3c108d90 2024-04-26T22:57:42.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.470+0000 7fcb41200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb3c10a940 0x7fcb3c1a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:42.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.470+0000 7fcb3b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb3c109f80 0x7fcb3c19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:42.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.470+0000 7fcb41200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb3c10a940 0x7fcb3c1a6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41484/0 (socket says 172.21.15.27:41484) 2024-04-26T22:57:42.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.470+0000 7fcb41200640 1 -- 172.21.15.27:0/1125241501 learned_addr learned my addr 172.21.15.27:0/1125241501 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:42.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.470+0000 7fcb3be00640 1 --2- 172.21.15.27:0/1125241501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb3c108d90 0x7fcb3c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:42.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.470+0000 7fcb41200640 1 -- 172.21.15.27:0/1125241501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb3c108d90 msgr2=0x7fcb3c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:42.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.470+0000 7fcb41200640 1 --2- 172.21.15.27:0/1125241501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb3c108d90 0x7fcb3c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.470+0000 7fcb41200640 1 -- 172.21.15.27:0/1125241501 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb3c109f80 msgr2=0x7fcb3c19f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:42.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.470+0000 7fcb41200640 1 --2- 172.21.15.27:0/1125241501 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb3c109f80 0x7fcb3c19f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.470+0000 7fcb41200640 1 -- 172.21.15.27:0/1125241501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3c1a6b00 con 0x7fcb3c10a940 2024-04-26T22:57:42.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.470+0000 7fcb3b400640 1 --2- 172.21.15.27:0/1125241501 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb3c109f80 0x7fcb3c19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:42.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.471+0000 7fcb41200640 1 --2- 172.21.15.27:0/1125241501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb3c10a940 0x7fcb3c1a6490 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400efd0 tx=0x7fcb3400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:42.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.471+0000 7fcb38c00640 1 -- 172.21.15.27:0/1125241501 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34003b70 con 0x7fcb3c10a940 2024-04-26T22:57:42.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.471+0000 7fcb38c00640 1 -- 172.21.15.27:0/1125241501 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb34003d10 con 0x7fcb3c10a940 2024-04-26T22:57:42.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.471+0000 7fcb41c00640 1 -- 172.21.15.27:0/1125241501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3c1a6df0 con 0x7fcb3c10a940 2024-04-26T22:57:42.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.471+0000 7fcb38c00640 1 -- 172.21.15.27:0/1125241501 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb340108a0 con 0x7fcb3c10a940 2024-04-26T22:57:42.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.471+0000 7fcb41c00640 1 -- 172.21.15.27:0/1125241501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb3c1a7260 con 0x7fcb3c10a940 2024-04-26T22:57:42.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.473+0000 7fcb41c00640 1 -- 172.21.15.27:0/1125241501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb3c110050 con 0x7fcb3c10a940 2024-04-26T22:57:42.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.475+0000 7fcb38c00640 1 -- 172.21.15.27:0/1125241501 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcb34004020 con 0x7fcb3c10a940 2024-04-26T22:57:42.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.476+0000 7fcb38c00640 1 --2- 172.21.15.27:0/1125241501 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcb200a3cf0 0x7fcb200a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:42.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.476+0000 7fcb38c00640 1 -- 172.21.15.27:0/1125241501 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fcb340d0dc0 con 0x7fcb3c10a940 2024-04-26T22:57:42.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.477+0000 7fcb3be00640 1 --2- 172.21.15.27:0/1125241501 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcb200a3cf0 0x7fcb200a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:42.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.477+0000 7fcb3be00640 1 --2- 172.21.15.27:0/1125241501 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcb200a3cf0 0x7fcb200a61b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c109de0 tx=0x7fcb2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:42.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.480+0000 7fcb38c00640 1 -- 172.21.15.27:0/1125241501 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcb3409d210 con 0x7fcb3c10a940 2024-04-26T22:57:42.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:42 smithi080 ceph-mon[32324]: pgmap v402: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:57:42.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:42 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2321767459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:57:42.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:42 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2150020547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:57:42.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:42 smithi094 ceph-mon[32889]: pgmap v402: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:57:42.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:42 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2321767459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:57:42.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:42 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2150020547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:57:42.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:42 smithi027 ceph-mon[28034]: pgmap v402: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:57:42.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:42 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2321767459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:57:42.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:42 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2150020547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:57:42.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.728+0000 7fcb41c00640 1 -- 172.21.15.27:0/1125241501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb3c02dd20 con 0x7fcb3c10a940 2024-04-26T22:57:42.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.730+0000 7fcb38c00640 1 -- 172.21.15.27:0/1125241501 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11850 (secure 0 0 0) 0x7fcb3409cb10 con 0x7fcb3c10a940 2024-04-26T22:57:42.730 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:57:42.731 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T22:55:25.265331+0000","modified":"2024-04-26T22:55:54.648644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24578,"mds_1":24577,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/1620677319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1620677319},{"type":"v1","addr":"172.21.15.94:6839","nonce":1620677319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.27:6841/1571355196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1571355196},{"type":"v1","addr":"172.21.15.27:6841","nonce":1571355196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24577":{"gid":24577,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.27:6839/3193530521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3193530521},{"type":"v1","addr":"172.21.15.27:6839","nonce":3193530521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/95952000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":95952000},{"type":"v1","addr":"172.21.15.80:6839","nonce":95952000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24578,"qdb_cluster":[24560,24577,24578]},"id":1}]} 2024-04-26T22:57:42.731 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-04-26T22:57:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.732+0000 7fcb41c00640 1 -- 172.21.15.27:0/1125241501 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcb200a3cf0 msgr2=0x7fcb200a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.732+0000 7fcb41c00640 1 --2- 172.21.15.27:0/1125241501 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcb200a3cf0 0x7fcb200a61b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c109de0 tx=0x7fcb2c005f90 comp rx=0 tx=0).stop 2024-04-26T22:57:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.732+0000 7fcb41c00640 1 -- 172.21.15.27:0/1125241501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb3c10a940 msgr2=0x7fcb3c1a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.732+0000 7fcb41c00640 1 --2- 172.21.15.27:0/1125241501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb3c10a940 0x7fcb3c1a6490 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400efd0 tx=0x7fcb3400c6a0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.732+0000 7fcb41c00640 1 -- 172.21.15.27:0/1125241501 shutdown_connections 2024-04-26T22:57:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.732+0000 7fcb41c00640 1 --2- 172.21.15.27:0/1125241501 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fcb200a3cf0 0x7fcb200a61b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.732+0000 7fcb41c00640 1 --2- 172.21.15.27:0/1125241501 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb3c109f80 0x7fcb3c19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.733+0000 7fcb41c00640 1 --2- 172.21.15.27:0/1125241501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb3c10a940 0x7fcb3c1a6490 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.733+0000 7fcb41c00640 1 --2- 172.21.15.27:0/1125241501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb3c108d90 0x7fcb3c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.733+0000 7fcb41c00640 1 -- 172.21.15.27:0/1125241501 >> 172.21.15.27:0/1125241501 conn(0x7fcb3c077d30 msgr2=0x7fcb3c10d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:42.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.733+0000 7fcb41c00640 1 -- 172.21.15.27:0/1125241501 shutdown_connections 2024-04-26T22:57:42.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.733+0000 7fcb41c00640 1 -- 172.21.15.27:0/1125241501 wait complete. 2024-04-26T22:57:42.752 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:57:42.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.967+0000 7ff47fe00640 1 -- 172.21.15.27:0/3175299119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4801020d0 msgr2=0x7ff480102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:42.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.967+0000 7ff47fe00640 1 --2- 172.21.15.27:0/3175299119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4801020d0 0x7ff480102970 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff478009ca0 tx=0x7ff47802a8b0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.967+0000 7ff47fe00640 1 -- 172.21.15.27:0/3175299119 shutdown_connections 2024-04-26T22:57:42.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.967+0000 7ff47fe00640 1 --2- 172.21.15.27:0/3175299119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff480101730 0x7ff480101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.967+0000 7ff47fe00640 1 --2- 172.21.15.27:0/3175299119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4801020d0 0x7ff480102970 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.967+0000 7ff47fe00640 1 --2- 172.21.15.27:0/3175299119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4801006c0 0x7ff480100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.967+0000 7ff47fe00640 1 -- 172.21.15.27:0/3175299119 >> 172.21.15.27:0/3175299119 conn(0x7ff4800fbf00 msgr2=0x7ff4800fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:42.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.967+0000 7ff47fe00640 1 -- 172.21.15.27:0/3175299119 shutdown_connections 2024-04-26T22:57:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.967+0000 7ff47fe00640 1 -- 172.21.15.27:0/3175299119 wait complete. 2024-04-26T22:57:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.968+0000 7ff47fe00640 1 Processor -- start 2024-04-26T22:57:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.968+0000 7ff47fe00640 1 -- start start 2024-04-26T22:57:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.969+0000 7ff47fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4801006c0 0x7ff48019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.969+0000 7ff47fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff480101730 0x7ff48019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.969+0000 7ff47fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4801020d0 0x7ff4801a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.969+0000 7ff47fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff480110490 con 0x7ff4801020d0 2024-04-26T22:57:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.969+0000 7ff47fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff480110310 con 0x7ff4801006c0 2024-04-26T22:57:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.969+0000 7ff47fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff480110610 con 0x7ff480101730 2024-04-26T22:57:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.969+0000 7ff47f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4801020d0 0x7ff4801a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.969+0000 7ff47f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4801020d0 0x7ff4801a1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41504/0 (socket says 172.21.15.27:41504) 2024-04-26T22:57:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.969+0000 7ff47f400640 1 -- 172.21.15.27:0/1263499702 learned_addr learned my addr 172.21.15.27:0/1263499702 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.969+0000 7ff47ea00640 1 --2- 172.21.15.27:0/1263499702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4801006c0 0x7ff48019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.969+0000 7ff47e000640 1 --2- 172.21.15.27:0/1263499702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff480101730 0x7ff48019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.970+0000 7ff47ea00640 1 -- 172.21.15.27:0/1263499702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff480101730 msgr2=0x7ff48019af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.970+0000 7ff47ea00640 1 --2- 172.21.15.27:0/1263499702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff480101730 0x7ff48019af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.970+0000 7ff47ea00640 1 -- 172.21.15.27:0/1263499702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4801020d0 msgr2=0x7ff4801a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.970+0000 7ff47ea00640 1 --2- 172.21.15.27:0/1263499702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4801020d0 0x7ff4801a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.970+0000 7ff47ea00640 1 -- 172.21.15.27:0/1263499702 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4801a2650 con 0x7ff4801006c0 2024-04-26T22:57:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.970+0000 7ff47ea00640 1 --2- 172.21.15.27:0/1263499702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4801006c0 0x7ff48019aa30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff47000d6d0 tx=0x7ff47000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.970+0000 7ff477400640 1 -- 172.21.15.27:0/1263499702 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff470017070 con 0x7ff4801006c0 2024-04-26T22:57:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.970+0000 7ff477400640 1 -- 172.21.15.27:0/1263499702 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff470009820 con 0x7ff4801006c0 2024-04-26T22:57:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.970+0000 7ff47fe00640 1 -- 172.21.15.27:0/1263499702 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4801a2940 con 0x7ff4801006c0 2024-04-26T22:57:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.970+0000 7ff477400640 1 -- 172.21.15.27:0/1263499702 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff470004c40 con 0x7ff4801006c0 2024-04-26T22:57:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.971+0000 7ff47fe00640 1 -- 172.21.15.27:0/1263499702 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4801a2ee0 con 0x7ff4801006c0 2024-04-26T22:57:42.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.972+0000 7ff47fe00640 1 -- 172.21.15.27:0/1263499702 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff48010cb10 con 0x7ff4801006c0 2024-04-26T22:57:42.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.974+0000 7ff477400640 1 -- 172.21.15.27:0/1263499702 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff470005220 con 0x7ff4801006c0 2024-04-26T22:57:42.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.975+0000 7ff477400640 1 --2- 172.21.15.27:0/1263499702 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4640a3d70 0x7ff4640a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:42.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.976+0000 7ff477400640 1 -- 172.21.15.27:0/1263499702 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7ff4700d0650 con 0x7ff4801006c0 2024-04-26T22:57:42.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.976+0000 7ff47e000640 1 --2- 172.21.15.27:0/1263499702 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4640a3d70 0x7ff4640a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:42.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.976+0000 7ff47e000640 1 --2- 172.21.15.27:0/1263499702 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4640a3d70 0x7ff4640a6230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0095f0 tx=0x7ff46c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:42.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:42.978+0000 7ff477400640 1 -- 172.21.15.27:0/1263499702 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff47009c470 con 0x7ff4801006c0 2024-04-26T22:57:43.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.189+0000 7ff47fe00640 1 -- 172.21.15.27:0/1263499702 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff48002dca0 con 0x7ff4801006c0 2024-04-26T22:57:43.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.191+0000 7ff477400640 1 -- 172.21.15.27:0/1263499702 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24902 (secure 0 0 0) 0x7ff4700a0770 con 0x7ff4801006c0 2024-04-26T22:57:43.193 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:57:43.193 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":94,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:57:05.902371+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4},{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:57:43.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.194+0000 7ff47fe00640 1 -- 172.21.15.27:0/1263499702 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4640a3d70 msgr2=0x7ff4640a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:43.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.194+0000 7ff47fe00640 1 --2- 172.21.15.27:0/1263499702 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4640a3d70 0x7ff4640a6230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0095f0 tx=0x7ff46c005f90 comp rx=0 tx=0).stop 2024-04-26T22:57:43.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.194+0000 7ff47fe00640 1 -- 172.21.15.27:0/1263499702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4801006c0 msgr2=0x7ff48019aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:43.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.194+0000 7ff47fe00640 1 --2- 172.21.15.27:0/1263499702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4801006c0 0x7ff48019aa30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff47000d6d0 tx=0x7ff47000db90 comp rx=0 tx=0).stop 2024-04-26T22:57:43.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.195+0000 7ff47fe00640 1 -- 172.21.15.27:0/1263499702 shutdown_connections 2024-04-26T22:57:43.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.195+0000 7ff47fe00640 1 --2- 172.21.15.27:0/1263499702 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff4640a3d70 0x7ff4640a6230 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.195+0000 7ff47fe00640 1 --2- 172.21.15.27:0/1263499702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4801020d0 0x7ff4801a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.195+0000 7ff47fe00640 1 --2- 172.21.15.27:0/1263499702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff480101730 0x7ff48019af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.195+0000 7ff47fe00640 1 --2- 172.21.15.27:0/1263499702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4801006c0 0x7ff48019aa30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.195+0000 7ff47fe00640 1 -- 172.21.15.27:0/1263499702 >> 172.21.15.27:0/1263499702 conn(0x7ff4800fbf00 msgr2=0x7ff4800fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:43.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.195+0000 7ff47fe00640 1 -- 172.21.15.27:0/1263499702 shutdown_connections 2024-04-26T22:57:43.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.195+0000 7ff47fe00640 1 -- 172.21.15.27:0/1263499702 wait complete. 2024-04-26T22:57:43.215 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:49:54.482679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:55:22.899174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:55:24.218295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T22:55:28.394547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T22:57:43.216 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:57:43.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.439+0000 7f655b800640 1 -- 172.21.15.27:0/1170857908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f655410c360 msgr2=0x7f655410c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:43.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.439+0000 7f655b800640 1 --2- 172.21.15.27:0/1170857908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f655410c360 0x7f655410c7e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6544009900 tx=0x7f654402a3f0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.439+0000 7f655b800640 1 -- 172.21.15.27:0/1170857908 shutdown_connections 2024-04-26T22:57:43.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.439+0000 7f655b800640 1 --2- 172.21.15.27:0/1170857908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f655410c360 0x7f655410c7e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.439+0000 7f655b800640 1 --2- 172.21.15.27:0/1170857908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655410cd20 0x7f65541135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.439+0000 7f655b800640 1 --2- 172.21.15.27:0/1170857908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f655410b170 0x7f655410b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.439+0000 7f655b800640 1 -- 172.21.15.27:0/1170857908 >> 172.21.15.27:0/1170857908 conn(0x7f6554071910 msgr2=0x7f6554073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:43.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.439+0000 7f655b800640 1 -- 172.21.15.27:0/1170857908 shutdown_connections 2024-04-26T22:57:43.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.439+0000 7f655b800640 1 -- 172.21.15.27:0/1170857908 wait complete. 2024-04-26T22:57:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.440+0000 7f655b800640 1 Processor -- start 2024-04-26T22:57:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.440+0000 7f655b800640 1 -- start start 2024-04-26T22:57:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.441+0000 7f655b800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f655410b170 0x7f65541a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.441+0000 7f655b800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f655410c360 0x7f65541a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.441+0000 7f655b800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655410cd20 0x7f65541a8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.441+0000 7f655b800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6554116d00 con 0x7f655410c360 2024-04-26T22:57:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.441+0000 7f655b800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6554116b80 con 0x7f655410b170 2024-04-26T22:57:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.441+0000 7f655b800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6554116e80 con 0x7f655410cd20 2024-04-26T22:57:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.441+0000 7f6558e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f655410b170 0x7f65541a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.441+0000 7f6553e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f655410c360 0x7f65541a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.441+0000 7f6559800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655410cd20 0x7f65541a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.441+0000 7f6558e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f655410b170 0x7f65541a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:58680/0 (socket says 172.21.15.27:58680) 2024-04-26T22:57:43.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.441+0000 7f6558e00640 1 -- 172.21.15.27:0/2689608703 learned_addr learned my addr 172.21.15.27:0/2689608703 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:43.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.442+0000 7f6559800640 1 -- 172.21.15.27:0/2689608703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f655410b170 msgr2=0x7f65541a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:43.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.442+0000 7f6559800640 1 --2- 172.21.15.27:0/2689608703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f655410b170 0x7f65541a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.442+0000 7f6559800640 1 -- 172.21.15.27:0/2689608703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f655410c360 msgr2=0x7f65541a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:43.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.442+0000 7f6559800640 1 --2- 172.21.15.27:0/2689608703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f655410c360 0x7f65541a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.442+0000 7f6559800640 1 -- 172.21.15.27:0/2689608703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65541a9000 con 0x7f655410cd20 2024-04-26T22:57:43.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.442+0000 7f6559800640 1 --2- 172.21.15.27:0/2689608703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655410cd20 0x7f65541a8990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f654c009fd0 tx=0x7f654c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:43.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.442+0000 7f6551600640 1 -- 172.21.15.27:0/2689608703 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c00ce20 con 0x7f655410cd20 2024-04-26T22:57:43.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.442+0000 7f655b800640 1 -- 172.21.15.27:0/2689608703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65541a92f0 con 0x7f655410cd20 2024-04-26T22:57:43.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.442+0000 7f6551600640 1 -- 172.21.15.27:0/2689608703 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f654c005020 con 0x7f655410cd20 2024-04-26T22:57:43.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.442+0000 7f6551600640 1 -- 172.21.15.27:0/2689608703 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c010690 con 0x7f655410cd20 2024-04-26T22:57:43.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.442+0000 7f655b800640 1 -- 172.21.15.27:0/2689608703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65541a9860 con 0x7f655410cd20 2024-04-26T22:57:43.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.443+0000 7f655b800640 1 -- 172.21.15.27:0/2689608703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6554112560 con 0x7f655410cd20 2024-04-26T22:57:43.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.446+0000 7f6551600640 1 -- 172.21.15.27:0/2689608703 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f654c0054d0 con 0x7f655410cd20 2024-04-26T22:57:43.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.447+0000 7f6551600640 1 --2- 172.21.15.27:0/2689608703 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f653c0a3cc0 0x7f653c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:43.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.447+0000 7f6551600640 1 -- 172.21.15.27:0/2689608703 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f654c0d1060 con 0x7f655410cd20 2024-04-26T22:57:43.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.447+0000 7f6558e00640 1 --2- 172.21.15.27:0/2689608703 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f653c0a3cc0 0x7f653c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:43.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.448+0000 7f6558e00640 1 --2- 172.21.15.27:0/2689608703 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f653c0a3cc0 0x7f653c0a6180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f65480095f0 tx=0x7f6548005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.452+0000 7f6551600640 1 -- 172.21.15.27:0/2689608703 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f654c09d450 con 0x7f655410cd20 2024-04-26T22:57:43.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:43 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2166759105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:57:43.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:43 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1125241501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:57:43.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:43 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2166759105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:57:43.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:43 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1125241501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:57:43.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:43 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2166759105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:57:43.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:43 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1125241501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:57:43.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.660+0000 7f655b800640 1 -- 172.21.15.27:0/2689608703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6554063530 con 0x7f655410cd20 2024-04-26T22:57:43.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.662+0000 7f6551600640 1 -- 172.21.15.27:0/2689608703 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f654c09cd50 con 0x7f655410cd20 2024-04-26T22:57:43.662 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T22:57:43.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.664+0000 7f655b800640 1 -- 172.21.15.27:0/2689608703 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f653c0a3cc0 msgr2=0x7f653c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:43.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.664+0000 7f655b800640 1 --2- 172.21.15.27:0/2689608703 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f653c0a3cc0 0x7f653c0a6180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f65480095f0 tx=0x7f6548005f90 comp rx=0 tx=0).stop 2024-04-26T22:57:43.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.664+0000 7f655b800640 1 -- 172.21.15.27:0/2689608703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655410cd20 msgr2=0x7f65541a8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:43.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.664+0000 7f655b800640 1 --2- 172.21.15.27:0/2689608703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655410cd20 0x7f65541a8990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f654c009fd0 tx=0x7f654c00ef10 comp rx=0 tx=0).stop 2024-04-26T22:57:43.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.665+0000 7f655b800640 1 -- 172.21.15.27:0/2689608703 shutdown_connections 2024-04-26T22:57:43.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.665+0000 7f655b800640 1 --2- 172.21.15.27:0/2689608703 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f653c0a3cc0 0x7f653c0a6180 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.665+0000 7f655b800640 1 --2- 172.21.15.27:0/2689608703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f655410cd20 0x7f65541a8990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.665+0000 7f655b800640 1 --2- 172.21.15.27:0/2689608703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f655410c360 0x7f65541a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.665+0000 7f655b800640 1 --2- 172.21.15.27:0/2689608703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f655410b170 0x7f65541a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.665+0000 7f655b800640 1 -- 172.21.15.27:0/2689608703 >> 172.21.15.27:0/2689608703 conn(0x7f6554071910 msgr2=0x7f655410f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:43.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.665+0000 7f655b800640 1 -- 172.21.15.27:0/2689608703 shutdown_connections 2024-04-26T22:57:43.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.665+0000 7f655b800640 1 -- 172.21.15.27:0/2689608703 wait complete. 2024-04-26T22:57:43.686 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:57:43.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.914+0000 7f230ee00640 1 -- 172.21.15.27:0/3356234785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2308108f90 msgr2=0x7f23081093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:43.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.914+0000 7f230ee00640 1 --2- 172.21.15.27:0/3356234785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2308108f90 0x7f23081093b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009900 tx=0x7f22f802a3f0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.914+0000 7f230ee00640 1 -- 172.21.15.27:0/3356234785 shutdown_connections 2024-04-26T22:57:43.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.914+0000 7f230ee00640 1 --2- 172.21.15.27:0/3356234785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f230810a180 0x7f230810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.914+0000 7f230ee00640 1 --2- 172.21.15.27:0/3356234785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f230810ab40 0x7f23081113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.914+0000 7f230ee00640 1 --2- 172.21.15.27:0/3356234785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2308108f90 0x7f23081093b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.914+0000 7f230ee00640 1 -- 172.21.15.27:0/3356234785 >> 172.21.15.27:0/3356234785 conn(0x7f2308077f40 msgr2=0x7f230807a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:43.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.914+0000 7f230ee00640 1 -- 172.21.15.27:0/3356234785 shutdown_connections 2024-04-26T22:57:43.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.914+0000 7f230ee00640 1 -- 172.21.15.27:0/3356234785 wait complete. 2024-04-26T22:57:43.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.915+0000 7f230ee00640 1 Processor -- start 2024-04-26T22:57:43.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.915+0000 7f230ee00640 1 -- start start 2024-04-26T22:57:43.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.916+0000 7f230ee00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2308108f90 0x7f230819f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:43.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.916+0000 7f230ee00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f230810a180 0x7f230819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:43.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.916+0000 7f230ee00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f230810ab40 0x7f23081a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:43.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.916+0000 7f230ee00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2308114b20 con 0x7f2308108f90 2024-04-26T22:57:43.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.916+0000 7f230ee00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f23081149a0 con 0x7f230810a180 2024-04-26T22:57:43.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.916+0000 7f230ee00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2308114ca0 con 0x7f230810ab40 2024-04-26T22:57:43.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.916+0000 7f2307e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2308108f90 0x7f230819f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:43.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.916+0000 7f2307e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2308108f90 0x7f230819f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41536/0 (socket says 172.21.15.27:41536) 2024-04-26T22:57:43.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.916+0000 7f2307400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f230810a180 0x7f230819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:43.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.916+0000 7f2307e00640 1 -- 172.21.15.27:0/1081534108 learned_addr learned my addr 172.21.15.27:0/1081534108 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:43.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.916+0000 7f230ce00640 1 --2- 172.21.15.27:0/1081534108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f230810ab40 0x7f23081a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:43.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.917+0000 7f2307400640 1 -- 172.21.15.27:0/1081534108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f230810ab40 msgr2=0x7f23081a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:43.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.917+0000 7f2307400640 1 --2- 172.21.15.27:0/1081534108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f230810ab40 0x7f23081a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.917+0000 7f2307400640 1 -- 172.21.15.27:0/1081534108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2308108f90 msgr2=0x7f230819f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:43.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.917+0000 7f2307400640 1 --2- 172.21.15.27:0/1081534108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2308108f90 0x7f230819f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:43.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.917+0000 7f2307400640 1 -- 172.21.15.27:0/1081534108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23081a6e20 con 0x7f230810a180 2024-04-26T22:57:43.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.917+0000 7f2307e00640 1 --2- 172.21.15.27:0/1081534108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2308108f90 0x7f230819f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:43.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.917+0000 7f2307400640 1 --2- 172.21.15.27:0/1081534108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f230810a180 0x7f230819f740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f22f400e9f0 tx=0x7f22f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:43.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.917+0000 7f2304c00640 1 -- 172.21.15.27:0/1081534108 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f400cd80 con 0x7f230810a180 2024-04-26T22:57:43.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.917+0000 7f2304c00640 1 -- 172.21.15.27:0/1081534108 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22f40035f0 con 0x7f230810a180 2024-04-26T22:57:43.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.917+0000 7f230ee00640 1 -- 172.21.15.27:0/1081534108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23081a7110 con 0x7f230810a180 2024-04-26T22:57:43.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.917+0000 7f2304c00640 1 -- 172.21.15.27:0/1081534108 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f4010670 con 0x7f230810a180 2024-04-26T22:57:43.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.918+0000 7f230ee00640 1 -- 172.21.15.27:0/1081534108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23081a7660 con 0x7f230810a180 2024-04-26T22:57:43.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.919+0000 7f230ee00640 1 -- 172.21.15.27:0/1081534108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2308110380 con 0x7f230810a180 2024-04-26T22:57:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.921+0000 7f2304c00640 1 -- 172.21.15.27:0/1081534108 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f22f40037a0 con 0x7f230810a180 2024-04-26T22:57:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.923+0000 7f2304c00640 1 --2- 172.21.15.27:0/1081534108 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f22ec0a3c40 0x7f22ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.923+0000 7f2304c00640 1 -- 172.21.15.27:0/1081534108 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f22f40d0c90 con 0x7f230810a180 2024-04-26T22:57:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.923+0000 7f2307e00640 1 --2- 172.21.15.27:0/1081534108 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f22ec0a3c40 0x7f22ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.924+0000 7f2307e00640 1 --2- 172.21.15.27:0/1081534108 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f22ec0a3c40 0x7f22ec0a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f22f8000c00 tx=0x7f22f8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:43.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:43.927+0000 7f2304c00640 1 -- 172.21.15.27:0/1081534108 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f22f409d080 con 0x7f230810a180 2024-04-26T22:57:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.140+0000 7f230ee00640 1 -- 172.21.15.27:0/1081534108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f230802dd20 con 0x7f230810a180 2024-04-26T22:57:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.141+0000 7f2304c00640 1 -- 172.21.15.27:0/1081534108 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f22f409c980 con 0x7f230810a180 2024-04-26T22:57:44.142 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T22:57:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.144+0000 7f230ee00640 1 -- 172.21.15.27:0/1081534108 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f22ec0a3c40 msgr2=0x7f22ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.144+0000 7f230ee00640 1 --2- 172.21.15.27:0/1081534108 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f22ec0a3c40 0x7f22ec0a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f22f8000c00 tx=0x7f22f8005a20 comp rx=0 tx=0).stop 2024-04-26T22:57:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.144+0000 7f230ee00640 1 -- 172.21.15.27:0/1081534108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f230810a180 msgr2=0x7f230819f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.144+0000 7f230ee00640 1 --2- 172.21.15.27:0/1081534108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f230810a180 0x7f230819f740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f22f400e9f0 tx=0x7f22f400eeb0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.144+0000 7f230ee00640 1 -- 172.21.15.27:0/1081534108 shutdown_connections 2024-04-26T22:57:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.144+0000 7f230ee00640 1 --2- 172.21.15.27:0/1081534108 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f22ec0a3c40 0x7f22ec0a6100 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.144+0000 7f230ee00640 1 --2- 172.21.15.27:0/1081534108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f230810a180 0x7f230819f740 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.144+0000 7f230ee00640 1 --2- 172.21.15.27:0/1081534108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f230810ab40 0x7f23081a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.144+0000 7f230ee00640 1 --2- 172.21.15.27:0/1081534108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2308108f90 0x7f230819f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.144+0000 7f230ee00640 1 -- 172.21.15.27:0/1081534108 >> 172.21.15.27:0/1081534108 conn(0x7f2308077f40 msgr2=0x7f230810d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.144+0000 7f230ee00640 1 -- 172.21.15.27:0/1081534108 shutdown_connections 2024-04-26T22:57:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.144+0000 7f230ee00640 1 -- 172.21.15.27:0/1081534108 wait complete. 2024-04-26T22:57:44.167 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:57:44.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.378+0000 7fedf3a00640 1 -- 172.21.15.27:0/4111154484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedec0772f0 msgr2=0x7fedec077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:44.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.378+0000 7fedf3a00640 1 --2- 172.21.15.27:0/4111154484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedec0772f0 0x7fedec077710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7feddc009ca0 tx=0x7feddc02a8c0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.378+0000 7fedf3a00640 1 -- 172.21.15.27:0/4111154484 shutdown_connections 2024-04-26T22:57:44.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.378+0000 7fedf3a00640 1 --2- 172.21.15.27:0/4111154484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedec078ed0 0x7fedec079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.378+0000 7fedf3a00640 1 --2- 172.21.15.27:0/4111154484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedec078530 0x7fedec078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.378+0000 7fedf3a00640 1 --2- 172.21.15.27:0/4111154484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedec0772f0 0x7fedec077710 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.378+0000 7fedf3a00640 1 -- 172.21.15.27:0/4111154484 >> 172.21.15.27:0/4111154484 conn(0x7fedec100120 msgr2=0x7fedec102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:44.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.379+0000 7fedf3a00640 1 -- 172.21.15.27:0/4111154484 shutdown_connections 2024-04-26T22:57:44.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.379+0000 7fedf3a00640 1 -- 172.21.15.27:0/4111154484 wait complete. 2024-04-26T22:57:44.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.379+0000 7fedf3a00640 1 Processor -- start 2024-04-26T22:57:44.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.380+0000 7fedf3a00640 1 -- start start 2024-04-26T22:57:44.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.380+0000 7fedf3a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedec0772f0 0x7fedec19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:44.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.380+0000 7fedf3a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedec078530 0x7fedec19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:44.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.380+0000 7fedf3a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedec078ed0 0x7fedec1a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:44.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.380+0000 7fedf3a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fedec114aa0 con 0x7fedec078ed0 2024-04-26T22:57:44.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.380+0000 7fedf3a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fedec114920 con 0x7fedec0772f0 2024-04-26T22:57:44.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.380+0000 7fedf3a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fedec114c20 con 0x7fedec078530 2024-04-26T22:57:44.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.380+0000 7fedf1a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedec078ed0 0x7fedec1a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:44.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.380+0000 7fedf1a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedec078ed0 0x7fedec1a6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41558/0 (socket says 172.21.15.27:41558) 2024-04-26T22:57:44.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.381+0000 7fedf1000640 1 --2- 172.21.15.27:0/3122343062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedec0772f0 0x7fedec19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:44.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.381+0000 7fedf1a00640 1 -- 172.21.15.27:0/3122343062 learned_addr learned my addr 172.21.15.27:0/3122343062 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:44.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.381+0000 7fedebe00640 1 --2- 172.21.15.27:0/3122343062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedec078530 0x7fedec19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:44.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.381+0000 7fedf1a00640 1 -- 172.21.15.27:0/3122343062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedec078530 msgr2=0x7fedec19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:44.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.381+0000 7fedf1a00640 1 --2- 172.21.15.27:0/3122343062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedec078530 0x7fedec19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.381+0000 7fedf1a00640 1 -- 172.21.15.27:0/3122343062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedec0772f0 msgr2=0x7fedec19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:44.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.381+0000 7fedf1a00640 1 --2- 172.21.15.27:0/3122343062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedec0772f0 0x7fedec19f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.381+0000 7fedf1a00640 1 -- 172.21.15.27:0/3122343062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedec1a6dc0 con 0x7fedec078ed0 2024-04-26T22:57:44.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.381+0000 7fedf1a00640 1 --2- 172.21.15.27:0/3122343062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedec078ed0 0x7fedec1a6750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fede400e970 tx=0x7fede400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:44.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.381+0000 7fede9600640 1 -- 172.21.15.27:0/3122343062 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede400ccc0 con 0x7fedec078ed0 2024-04-26T22:57:44.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.382+0000 7fedf3a00640 1 -- 172.21.15.27:0/3122343062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedec1a70b0 con 0x7fedec078ed0 2024-04-26T22:57:44.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.382+0000 7fede9600640 1 -- 172.21.15.27:0/3122343062 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede400ce60 con 0x7fedec078ed0 2024-04-26T22:57:44.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.382+0000 7fede9600640 1 -- 172.21.15.27:0/3122343062 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4018430 con 0x7fedec078ed0 2024-04-26T22:57:44.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.382+0000 7fedf3a00640 1 -- 172.21.15.27:0/3122343062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedec1a75b0 con 0x7fedec078ed0 2024-04-26T22:57:44.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.383+0000 7fedf3a00640 1 -- 172.21.15.27:0/3122343062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedec1116f0 con 0x7fedec078ed0 2024-04-26T22:57:44.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.385+0000 7fede9600640 1 -- 172.21.15.27:0/3122343062 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fede4004f70 con 0x7fedec078ed0 2024-04-26T22:57:44.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.387+0000 7fede9600640 1 --2- 172.21.15.27:0/3122343062 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fedd00a3c40 0x7fedd00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:44.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.387+0000 7fede9600640 1 -- 172.21.15.27:0/3122343062 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fede40d0f30 con 0x7fedec078ed0 2024-04-26T22:57:44.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.387+0000 7fedf1000640 1 --2- 172.21.15.27:0/3122343062 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fedd00a3c40 0x7fedd00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:44.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.388+0000 7fedf1000640 1 --2- 172.21.15.27:0/3122343062 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fedd00a3c40 0x7fedd00a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feddc005e30 tx=0x7feddc005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:44.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.391+0000 7fede9600640 1 -- 172.21.15.27:0/3122343062 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fede409d320 con 0x7fedec078ed0 2024-04-26T22:57:44.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:44 smithi080 ceph-mon[32324]: pgmap v403: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-26T22:57:44.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:44 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1263499702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:57:44.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:44 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2689608703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:57:44.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:44 smithi094 ceph-mon[32889]: pgmap v403: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-26T22:57:44.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:44 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1263499702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:57:44.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:44 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2689608703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:57:44.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.597+0000 7fedf3a00640 1 -- 172.21.15.27:0/3122343062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fedec063530 con 0x7fedec078ed0 2024-04-26T22:57:44.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.598+0000 7fede9600640 1 -- 172.21.15.27:0/3122343062 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fede4087020 con 0x7fedec078ed0 2024-04-26T22:57:44.598 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-26T22:57:44.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.600+0000 7fedf3a00640 1 -- 172.21.15.27:0/3122343062 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fedd00a3c40 msgr2=0x7fedd00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:44.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.600+0000 7fedf3a00640 1 --2- 172.21.15.27:0/3122343062 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fedd00a3c40 0x7fedd00a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feddc005e30 tx=0x7feddc005dc0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.601+0000 7fedf3a00640 1 -- 172.21.15.27:0/3122343062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedec078ed0 msgr2=0x7fedec1a6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:44.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.601+0000 7fedf3a00640 1 --2- 172.21.15.27:0/3122343062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedec078ed0 0x7fedec1a6750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fede400e970 tx=0x7fede400ed70 comp rx=0 tx=0).stop 2024-04-26T22:57:44.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.601+0000 7fedf3a00640 1 -- 172.21.15.27:0/3122343062 shutdown_connections 2024-04-26T22:57:44.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.601+0000 7fedf3a00640 1 --2- 172.21.15.27:0/3122343062 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fedd00a3c40 0x7fedd00a6100 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.601+0000 7fedf3a00640 1 --2- 172.21.15.27:0/3122343062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedec078ed0 0x7fedec1a6750 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.601+0000 7fedf3a00640 1 --2- 172.21.15.27:0/3122343062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedec078530 0x7fedec19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.601+0000 7fedf3a00640 1 --2- 172.21.15.27:0/3122343062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedec0772f0 0x7fedec19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.601+0000 7fedf3a00640 1 -- 172.21.15.27:0/3122343062 >> 172.21.15.27:0/3122343062 conn(0x7fedec100120 msgr2=0x7fedec101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:44.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.601+0000 7fedf3a00640 1 -- 172.21.15.27:0/3122343062 shutdown_connections 2024-04-26T22:57:44.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.601+0000 7fedf3a00640 1 -- 172.21.15.27:0/3122343062 wait complete. 2024-04-26T22:57:44.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:44 smithi027 ceph-mon[28034]: pgmap v403: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-26T22:57:44.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:44 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1263499702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:57:44.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:44 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2689608703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:57:44.621 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T22:57:44.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.840+0000 7fba21e00640 1 -- 172.21.15.27:0/816651669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba1c103970 msgr2=0x7fba1c105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:44.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.840+0000 7fba21e00640 1 --2- 172.21.15.27:0/816651669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba1c103970 0x7fba1c105de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009900 tx=0x7fba0c02a3f0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.841+0000 7fba21e00640 1 -- 172.21.15.27:0/816651669 shutdown_connections 2024-04-26T22:57:44.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.841+0000 7fba21e00640 1 --2- 172.21.15.27:0/816651669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba1c106320 0x7fba1c1067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.841+0000 7fba21e00640 1 --2- 172.21.15.27:0/816651669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba1c103970 0x7fba1c105de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.841+0000 7fba21e00640 1 --2- 172.21.15.27:0/816651669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba1c101000 0x7fba1c103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.841+0000 7fba21e00640 1 -- 172.21.15.27:0/816651669 >> 172.21.15.27:0/816651669 conn(0x7fba1c0fcc60 msgr2=0x7fba1c0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:44.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.841+0000 7fba21e00640 1 -- 172.21.15.27:0/816651669 shutdown_connections 2024-04-26T22:57:44.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.841+0000 7fba21e00640 1 -- 172.21.15.27:0/816651669 wait complete. 2024-04-26T22:57:44.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.842+0000 7fba21e00640 1 Processor -- start 2024-04-26T22:57:44.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.842+0000 7fba21e00640 1 -- start start 2024-04-26T22:57:44.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.842+0000 7fba21e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba1c101000 0x7fba1c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:44.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.842+0000 7fba21e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba1c103970 0x7fba1c19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:44.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.842+0000 7fba21e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba1c106320 0x7fba1c1a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:44.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.842+0000 7fba21e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1148e0 con 0x7fba1c106320 2024-04-26T22:57:44.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.842+0000 7fba21e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c114760 con 0x7fba1c103970 2024-04-26T22:57:44.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.842+0000 7fba21e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c114a60 con 0x7fba1c101000 2024-04-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.842+0000 7fba20a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba1c101000 0x7fba1c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.842+0000 7fba21400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba1c106320 0x7fba1c1a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.843+0000 7fba20a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba1c101000 0x7fba1c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:44470/0 (socket says 172.21.15.27:44470) 2024-04-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.843+0000 7fba20a00640 1 -- 172.21.15.27:0/4219045126 learned_addr learned my addr 172.21.15.27:0/4219045126 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.843+0000 7fba13e00640 1 --2- 172.21.15.27:0/4219045126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba1c103970 0x7fba1c19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.843+0000 7fba20a00640 1 -- 172.21.15.27:0/4219045126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba1c103970 msgr2=0x7fba1c19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.843+0000 7fba20a00640 1 --2- 172.21.15.27:0/4219045126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba1c103970 0x7fba1c19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.843+0000 7fba20a00640 1 -- 172.21.15.27:0/4219045126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba1c106320 msgr2=0x7fba1c1a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.843+0000 7fba20a00640 1 --2- 172.21.15.27:0/4219045126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba1c106320 0x7fba1c1a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.843+0000 7fba20a00640 1 -- 172.21.15.27:0/4219045126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c1a6bb0 con 0x7fba1c101000 2024-04-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.843+0000 7fba21400640 1 --2- 172.21.15.27:0/4219045126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba1c106320 0x7fba1c1a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.843+0000 7fba20a00640 1 --2- 172.21.15.27:0/4219045126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba1c101000 0x7fba1c19ef90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fba0400e9f0 tx=0x7fba0400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.844+0000 7fba1a000640 1 -- 172.21.15.27:0/4219045126 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0400ccc0 con 0x7fba1c101000 2024-04-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.844+0000 7fba1a000640 1 -- 172.21.15.27:0/4219045126 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba040108e0 con 0x7fba1c101000 2024-04-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.844+0000 7fba21e00640 1 -- 172.21.15.27:0/4219045126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1c1a6ea0 con 0x7fba1c101000 2024-04-26T22:57:44.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.844+0000 7fba1a000640 1 -- 172.21.15.27:0/4219045126 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04018880 con 0x7fba1c101000 2024-04-26T22:57:44.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.844+0000 7fba21e00640 1 -- 172.21.15.27:0/4219045126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1c1a7310 con 0x7fba1c101000 2024-04-26T22:57:44.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.845+0000 7fba21e00640 1 -- 172.21.15.27:0/4219045126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba1c0fe750 con 0x7fba1c101000 2024-04-26T22:57:44.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.847+0000 7fba1a000640 1 -- 172.21.15.27:0/4219045126 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fba04010430 con 0x7fba1c101000 2024-04-26T22:57:44.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.849+0000 7fba1a000640 1 --2- 172.21.15.27:0/4219045126 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb9fc0a3cc0 0x7fb9fc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:44.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.849+0000 7fba1a000640 1 -- 172.21.15.27:0/4219045126 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fba040d0d40 con 0x7fba1c101000 2024-04-26T22:57:44.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.849+0000 7fba13e00640 1 --2- 172.21.15.27:0/4219045126 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb9fc0a3cc0 0x7fb9fc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:44.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.850+0000 7fba13e00640 1 --2- 172.21.15.27:0/4219045126 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb9fc0a3cc0 0x7fb9fc0a6180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fba0c031500 tx=0x7fba0c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:44.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:44.851+0000 7fba1a000640 1 -- 172.21.15.27:0/4219045126 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fba0409d130 con 0x7fba1c101000 2024-04-26T22:57:45.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.061+0000 7fba21e00640 1 -- 172.21.15.27:0/4219045126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fba1c02dd40 con 0x7fba1c101000 2024-04-26T22:57:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.062+0000 7fba1a000640 1 -- 172.21.15.27:0/4219045126 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fba0409ca30 con 0x7fba1c101000 2024-04-26T22:57:45.062 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T22:57:45.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.064+0000 7fba21e00640 1 -- 172.21.15.27:0/4219045126 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb9fc0a3cc0 msgr2=0x7fb9fc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:45.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.064+0000 7fba21e00640 1 --2- 172.21.15.27:0/4219045126 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb9fc0a3cc0 0x7fb9fc0a6180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fba0c031500 tx=0x7fba0c005a20 comp rx=0 tx=0).stop 2024-04-26T22:57:45.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.064+0000 7fba21e00640 1 -- 172.21.15.27:0/4219045126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba1c101000 msgr2=0x7fba1c19ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:45.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.064+0000 7fba21e00640 1 --2- 172.21.15.27:0/4219045126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba1c101000 0x7fba1c19ef90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fba0400e9f0 tx=0x7fba0400edf0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.065+0000 7fba21e00640 1 -- 172.21.15.27:0/4219045126 shutdown_connections 2024-04-26T22:57:45.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.065+0000 7fba21e00640 1 --2- 172.21.15.27:0/4219045126 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb9fc0a3cc0 0x7fb9fc0a6180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.065+0000 7fba21e00640 1 --2- 172.21.15.27:0/4219045126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba1c103970 0x7fba1c19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.065+0000 7fba21e00640 1 --2- 172.21.15.27:0/4219045126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba1c106320 0x7fba1c1a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.065+0000 7fba21e00640 1 --2- 172.21.15.27:0/4219045126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba1c101000 0x7fba1c19ef90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.065+0000 7fba21e00640 1 -- 172.21.15.27:0/4219045126 >> 172.21.15.27:0/4219045126 conn(0x7fba1c0fcc60 msgr2=0x7fba1c101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:45.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.065+0000 7fba21e00640 1 -- 172.21.15.27:0/4219045126 shutdown_connections 2024-04-26T22:57:45.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.065+0000 7fba21e00640 1 -- 172.21.15.27:0/4219045126 wait complete. 2024-04-26T22:57:45.085 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:57:45.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.300+0000 7f0138200640 1 -- 172.21.15.27:0/865899132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01301011e0 msgr2=0x7f0130103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:45.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.300+0000 7f0138200640 1 --2- 172.21.15.27:0/865899132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01301011e0 0x7f0130103610 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0120009be0 tx=0x7f012002a6c0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.300+0000 7f0138200640 1 -- 172.21.15.27:0/865899132 shutdown_connections 2024-04-26T22:57:45.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.300+0000 7f0138200640 1 --2- 172.21.15.27:0/865899132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0130106500 0x7f0130106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.300+0000 7f0138200640 1 --2- 172.21.15.27:0/865899132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130103b50 0x7f0130105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.300+0000 7f0138200640 1 --2- 172.21.15.27:0/865899132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01301011e0 0x7f0130103610 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.300+0000 7f0138200640 1 -- 172.21.15.27:0/865899132 >> 172.21.15.27:0/865899132 conn(0x7f01300fce00 msgr2=0x7f01300ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:45.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.300+0000 7f0138200640 1 -- 172.21.15.27:0/865899132 shutdown_connections 2024-04-26T22:57:45.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.300+0000 7f0138200640 1 -- 172.21.15.27:0/865899132 wait complete. 2024-04-26T22:57:45.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.301+0000 7f0138200640 1 Processor -- start 2024-04-26T22:57:45.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.301+0000 7f0138200640 1 -- start start 2024-04-26T22:57:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.302+0000 7f0138200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01301011e0 0x7f013019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.302+0000 7f0138200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0130103b50 0x7f013019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.302+0000 7f0138200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130106500 0x7f01301a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.302+0000 7f0138200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0130114ac0 con 0x7f0130106500 2024-04-26T22:57:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.302+0000 7f0138200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0130114940 con 0x7f01301011e0 2024-04-26T22:57:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.302+0000 7f0138200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0130114c40 con 0x7f0130103b50 2024-04-26T22:57:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.302+0000 7f0136200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130106500 0x7f01301a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.302+0000 7f0135800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01301011e0 0x7f013019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.302+0000 7f0136200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130106500 0x7f01301a66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41600/0 (socket says 172.21.15.27:41600) 2024-04-26T22:57:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.302+0000 7f0136200640 1 -- 172.21.15.27:0/2388106975 learned_addr learned my addr 172.21.15.27:0/2388106975 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.302+0000 7f0135800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01301011e0 0x7f013019f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:58758/0 (socket says 172.21.15.27:58758) 2024-04-26T22:57:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.302+0000 7f0134e00640 1 --2- 172.21.15.27:0/2388106975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0130103b50 0x7f013019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.303+0000 7f0136200640 1 -- 172.21.15.27:0/2388106975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0130103b50 msgr2=0x7f013019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.303+0000 7f0136200640 1 --2- 172.21.15.27:0/2388106975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0130103b50 0x7f013019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.303+0000 7f0136200640 1 -- 172.21.15.27:0/2388106975 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01301011e0 msgr2=0x7f013019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:45.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.303+0000 7f0136200640 1 --2- 172.21.15.27:0/2388106975 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01301011e0 0x7f013019f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.303+0000 7f0136200640 1 -- 172.21.15.27:0/2388106975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01301a6d20 con 0x7f0130106500 2024-04-26T22:57:45.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.303+0000 7f0135800640 1 --2- 172.21.15.27:0/2388106975 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01301011e0 0x7f013019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:45.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.303+0000 7f0136200640 1 --2- 172.21.15.27:0/2388106975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130106500 0x7f01301a66b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f012800e970 tx=0x7f012800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:45.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.303+0000 7f012e000640 1 -- 172.21.15.27:0/2388106975 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012800ccc0 con 0x7f0130106500 2024-04-26T22:57:45.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.303+0000 7f012e000640 1 -- 172.21.15.27:0/2388106975 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f012800ce60 con 0x7f0130106500 2024-04-26T22:57:45.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.303+0000 7f012e000640 1 -- 172.21.15.27:0/2388106975 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128010430 con 0x7f0130106500 2024-04-26T22:57:45.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.303+0000 7f0138200640 1 -- 172.21.15.27:0/2388106975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01301a6fb0 con 0x7f0130106500 2024-04-26T22:57:45.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.303+0000 7f0138200640 1 -- 172.21.15.27:0/2388106975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01301a7480 con 0x7f0130106500 2024-04-26T22:57:45.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.305+0000 7f0138200640 1 -- 172.21.15.27:0/2388106975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0130069c30 con 0x7f0130106500 2024-04-26T22:57:45.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.307+0000 7f012e000640 1 -- 172.21.15.27:0/2388106975 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0128004f20 con 0x7f0130106500 2024-04-26T22:57:45.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.308+0000 7f012e000640 1 --2- 172.21.15.27:0/2388106975 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f01140a3c40 0x7f01140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:45.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.308+0000 7f012e000640 1 -- 172.21.15.27:0/2388106975 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f01280d0ca0 con 0x7f0130106500 2024-04-26T22:57:45.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.309+0000 7f0135800640 1 --2- 172.21.15.27:0/2388106975 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f01140a3c40 0x7f01140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:45.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.309+0000 7f0135800640 1 --2- 172.21.15.27:0/2388106975 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f01140a3c40 0x7f01140a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f012002acb0 tx=0x7f0120005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:45.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.315+0000 7f012e000640 1 -- 172.21.15.27:0/2388106975 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f012809d090 con 0x7f0130106500 2024-04-26T22:57:45.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:45 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1081534108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:57:45.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:45 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3122343062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:57:45.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:45 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/4219045126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:57:45.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:45 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1081534108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:57:45.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:45 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3122343062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:57:45.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:45 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/4219045126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:57:45.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:45 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1081534108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:57:45.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:45 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3122343062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:57:45.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:45 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/4219045126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:57:45.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.564+0000 7f0138200640 1 -- 172.21.15.27:0/2388106975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f013002dad0 con 0x7f0130106500 2024-04-26T22:57:45.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.566+0000 7f012e000640 1 -- 172.21.15.27:0/2388106975 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11850 (secure 0 0 0) 0x7f012809c990 con 0x7f0130106500 2024-04-26T22:57:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:57:45.567 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T22:55:25.265331+0000","modified":"2024-04-26T22:55:54.648644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24578,"mds_1":24577,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/1620677319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1620677319},{"type":"v1","addr":"172.21.15.94:6839","nonce":1620677319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.27:6841/1571355196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1571355196},{"type":"v1","addr":"172.21.15.27:6841","nonce":1571355196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24577":{"gid":24577,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.27:6839/3193530521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3193530521},{"type":"v1","addr":"172.21.15.27:6839","nonce":3193530521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/95952000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":95952000},{"type":"v1","addr":"172.21.15.80:6839","nonce":95952000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24578,"qdb_cluster":[24560,24577,24578]},"id":1}]} 2024-04-26T22:57:45.568 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-04-26T22:57:45.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.569+0000 7f0138200640 1 -- 172.21.15.27:0/2388106975 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f01140a3c40 msgr2=0x7f01140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:45.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.569+0000 7f0138200640 1 --2- 172.21.15.27:0/2388106975 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f01140a3c40 0x7f01140a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f012002acb0 tx=0x7f0120005fd0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.570+0000 7f0138200640 1 -- 172.21.15.27:0/2388106975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130106500 msgr2=0x7f01301a66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:45.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.570+0000 7f0138200640 1 --2- 172.21.15.27:0/2388106975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130106500 0x7f01301a66b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f012800e970 tx=0x7f012800ed70 comp rx=0 tx=0).stop 2024-04-26T22:57:45.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.570+0000 7f0138200640 1 -- 172.21.15.27:0/2388106975 shutdown_connections 2024-04-26T22:57:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.570+0000 7f0138200640 1 --2- 172.21.15.27:0/2388106975 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f01140a3c40 0x7f01140a6100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.570+0000 7f0138200640 1 --2- 172.21.15.27:0/2388106975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130106500 0x7f01301a66b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.570+0000 7f0138200640 1 --2- 172.21.15.27:0/2388106975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0130103b50 0x7f013019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.570+0000 7f0138200640 1 --2- 172.21.15.27:0/2388106975 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01301011e0 0x7f013019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.570+0000 7f0138200640 1 -- 172.21.15.27:0/2388106975 >> 172.21.15.27:0/2388106975 conn(0x7f01300fce00 msgr2=0x7f01301044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.570+0000 7f0138200640 1 -- 172.21.15.27:0/2388106975 shutdown_connections 2024-04-26T22:57:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.570+0000 7f0138200640 1 -- 172.21.15.27:0/2388106975 wait complete. 2024-04-26T22:57:45.589 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:57:45.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.808+0000 7ff942e00640 1 -- 172.21.15.27:0/803455354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff93c102530 msgr2=0x7ff93c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:45.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.808+0000 7ff942e00640 1 --2- 172.21.15.27:0/803455354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff93c102530 0x7ff93c1029b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff934009ca0 tx=0x7ff93402a8c0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.808+0000 7ff942e00640 1 -- 172.21.15.27:0/803455354 shutdown_connections 2024-04-26T22:57:45.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.808+0000 7ff942e00640 1 --2- 172.21.15.27:0/803455354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff93c102530 0x7ff93c1029b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.808+0000 7ff942e00640 1 --2- 172.21.15.27:0/803455354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93c101b90 0x7ff93c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.808+0000 7ff942e00640 1 --2- 172.21.15.27:0/803455354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff93c101230 0x7ff93c101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.808+0000 7ff942e00640 1 -- 172.21.15.27:0/803455354 >> 172.21.15.27:0/803455354 conn(0x7ff93c0fcdd0 msgr2=0x7ff93c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:45.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.809+0000 7ff942e00640 1 -- 172.21.15.27:0/803455354 shutdown_connections 2024-04-26T22:57:45.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.809+0000 7ff942e00640 1 -- 172.21.15.27:0/803455354 wait complete. 2024-04-26T22:57:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.810+0000 7ff942e00640 1 Processor -- start 2024-04-26T22:57:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.810+0000 7ff942e00640 1 -- start start 2024-04-26T22:57:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.810+0000 7ff942e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff93c101230 0x7ff93c19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.810+0000 7ff942e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff93c101b90 0x7ff93c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.810+0000 7ff942e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93c102530 0x7ff93c1a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.810+0000 7ff942e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c1106d0 con 0x7ff93c101230 2024-04-26T22:57:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.810+0000 7ff942e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c110550 con 0x7ff93c101b90 2024-04-26T22:57:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.810+0000 7ff942e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c110850 con 0x7ff93c102530 2024-04-26T22:57:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.810+0000 7ff93be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff93c101230 0x7ff93c19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.810+0000 7ff93be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff93c101230 0x7ff93c19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41618/0 (socket says 172.21.15.27:41618) 2024-04-26T22:57:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.810+0000 7ff93be00640 1 -- 172.21.15.27:0/107417072 learned_addr learned my addr 172.21.15.27:0/107417072 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.810+0000 7ff933400640 1 --2- 172.21.15.27:0/107417072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff93c101b90 0x7ff93c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.811+0000 7ff940e00640 1 --2- 172.21.15.27:0/107417072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93c102530 0x7ff93c1a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.811+0000 7ff93be00640 1 -- 172.21.15.27:0/107417072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93c102530 msgr2=0x7ff93c1a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.811+0000 7ff93be00640 1 --2- 172.21.15.27:0/107417072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93c102530 0x7ff93c1a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.811+0000 7ff93be00640 1 -- 172.21.15.27:0/107417072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff93c101b90 msgr2=0x7ff93c19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.811+0000 7ff93be00640 1 --2- 172.21.15.27:0/107417072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff93c101b90 0x7ff93c19b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.811+0000 7ff93be00640 1 -- 172.21.15.27:0/107417072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93c1a2940 con 0x7ff93c101230 2024-04-26T22:57:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.811+0000 7ff93be00640 1 --2- 172.21.15.27:0/107417072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff93c101230 0x7ff93c19ad30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00e9f0 tx=0x7ff92c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.812+0000 7ff939600640 1 -- 172.21.15.27:0/107417072 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c00cd80 con 0x7ff93c101230 2024-04-26T22:57:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.812+0000 7ff942e00640 1 -- 172.21.15.27:0/107417072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93c1a2c30 con 0x7ff93c101230 2024-04-26T22:57:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.812+0000 7ff939600640 1 -- 172.21.15.27:0/107417072 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff92c0108e0 con 0x7ff93c101230 2024-04-26T22:57:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.812+0000 7ff939600640 1 -- 172.21.15.27:0/107417072 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c018980 con 0x7ff93c101230 2024-04-26T22:57:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.812+0000 7ff942e00640 1 -- 172.21.15.27:0/107417072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff93c1a31d0 con 0x7ff93c101230 2024-04-26T22:57:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.813+0000 7ff942e00640 1 -- 172.21.15.27:0/107417072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff93c10c0e0 con 0x7ff93c101230 2024-04-26T22:57:45.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.816+0000 7ff939600640 1 -- 172.21.15.27:0/107417072 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff92c010430 con 0x7ff93c101230 2024-04-26T22:57:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.817+0000 7ff939600640 1 --2- 172.21.15.27:0/107417072 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff9200a3c40 0x7ff9200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.817+0000 7ff939600640 1 -- 172.21.15.27:0/107417072 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7ff92c0d1230 con 0x7ff93c101230 2024-04-26T22:57:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.817+0000 7ff933400640 1 --2- 172.21.15.27:0/107417072 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff9200a3c40 0x7ff9200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.818+0000 7ff933400640 1 --2- 172.21.15.27:0/107417072 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff9200a3c40 0x7ff9200a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff924009bd0 tx=0x7ff924005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:45.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:45.819+0000 7ff939600640 1 -- 172.21.15.27:0/107417072 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff92c09cfd0 con 0x7ff93c101230 2024-04-26T22:57:46.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.029+0000 7ff942e00640 1 -- 172.21.15.27:0/107417072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff93c02dd30 con 0x7ff93c101230 2024-04-26T22:57:46.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.034+0000 7ff939600640 1 -- 172.21.15.27:0/107417072 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24902 (secure 0 0 0) 0x7ff92c0a12d0 con 0x7ff93c101230 2024-04-26T22:57:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:57:46.035 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":94,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:57:05.902371+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4},{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:57:46.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.037+0000 7ff942e00640 1 -- 172.21.15.27:0/107417072 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff9200a3c40 msgr2=0x7ff9200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:46.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.037+0000 7ff942e00640 1 --2- 172.21.15.27:0/107417072 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff9200a3c40 0x7ff9200a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff924009bd0 tx=0x7ff924005f90 comp rx=0 tx=0).stop 2024-04-26T22:57:46.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.037+0000 7ff942e00640 1 -- 172.21.15.27:0/107417072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff93c101230 msgr2=0x7ff93c19ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:46.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.037+0000 7ff942e00640 1 --2- 172.21.15.27:0/107417072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff93c101230 0x7ff93c19ad30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00e9f0 tx=0x7ff92c00eeb0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.037+0000 7ff942e00640 1 -- 172.21.15.27:0/107417072 shutdown_connections 2024-04-26T22:57:46.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.037+0000 7ff942e00640 1 --2- 172.21.15.27:0/107417072 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7ff9200a3c40 0x7ff9200a6100 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.037+0000 7ff942e00640 1 --2- 172.21.15.27:0/107417072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff93c101b90 0x7ff93c19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.037+0000 7ff942e00640 1 --2- 172.21.15.27:0/107417072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93c102530 0x7ff93c1a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.037+0000 7ff942e00640 1 --2- 172.21.15.27:0/107417072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff93c101230 0x7ff93c19ad30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.037+0000 7ff942e00640 1 -- 172.21.15.27:0/107417072 >> 172.21.15.27:0/107417072 conn(0x7ff93c0fcdd0 msgr2=0x7ff93c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:46.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.038+0000 7ff942e00640 1 -- 172.21.15.27:0/107417072 shutdown_connections 2024-04-26T22:57:46.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.038+0000 7ff942e00640 1 -- 172.21.15.27:0/107417072 wait complete. 2024-04-26T22:57:46.058 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:57:46.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.272+0000 7f52da400640 1 -- 172.21.15.27:0/3115261769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52d410a1a0 msgr2=0x7f52d410a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:46.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.272+0000 7f52da400640 1 --2- 172.21.15.27:0/3115261769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52d410a1a0 0x7f52d410a620 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f52c0009900 tx=0x7f52c002a3f0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.272+0000 7f52da400640 1 -- 172.21.15.27:0/3115261769 shutdown_connections 2024-04-26T22:57:46.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.272+0000 7f52da400640 1 --2- 172.21.15.27:0/3115261769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52d410a1a0 0x7f52d410a620 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.272+0000 7f52da400640 1 --2- 172.21.15.27:0/3115261769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52d410ab60 0x7f52d41113f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.272+0000 7f52da400640 1 --2- 172.21.15.27:0/3115261769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d4108fb0 0x7f52d41093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.272+0000 7f52da400640 1 -- 172.21.15.27:0/3115261769 >> 172.21.15.27:0/3115261769 conn(0x7f52d4077f40 msgr2=0x7f52d407a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:46.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.272+0000 7f52da400640 1 -- 172.21.15.27:0/3115261769 shutdown_connections 2024-04-26T22:57:46.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.272+0000 7f52da400640 1 -- 172.21.15.27:0/3115261769 wait complete. 2024-04-26T22:57:46.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.273+0000 7f52da400640 1 Processor -- start 2024-04-26T22:57:46.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.273+0000 7f52da400640 1 -- start start 2024-04-26T22:57:46.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.274+0000 7f52da400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52d4108fb0 0x7f52d419f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.274+0000 7f52da400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52d410a1a0 0x7f52d419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.274+0000 7f52da400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d410ab60 0x7f52d41a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.274+0000 7f52da400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f52d4114b40 con 0x7f52d410ab60 2024-04-26T22:57:46.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.274+0000 7f52da400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f52d41149c0 con 0x7f52d410a1a0 2024-04-26T22:57:46.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.274+0000 7f52da400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f52d4114cc0 con 0x7f52d4108fb0 2024-04-26T22:57:46.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.274+0000 7f52d3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d410ab60 0x7f52d41a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.274+0000 7f52d2a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52d410a1a0 0x7f52d419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.274+0000 7f52d2a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52d410a1a0 0x7f52d419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:58788/0 (socket says 172.21.15.27:58788) 2024-04-26T22:57:46.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.274+0000 7f52d3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d410ab60 0x7f52d41a6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41630/0 (socket says 172.21.15.27:41630) 2024-04-26T22:57:46.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.274+0000 7f52d2a00640 1 -- 172.21.15.27:0/377749968 learned_addr learned my addr 172.21.15.27:0/377749968 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:46.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.274+0000 7f52d3400640 1 --2- 172.21.15.27:0/377749968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52d4108fb0 0x7f52d419f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.275+0000 7f52d3e00640 1 -- 172.21.15.27:0/377749968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52d4108fb0 msgr2=0x7f52d419f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:46.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.275+0000 7f52d3e00640 1 --2- 172.21.15.27:0/377749968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52d4108fb0 0x7f52d419f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.275+0000 7f52d3e00640 1 -- 172.21.15.27:0/377749968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52d410a1a0 msgr2=0x7f52d419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:46.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.275+0000 7f52d3e00640 1 --2- 172.21.15.27:0/377749968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52d410a1a0 0x7f52d419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.275+0000 7f52d3e00640 1 -- 172.21.15.27:0/377749968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d41a6dd0 con 0x7f52d410ab60 2024-04-26T22:57:46.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.275+0000 7f52d2a00640 1 --2- 172.21.15.27:0/377749968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52d410a1a0 0x7f52d419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:46.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.275+0000 7f52d3e00640 1 --2- 172.21.15.27:0/377749968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d410ab60 0x7f52d41a6760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00ec50 tx=0x7f52cc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:46.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.275+0000 7f52cbe00640 1 -- 172.21.15.27:0/377749968 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc003b70 con 0x7f52d410ab60 2024-04-26T22:57:46.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.275+0000 7f52da400640 1 -- 172.21.15.27:0/377749968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d41a70c0 con 0x7f52d410ab60 2024-04-26T22:57:46.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.275+0000 7f52cbe00640 1 -- 172.21.15.27:0/377749968 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52cc003d10 con 0x7f52d410ab60 2024-04-26T22:57:46.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.275+0000 7f52da400640 1 -- 172.21.15.27:0/377749968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d41a7610 con 0x7f52d410ab60 2024-04-26T22:57:46.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.275+0000 7f52cbe00640 1 -- 172.21.15.27:0/377749968 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0107d0 con 0x7f52d410ab60 2024-04-26T22:57:46.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.277+0000 7f52da400640 1 -- 172.21.15.27:0/377749968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d4110270 con 0x7f52d410ab60 2024-04-26T22:57:46.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.279+0000 7f52cbe00640 1 -- 172.21.15.27:0/377749968 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f52cc003fe0 con 0x7f52d410ab60 2024-04-26T22:57:46.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.280+0000 7f52cbe00640 1 --2- 172.21.15.27:0/377749968 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f52b80a3c40 0x7f52b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.280+0000 7f52cbe00640 1 -- 172.21.15.27:0/377749968 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f52cc0d0e80 con 0x7f52d410ab60 2024-04-26T22:57:46.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.281+0000 7f52d3400640 1 --2- 172.21.15.27:0/377749968 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f52b80a3c40 0x7f52b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.281+0000 7f52d3400640 1 --2- 172.21.15.27:0/377749968 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f52b80a3c40 0x7f52b80a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f52d410a000 tx=0x7f52c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:46.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.285+0000 7f52cbe00640 1 -- 172.21.15.27:0/377749968 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f52cc09d270 con 0x7f52d410ab60 2024-04-26T22:57:46.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:46 smithi027 ceph-mon[28034]: pgmap v404: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-26T22:57:46.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:46 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2388106975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:57:46.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:46 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/107417072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:57:46.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:46 smithi080 ceph-mon[32324]: pgmap v404: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-26T22:57:46.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:46 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2388106975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:57:46.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:46 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/107417072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:57:46.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:46 smithi094 ceph-mon[32889]: pgmap v404: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-26T22:57:46.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:46 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2388106975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:57:46.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:46 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/107417072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:57:46.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.545+0000 7f52da400640 1 -- 172.21.15.27:0/377749968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52d402dad0 con 0x7f52d410ab60 2024-04-26T22:57:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.546+0000 7f52cbe00640 1 -- 172.21.15.27:0/377749968 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11850 (secure 0 0 0) 0x7f52cc09cb70 con 0x7f52d410ab60 2024-04-26T22:57:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:57:46.547 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T22:55:25.265331+0000","modified":"2024-04-26T22:55:54.648644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24578,"mds_1":24577,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/1620677319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1620677319},{"type":"v1","addr":"172.21.15.94:6839","nonce":1620677319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.27:6841/1571355196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1571355196},{"type":"v1","addr":"172.21.15.27:6841","nonce":1571355196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24577":{"gid":24577,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.27:6839/3193530521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3193530521},{"type":"v1","addr":"172.21.15.27:6839","nonce":3193530521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/95952000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":95952000},{"type":"v1","addr":"172.21.15.80:6839","nonce":95952000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24578,"qdb_cluster":[24560,24577,24578]},"id":1}]} 2024-04-26T22:57:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-04-26T22:57:46.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.549+0000 7f52da400640 1 -- 172.21.15.27:0/377749968 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f52b80a3c40 msgr2=0x7f52b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:46.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.549+0000 7f52da400640 1 --2- 172.21.15.27:0/377749968 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f52b80a3c40 0x7f52b80a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f52d410a000 tx=0x7f52c4005f90 comp rx=0 tx=0).stop 2024-04-26T22:57:46.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.549+0000 7f52da400640 1 -- 172.21.15.27:0/377749968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d410ab60 msgr2=0x7f52d41a6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:46.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.549+0000 7f52da400640 1 --2- 172.21.15.27:0/377749968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d410ab60 0x7f52d41a6760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00ec50 tx=0x7f52cc00c6a0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.549+0000 7f52da400640 1 -- 172.21.15.27:0/377749968 shutdown_connections 2024-04-26T22:57:46.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.549+0000 7f52da400640 1 --2- 172.21.15.27:0/377749968 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f52b80a3c40 0x7f52b80a6100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.549+0000 7f52da400640 1 --2- 172.21.15.27:0/377749968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52d410a1a0 0x7f52d419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.549+0000 7f52da400640 1 --2- 172.21.15.27:0/377749968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d410ab60 0x7f52d41a6760 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.549+0000 7f52da400640 1 --2- 172.21.15.27:0/377749968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52d4108fb0 0x7f52d419f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.549+0000 7f52da400640 1 -- 172.21.15.27:0/377749968 >> 172.21.15.27:0/377749968 conn(0x7f52d4077f40 msgr2=0x7f52d410d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:46.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.549+0000 7f52da400640 1 -- 172.21.15.27:0/377749968 shutdown_connections 2024-04-26T22:57:46.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.550+0000 7f52da400640 1 -- 172.21.15.27:0/377749968 wait complete. 2024-04-26T22:57:46.570 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T22:55:25.265331+0000', 'modified': '2024-04-26T22:55:54.648644+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24578, 'mds_1': 24577, 'mds_2': 24560}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.80:6837/3963089995', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6836', 'nonce': 3963089995}, {'type': 'v1', 'addr': '172.21.15.80:6837', 'nonce': 3963089995}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6837/3367319222', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6836', 'nonce': 3367319222}, {'type': 'v1', 'addr': '172.21.15.94:6837', 'nonce': 3367319222}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24559': {'gid': 24559, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6839/1620677319', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6838', 'nonce': 1620677319}, {'type': 'v1', 'addr': '172.21.15.94:6839', 'nonce': 1620677319}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24560': {'gid': 24560, 'name': 'a', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.27:6841/1571355196', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6840', 'nonce': 1571355196}, {'type': 'v1', 'addr': '172.21.15.27:6841', 'nonce': 1571355196}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24577': {'gid': 24577, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.27:6839/3193530521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6838', 'nonce': 3193530521}, {'type': 'v1', 'addr': '172.21.15.27:6839', 'nonce': 3193530521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6839/95952000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6838', 'nonce': 95952000}, {'type': 'v1', 'addr': '172.21.15.80:6839', 'nonce': 95952000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24578, 'qdb_cluster': [24560, 24577, 24578]} 2024-04-26T22:57:46.570 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T22:57:46.570 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T22:57:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.790+0000 7fa0fa400640 1 -- 172.21.15.27:0/2152252631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0f4104350 msgr2=0x7fa0f410abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.790+0000 7fa0fa400640 1 --2- 172.21.15.27:0/2152252631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0f4104350 0x7fa0f410abe0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009ca0 tx=0x7fa0ec02a8b0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.790+0000 7fa0fa400640 1 -- 172.21.15.27:0/2152252631 shutdown_connections 2024-04-26T22:57:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.790+0000 7fa0fa400640 1 --2- 172.21.15.27:0/2152252631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0f4104350 0x7fa0f410abe0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.790+0000 7fa0fa400640 1 --2- 172.21.15.27:0/2152252631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0f4103990 0x7fa0f4103e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.790+0000 7fa0fa400640 1 --2- 172.21.15.27:0/2152252631 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0f41027a0 0x7fa0f4102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.790+0000 7fa0fa400640 1 -- 172.21.15.27:0/2152252631 >> 172.21.15.27:0/2152252631 conn(0x7fa0f40fdd10 msgr2=0x7fa0f4100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.790+0000 7fa0fa400640 1 -- 172.21.15.27:0/2152252631 shutdown_connections 2024-04-26T22:57:46.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.790+0000 7fa0fa400640 1 -- 172.21.15.27:0/2152252631 wait complete. 2024-04-26T22:57:46.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.792+0000 7fa0fa400640 1 Processor -- start 2024-04-26T22:57:46.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.792+0000 7fa0fa400640 1 -- start start 2024-04-26T22:57:46.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.793+0000 7fa0fa400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0f41027a0 0x7fa0f419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.793+0000 7fa0fa400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0f4103990 0x7fa0f419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.793+0000 7fa0fa400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0f4104350 0x7fa0f41a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.793+0000 7fa0fa400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f410e2b0 con 0x7fa0f4104350 2024-04-26T22:57:46.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.793+0000 7fa0fa400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f410e130 con 0x7fa0f41027a0 2024-04-26T22:57:46.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.793+0000 7fa0fa400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f410e430 con 0x7fa0f4103990 2024-04-26T22:57:46.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.793+0000 7fa0f9a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0f4104350 0x7fa0f41a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.793+0000 7fa0f9a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0f4104350 0x7fa0f41a2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41652/0 (socket says 172.21.15.27:41652) 2024-04-26T22:57:46.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.793+0000 7fa0f9a00640 1 -- 172.21.15.27:0/768333413 learned_addr learned my addr 172.21.15.27:0/768333413 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:46.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.793+0000 7fa0f3e00640 1 --2- 172.21.15.27:0/768333413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0f4103990 0x7fa0f419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.793+0000 7fa0f9000640 1 --2- 172.21.15.27:0/768333413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0f41027a0 0x7fa0f419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.794+0000 7fa0f9a00640 1 -- 172.21.15.27:0/768333413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0f4103990 msgr2=0x7fa0f419b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:46.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.794+0000 7fa0f9a00640 1 --2- 172.21.15.27:0/768333413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0f4103990 0x7fa0f419b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.794+0000 7fa0f9a00640 1 -- 172.21.15.27:0/768333413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0f41027a0 msgr2=0x7fa0f419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:46.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.794+0000 7fa0f9a00640 1 --2- 172.21.15.27:0/768333413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0f41027a0 0x7fa0f419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.794+0000 7fa0f9a00640 1 -- 172.21.15.27:0/768333413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f41a27b0 con 0x7fa0f4104350 2024-04-26T22:57:46.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.794+0000 7fa0f9000640 1 --2- 172.21.15.27:0/768333413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0f41027a0 0x7fa0f419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:46.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.794+0000 7fa0f9a00640 1 --2- 172.21.15.27:0/768333413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0f4104350 0x7fa0f41a2140 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec005e30 tx=0x7fa0ec003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:46.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.794+0000 7fa0f1600640 1 -- 172.21.15.27:0/768333413 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec004240 con 0x7fa0f4104350 2024-04-26T22:57:46.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.794+0000 7fa0fa400640 1 -- 172.21.15.27:0/768333413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f41a2a40 con 0x7fa0f4104350 2024-04-26T22:57:46.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.794+0000 7fa0f1600640 1 -- 172.21.15.27:0/768333413 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0ec040070 con 0x7fa0f4104350 2024-04-26T22:57:46.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.794+0000 7fa0f1600640 1 -- 172.21.15.27:0/768333413 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec045bd0 con 0x7fa0f4104350 2024-04-26T22:57:46.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.795+0000 7fa0fa400640 1 -- 172.21.15.27:0/768333413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f41a2fe0 con 0x7fa0f4104350 2024-04-26T22:57:46.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.798+0000 7fa0fe781740 1 Processor -- start 2024-04-26T22:57:46.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.798+0000 7fa0f1600640 1 -- 172.21.15.27:0/768333413 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa0ec0043e0 con 0x7fa0f4104350 2024-04-26T22:57:46.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.798+0000 7fa0fe781740 1 -- start start 2024-04-26T22:57:46.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.799+0000 7fa0fe781740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe7c2c0 0x55f25fe7c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.799+0000 7fa0fe781740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 0x55f25fe817b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.799+0000 7fa0fe781740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe81e20 0x55f25fe842b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.799+0000 7fa0fe781740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f25fde0600 con 0x55f25fe7ccb0 2024-04-26T22:57:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.799+0000 7fa0fe781740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55f25fd7d100 con 0x55f25fe7c2c0 2024-04-26T22:57:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.799+0000 7fa0fe781740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f25fddc5c0 con 0x55f25fe81e20 2024-04-26T22:57:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.799+0000 7fa0f3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe7c2c0 0x55f25fe7c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.799+0000 7fa0f3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe7c2c0 0x55f25fe7c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:58816/0 (socket says 172.21.15.27:58816) 2024-04-26T22:57:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.799+0000 7fa0f3e00640 1 -- 172.21.15.27:0/3114805652 learned_addr learned my addr 172.21.15.27:0/3114805652 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.799+0000 7fa0f9000640 1 --2- 172.21.15.27:0/3114805652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 0x55f25fe817b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.799+0000 7fa0f3e00640 1 --2- 172.21.15.27:0/3114805652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe81e20 0x55f25fe842b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.800+0000 7fa0f3e00640 1 -- 172.21.15.27:0/3114805652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe7c2c0 msgr2=0x55f25fe7c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.800+0000 7fa0f3e00640 1 --2- 172.21.15.27:0/3114805652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe7c2c0 0x55f25fe7c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.800+0000 7fa0f3e00640 1 -- 172.21.15.27:0/3114805652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 msgr2=0x55f25fe817b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.800+0000 7fa0f3e00640 1 --2- 172.21.15.27:0/3114805652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 0x55f25fe817b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.800+0000 7fa0f3e00640 1 -- 172.21.15.27:0/3114805652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f25faf22d0 con 0x55f25fe81e20 2024-04-26T22:57:46.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.800+0000 7fa0f9000640 1 --2- 172.21.15.27:0/3114805652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 0x55f25fe817b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:46.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.800+0000 7fa0f3e00640 1 --2- 172.21.15.27:0/3114805652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe81e20 0x55f25fe842b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009fd0 tx=0x7fa0e0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=326977e50ad78e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:46.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.800+0000 7fa0f1600640 1 --2- 172.21.15.27:0/768333413 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa0dc0a3cb0 0x7fa0dc0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.800+0000 7fa0e6000640 1 -- 172.21.15.27:0/3114805652 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0011d70 con 0x55f25fe81e20 2024-04-26T22:57:46.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.800+0000 7fa0e6000640 1 -- 172.21.15.27:0/3114805652 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e00054d0 con 0x55f25fe81e20 2024-04-26T22:57:46.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.800+0000 7fa0e6000640 1 -- 172.21.15.27:0/3114805652 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e000f790 con 0x55f25fe81e20 2024-04-26T22:57:46.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.800+0000 7fa0f1600640 1 -- 172.21.15.27:0/768333413 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fa0ec0f1ad0 con 0x7fa0f4104350 2024-04-26T22:57:46.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.800+0000 7fa0f9000640 1 --2- 172.21.15.27:0/768333413 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa0dc0a3cb0 0x7fa0dc0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.801+0000 7fa0fe781740 1 -- 172.21.15.27:0/3114805652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe81e20 msgr2=0x55f25fe842b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:46.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.801+0000 7fa0fe781740 1 --2- 172.21.15.27:0/3114805652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe81e20 0x55f25fe842b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009fd0 tx=0x7fa0e0005f90 comp rx=0 tx=0).stop 2024-04-26T22:57:46.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.801+0000 7fa0f9000640 1 --2- 172.21.15.27:0/768333413 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa0dc0a3cb0 0x7fa0dc0a6170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009fd0 tx=0x7fa0e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:46.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.801+0000 7fa0fe781740 1 -- 172.21.15.27:0/3114805652 shutdown_connections 2024-04-26T22:57:46.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.801+0000 7fa0fe781740 1 --2- 172.21.15.27:0/3114805652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe81e20 0x55f25fe842b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.801+0000 7fa0fe781740 1 --2- 172.21.15.27:0/3114805652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 0x55f25fe817b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.801+0000 7fa0fe781740 1 --2- 172.21.15.27:0/3114805652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe7c2c0 0x55f25fe7c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.801+0000 7fa0fe781740 1 -- 172.21.15.27:0/3114805652 >> 172.21.15.27:0/3114805652 conn(0x55f25fe10d10 msgr2=0x55f25fe77d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:46.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.801+0000 7fa0fe781740 1 -- 172.21.15.27:0/3114805652 shutdown_connections 2024-04-26T22:57:46.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.801+0000 7fa0fe781740 1 -- 172.21.15.27:0/3114805652 wait complete. 2024-04-26T22:57:46.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.802+0000 7fa0fe781740 1 -- start start 2024-04-26T22:57:46.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.802+0000 7fa0fe781740 1 Processor -- start 2024-04-26T22:57:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0fe781740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 0x55f25fe86770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0fe781740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 0x55f25fe86cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0fe781740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe81e20 0x55f25fe8dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0fe781740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f25fe84dd0 con 0x55f25fe7ccb0 2024-04-26T22:57:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0fe781740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55f25fd67f70 con 0x55f25fe81e20 2024-04-26T22:57:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0fe781740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f25fe84f50 con 0x55f25fe7c2c0 2024-04-26T22:57:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0fe781740 10 client.0 _ll_register_callbacks cb 0x55f25fdd6480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:57:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0f3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 0x55f25fe86cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0f9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 0x55f25fe86770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0f9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 0x55f25fe86770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:44562/0 (socket says 172.21.15.27:44562) 2024-04-26T22:57:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0f9000640 1 -- 172.21.15.27:0/4123792899 learned_addr learned my addr 172.21.15.27:0/4123792899 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0f9a00640 1 --2- 172.21.15.27:0/4123792899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe81e20 0x55f25fe8dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0f3e00640 1 -- 172.21.15.27:0/4123792899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 msgr2=0x55f25fe86770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0f3e00640 1 --2- 172.21.15.27:0/4123792899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 0x55f25fe86770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0f3e00640 1 -- 172.21.15.27:0/4123792899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe81e20 msgr2=0x55f25fe8dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0f3e00640 1 --2- 172.21.15.27:0/4123792899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe81e20 0x55f25fe8dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.803+0000 7fa0f3e00640 1 -- 172.21.15.27:0/4123792899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f25fbbc6b0 con 0x55f25fe7ccb0 2024-04-26T22:57:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.804+0000 7fa0f3e00640 1 --2- 172.21.15.27:0/4123792899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 0x55f25fe86cb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa0e001e000 tx=0x7fa0e0004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d31d61851007a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.804+0000 7fa0daa00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T22:57:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.804+0000 7fa0daa00640 1 -- 172.21.15.27:0/4123792899 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0005020 con 0x55f25fe7ccb0 2024-04-26T22:57:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.804+0000 7fa0fe781740 1 -- 172.21.15.27:0/4123792899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e00054d0 con 0x55f25fe7ccb0 2024-04-26T22:57:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.804+0000 7fa0daa00640 1 -- 172.21.15.27:0/4123792899 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fa0e00051c0 con 0x55f25fe7ccb0 2024-04-26T22:57:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.804+0000 7fa0daa00640 1 -- 172.21.15.27:0/4123792899 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa0e0004050 con 0x55f25fe7ccb0 2024-04-26T22:57:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.804+0000 7fa0fe781740 10 client.15054 fetch_fsmap learned FSMap version 24 2024-04-26T22:57:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.804+0000 7fa0fe781740 1 -- 172.21.15.27:0/4123792899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f25fe90cd0 con 0x55f25fe7ccb0 2024-04-26T22:57:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.805+0000 7fa0daa00640 1 -- 172.21.15.27:0/4123792899 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e00054d0 con 0x55f25fe7ccb0 2024-04-26T22:57:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.805+0000 7fa0daa00640 1 -- 172.21.15.27:0/4123792899 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fa0e00035f0 con 0x55f25fe7ccb0 2024-04-26T22:57:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.805+0000 7fa0fe781740 10 client.15054 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T22:57:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.805+0000 7fa0fe781740 10 client.15054 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T22:57:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.805+0000 7fa0fe781740 20 client.15054 populate_metadata read hostname 'smithi027' 2024-04-26T22:57:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.805+0000 7fa0fe781740 1 --2- 172.21.15.27:0/4123792899 >> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] conn(0x55f25fe930c0 0x55f25fe94530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.805+0000 7fa0fe781740 4 client.15054 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:57:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.805+0000 7fa0fe781740 1 -- 172.21.15.27:0/4123792899 --> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f25fdee210 con 0x55f25fe930c0 2024-04-26T22:57:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.806+0000 7fa0f9000640 1 --2- 172.21.15.27:0/4123792899 >> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] conn(0x55f25fe930c0 0x55f25fe94530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.806+0000 7fa0f9000640 1 --2- 172.21.15.27:0/4123792899 >> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] conn(0x55f25fe930c0 0x55f25fe94530 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3cf78c734c6e75b9 server_cookie=a798b9e8194a2e2a in_seq=0 out_seq=0 2024-04-26T22:57:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.806+0000 7fa0daa00640 10 client.15054 ms_handle_connect on v2:172.21.15.27:6840/1571355196 2024-04-26T22:57:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.807+0000 7fa0daa00640 1 -- 172.21.15.27:0/4123792899 <== mds.2 v2:172.21.15.27:6840/1571355196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f25fdee210 con 0x55f25fe930c0 2024-04-26T22:57:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.807+0000 7fa0daa00640 10 client.15054 handle_command_reply: tid=0 2024-04-26T22:57:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.807+0000 7fa0fe781740 1 client.15054 shutdown 2024-04-26T22:57:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.807+0000 7fa0fe781740 1 -- 172.21.15.27:0/4123792899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 msgr2=0x55f25fe86cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.807+0000 7fa0fe781740 1 --2- 172.21.15.27:0/4123792899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 0x55f25fe86cb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa0e001e000 tx=0x7fa0e0004920 comp rx=0 tx=0).stop 2024-04-26T22:57:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.808+0000 7fa0fe781740 1 -- 172.21.15.27:0/4123792899 shutdown_connections 2024-04-26T22:57:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.808+0000 7fa0fe781740 1 --2- 172.21.15.27:0/4123792899 >> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] conn(0x55f25fe930c0 0x55f25fe94530 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.808+0000 7fa0fe781740 1 --2- 172.21.15.27:0/4123792899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe81e20 0x55f25fe8dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.808+0000 7fa0fe781740 1 --2- 172.21.15.27:0/4123792899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 0x55f25fe86cb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.808+0000 7fa0fe781740 1 --2- 172.21.15.27:0/4123792899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 0x55f25fe86770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.808+0000 7fa0daa00640 0 client.15054 ms_handle_reset on v2:172.21.15.27:6840/1571355196 2024-04-26T22:57:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.808+0000 7fa0fe781740 1 -- 172.21.15.27:0/4123792899 >> 172.21.15.27:0/4123792899 conn(0x55f25fe10d10 msgr2=0x55f25fe76f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.808+0000 7fa0fe781740 1 -- 172.21.15.27:0/4123792899 shutdown_connections 2024-04-26T22:57:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.808+0000 7fa0fe781740 1 -- 172.21.15.27:0/4123792899 wait complete. 2024-04-26T22:57:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.808+0000 7fa0fe781740 20 client.15054 trim_cache size 0 max 16384 2024-04-26T22:57:46.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.828+0000 7fa0fe781740 1 Processor -- start 2024-04-26T22:57:46.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.828+0000 7fa0fe781740 1 -- start start 2024-04-26T22:57:46.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.828+0000 7fa0fe781740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 0x55f25fe7a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.828+0000 7fa0fe781740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe7ccb0 0x55f25fe8dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.828+0000 7fa0fe781740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe81e20 0x55f25fe7ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.828+0000 7fa0fe781740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f25fe92870 con 0x55f25fe81e20 2024-04-26T22:57:46.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.828+0000 7fa0fe781740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55f25fe92f40 con 0x55f25fe7ccb0 2024-04-26T22:57:46.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.828+0000 7fa0fe781740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f25fe93610 con 0x55f25fe7c2c0 2024-04-26T22:57:46.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.829+0000 7fa0f3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe7ccb0 0x55f25fe8dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.829+0000 7fa0f9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 0x55f25fe7a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.829+0000 7fa0f3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe7ccb0 0x55f25fe8dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:58832/0 (socket says 172.21.15.27:58832) 2024-04-26T22:57:46.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.829+0000 7fa0f3e00640 1 -- 172.21.15.27:0/1850806330 learned_addr learned my addr 172.21.15.27:0/1850806330 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:46.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.829+0000 7fa0f9a00640 1 --2- 172.21.15.27:0/1850806330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe81e20 0x55f25fe7ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.829+0000 7fa0f9000640 1 -- 172.21.15.27:0/1850806330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe7ccb0 msgr2=0x55f25fe8dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:46.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.829+0000 7fa0f9000640 1 --2- 172.21.15.27:0/1850806330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe7ccb0 0x55f25fe8dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.829+0000 7fa0f9000640 1 -- 172.21.15.27:0/1850806330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe81e20 msgr2=0x55f25fe7ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:46.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.829+0000 7fa0f9000640 1 --2- 172.21.15.27:0/1850806330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe81e20 0x55f25fe7ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.829+0000 7fa0f9000640 1 -- 172.21.15.27:0/1850806330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f25fe73460 con 0x55f25fe7c2c0 2024-04-26T22:57:46.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.829+0000 7fa0f9a00640 1 --2- 172.21.15.27:0/1850806330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe81e20 0x55f25fe7ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:57:46.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.829+0000 7fa0f3e00640 1 --2- 172.21.15.27:0/1850806330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe7ccb0 0x55f25fe8dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:46.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.830+0000 7fa0f9000640 1 --2- 172.21.15.27:0/1850806330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 0x55f25fe7a900 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55f25fe956b0 tx=0x7fa0e8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc554d70974dcfa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:46.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.830+0000 7fa0e5600640 1 -- 172.21.15.27:0/1850806330 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e802b420 con 0x55f25fe7c2c0 2024-04-26T22:57:46.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.830+0000 7fa0e5600640 1 -- 172.21.15.27:0/1850806330 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e8005020 con 0x55f25fe7c2c0 2024-04-26T22:57:46.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.830+0000 7fa0e5600640 1 -- 172.21.15.27:0/1850806330 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80047a0 con 0x55f25fe7c2c0 2024-04-26T22:57:46.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.830+0000 7fa0fe781740 1 -- 172.21.15.27:0/1850806330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 msgr2=0x55f25fe7a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:46.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.830+0000 7fa0fe781740 1 --2- 172.21.15.27:0/1850806330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 0x55f25fe7a900 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55f25fe956b0 tx=0x7fa0e8017000 comp rx=0 tx=0).stop 2024-04-26T22:57:46.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.830+0000 7fa0fe781740 1 -- 172.21.15.27:0/1850806330 shutdown_connections 2024-04-26T22:57:46.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.830+0000 7fa0fe781740 1 --2- 172.21.15.27:0/1850806330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe7ccb0 0x55f25fe8dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.830+0000 7fa0fe781740 1 --2- 172.21.15.27:0/1850806330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe81e20 0x55f25fe7ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.830+0000 7fa0fe781740 1 --2- 172.21.15.27:0/1850806330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 0x55f25fe7a900 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.830+0000 7fa0fe781740 1 -- 172.21.15.27:0/1850806330 >> 172.21.15.27:0/1850806330 conn(0x55f25fe10d10 msgr2=0x55f25fe7e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:46.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.831+0000 7fa0fe781740 1 -- 172.21.15.27:0/1850806330 shutdown_connections 2024-04-26T22:57:46.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.831+0000 7fa0fe781740 1 -- 172.21.15.27:0/1850806330 wait complete. 2024-04-26T22:57:46.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.831+0000 7fa0fe781740 1 -- start start 2024-04-26T22:57:46.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.831+0000 7fa0fe781740 1 Processor -- start 2024-04-26T22:57:46.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0fe781740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 0x55f25fe89580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0fe781740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 0x55f25fe89ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0fe781740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe81e20 0x55f25fe8cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0fe781740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f25fe943b0 con 0x55f25fe7ccb0 2024-04-26T22:57:46.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0fe781740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55f25fe93ce0 con 0x55f25fe81e20 2024-04-26T22:57:46.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0fe781740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f25fe94a80 con 0x55f25fe7c2c0 2024-04-26T22:57:46.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0fe781740 10 client.0 _ll_register_callbacks cb 0x7fa0e8009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:57:46.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0f3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 0x55f25fe89ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0f3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 0x55f25fe89ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41668/0 (socket says 172.21.15.27:41668) 2024-04-26T22:57:46.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0f3e00640 1 -- 172.21.15.27:0/1702680365 learned_addr learned my addr 172.21.15.27:0/1702680365 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:46.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0f9000640 1 --2- 172.21.15.27:0/1702680365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 0x55f25fe89580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0f9a00640 1 --2- 172.21.15.27:0/1702680365 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe81e20 0x55f25fe8cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0f3e00640 1 -- 172.21.15.27:0/1702680365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 msgr2=0x55f25fe89580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:46.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0f3e00640 1 --2- 172.21.15.27:0/1702680365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 0x55f25fe89580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0f3e00640 1 -- 172.21.15.27:0/1702680365 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe81e20 msgr2=0x55f25fe8cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:46.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0f3e00640 1 --2- 172.21.15.27:0/1702680365 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe81e20 0x55f25fe8cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.832+0000 7fa0f3e00640 1 -- 172.21.15.27:0/1702680365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f25fe8d0d0 con 0x55f25fe7ccb0 2024-04-26T22:57:46.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.833+0000 7fa0f3e00640 1 --2- 172.21.15.27:0/1702680365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 0x55f25fe89ac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0006010 tx=0x7fa0e0009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4598b201416f125 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:46.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.833+0000 7fa0daa00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T22:57:46.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.833+0000 7fa0daa00640 1 -- 172.21.15.27:0/1702680365 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e003b060 con 0x55f25fe7ccb0 2024-04-26T22:57:46.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.833+0000 7fa0daa00640 1 -- 172.21.15.27:0/1702680365 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fa0e0003c00 con 0x55f25fe7ccb0 2024-04-26T22:57:46.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.833+0000 7fa0fe781740 1 -- 172.21.15.27:0/1702680365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e8005020 con 0x55f25fe7ccb0 2024-04-26T22:57:46.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.833+0000 7fa0daa00640 1 -- 172.21.15.27:0/1702680365 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa0e0003e00 con 0x55f25fe7ccb0 2024-04-26T22:57:46.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.833+0000 7fa0fe781740 10 client.15060 fetch_fsmap learned FSMap version 24 2024-04-26T22:57:46.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.833+0000 7fa0fe781740 1 -- 172.21.15.27:0/1702680365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f25ff019a0 con 0x55f25fe7ccb0 2024-04-26T22:57:46.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.834+0000 7fa0daa00640 1 -- 172.21.15.27:0/1702680365 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e8005020 con 0x55f25fe7ccb0 2024-04-26T22:57:46.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.834+0000 7fa0daa00640 1 -- 172.21.15.27:0/1702680365 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fa0e0005190 con 0x55f25fe7ccb0 2024-04-26T22:57:46.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.834+0000 7fa0fe781740 10 client.15060 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T22:57:46.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.834+0000 7fa0fe781740 10 client.15060 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T22:57:46.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.834+0000 7fa0fe781740 20 client.15060 populate_metadata read hostname 'smithi027' 2024-04-26T22:57:46.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.834+0000 7fa0fe781740 1 --2- 172.21.15.27:0/1702680365 >> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] conn(0x55f25ff065d0 0x55f25ff052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:46.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.834+0000 7fa0fe781740 4 client.15060 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T22:57:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.834+0000 7fa0fe781740 1 -- 172.21.15.27:0/1702680365 --> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] -- command(tid 0: {"prefix": "status"}) -- 0x55f25fde8080 con 0x55f25ff065d0 2024-04-26T22:57:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.834+0000 7fa0f9000640 1 --2- 172.21.15.27:0/1702680365 >> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] conn(0x55f25ff065d0 0x55f25ff052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.835+0000 7fa0f9000640 1 --2- 172.21.15.27:0/1702680365 >> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] conn(0x55f25ff065d0 0x55f25ff052e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68f98d0b0b511a1a server_cookie=93b733247222d91d in_seq=0 out_seq=0 2024-04-26T22:57:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.835+0000 7fa0daa00640 10 client.15060 ms_handle_connect on v2:172.21.15.27:6840/1571355196 2024-04-26T22:57:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.835+0000 7fa0daa00640 1 -- 172.21.15.27:0/1702680365 <== mds.2 v2:172.21.15.27:6840/1571355196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f25fde8080 con 0x55f25ff065d0 2024-04-26T22:57:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.835+0000 7fa0daa00640 10 client.15060 handle_command_reply: tid=0 2024-04-26T22:57:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.835+0000 7fa0fe781740 1 client.15060 shutdown 2024-04-26T22:57:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.835+0000 7fa0fe781740 1 -- 172.21.15.27:0/1702680365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 msgr2=0x55f25fe89ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.835+0000 7fa0fe781740 1 --2- 172.21.15.27:0/1702680365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 0x55f25fe89ac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0006010 tx=0x7fa0e0009b00 comp rx=0 tx=0).stop 2024-04-26T22:57:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.836+0000 7fa0fe781740 1 -- 172.21.15.27:0/1702680365 shutdown_connections 2024-04-26T22:57:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.836+0000 7fa0fe781740 1 --2- 172.21.15.27:0/1702680365 >> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] conn(0x55f25ff065d0 0x55f25ff052e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.836+0000 7fa0fe781740 1 --2- 172.21.15.27:0/1702680365 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f25fe81e20 0x55f25fe8cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.836+0000 7fa0fe781740 1 --2- 172.21.15.27:0/1702680365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f25fe7ccb0 0x55f25fe89ac0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.836+0000 7fa0fe781740 1 --2- 172.21.15.27:0/1702680365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25fe7c2c0 0x55f25fe89580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.836+0000 7fa0daa00640 0 client.15060 ms_handle_reset on v2:172.21.15.27:6840/1571355196 2024-04-26T22:57:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.836+0000 7fa0fe781740 1 -- 172.21.15.27:0/1702680365 >> 172.21.15.27:0/1702680365 conn(0x55f25fe10d10 msgr2=0x55f25fe85380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.836+0000 7fa0fe781740 1 -- 172.21.15.27:0/1702680365 shutdown_connections 2024-04-26T22:57:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.836+0000 7fa0fe781740 1 -- 172.21.15.27:0/1702680365 wait complete. 2024-04-26T22:57:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.836+0000 7fa0fe781740 20 client.15060 trim_cache size 0 max 16384 2024-04-26T22:57:46.840 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T22:57:46.840 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "95ebdda6-041e-11ef-bc93-c7b262605968", 2024-04-26T22:57:46.840 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-04-26T22:57:46.840 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24560, 2024-04-26T22:57:46.840 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T22:57:46.840 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T22:57:46.841 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T22:57:46.841 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 113.20487573699999, 2024-04-26T22:57:46.841 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 24, 2024-04-26T22:57:46.841 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 94, 2024-04-26T22:57:46.841 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T22:57:46.841 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 151.15268555099999 2024-04-26T22:57:46.841 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T22:57:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.837+0000 7fa0fa400640 1 -- 172.21.15.27:0/768333413 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa0dc0a3cb0 msgr2=0x7fa0dc0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:46.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.837+0000 7fa0fa400640 1 --2- 172.21.15.27:0/768333413 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa0dc0a3cb0 0x7fa0dc0a6170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009fd0 tx=0x7fa0e8005f90 comp rx=0 tx=0).stop 2024-04-26T22:57:46.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.837+0000 7fa0fa400640 1 -- 172.21.15.27:0/768333413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0f4104350 msgr2=0x7fa0f41a2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:46.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.837+0000 7fa0fa400640 1 --2- 172.21.15.27:0/768333413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0f4104350 0x7fa0f41a2140 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec005e30 tx=0x7fa0ec003c00 comp rx=0 tx=0).stop 2024-04-26T22:57:46.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.837+0000 7fa0fa400640 1 -- 172.21.15.27:0/768333413 shutdown_connections 2024-04-26T22:57:46.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.837+0000 7fa0fa400640 1 --2- 172.21.15.27:0/768333413 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa0dc0a3cb0 0x7fa0dc0a6170 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.837+0000 7fa0fa400640 1 --2- 172.21.15.27:0/768333413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0f4104350 0x7fa0f41a2140 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.837+0000 7fa0fa400640 1 --2- 172.21.15.27:0/768333413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0f4103990 0x7fa0f419b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.837+0000 7fa0fa400640 1 --2- 172.21.15.27:0/768333413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0f41027a0 0x7fa0f419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:46.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.837+0000 7fa0fa400640 1 -- 172.21.15.27:0/768333413 >> 172.21.15.27:0/768333413 conn(0x7fa0f40fdd10 msgr2=0x7fa0f4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:46.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.837+0000 7fa0fa400640 1 -- 172.21.15.27:0/768333413 shutdown_connections 2024-04-26T22:57:46.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:46.837+0000 7fa0fa400640 1 -- 172.21.15.27:0/768333413 wait complete. 2024-04-26T22:57:46.854 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-26T22:57:47.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.057+0000 7f7928c00640 1 -- 172.21.15.27:0/3624890342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79241087d0 msgr2=0x7f792410ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.057+0000 7f7928c00640 1 --2- 172.21.15.27:0/3624890342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79241087d0 0x7f792410ac20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f791c009be0 tx=0x7f791c02a6d0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.057+0000 7f7928c00640 1 -- 172.21.15.27:0/3624890342 shutdown_connections 2024-04-26T22:57:47.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.057+0000 7f7928c00640 1 --2- 172.21.15.27:0/3624890342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79241087d0 0x7f792410ac20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.057+0000 7f7928c00640 1 --2- 172.21.15.27:0/3624890342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7924105ea0 0x7f7924108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.057+0000 7f7928c00640 1 --2- 172.21.15.27:0/3624890342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7924103570 0x7f7924105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.057+0000 7f7928c00640 1 -- 172.21.15.27:0/3624890342 >> 172.21.15.27:0/3624890342 conn(0x7f79240fcfa0 msgr2=0x7f79240ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:47.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.057+0000 7f7928c00640 1 -- 172.21.15.27:0/3624890342 shutdown_connections 2024-04-26T22:57:47.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.057+0000 7f7928c00640 1 -- 172.21.15.27:0/3624890342 wait complete. 2024-04-26T22:57:47.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.058+0000 7f7928c00640 1 Processor -- start 2024-04-26T22:57:47.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.058+0000 7f7928c00640 1 -- start start 2024-04-26T22:57:47.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.058+0000 7f7928c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7924103570 0x7f792419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.058+0000 7f7928c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7924105ea0 0x7f792419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.059+0000 7f7928c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79241087d0 0x7f79241a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.059+0000 7f7928c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f792410e540 con 0x7f7924103570 2024-04-26T22:57:47.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.059+0000 7f7928c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f792410e3c0 con 0x7f7924105ea0 2024-04-26T22:57:47.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.059+0000 7f7928c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f792410e6c0 con 0x7f79241087d0 2024-04-26T22:57:47.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.059+0000 7f7921e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7924103570 0x7f792419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.059+0000 7f7921e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7924103570 0x7f792419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41692/0 (socket says 172.21.15.27:41692) 2024-04-26T22:57:47.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.059+0000 7f7921400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7924105ea0 0x7f792419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.059+0000 7f7921e00640 1 -- 172.21.15.27:0/2694967979 learned_addr learned my addr 172.21.15.27:0/2694967979 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:47.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.059+0000 7f7922800640 1 --2- 172.21.15.27:0/2694967979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79241087d0 0x7f79241a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.059+0000 7f7921e00640 1 -- 172.21.15.27:0/2694967979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79241087d0 msgr2=0x7f79241a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.059+0000 7f7921e00640 1 --2- 172.21.15.27:0/2694967979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79241087d0 0x7f79241a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.059+0000 7f7921e00640 1 -- 172.21.15.27:0/2694967979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7924105ea0 msgr2=0x7f792419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.059+0000 7f7921e00640 1 --2- 172.21.15.27:0/2694967979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7924105ea0 0x7f792419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.059+0000 7f7921e00640 1 -- 172.21.15.27:0/2694967979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79241a2a10 con 0x7f7924103570 2024-04-26T22:57:47.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.059+0000 7f7921400640 1 --2- 172.21.15.27:0/2694967979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7924105ea0 0x7f792419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:47.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.060+0000 7f7921e00640 1 --2- 172.21.15.27:0/2694967979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7924103570 0x7f792419adf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f791400e9c0 tx=0x7f791400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:47.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.060+0000 7f791aa00640 1 -- 172.21.15.27:0/2694967979 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791400ccc0 con 0x7f7924103570 2024-04-26T22:57:47.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.060+0000 7f7928c00640 1 -- 172.21.15.27:0/2694967979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79241a2d00 con 0x7f7924103570 2024-04-26T22:57:47.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.060+0000 7f791aa00640 1 -- 172.21.15.27:0/2694967979 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79140054d0 con 0x7f7924103570 2024-04-26T22:57:47.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.060+0000 7f791aa00640 1 -- 172.21.15.27:0/2694967979 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914010430 con 0x7f7924103570 2024-04-26T22:57:47.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.060+0000 7f7928c00640 1 -- 172.21.15.27:0/2694967979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79241a31e0 con 0x7f7924103570 2024-04-26T22:57:47.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.064+0000 7f792bbcd740 1 Processor -- start 2024-04-26T22:57:47.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.064+0000 7f791aa00640 1 -- 172.21.15.27:0/2694967979 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7914004f80 con 0x7f7924103570 2024-04-26T22:57:47.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.064+0000 7f792bbcd740 1 -- start start 2024-04-26T22:57:47.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.064+0000 7f792bbcd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7dc0f0 0x56547b7dc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.064+0000 7f792bbcd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.064+0000 7f792bbcd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 0x56547b7e40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.064+0000 7f792bbcd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56547b740600 con 0x56547b7e1c50 2024-04-26T22:57:47.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.064+0000 7f792bbcd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x56547b6dd100 con 0x56547b7dcae0 2024-04-26T22:57:47.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.064+0000 7f792bbcd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56547b73c5c0 con 0x56547b7dc0f0 2024-04-26T22:57:47.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.065+0000 7f7921400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 0x56547b7e40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.065+0000 7f7921e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.065+0000 7f7922800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7dc0f0 0x56547b7dc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.065+0000 7f7921400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 0x56547b7e40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41704/0 (socket says 172.21.15.27:41704) 2024-04-26T22:57:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.065+0000 7f7921400640 1 -- 172.21.15.27:0/533939769 learned_addr learned my addr 172.21.15.27:0/533939769 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.065+0000 7f7921e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:58866/0 (socket says 172.21.15.27:58866) 2024-04-26T22:57:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.065+0000 7f7921400640 1 -- 172.21.15.27:0/533939769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7dc0f0 msgr2=0x56547b7dc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.065+0000 7f7921400640 1 --2- 172.21.15.27:0/533939769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7dc0f0 0x56547b7dc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.065+0000 7f7921400640 1 -- 172.21.15.27:0/533939769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 msgr2=0x56547b7e15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.065+0000 7f7921400640 1 --2- 172.21.15.27:0/533939769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.065+0000 7f7921400640 1 -- 172.21.15.27:0/533939769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56547b4522d0 con 0x56547b7e1c50 2024-04-26T22:57:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.065+0000 7f7921400640 1 --2- 172.21.15.27:0/533939769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 0x56547b7e40e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f79100114f0 tx=0x7f79100119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab99c0f648326651 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.066+0000 7f791aa00640 1 --2- 172.21.15.27:0/2694967979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f79080a3c40 0x7f79080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.066+0000 7f790f000640 1 -- 172.21.15.27:0/533939769 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910005020 con 0x56547b7e1c50 2024-04-26T22:57:47.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.066+0000 7f791aa00640 1 -- 172.21.15.27:0/2694967979 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f79140d0db0 con 0x7f7924103570 2024-04-26T22:57:47.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.066+0000 7f790f000640 1 -- 172.21.15.27:0/533939769 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79100047a0 con 0x56547b7e1c50 2024-04-26T22:57:47.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.066+0000 7f790f000640 1 -- 172.21.15.27:0/533939769 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791000f840 con 0x56547b7e1c50 2024-04-26T22:57:47.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.066+0000 7f7922800640 1 --2- 172.21.15.27:0/2694967979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f79080a3c40 0x7f79080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.067+0000 7f792bbcd740 1 -- 172.21.15.27:0/533939769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 msgr2=0x56547b7e40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.067+0000 7f792bbcd740 1 --2- 172.21.15.27:0/533939769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 0x56547b7e40e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f79100114f0 tx=0x7f79100119b0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.067+0000 7f7922800640 1 --2- 172.21.15.27:0/2694967979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f79080a3c40 0x7f79080a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f791c02abf0 tx=0x7f791c030990 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:47.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.067+0000 7f792bbcd740 1 -- 172.21.15.27:0/533939769 shutdown_connections 2024-04-26T22:57:47.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.067+0000 7f792bbcd740 1 --2- 172.21.15.27:0/533939769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.067+0000 7f792bbcd740 1 --2- 172.21.15.27:0/533939769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 0x56547b7e40e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.067+0000 7f792bbcd740 1 --2- 172.21.15.27:0/533939769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7dc0f0 0x56547b7dc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.067+0000 7f792bbcd740 1 -- 172.21.15.27:0/533939769 >> 172.21.15.27:0/533939769 conn(0x56547b770d10 msgr2=0x56547b7d7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:47.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.067+0000 7f792bbcd740 1 -- 172.21.15.27:0/533939769 shutdown_connections 2024-04-26T22:57:47.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.067+0000 7f792bbcd740 1 -- 172.21.15.27:0/533939769 wait complete. 2024-04-26T22:57:47.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.068+0000 7f792bbcd740 1 -- start start 2024-04-26T22:57:47.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.068+0000 7f792bbcd740 1 Processor -- start 2024-04-26T22:57:47.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.068+0000 7f792bbcd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7dc0f0 0x56547b7e6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.068+0000 7f792bbcd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.068+0000 7f792bbcd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7e1c50 0x56547b7edac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.068+0000 7f792bbcd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56547b7e4c00 con 0x56547b7dc0f0 2024-04-26T22:57:47.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.068+0000 7f792bbcd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x56547b6c7f70 con 0x56547b7dcae0 2024-04-26T22:57:47.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.068+0000 7f792bbcd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56547b7e4d80 con 0x56547b7e1c50 2024-04-26T22:57:47.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.068+0000 7f792bbcd740 10 client.0 _ll_register_callbacks cb 0x56547b736480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:57:47.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.069+0000 7f7921e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7dc0f0 0x56547b7e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.069+0000 7f7921400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.069+0000 7f7922800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7e1c50 0x56547b7edac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.069+0000 7f7921e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7dc0f0 0x56547b7e6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41718/0 (socket says 172.21.15.27:41718) 2024-04-26T22:57:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.069+0000 7f7921400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:58880/0 (socket says 172.21.15.27:58880) 2024-04-26T22:57:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.069+0000 7f7921e00640 1 -- 172.21.15.27:0/599219441 learned_addr learned my addr 172.21.15.27:0/599219441 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.069+0000 7f7921400640 1 -- 172.21.15.27:0/599219441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7e1c50 msgr2=0x56547b7edac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.069+0000 7f7921400640 1 --2- 172.21.15.27:0/599219441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7e1c50 0x56547b7edac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.069+0000 7f7921400640 1 -- 172.21.15.27:0/599219441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7dc0f0 msgr2=0x56547b7e6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.069+0000 7f7921400640 1 --2- 172.21.15.27:0/599219441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7dc0f0 0x56547b7e6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.069+0000 7f7921400640 1 -- 172.21.15.27:0/599219441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56547b51c6b0 con 0x56547b7dcae0 2024-04-26T22:57:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.069+0000 7f7921e00640 1 --2- 172.21.15.27:0/599219441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7dc0f0 0x56547b7e6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.069+0000 7f7921400640 1 --2- 172.21.15.27:0/599219441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e6a80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f791001b040 tx=0x7f7910005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e27a09efea66d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.070+0000 7f7903400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T22:57:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.070+0000 7f7903400640 1 -- 172.21.15.27:0/599219441 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910028070 con 0x56547b7dcae0 2024-04-26T22:57:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.070+0000 7f7903400640 1 -- 172.21.15.27:0/599219441 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f7910004020 con 0x56547b7dcae0 2024-04-26T22:57:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.070+0000 7f792bbcd740 1 -- 172.21.15.27:0/599219441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79100047a0 con 0x56547b7dcae0 2024-04-26T22:57:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.070+0000 7f7903400640 1 -- 172.21.15.27:0/599219441 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7910033050 con 0x56547b7dcae0 2024-04-26T22:57:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.070+0000 7f792bbcd740 10 client.25172 fetch_fsmap learned FSMap version 24 2024-04-26T22:57:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.070+0000 7f792bbcd740 1 -- 172.21.15.27:0/599219441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56547b7f0aa0 con 0x56547b7dcae0 2024-04-26T22:57:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.071+0000 7f7903400640 1 -- 172.21.15.27:0/599219441 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79100047a0 con 0x56547b7dcae0 2024-04-26T22:57:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.071+0000 7f7903400640 1 -- 172.21.15.27:0/599219441 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f791000fc90 con 0x56547b7dcae0 2024-04-26T22:57:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.071+0000 7f792bbcd740 10 client.25172 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T22:57:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.071+0000 7f792bbcd740 10 client.25172 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-26T22:57:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.071+0000 7f792bbcd740 20 client.25172 populate_metadata read hostname 'smithi027' 2024-04-26T22:57:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.071+0000 7f792bbcd740 1 --2- 172.21.15.27:0/599219441 >> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] conn(0x56547b7f2f10 0x56547b7f4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.071+0000 7f792bbcd740 4 client.25172 mds_command: new command op to 24577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:57:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.071+0000 7f792bbcd740 1 -- 172.21.15.27:0/599219441 --> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56547b74e210 con 0x56547b7f2f10 2024-04-26T22:57:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.072+0000 7f7921e00640 1 --2- 172.21.15.27:0/599219441 >> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] conn(0x56547b7f2f10 0x56547b7f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.072+0000 7f7921e00640 1 --2- 172.21.15.27:0/599219441 >> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] conn(0x56547b7f2f10 0x56547b7f4380 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1145211eddeec354 server_cookie=96079b6942bdd86 in_seq=0 out_seq=0 2024-04-26T22:57:47.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.072+0000 7f7903400640 10 client.25172 ms_handle_connect on v2:172.21.15.27:6838/3193530521 2024-04-26T22:57:47.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.073+0000 7f7903400640 1 -- 172.21.15.27:0/599219441 <== mds.1 v2:172.21.15.27:6838/3193530521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56547b74e210 con 0x56547b7f2f10 2024-04-26T22:57:47.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.073+0000 7f7903400640 10 client.25172 handle_command_reply: tid=0 2024-04-26T22:57:47.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.073+0000 7f792bbcd740 1 client.25172 shutdown 2024-04-26T22:57:47.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.073+0000 7f792bbcd740 1 -- 172.21.15.27:0/599219441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 msgr2=0x56547b7e6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.073+0000 7f792bbcd740 1 --2- 172.21.15.27:0/599219441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e6a80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f791001b040 tx=0x7f7910005230 comp rx=0 tx=0).stop 2024-04-26T22:57:47.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.073+0000 7f792bbcd740 1 -- 172.21.15.27:0/599219441 shutdown_connections 2024-04-26T22:57:47.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.073+0000 7f792bbcd740 1 --2- 172.21.15.27:0/599219441 >> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] conn(0x56547b7f2f10 0x56547b7f4380 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.073+0000 7f792bbcd740 1 --2- 172.21.15.27:0/599219441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e6a80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.073+0000 7f792bbcd740 1 --2- 172.21.15.27:0/599219441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7e1c50 0x56547b7edac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.073+0000 7f7903400640 0 client.25172 ms_handle_reset on v2:172.21.15.27:6838/3193530521 2024-04-26T22:57:47.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.073+0000 7f792bbcd740 1 --2- 172.21.15.27:0/599219441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7dc0f0 0x56547b7e6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.073+0000 7f792bbcd740 1 -- 172.21.15.27:0/599219441 >> 172.21.15.27:0/599219441 conn(0x56547b770d10 msgr2=0x56547b7d6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:47.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.074+0000 7f792bbcd740 1 -- 172.21.15.27:0/599219441 shutdown_connections 2024-04-26T22:57:47.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.074+0000 7f792bbcd740 1 -- 172.21.15.27:0/599219441 wait complete. 2024-04-26T22:57:47.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.074+0000 7f792bbcd740 20 client.25172 trim_cache size 0 max 16384 2024-04-26T22:57:47.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.094+0000 7f792bbcd740 1 Processor -- start 2024-04-26T22:57:47.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.094+0000 7f792bbcd740 1 -- start start 2024-04-26T22:57:47.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.095+0000 7f792bbcd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7dc0f0 0x56547b7e4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.095+0000 7f792bbcd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7eda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.095+0000 7f792bbcd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 0x56547b7ddee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.095+0000 7f792bbcd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56547b7f26c0 con 0x56547b7e1c50 2024-04-26T22:57:47.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.095+0000 7f792bbcd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x56547b7f2d90 con 0x56547b7dcae0 2024-04-26T22:57:47.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.095+0000 7f792bbcd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56547b7f3460 con 0x56547b7dc0f0 2024-04-26T22:57:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.095+0000 7f7921400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7eda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.095+0000 7f7921e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7dc0f0 0x56547b7e4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.095+0000 7f7922800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 0x56547b7ddee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.095+0000 7f7921400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7eda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:58884/0 (socket says 172.21.15.27:58884) 2024-04-26T22:57:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.095+0000 7f7921400640 1 -- 172.21.15.27:0/257053480 learned_addr learned my addr 172.21.15.27:0/257053480 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.095+0000 7f7921400640 1 -- 172.21.15.27:0/257053480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7dc0f0 msgr2=0x56547b7e4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.095+0000 7f7921400640 1 --2- 172.21.15.27:0/257053480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7dc0f0 0x56547b7e4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.095+0000 7f7921400640 1 -- 172.21.15.27:0/257053480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 msgr2=0x56547b7ddee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.095+0000 7f7921400640 1 --2- 172.21.15.27:0/257053480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 0x56547b7ddee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.095+0000 7f7921400640 1 -- 172.21.15.27:0/257053480 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56547b7dcec0 con 0x56547b7dcae0 2024-04-26T22:57:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.096+0000 7f7921400640 1 --2- 172.21.15.27:0/257053480 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7eda80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7910009730 tx=0x7f7910009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3131d4b2b83f88db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.096+0000 7f790e600640 1 -- 172.21.15.27:0/257053480 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910028040 con 0x56547b7dcae0 2024-04-26T22:57:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.096+0000 7f790e600640 1 -- 172.21.15.27:0/257053480 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7910005530 con 0x56547b7dcae0 2024-04-26T22:57:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.096+0000 7f790e600640 1 -- 172.21.15.27:0/257053480 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100246c0 con 0x56547b7dcae0 2024-04-26T22:57:47.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.097+0000 7f792bbcd740 1 -- 172.21.15.27:0/257053480 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 msgr2=0x56547b7eda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.097+0000 7f792bbcd740 1 --2- 172.21.15.27:0/257053480 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7eda80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7910009730 tx=0x7f7910009ea0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.097+0000 7f792bbcd740 1 -- 172.21.15.27:0/257053480 shutdown_connections 2024-04-26T22:57:47.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.097+0000 7f792bbcd740 1 --2- 172.21.15.27:0/257053480 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7eda80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.097+0000 7f792bbcd740 1 --2- 172.21.15.27:0/257053480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 0x56547b7ddee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.097+0000 7f792bbcd740 1 --2- 172.21.15.27:0/257053480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7dc0f0 0x56547b7e4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.097+0000 7f792bbcd740 1 -- 172.21.15.27:0/257053480 >> 172.21.15.27:0/257053480 conn(0x56547b770d10 msgr2=0x56547b7f1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:47.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.097+0000 7f792bbcd740 1 -- 172.21.15.27:0/257053480 shutdown_connections 2024-04-26T22:57:47.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.097+0000 7f792bbcd740 1 -- 172.21.15.27:0/257053480 wait complete. 2024-04-26T22:57:47.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.097+0000 7f792bbcd740 1 -- start start 2024-04-26T22:57:47.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.097+0000 7f792bbcd740 1 Processor -- start 2024-04-26T22:57:47.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.098+0000 7f792bbcd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7dc0f0 0x56547b7e9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.098+0000 7f792bbcd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.098+0000 7f792bbcd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 0x56547b7ec3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.098+0000 7f792bbcd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56547b7f4200 con 0x56547b7e1c50 2024-04-26T22:57:47.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.098+0000 7f792bbcd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x56547b7f3b30 con 0x56547b7dcae0 2024-04-26T22:57:47.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.098+0000 7f792bbcd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56547b7f48d0 con 0x56547b7dc0f0 2024-04-26T22:57:47.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.098+0000 7f792bbcd740 10 client.0 _ll_register_callbacks cb 0x7f79140d47d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:57:47.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.098+0000 7f7922800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 0x56547b7ec3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.098+0000 7f7921400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.098+0000 7f7921400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:58886/0 (socket says 172.21.15.27:58886) 2024-04-26T22:57:47.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.098+0000 7f7921400640 1 -- 172.21.15.27:0/1715255146 learned_addr learned my addr 172.21.15.27:0/1715255146 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:47.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.098+0000 7f7922800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 0x56547b7ec3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41744/0 (socket says 172.21.15.27:41744) 2024-04-26T22:57:47.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.098+0000 7f7921e00640 1 --2- 172.21.15.27:0/1715255146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7dc0f0 0x56547b7e9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.099+0000 7f7921400640 1 -- 172.21.15.27:0/1715255146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7dc0f0 msgr2=0x56547b7e9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:47.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.099+0000 7f7921400640 1 --2- 172.21.15.27:0/1715255146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7dc0f0 0x56547b7e9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.099+0000 7f7921400640 1 -- 172.21.15.27:0/1715255146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 msgr2=0x56547b7ec3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:47.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.099+0000 7f7921400640 1 --2- 172.21.15.27:0/1715255146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 0x56547b7ec3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.099+0000 7f7921400640 1 -- 172.21.15.27:0/1715255146 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56547b7eca30 con 0x56547b7dcae0 2024-04-26T22:57:47.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.099+0000 7f7922800640 1 --2- 172.21.15.27:0/1715255146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 0x56547b7ec3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:47.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.099+0000 7f7921400640 1 --2- 172.21.15.27:0/1715255146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e9560 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f79100059f0 tx=0x7f7910005af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4709902c6d8bc1d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:47.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.099+0000 7f7903400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T22:57:47.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.099+0000 7f7903400640 1 -- 172.21.15.27:0/1715255146 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791003c070 con 0x56547b7dcae0 2024-04-26T22:57:47.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.099+0000 7f7903400640 1 -- 172.21.15.27:0/1715255146 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f791000f820 con 0x56547b7dcae0 2024-04-26T22:57:47.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.099+0000 7f792bbcd740 1 -- 172.21.15.27:0/1715255146 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7910005530 con 0x56547b7dcae0 2024-04-26T22:57:47.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.100+0000 7f7903400640 1 -- 172.21.15.27:0/1715255146 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f791004a050 con 0x56547b7dcae0 2024-04-26T22:57:47.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.100+0000 7f792bbcd740 10 client.25178 fetch_fsmap learned FSMap version 24 2024-04-26T22:57:47.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.100+0000 7f792bbcd740 1 -- 172.21.15.27:0/1715255146 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56547b7edcf0 con 0x56547b7dcae0 2024-04-26T22:57:47.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.100+0000 7f7903400640 1 -- 172.21.15.27:0/1715255146 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7910005530 con 0x56547b7dcae0 2024-04-26T22:57:47.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.100+0000 7f7903400640 1 -- 172.21.15.27:0/1715255146 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7910038c30 con 0x56547b7dcae0 2024-04-26T22:57:47.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.100+0000 7f792bbcd740 10 client.25178 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T22:57:47.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.100+0000 7f792bbcd740 10 client.25178 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-26T22:57:47.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.100+0000 7f792bbcd740 20 client.25178 populate_metadata read hostname 'smithi027' 2024-04-26T22:57:47.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.100+0000 7f792bbcd740 1 --2- 172.21.15.27:0/1715255146 >> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] conn(0x56547b85d590 0x56547b85c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.100+0000 7f792bbcd740 4 client.25178 mds_command: new command op to 24577 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T22:57:47.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.100+0000 7f792bbcd740 1 -- 172.21.15.27:0/1715255146 --> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] -- command(tid 0: {"prefix": "status"}) -- 0x56547b748080 con 0x56547b85d590 2024-04-26T22:57:47.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.101+0000 7f7921e00640 1 --2- 172.21.15.27:0/1715255146 >> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] conn(0x56547b85d590 0x56547b85c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.101+0000 7f7921e00640 1 --2- 172.21.15.27:0/1715255146 >> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] conn(0x56547b85d590 0x56547b85c2a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47ce359ac70ade69 server_cookie=ad74effa7a9d4d06 in_seq=0 out_seq=0 2024-04-26T22:57:47.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.101+0000 7f7903400640 10 client.25178 ms_handle_connect on v2:172.21.15.27:6838/3193530521 2024-04-26T22:57:47.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.102+0000 7f7903400640 1 -- 172.21.15.27:0/1715255146 <== mds.1 v2:172.21.15.27:6838/3193530521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56547b748080 con 0x56547b85d590 2024-04-26T22:57:47.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.102+0000 7f7903400640 10 client.25178 handle_command_reply: tid=0 2024-04-26T22:57:47.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.102+0000 7f792bbcd740 1 client.25178 shutdown 2024-04-26T22:57:47.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.102+0000 7f792bbcd740 1 -- 172.21.15.27:0/1715255146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 msgr2=0x56547b7e9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.102+0000 7f792bbcd740 1 --2- 172.21.15.27:0/1715255146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e9560 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f79100059f0 tx=0x7f7910005af0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.102+0000 7f792bbcd740 1 -- 172.21.15.27:0/1715255146 shutdown_connections 2024-04-26T22:57:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.102+0000 7f792bbcd740 1 --2- 172.21.15.27:0/1715255146 >> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] conn(0x56547b85d590 0x56547b85c2a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.102+0000 7f792bbcd740 1 --2- 172.21.15.27:0/1715255146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56547b7dcae0 0x56547b7e9560 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.102+0000 7f792bbcd740 1 --2- 172.21.15.27:0/1715255146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56547b7e1c50 0x56547b7ec3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.102+0000 7f792bbcd740 1 --2- 172.21.15.27:0/1715255146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56547b7dc0f0 0x56547b7e9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.102+0000 7f792bbcd740 1 -- 172.21.15.27:0/1715255146 >> 172.21.15.27:0/1715255146 conn(0x56547b770d10 msgr2=0x56547b7e4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.102+0000 7f7903400640 0 client.25178 ms_handle_reset on v2:172.21.15.27:6838/3193530521 2024-04-26T22:57:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.102+0000 7f792bbcd740 1 -- 172.21.15.27:0/1715255146 shutdown_connections 2024-04-26T22:57:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.102+0000 7f792bbcd740 1 -- 172.21.15.27:0/1715255146 wait complete. 2024-04-26T22:57:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.102+0000 7f792bbcd740 20 client.25178 trim_cache size 0 max 16384 2024-04-26T22:57:47.108 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T22:57:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "95ebdda6-041e-11ef-bc93-c7b262605968", 2024-04-26T22:57:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-04-26T22:57:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24577, 2024-04-26T22:57:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T22:57:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T22:57:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T22:57:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 114.49518218599999, 2024-04-26T22:57:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 24, 2024-04-26T22:57:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 94, 2024-04-26T22:57:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T22:57:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 155.831928726 2024-04-26T22:57:47.109 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T22:57:47.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.103+0000 7f7928c00640 1 -- 172.21.15.27:0/2694967979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f79080a3c40 msgr2=0x7f79080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.103+0000 7f7928c00640 1 --2- 172.21.15.27:0/2694967979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f79080a3c40 0x7f79080a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f791c02abf0 tx=0x7f791c030990 comp rx=0 tx=0).stop 2024-04-26T22:57:47.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.103+0000 7f7928c00640 1 -- 172.21.15.27:0/2694967979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7924103570 msgr2=0x7f792419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.103+0000 7f7928c00640 1 --2- 172.21.15.27:0/2694967979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7924103570 0x7f792419adf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f791400e9c0 tx=0x7f791400ee80 comp rx=0 tx=0).stop 2024-04-26T22:57:47.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.103+0000 7f7928c00640 1 -- 172.21.15.27:0/2694967979 shutdown_connections 2024-04-26T22:57:47.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.103+0000 7f7928c00640 1 --2- 172.21.15.27:0/2694967979 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f79080a3c40 0x7f79080a6100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.103+0000 7f7928c00640 1 --2- 172.21.15.27:0/2694967979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7924105ea0 0x7f792419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.103+0000 7f7928c00640 1 --2- 172.21.15.27:0/2694967979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79241087d0 0x7f79241a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.103+0000 7f7928c00640 1 --2- 172.21.15.27:0/2694967979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7924103570 0x7f792419adf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.103+0000 7f7928c00640 1 -- 172.21.15.27:0/2694967979 >> 172.21.15.27:0/2694967979 conn(0x7f79240fcfa0 msgr2=0x7f7924102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:47.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.103+0000 7f7928c00640 1 -- 172.21.15.27:0/2694967979 shutdown_connections 2024-04-26T22:57:47.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.103+0000 7f7928c00640 1 -- 172.21.15.27:0/2694967979 wait complete. 2024-04-26T22:57:47.122 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T22:57:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.331+0000 7fe6e1e00640 1 -- 172.21.15.27:0/1099505101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6dc078ed0 msgr2=0x7fe6dc079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.331+0000 7fe6e1e00640 1 --2- 172.21.15.27:0/1099505101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6dc078ed0 0x7fe6dc079770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009ca0 tx=0x7fe6d402a8c0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.332+0000 7fe6e1e00640 1 -- 172.21.15.27:0/1099505101 shutdown_connections 2024-04-26T22:57:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.332+0000 7fe6e1e00640 1 --2- 172.21.15.27:0/1099505101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6dc078530 0x7fe6dc078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.332+0000 7fe6e1e00640 1 --2- 172.21.15.27:0/1099505101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6dc078ed0 0x7fe6dc079770 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.332+0000 7fe6e1e00640 1 --2- 172.21.15.27:0/1099505101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6dc0772f0 0x7fe6dc077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.332+0000 7fe6e1e00640 1 -- 172.21.15.27:0/1099505101 >> 172.21.15.27:0/1099505101 conn(0x7fe6dc0fff40 msgr2=0x7fe6dc102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:47.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.332+0000 7fe6e1e00640 1 -- 172.21.15.27:0/1099505101 shutdown_connections 2024-04-26T22:57:47.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.332+0000 7fe6e1e00640 1 -- 172.21.15.27:0/1099505101 wait complete. 2024-04-26T22:57:47.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.333+0000 7fe6e1e00640 1 Processor -- start 2024-04-26T22:57:47.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.333+0000 7fe6e1e00640 1 -- start start 2024-04-26T22:57:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.334+0000 7fe6e1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6dc0772f0 0x7fe6dc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.334+0000 7fe6e1e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6dc078530 0x7fe6dc19f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.334+0000 7fe6e1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6dc078ed0 0x7fe6dc1a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.334+0000 7fe6e1e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc1148d0 con 0x7fe6dc0772f0 2024-04-26T22:57:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.334+0000 7fe6e1e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc114750 con 0x7fe6dc078530 2024-04-26T22:57:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.334+0000 7fe6e1e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc114a50 con 0x7fe6dc078ed0 2024-04-26T22:57:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.334+0000 7fe6e0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6dc0772f0 0x7fe6dc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.334+0000 7fe6e0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6dc0772f0 0x7fe6dc19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41762/0 (socket says 172.21.15.27:41762) 2024-04-26T22:57:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.334+0000 7fe6e0a00640 1 -- 172.21.15.27:0/3307413291 learned_addr learned my addr 172.21.15.27:0/3307413291 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.334+0000 7fe6dbe00640 1 --2- 172.21.15.27:0/3307413291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6dc078530 0x7fe6dc19f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.334+0000 7fe6e1400640 1 --2- 172.21.15.27:0/3307413291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6dc078ed0 0x7fe6dc1a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.335+0000 7fe6dbe00640 1 -- 172.21.15.27:0/3307413291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6dc078ed0 msgr2=0x7fe6dc1a64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.335+0000 7fe6dbe00640 1 --2- 172.21.15.27:0/3307413291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6dc078ed0 0x7fe6dc1a64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.335+0000 7fe6dbe00640 1 -- 172.21.15.27:0/3307413291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6dc0772f0 msgr2=0x7fe6dc19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.335+0000 7fe6dbe00640 1 --2- 172.21.15.27:0/3307413291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6dc0772f0 0x7fe6dc19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.335+0000 7fe6dbe00640 1 -- 172.21.15.27:0/3307413291 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc1a6b50 con 0x7fe6dc078530 2024-04-26T22:57:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.335+0000 7fe6e0a00640 1 --2- 172.21.15.27:0/3307413291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6dc0772f0 0x7fe6dc19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.335+0000 7fe6dbe00640 1 --2- 172.21.15.27:0/3307413291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6dc078530 0x7fe6dc19f470 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e9f0 tx=0x7fe6c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:47.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.335+0000 7fe6d9600640 1 -- 172.21.15.27:0/3307413291 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c800ccc0 con 0x7fe6dc078530 2024-04-26T22:57:47.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.335+0000 7fe6d9600640 1 -- 172.21.15.27:0/3307413291 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6c80108e0 con 0x7fe6dc078530 2024-04-26T22:57:47.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.335+0000 7fe6d9600640 1 -- 172.21.15.27:0/3307413291 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8018880 con 0x7fe6dc078530 2024-04-26T22:57:47.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.335+0000 7fe6e1e00640 1 -- 172.21.15.27:0/3307413291 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6dc1a6e40 con 0x7fe6dc078530 2024-04-26T22:57:47.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.336+0000 7fe6e1e00640 1 -- 172.21.15.27:0/3307413291 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6dc1a7350 con 0x7fe6dc078530 2024-04-26T22:57:47.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.340+0000 7fe6e63b9740 1 Processor -- start 2024-04-26T22:57:47.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.340+0000 7fe6e63b9740 1 -- start start 2024-04-26T22:57:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.341+0000 7fe6e63b9740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cf2c0 0x5565282cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.341+0000 7fe6e63b9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 0x5565282d47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.341+0000 7fe6e63b9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282d4e20 0x5565282d72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.341+0000 7fe6d9600640 1 -- 172.21.15.27:0/3307413291 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe6c8010430 con 0x7fe6dc078530 2024-04-26T22:57:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.341+0000 7fe6e63b9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556528233600 con 0x5565282cfcb0 2024-04-26T22:57:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.341+0000 7fe6e63b9740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5565281d0100 con 0x5565282cf2c0 2024-04-26T22:57:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.341+0000 7fe6e63b9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55652822f5c0 con 0x5565282d4e20 2024-04-26T22:57:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.341+0000 7fe6e0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 0x5565282d47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.341+0000 7fe6e0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 0x5565282d47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41770/0 (socket says 172.21.15.27:41770) 2024-04-26T22:57:47.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.341+0000 7fe6e0a00640 1 -- 172.21.15.27:0/2300773452 learned_addr learned my addr 172.21.15.27:0/2300773452 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:47.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.341+0000 7fe6e1400640 1 --2- 172.21.15.27:0/2300773452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cf2c0 0x5565282cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.341+0000 7fe6dbe00640 1 --2- 172.21.15.27:0/2300773452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282d4e20 0x5565282d72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.342+0000 7fe6e1400640 1 -- 172.21.15.27:0/2300773452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282d4e20 msgr2=0x5565282d72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:47.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.342+0000 7fe6e1400640 1 --2- 172.21.15.27:0/2300773452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282d4e20 0x5565282d72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.342+0000 7fe6e1400640 1 -- 172.21.15.27:0/2300773452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 msgr2=0x5565282d47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:47.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.342+0000 7fe6e1400640 1 --2- 172.21.15.27:0/2300773452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 0x5565282d47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.342+0000 7fe6e1400640 1 -- 172.21.15.27:0/2300773452 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556527f452d0 con 0x5565282cf2c0 2024-04-26T22:57:47.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.342+0000 7fe6e0a00640 1 --2- 172.21.15.27:0/2300773452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 0x5565282d47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:47.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.342+0000 7fe6e1400640 1 --2- 172.21.15.27:0/2300773452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cf2c0 0x5565282cf6e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4005e30 tx=0x7fe6d4005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6662cbc826d7f114 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:47.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.342+0000 7fe6cdc00640 1 -- 172.21.15.27:0/2300773452 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4031650 con 0x5565282cf2c0 2024-04-26T22:57:47.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.342+0000 7fe6cdc00640 1 -- 172.21.15.27:0/2300773452 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6d4036030 con 0x5565282cf2c0 2024-04-26T22:57:47.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.342+0000 7fe6cdc00640 1 -- 172.21.15.27:0/2300773452 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4004790 con 0x5565282cf2c0 2024-04-26T22:57:47.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.343+0000 7fe6d9600640 1 --2- 172.21.15.27:0/3307413291 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6c40a3cb0 0x7fe6c40a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.343+0000 7fe6d9600640 1 -- 172.21.15.27:0/3307413291 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fe6c80d0c60 con 0x7fe6dc078530 2024-04-26T22:57:47.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.343+0000 7fe6e0a00640 1 --2- 172.21.15.27:0/3307413291 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6c40a3cb0 0x7fe6c40a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.343+0000 7fe6e63b9740 1 -- 172.21.15.27:0/2300773452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cf2c0 msgr2=0x5565282cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.343+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/2300773452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cf2c0 0x5565282cf6e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4005e30 tx=0x7fe6d4005ef0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.343+0000 7fe6e63b9740 1 -- 172.21.15.27:0/2300773452 shutdown_connections 2024-04-26T22:57:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.343+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/2300773452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282d4e20 0x5565282d72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.343+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/2300773452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 0x5565282d47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.343+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/2300773452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cf2c0 0x5565282cf6e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.343+0000 7fe6e63b9740 1 -- 172.21.15.27:0/2300773452 >> 172.21.15.27:0/2300773452 conn(0x556528263d10 msgr2=0x5565282cad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.343+0000 7fe6e63b9740 1 -- 172.21.15.27:0/2300773452 shutdown_connections 2024-04-26T22:57:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.343+0000 7fe6e63b9740 1 -- 172.21.15.27:0/2300773452 wait complete. 2024-04-26T22:57:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.344+0000 7fe6e0a00640 1 --2- 172.21.15.27:0/3307413291 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6c40a3cb0 0x7fe6c40a6170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5565282cbbf0 tx=0x7fe6d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.344+0000 7fe6e63b9740 1 -- start start 2024-04-26T22:57:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.344+0000 7fe6e63b9740 1 Processor -- start 2024-04-26T22:57:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6e63b9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282cf2c0 0x5565282d95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6e63b9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 0x5565282d9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6e63b9740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282d4e20 0x5565282e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6e63b9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5565282d7bf0 con 0x5565282cfcb0 2024-04-26T22:57:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6e63b9740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5565281baf70 con 0x5565282d4e20 2024-04-26T22:57:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6e63b9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5565282d7d70 con 0x5565282cf2c0 2024-04-26T22:57:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6e63b9740 10 client.0 _ll_register_callbacks cb 0x556528229480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:57:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6e0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282cf2c0 0x5565282d95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6dbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 0x5565282d9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6e0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282cf2c0 0x5565282d95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:44670/0 (socket says 172.21.15.27:44670) 2024-04-26T22:57:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6e0a00640 1 -- 172.21.15.27:0/1970883843 learned_addr learned my addr 172.21.15.27:0/1970883843 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6dbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 0x5565282d9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41780/0 (socket says 172.21.15.27:41780) 2024-04-26T22:57:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6e1400640 1 --2- 172.21.15.27:0/1970883843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282d4e20 0x5565282e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6e0a00640 1 -- 172.21.15.27:0/1970883843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282d4e20 msgr2=0x5565282e0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6e0a00640 1 --2- 172.21.15.27:0/1970883843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282d4e20 0x5565282e0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6e0a00640 1 -- 172.21.15.27:0/1970883843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 msgr2=0x5565282d9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:47.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6e0a00640 1 --2- 172.21.15.27:0/1970883843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 0x5565282d9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.345+0000 7fe6e0a00640 1 -- 172.21.15.27:0/1970883843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55652800f6b0 con 0x5565282cf2c0 2024-04-26T22:57:47.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.346+0000 7fe6e0a00640 1 --2- 172.21.15.27:0/1970883843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282cf2c0 0x5565282d95f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009780 tx=0x7fe6d0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d5f29daafee90ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:47.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.346+0000 7fe6c2000640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T22:57:47.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.346+0000 7fe6c2000640 1 -- 172.21.15.27:0/1970883843 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d000fbe0 con 0x5565282cf2c0 2024-04-26T22:57:47.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.346+0000 7fe6c2000640 1 -- 172.21.15.27:0/1970883843 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fe6d000fd80 con 0x5565282cf2c0 2024-04-26T22:57:47.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.346+0000 7fe6e63b9740 1 -- 172.21.15.27:0/1970883843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6d4036030 con 0x5565282cf2c0 2024-04-26T22:57:47.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.346+0000 7fe6c2000640 1 -- 172.21.15.27:0/1970883843 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe6d0032050 con 0x5565282cf2c0 2024-04-26T22:57:47.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.346+0000 7fe6e63b9740 10 client.25216 fetch_fsmap learned FSMap version 24 2024-04-26T22:57:47.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.346+0000 7fe6e63b9740 1 -- 172.21.15.27:0/1970883843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5565282e3bb0 con 0x5565282cf2c0 2024-04-26T22:57:47.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.347+0000 7fe6c2000640 1 -- 172.21.15.27:0/1970883843 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6d4036030 con 0x5565282cf2c0 2024-04-26T22:57:47.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.347+0000 7fe6c2000640 1 -- 172.21.15.27:0/1970883843 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe6d002a780 con 0x5565282cf2c0 2024-04-26T22:57:47.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.347+0000 7fe6e63b9740 10 client.25216 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T22:57:47.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.347+0000 7fe6e63b9740 10 client.25216 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T22:57:47.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.347+0000 7fe6e63b9740 20 client.25216 populate_metadata read hostname 'smithi027' 2024-04-26T22:57:47.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.347+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/1970883843 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5565282e6310 0x5565282e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.347+0000 7fe6e63b9740 4 client.25216 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:57:47.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.347+0000 7fe6e63b9740 1 -- 172.21.15.27:0/1970883843 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556528241210 con 0x5565282e6310 2024-04-26T22:57:47.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.348+0000 7fe6dbe00640 1 --2- 172.21.15.27:0/1970883843 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5565282e6310 0x5565282e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.348+0000 7fe6dbe00640 1 --2- 172.21.15.27:0/1970883843 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5565282e6310 0x5565282e7780 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dad89f3b1c1ceaac server_cookie=3db4de76641fb459 in_seq=0 out_seq=0 2024-04-26T22:57:47.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.348+0000 7fe6c2000640 10 client.25216 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:57:47.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.349+0000 7fe6c2000640 1 -- 172.21.15.27:0/1970883843 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556528241210 con 0x5565282e6310 2024-04-26T22:57:47.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.349+0000 7fe6c2000640 10 client.25216 handle_command_reply: tid=0 2024-04-26T22:57:47.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.349+0000 7fe6e63b9740 1 client.25216 shutdown 2024-04-26T22:57:47.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.349+0000 7fe6e63b9740 1 -- 172.21.15.27:0/1970883843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282cf2c0 msgr2=0x5565282d95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.349+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/1970883843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282cf2c0 0x5565282d95f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009780 tx=0x7fe6d0004730 comp rx=0 tx=0).stop 2024-04-26T22:57:47.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.349+0000 7fe6e63b9740 1 -- 172.21.15.27:0/1970883843 shutdown_connections 2024-04-26T22:57:47.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.349+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/1970883843 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5565282e6310 0x5565282e7780 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.349+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/1970883843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282d4e20 0x5565282e0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.349+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/1970883843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 0x5565282d9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.349+0000 7fe6c2000640 0 client.25216 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:57:47.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.349+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/1970883843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282cf2c0 0x5565282d95f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.349+0000 7fe6e63b9740 1 -- 172.21.15.27:0/1970883843 >> 172.21.15.27:0/1970883843 conn(0x556528263d10 msgr2=0x5565282c9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:47.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.350+0000 7fe6e63b9740 1 -- 172.21.15.27:0/1970883843 shutdown_connections 2024-04-26T22:57:47.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.350+0000 7fe6e63b9740 1 -- 172.21.15.27:0/1970883843 wait complete. 2024-04-26T22:57:47.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.350+0000 7fe6e63b9740 20 client.25216 trim_cache size 0 max 16384 2024-04-26T22:57:47.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.370+0000 7fe6e63b9740 1 Processor -- start 2024-04-26T22:57:47.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.370+0000 7fe6e63b9740 1 -- start start 2024-04-26T22:57:47.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.370+0000 7fe6e63b9740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cf2c0 0x5565282ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.370+0000 7fe6e63b9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 0x5565282cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.370+0000 7fe6e63b9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282d4e20 0x5565282e0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.370+0000 7fe6e63b9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5565282e5ac0 con 0x5565282cfcb0 2024-04-26T22:57:47.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.370+0000 7fe6e63b9740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5565282e6190 con 0x5565282cf2c0 2024-04-26T22:57:47.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.370+0000 7fe6e63b9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5565282e6860 con 0x5565282d4e20 2024-04-26T22:57:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.371+0000 7fe6dbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 0x5565282cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.371+0000 7fe6e0a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cf2c0 0x5565282ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.371+0000 7fe6e0a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cf2c0 0x5565282ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:58930/0 (socket says 172.21.15.27:58930) 2024-04-26T22:57:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.371+0000 7fe6e0a00640 1 -- 172.21.15.27:0/1953701225 learned_addr learned my addr 172.21.15.27:0/1953701225 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.371+0000 7fe6dbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 0x5565282cad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41790/0 (socket says 172.21.15.27:41790) 2024-04-26T22:57:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.371+0000 7fe6e1400640 1 --2- 172.21.15.27:0/1953701225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282d4e20 0x5565282e0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.371+0000 7fe6e0a00640 1 -- 172.21.15.27:0/1953701225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282d4e20 msgr2=0x5565282e0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.371+0000 7fe6e0a00640 1 --2- 172.21.15.27:0/1953701225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282d4e20 0x5565282e0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.371+0000 7fe6e0a00640 1 -- 172.21.15.27:0/1953701225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 msgr2=0x5565282cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.371+0000 7fe6e0a00640 1 --2- 172.21.15.27:0/1953701225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 0x5565282cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.371+0000 7fe6e0a00640 1 -- 172.21.15.27:0/1953701225 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565282da710 con 0x5565282cf2c0 2024-04-26T22:57:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.371+0000 7fe6e1400640 1 --2- 172.21.15.27:0/1953701225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282d4e20 0x5565282e0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:57:47.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.371+0000 7fe6e0a00640 1 --2- 172.21.15.27:0/1953701225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cf2c0 0x5565282ca7c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000f800 tx=0x7fe6d0020410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b79eb7d8cbf7f944 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:47.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.372+0000 7fe6cd200640 1 -- 172.21.15.27:0/1953701225 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d003a020 con 0x5565282cf2c0 2024-04-26T22:57:47.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.372+0000 7fe6cd200640 1 -- 172.21.15.27:0/1953701225 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6d002f040 con 0x5565282cf2c0 2024-04-26T22:57:47.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.372+0000 7fe6cd200640 1 -- 172.21.15.27:0/1953701225 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0003610 con 0x5565282cf2c0 2024-04-26T22:57:47.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.372+0000 7fe6e63b9740 1 -- 172.21.15.27:0/1953701225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cf2c0 msgr2=0x5565282ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.372+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/1953701225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cf2c0 0x5565282ca7c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000f800 tx=0x7fe6d0020410 comp rx=0 tx=0).stop 2024-04-26T22:57:47.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.372+0000 7fe6e63b9740 1 -- 172.21.15.27:0/1953701225 shutdown_connections 2024-04-26T22:57:47.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.372+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/1953701225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282d4e20 0x5565282e0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.372+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/1953701225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cfcb0 0x5565282cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.372+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/1953701225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cf2c0 0x5565282ca7c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.372+0000 7fe6e63b9740 1 -- 172.21.15.27:0/1953701225 >> 172.21.15.27:0/1953701225 conn(0x556528263d10 msgr2=0x5565282e5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:47.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.373+0000 7fe6e63b9740 1 -- 172.21.15.27:0/1953701225 shutdown_connections 2024-04-26T22:57:47.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.373+0000 7fe6e63b9740 1 -- 172.21.15.27:0/1953701225 wait complete. 2024-04-26T22:57:47.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.373+0000 7fe6e63b9740 1 -- start start 2024-04-26T22:57:47.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.373+0000 7fe6e63b9740 1 Processor -- start 2024-04-26T22:57:47.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6e63b9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cf2c0 0x5565282dc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6e63b9740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cfcb0 0x5565282dc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6e63b9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282d4e20 0x5565282df530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6e63b9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5565282e7600 con 0x5565282cf2c0 2024-04-26T22:57:47.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6e63b9740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5565282e6f30 con 0x5565282cfcb0 2024-04-26T22:57:47.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6e63b9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5565282e7cd0 con 0x5565282d4e20 2024-04-26T22:57:47.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6e63b9740 10 client.0 _ll_register_callbacks cb 0x7fe6d0012730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:57:47.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6e0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cf2c0 0x5565282dc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6dbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cfcb0 0x5565282dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6dbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cfcb0 0x5565282dc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:58932/0 (socket says 172.21.15.27:58932) 2024-04-26T22:57:47.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6dbe00640 1 -- 172.21.15.27:0/166957110 learned_addr learned my addr 172.21.15.27:0/166957110 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:47.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6e0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cf2c0 0x5565282dc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41792/0 (socket says 172.21.15.27:41792) 2024-04-26T22:57:47.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6e1400640 1 --2- 172.21.15.27:0/166957110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282d4e20 0x5565282df530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6dbe00640 1 -- 172.21.15.27:0/166957110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282d4e20 msgr2=0x5565282df530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:47.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6dbe00640 1 --2- 172.21.15.27:0/166957110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282d4e20 0x5565282df530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6dbe00640 1 -- 172.21.15.27:0/166957110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cf2c0 msgr2=0x5565282dc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:57:47.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6dbe00640 1 --2- 172.21.15.27:0/166957110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cf2c0 0x5565282dc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.374+0000 7fe6dbe00640 1 -- 172.21.15.27:0/166957110 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565282dfb20 con 0x5565282cfcb0 2024-04-26T22:57:47.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.375+0000 7fe6dbe00640 1 --2- 172.21.15.27:0/166957110 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cfcb0 0x5565282dc580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8098e60 tx=0x7fe6c8085f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b2569629dbe8d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:47.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.375+0000 7fe6c2000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T22:57:47.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.375+0000 7fe6c2000640 1 -- 172.21.15.27:0/166957110 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c809ca00 con 0x5565282cfcb0 2024-04-26T22:57:47.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.375+0000 7fe6c2000640 1 -- 172.21.15.27:0/166957110 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fe6c8010a90 con 0x5565282cfcb0 2024-04-26T22:57:47.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.375+0000 7fe6e63b9740 1 -- 172.21.15.27:0/166957110 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6d002f040 con 0x5565282cfcb0 2024-04-26T22:57:47.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.375+0000 7fe6c2000640 1 -- 172.21.15.27:0/166957110 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe6c80b8050 con 0x5565282cfcb0 2024-04-26T22:57:47.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.375+0000 7fe6e63b9740 10 client.25196 fetch_fsmap learned FSMap version 24 2024-04-26T22:57:47.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.375+0000 7fe6e63b9740 1 -- 172.21.15.27:0/166957110 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55652834c2c0 con 0x5565282cfcb0 2024-04-26T22:57:47.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.375+0000 7fe6c2000640 1 -- 172.21.15.27:0/166957110 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6d002f040 con 0x5565282cfcb0 2024-04-26T22:57:47.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.376+0000 7fe6c2000640 1 -- 172.21.15.27:0/166957110 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe6c80ce3d0 con 0x5565282cfcb0 2024-04-26T22:57:47.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.376+0000 7fe6e63b9740 10 client.25196 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T22:57:47.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.376+0000 7fe6e63b9740 10 client.25196 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T22:57:47.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.376+0000 7fe6e63b9740 20 client.25196 populate_metadata read hostname 'smithi027' 2024-04-26T22:57:47.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.376+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/166957110 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x556528350d40 0x55652834d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:47.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.376+0000 7fe6e63b9740 4 client.25196 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T22:57:47.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.376+0000 7fe6e63b9740 1 -- 172.21.15.27:0/166957110 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "status"}) -- 0x55652823b080 con 0x556528350d40 2024-04-26T22:57:47.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.376+0000 7fe6e0a00640 1 --2- 172.21.15.27:0/166957110 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x556528350d40 0x55652834d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:47.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.377+0000 7fe6e0a00640 1 --2- 172.21.15.27:0/166957110 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x556528350d40 0x55652834d850 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=374966188a76c897 server_cookie=19d5900e40736bd8 in_seq=0 out_seq=0 2024-04-26T22:57:47.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.377+0000 7fe6c2000640 10 client.25196 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:57:47.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.377+0000 7fe6c2000640 1 -- 172.21.15.27:0/166957110 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55652823b080 con 0x556528350d40 2024-04-26T22:57:47.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.377+0000 7fe6c2000640 10 client.25196 handle_command_reply: tid=0 2024-04-26T22:57:47.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.377+0000 7fe6e63b9740 1 client.25196 shutdown 2024-04-26T22:57:47.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.377+0000 7fe6e63b9740 1 -- 172.21.15.27:0/166957110 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cfcb0 msgr2=0x5565282dc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.377+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/166957110 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cfcb0 0x5565282dc580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8098e60 tx=0x7fe6c8085f90 comp rx=0 tx=0).stop 2024-04-26T22:57:47.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.377+0000 7fe6e63b9740 1 -- 172.21.15.27:0/166957110 shutdown_connections 2024-04-26T22:57:47.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.377+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/166957110 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x556528350d40 0x55652834d850 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.377+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/166957110 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565282cfcb0 0x5565282dc580 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.377+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/166957110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565282d4e20 0x5565282df530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.377+0000 7fe6e63b9740 1 --2- 172.21.15.27:0/166957110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565282cf2c0 0x5565282dc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.377+0000 7fe6c2000640 0 client.25196 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:57:47.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.377+0000 7fe6e63b9740 1 -- 172.21.15.27:0/166957110 >> 172.21.15.27:0/166957110 conn(0x556528263d10 msgr2=0x5565282ca7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:47.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.377+0000 7fe6e63b9740 1 -- 172.21.15.27:0/166957110 shutdown_connections 2024-04-26T22:57:47.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.377+0000 7fe6e63b9740 1 -- 172.21.15.27:0/166957110 wait complete. 2024-04-26T22:57:47.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.378+0000 7fe6e63b9740 20 client.25196 trim_cache size 0 max 16384 2024-04-26T22:57:47.383 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T22:57:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "95ebdda6-041e-11ef-bc93-c7b262605968", 2024-04-26T22:57:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-04-26T22:57:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24578, 2024-04-26T22:57:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T22:57:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T22:57:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T22:57:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 141.133349192, 2024-04-26T22:57:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 24, 2024-04-26T22:57:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 94, 2024-04-26T22:57:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T22:57:47.384 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 148.853760247 2024-04-26T22:57:47.384 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T22:57:47.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.378+0000 7fe6e1e00640 1 -- 172.21.15.27:0/3307413291 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6c40a3cb0 msgr2=0x7fe6c40a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.378+0000 7fe6e1e00640 1 --2- 172.21.15.27:0/3307413291 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6c40a3cb0 0x7fe6c40a6170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5565282cbbf0 tx=0x7fe6d0005f90 comp rx=0 tx=0).stop 2024-04-26T22:57:47.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.378+0000 7fe6e1e00640 1 -- 172.21.15.27:0/3307413291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6dc078530 msgr2=0x7fe6dc19f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:47.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.378+0000 7fe6e1e00640 1 --2- 172.21.15.27:0/3307413291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6dc078530 0x7fe6dc19f470 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e9f0 tx=0x7fe6c800eeb0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.378+0000 7fe6e1e00640 1 -- 172.21.15.27:0/3307413291 shutdown_connections 2024-04-26T22:57:47.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.378+0000 7fe6e1e00640 1 --2- 172.21.15.27:0/3307413291 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe6c40a3cb0 0x7fe6c40a6170 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.378+0000 7fe6e1e00640 1 --2- 172.21.15.27:0/3307413291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6dc078530 0x7fe6dc19f470 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.378+0000 7fe6e1e00640 1 --2- 172.21.15.27:0/3307413291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6dc078ed0 0x7fe6dc1a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.378+0000 7fe6e1e00640 1 --2- 172.21.15.27:0/3307413291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6dc0772f0 0x7fe6dc19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:47.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.378+0000 7fe6e1e00640 1 -- 172.21.15.27:0/3307413291 >> 172.21.15.27:0/3307413291 conn(0x7fe6dc0fff40 msgr2=0x7fe6dc101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:47.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.378+0000 7fe6e1e00640 1 -- 172.21.15.27:0/3307413291 shutdown_connections 2024-04-26T22:57:47.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:47.378+0000 7fe6e1e00640 1 -- 172.21.15.27:0/3307413291 wait complete. 2024-04-26T22:57:47.394 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:47 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/377749968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:57:47.397 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T22:57:47.398 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T22:57:47.407 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T22:57:47.408 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T22:57:47.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:47 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/377749968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:57:47.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:47 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/377749968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:57:47.880 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:57:48.305 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:48 smithi027 ceph-mon[28034]: pgmap v405: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-26T22:57:48.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:48 smithi080 ceph-mon[32324]: pgmap v405: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-26T22:57:48.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:48 smithi094 ceph-mon[32889]: pgmap v405: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-26T22:57:50.348 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:50 smithi027 ceph-mon[28034]: pgmap v406: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-26T22:57:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.514+0000 7f1f66400700 1 -- 172.21.15.27:0/1739104702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f601056c0 msgr2=0x7f1f60105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.514+0000 7f1f66400700 1 --2- 172.21.15.27:0/1739104702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f601056c0 0x7f1f60105ae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1f500099c0 tx=0x7f1f50009cd0 comp rx=0 tx=0).stop 2024-04-26T22:57:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.514+0000 7f1f66400700 1 -- 172.21.15.27:0/1739104702 shutdown_connections 2024-04-26T22:57:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.514+0000 7f1f66400700 1 --2- 172.21.15.27:0/1739104702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f60107260 0x7f1f6010db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.514+0000 7f1f66400700 1 --2- 172.21.15.27:0/1739104702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f601056c0 0x7f1f60105ae0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.514+0000 7f1f66400700 1 --2- 172.21.15.27:0/1739104702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f601068c0 0x7f1f60106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.514+0000 7f1f66400700 1 -- 172.21.15.27:0/1739104702 >> 172.21.15.27:0/1739104702 conn(0x7f1f60100c80 msgr2=0x7f1f601030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.514+0000 7f1f66400700 1 -- 172.21.15.27:0/1739104702 shutdown_connections 2024-04-26T22:57:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.514+0000 7f1f66400700 1 -- 172.21.15.27:0/1739104702 wait complete. 2024-04-26T22:57:50.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.515+0000 7f1f66400700 1 Processor -- start 2024-04-26T22:57:50.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.515+0000 7f1f66400700 1 -- start start 2024-04-26T22:57:50.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.516+0000 7f1f66400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f601056c0 0x7f1f6019b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:50.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.516+0000 7f1f66400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f601068c0 0x7f1f6019bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.516+0000 7f1f66400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f60107260 0x7f1f601a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.516+0000 7f1f66400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1f601111a0 con 0x7f1f601056c0 2024-04-26T22:57:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.516+0000 7f1f66400700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1f60111020 con 0x7f1f601068c0 2024-04-26T22:57:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.516+0000 7f1f66400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f60111320 con 0x7f1f60107260 2024-04-26T22:57:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.516+0000 7f1f5f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f601056c0 0x7f1f6019b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.516+0000 7f1f5f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f601056c0 0x7f1f6019b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42262/0 (socket says 172.21.15.27:42262) 2024-04-26T22:57:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.516+0000 7f1f5f400700 1 -- 172.21.15.27:0/2950532386 learned_addr learned my addr 172.21.15.27:0/2950532386 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.516+0000 7f1f5ea00700 1 --2- 172.21.15.27:0/2950532386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f601068c0 0x7f1f6019bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.516+0000 7f1f5fe00700 1 --2- 172.21.15.27:0/2950532386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f60107260 0x7f1f601a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.517+0000 7f1f5f400700 1 -- 172.21.15.27:0/2950532386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f60107260 msgr2=0x7f1f601a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.517+0000 7f1f5f400700 1 --2- 172.21.15.27:0/2950532386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f60107260 0x7f1f601a2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.517+0000 7f1f5f400700 1 -- 172.21.15.27:0/2950532386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f601068c0 msgr2=0x7f1f6019bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:50.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.517+0000 7f1f5f400700 1 --2- 172.21.15.27:0/2950532386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f601068c0 0x7f1f6019bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:50.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.517+0000 7f1f5f400700 1 -- 172.21.15.27:0/2950532386 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f601a34a0 con 0x7f1f601056c0 2024-04-26T22:57:50.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.517+0000 7f1f5f400700 1 --2- 172.21.15.27:0/2950532386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f601056c0 0x7f1f6019b840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000b580 tx=0x7f1f50003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:50.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.517+0000 7f1f57e00700 1 -- 172.21.15.27:0/2950532386 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50009e60 con 0x7f1f601056c0 2024-04-26T22:57:50.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.517+0000 7f1f57e00700 1 -- 172.21.15.27:0/2950532386 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f50005570 con 0x7f1f601056c0 2024-04-26T22:57:50.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.518+0000 7f1f66400700 1 -- 172.21.15.27:0/2950532386 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f601a3730 con 0x7f1f601056c0 2024-04-26T22:57:50.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.518+0000 7f1f57e00700 1 -- 172.21.15.27:0/2950532386 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000fe00 con 0x7f1f601056c0 2024-04-26T22:57:50.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.518+0000 7f1f66400700 1 -- 172.21.15.27:0/2950532386 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f601a3be0 con 0x7f1f601056c0 2024-04-26T22:57:50.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.519+0000 7f1f66400700 1 -- 172.21.15.27:0/2950532386 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f6006a110 con 0x7f1f601056c0 2024-04-26T22:57:50.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:50 smithi080 ceph-mon[32324]: pgmap v406: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-26T22:57:50.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.521+0000 7f1f57e00700 1 -- 172.21.15.27:0/2950532386 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1f5000f4f0 con 0x7f1f601056c0 2024-04-26T22:57:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.522+0000 7f1f57e00700 1 --2- 172.21.15.27:0/2950532386 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f1f440a3d50 0x7f1f440a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.523+0000 7f1f57e00700 1 -- 172.21.15.27:0/2950532386 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f1f500d1750 con 0x7f1f601056c0 2024-04-26T22:57:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.523+0000 7f1f5ea00700 1 --2- 172.21.15.27:0/2950532386 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f1f440a3d50 0x7f1f440a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.523+0000 7f1f5ea00700 1 --2- 172.21.15.27:0/2950532386 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f1f440a3d50 0x7f1f440a6210 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009c60 tx=0x7f1f4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:50.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.527+0000 7f1f57e00700 1 -- 172.21.15.27:0/2950532386 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1f50013070 con 0x7f1f601056c0 2024-04-26T22:57:50.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:50 smithi094 ceph-mon[32889]: pgmap v406: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-26T22:57:50.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.788+0000 7f1f66400700 1 -- 172.21.15.27:0/2950532386 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f1f60063860 con 0x7f1f440a3d50 2024-04-26T22:57:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.791+0000 7f1f57e00700 1 -- 172.21.15.27:0/2950532386 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f1f60063860 con 0x7f1f440a3d50 2024-04-26T22:57:50.792 INFO:teuthology.orchestra.run.smithi027.stdout:Backend: cephadm 2024-04-26T22:57:50.792 INFO:teuthology.orchestra.run.smithi027.stdout:Available: Yes 2024-04-26T22:57:50.792 INFO:teuthology.orchestra.run.smithi027.stdout:Paused: No 2024-04-26T22:57:50.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.794+0000 7f1f66400700 1 -- 172.21.15.27:0/2950532386 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f1f440a3d50 msgr2=0x7f1f440a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:50.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.794+0000 7f1f66400700 1 --2- 172.21.15.27:0/2950532386 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f1f440a3d50 0x7f1f440a6210 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009c60 tx=0x7f1f4c009260 comp rx=0 tx=0).stop 2024-04-26T22:57:50.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.794+0000 7f1f66400700 1 -- 172.21.15.27:0/2950532386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f601056c0 msgr2=0x7f1f6019b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.794+0000 7f1f66400700 1 --2- 172.21.15.27:0/2950532386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f601056c0 0x7f1f6019b840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000b580 tx=0x7f1f50003af0 comp rx=0 tx=0).stop 2024-04-26T22:57:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.795+0000 7f1f66400700 1 -- 172.21.15.27:0/2950532386 shutdown_connections 2024-04-26T22:57:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.795+0000 7f1f66400700 1 --2- 172.21.15.27:0/2950532386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f60107260 0x7f1f601a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.795+0000 7f1f66400700 1 --2- 172.21.15.27:0/2950532386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f601056c0 0x7f1f6019b840 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.795+0000 7f1f66400700 1 --2- 172.21.15.27:0/2950532386 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f1f440a3d50 0x7f1f440a6210 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.795+0000 7f1f66400700 1 --2- 172.21.15.27:0/2950532386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f601068c0 0x7f1f6019bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.795+0000 7f1f66400700 1 -- 172.21.15.27:0/2950532386 >> 172.21.15.27:0/2950532386 conn(0x7f1f60100c80 msgr2=0x7f1f60109c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.795+0000 7f1f66400700 1 -- 172.21.15.27:0/2950532386 shutdown_connections 2024-04-26T22:57:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:50.795+0000 7f1f66400700 1 -- 172.21.15.27:0/2950532386 wait complete. 2024-04-26T22:57:51.878 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T22:57:52.363 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:57:52.515 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:52 smithi027 ceph-mon[28034]: pgmap v407: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T22:57:52.515 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:52 smithi027 ceph-mon[28034]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:57:52.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:52 smithi080 ceph-mon[32324]: pgmap v407: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T22:57:52.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:52 smithi080 ceph-mon[32324]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:57:52.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:52 smithi094 ceph-mon[32889]: pgmap v407: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T22:57:52.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:52 smithi094 ceph-mon[32889]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:57:54.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:54 smithi027 ceph-mon[28034]: pgmap v408: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-26T22:57:54.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:54 smithi080 ceph-mon[32324]: pgmap v408: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-26T22:57:54.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:54 smithi094 ceph-mon[32889]: pgmap v408: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-26T22:57:54.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.952+0000 7f19b5600700 1 -- 172.21.15.27:0/2939780583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19b0107040 msgr2=0x7f19b01074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:54.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.952+0000 7f19b5600700 1 --2- 172.21.15.27:0/2939780583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19b0107040 0x7f19b01074c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f19a800b0a0 tx=0x7f19a800b3b0 comp rx=0 tx=0).stop 2024-04-26T22:57:54.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.952+0000 7f19b5600700 1 -- 172.21.15.27:0/2939780583 shutdown_connections 2024-04-26T22:57:54.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.952+0000 7f19b5600700 1 --2- 172.21.15.27:0/2939780583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19b0107040 0x7f19b01074c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:54.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.952+0000 7f19b5600700 1 --2- 172.21.15.27:0/2939780583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19b0101d00 0x7f19b0104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:54.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.952+0000 7f19b5600700 1 --2- 172.21.15.27:0/2939780583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19b0104670 0x7f19b0106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:54.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.952+0000 7f19b5600700 1 -- 172.21.15.27:0/2939780583 >> 172.21.15.27:0/2939780583 conn(0x7f19b00fd920 msgr2=0x7f19b00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:54.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.952+0000 7f19b5600700 1 -- 172.21.15.27:0/2939780583 shutdown_connections 2024-04-26T22:57:54.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.952+0000 7f19b5600700 1 -- 172.21.15.27:0/2939780583 wait complete. 2024-04-26T22:57:54.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.954+0000 7f19b5600700 1 Processor -- start 2024-04-26T22:57:54.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.954+0000 7f19b5600700 1 -- start start 2024-04-26T22:57:54.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.955+0000 7f19b5600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19b0101d00 0x7f19b019fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:54.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.955+0000 7f19b5600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19b0104670 0x7f19b01a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:54.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.955+0000 7f19b5600700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19b0107040 0x7f19b01a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:54.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.955+0000 7f19b5600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f19b0115490 con 0x7f19b0104670 2024-04-26T22:57:54.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.955+0000 7f19b5600700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f19b0115310 con 0x7f19b0107040 2024-04-26T22:57:54.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.955+0000 7f19b5600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f19b0115610 con 0x7f19b0101d00 2024-04-26T22:57:54.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.955+0000 7f19aea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19b0101d00 0x7f19b019fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.955+0000 7f19a6000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19b0104670 0x7f19b01a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.955+0000 7f19aea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19b0101d00 0x7f19b019fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:55226/0 (socket says 172.21.15.27:55226) 2024-04-26T22:57:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.955+0000 7f19aea00700 1 -- 172.21.15.27:0/716404110 learned_addr learned my addr 172.21.15.27:0/716404110 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.955+0000 7f19af400700 1 --2- 172.21.15.27:0/716404110 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19b0107040 0x7f19b01a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.956+0000 7f19aea00700 1 -- 172.21.15.27:0/716404110 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19b0107040 msgr2=0x7f19b01a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.956+0000 7f19aea00700 1 --2- 172.21.15.27:0/716404110 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19b0107040 0x7f19b01a70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.956+0000 7f19aea00700 1 -- 172.21.15.27:0/716404110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19b0104670 msgr2=0x7f19b01a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.956+0000 7f19aea00700 1 --2- 172.21.15.27:0/716404110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19b0104670 0x7f19b01a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.956+0000 7f19aea00700 1 -- 172.21.15.27:0/716404110 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b01a7760 con 0x7f19b0101d00 2024-04-26T22:57:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.956+0000 7f19aea00700 1 --2- 172.21.15.27:0/716404110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19b0101d00 0x7f19b019fb50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f19a000e9f0 tx=0x7f19a000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.956+0000 7f19a7e00700 1 -- 172.21.15.27:0/716404110 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a000cbd0 con 0x7f19b0101d00 2024-04-26T22:57:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.956+0000 7f19b5600700 1 -- 172.21.15.27:0/716404110 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b01a7a50 con 0x7f19b0101d00 2024-04-26T22:57:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.957+0000 7f19a7e00700 1 -- 172.21.15.27:0/716404110 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a000ee90 con 0x7f19b0101d00 2024-04-26T22:57:54.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.957+0000 7f19b5600700 1 -- 172.21.15.27:0/716404110 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b01a7f90 con 0x7f19b0101d00 2024-04-26T22:57:54.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.957+0000 7f19a7e00700 1 -- 172.21.15.27:0/716404110 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a00055d0 con 0x7f19b0101d00 2024-04-26T22:57:54.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.958+0000 7f19b5600700 1 -- 172.21.15.27:0/716404110 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f198c005360 con 0x7f19b0101d00 2024-04-26T22:57:54.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.960+0000 7f19a7e00700 1 -- 172.21.15.27:0/716404110 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f19a0004750 con 0x7f19b0101d00 2024-04-26T22:57:54.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.961+0000 7f19a7e00700 1 --2- 172.21.15.27:0/716404110 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f19940a3d50 0x7f19940a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:54.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.961+0000 7f19a7e00700 1 -- 172.21.15.27:0/716404110 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f19a00d0c90 con 0x7f19b0101d00 2024-04-26T22:57:54.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.962+0000 7f19a6000700 1 --2- 172.21.15.27:0/716404110 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f19940a3d50 0x7f19940a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:54.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.962+0000 7f19a6000700 1 --2- 172.21.15.27:0/716404110 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f19940a3d50 0x7f19940a6210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1998009680 tx=0x7f1998009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:54.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:54.964+0000 7f19a7e00700 1 -- 172.21.15.27:0/716404110 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f19a009d080 con 0x7f19b0101d00 2024-04-26T22:57:55.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:55.209+0000 7f19b5600700 1 -- 172.21.15.27:0/716404110 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f198c000bf0 con 0x7f19940a3d50 2024-04-26T22:57:55.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:55.259+0000 7f19a7e00700 1 -- 172.21.15.27:0/716404110 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f198c000bf0 con 0x7f19940a3d50 2024-04-26T22:57:55.358 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T22:57:55.358 INFO:teuthology.orchestra.run.smithi027.stdout:mds.a smithi027 running (2m) 35s ago 2m 16.2M - 19.0.0-3306-gbc612ffb 55dc54019ad8 3666b5925926 2024-04-26T22:57:55.358 INFO:teuthology.orchestra.run.smithi027.stdout:mds.b smithi080 running (2m) 87s ago 2m 28.4M - 19.0.0-3306-gbc612ffb 55dc54019ad8 4f14a8823937 2024-04-26T22:57:55.358 INFO:teuthology.orchestra.run.smithi027.stdout:mds.c smithi094 running (2m) 87s ago 2m 18.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 fa1e2683a695 2024-04-26T22:57:55.358 INFO:teuthology.orchestra.run.smithi027.stdout:mds.d smithi027 running (3m) 35s ago 3m 12.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 abcc565af192 2024-04-26T22:57:55.358 INFO:teuthology.orchestra.run.smithi027.stdout:mds.e smithi080 running (3m) 87s ago 3m 13.7M - 19.0.0-3306-gbc612ffb 55dc54019ad8 cb6cb4468c2c 2024-04-26T22:57:55.358 INFO:teuthology.orchestra.run.smithi027.stdout:mds.f smithi094 running (2m) 87s ago 2m 14.0M - 19.0.0-3306-gbc612ffb 55dc54019ad8 885bd0a555de 2024-04-26T22:57:55.358 INFO:teuthology.orchestra.run.smithi027.stdout:mds.g smithi027 running (2m) 35s ago 2m 16.4M - 19.0.0-3306-gbc612ffb 55dc54019ad8 d8edcbaf187c 2024-04-26T22:57:55.359 INFO:teuthology.orchestra.run.smithi027.stdout:mds.h smithi080 running (2m) 87s ago 2m 18.3M - 19.0.0-3306-gbc612ffb 55dc54019ad8 52d0925938a3 2024-04-26T22:57:55.359 INFO:teuthology.orchestra.run.smithi027.stdout:mds.i smithi094 running (2m) 87s ago 2m 16.9M - 19.0.0-3306-gbc612ffb 55dc54019ad8 d19201f63fdd 2024-04-26T22:57:55.359 INFO:teuthology.orchestra.run.smithi027.stdout:mds.j smithi027 running (3m) 35s ago 3m 12.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 03423de96aae 2024-04-26T22:57:55.359 INFO:teuthology.orchestra.run.smithi027.stdout:mds.k smithi080 running (3m) 87s ago 3m 14.1M - 19.0.0-3306-gbc612ffb 55dc54019ad8 7012fecda154 2024-04-26T22:57:55.359 INFO:teuthology.orchestra.run.smithi027.stdout:mds.l smithi094 running (3m) 87s ago 3m 16.3M - 19.0.0-3306-gbc612ffb 55dc54019ad8 c369057cfe5b 2024-04-26T22:57:55.359 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.x smithi027 *:9283,8765 running (11m) 35s ago 11m 518M - 19.0.0-3306-gbc612ffb 55dc54019ad8 a588d685001b 2024-04-26T22:57:55.359 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.y smithi080 *:8443,8765 running (9m) 87s ago 9m 453M - 19.0.0-3306-gbc612ffb 55dc54019ad8 6fd3b77644f8 2024-04-26T22:57:55.359 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.z smithi094 *:8443,8765 running (9m) 87s ago 9m 455M - 19.0.0-3306-gbc612ffb 55dc54019ad8 ea2b3c353262 2024-04-26T22:57:55.359 INFO:teuthology.orchestra.run.smithi027.stdout:mon.a smithi027 running (11m) 35s ago 12m 48.8M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 a43cb86680a7 2024-04-26T22:57:55.359 INFO:teuthology.orchestra.run.smithi027.stdout:mon.b smithi080 running (9m) 87s ago 9m 40.9M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 fe1b7809b5c1 2024-04-26T22:57:55.359 INFO:teuthology.orchestra.run.smithi027.stdout:mon.c smithi094 running (9m) 87s ago 9m 42.2M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 d9d25be2ae96 2024-04-26T22:57:55.359 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (8m) 35s ago 8m 56.2M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 c65a63d6aba5 2024-04-26T22:57:55.359 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi080 running (8m) 87s ago 8m 57.7M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 f44ba4c2285e 2024-04-26T22:57:55.359 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi094 running (8m) 87s ago 8m 59.9M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 ba169acef7fe 2024-04-26T22:57:55.360 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi027 running (7m) 35s ago 7m 55.9M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 288f76c469b9 2024-04-26T22:57:55.360 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi080 running (7m) 87s ago 7m 45.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 aef85c85f295 2024-04-26T22:57:55.360 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi094 running (6m) 87s ago 6m 56.7M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 01742e580df4 2024-04-26T22:57:55.360 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 smithi027 running (6m) 35s ago 6m 41.8M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 5a5a214bd945 2024-04-26T22:57:55.360 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 smithi080 running (5m) 87s ago 5m 44.8M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 0cbad678a989 2024-04-26T22:57:55.360 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 smithi094 running (5m) 87s ago 5m 54.1M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 70399ea6c0be 2024-04-26T22:57:55.360 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 smithi027 running (4m) 35s ago 4m 41.9M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 a80701b2cb26 2024-04-26T22:57:55.360 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 smithi080 running (4m) 87s ago 4m 42.9M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 c88caadc1b0b 2024-04-26T22:57:55.360 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 smithi094 running (3m) 87s ago 3m 53.8M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 c1c7bc0edf8e 2024-04-26T22:57:55.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:55.262+0000 7f19b5600700 1 -- 172.21.15.27:0/716404110 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f19940a3d50 msgr2=0x7f19940a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:55.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:55.262+0000 7f19b5600700 1 --2- 172.21.15.27:0/716404110 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f19940a3d50 0x7f19940a6210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1998009680 tx=0x7f1998009260 comp rx=0 tx=0).stop 2024-04-26T22:57:55.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:55.262+0000 7f19b5600700 1 -- 172.21.15.27:0/716404110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19b0101d00 msgr2=0x7f19b019fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:55.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:55.262+0000 7f19b5600700 1 --2- 172.21.15.27:0/716404110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19b0101d00 0x7f19b019fb50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f19a000e9f0 tx=0x7f19a000ed00 comp rx=0 tx=0).stop 2024-04-26T22:57:55.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:55.262+0000 7f19b5600700 1 -- 172.21.15.27:0/716404110 shutdown_connections 2024-04-26T22:57:55.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:55.262+0000 7f19b5600700 1 --2- 172.21.15.27:0/716404110 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f19940a3d50 0x7f19940a6210 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:55.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:55.262+0000 7f19b5600700 1 --2- 172.21.15.27:0/716404110 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19b0107040 0x7f19b01a70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:55.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:55.262+0000 7f19b5600700 1 --2- 172.21.15.27:0/716404110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19b0101d00 0x7f19b019fb50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:55.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:55.262+0000 7f19b5600700 1 --2- 172.21.15.27:0/716404110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19b0104670 0x7f19b01a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:55.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:55.262+0000 7f19b5600700 1 -- 172.21.15.27:0/716404110 >> 172.21.15.27:0/716404110 conn(0x7f19b00fd920 msgr2=0x7f19b0102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:55.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:55.263+0000 7f19b5600700 1 -- 172.21.15.27:0/716404110 shutdown_connections 2024-04-26T22:57:55.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:55.263+0000 7f19b5600700 1 -- 172.21.15.27:0/716404110 wait complete. 2024-04-26T22:57:56.357 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T22:57:56.441 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:56 smithi027 ceph-mon[28034]: pgmap v409: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:56.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:56 smithi027 ceph-mon[28034]: from='client.25234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:57:56.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:56 smithi080 ceph-mon[32324]: pgmap v409: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:56.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:56 smithi080 ceph-mon[32324]: from='client.25234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:57:56.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:56 smithi094 ceph-mon[32889]: pgmap v409: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:57:56.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:56 smithi094 ceph-mon[32889]: from='client.25234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:57:56.837 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:57:57.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:57 smithi027 ceph-mon[28034]: pgmap v410: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:57:57.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:57 smithi080 ceph-mon[32324]: pgmap v410: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:57:57.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:57 smithi094 ceph-mon[32889]: pgmap v410: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:57:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.257+0000 7faf20000700 1 -- 172.21.15.27:0/2275709508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faf181056c0 msgr2=0x7faf18105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.257+0000 7faf20000700 1 --2- 172.21.15.27:0/2275709508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faf181056c0 0x7faf18105ae0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009920 tx=0x7faf0c009c30 comp rx=0 tx=0).stop 2024-04-26T22:57:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.257+0000 7faf20000700 1 -- 172.21.15.27:0/2275709508 shutdown_connections 2024-04-26T22:57:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.257+0000 7faf20000700 1 --2- 172.21.15.27:0/2275709508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf18107260 0x7faf1810db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.257+0000 7faf20000700 1 --2- 172.21.15.27:0/2275709508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faf181056c0 0x7faf18105ae0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.257+0000 7faf20000700 1 --2- 172.21.15.27:0/2275709508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf181068c0 0x7faf18106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.257+0000 7faf20000700 1 -- 172.21.15.27:0/2275709508 >> 172.21.15.27:0/2275709508 conn(0x7faf18100c80 msgr2=0x7faf181030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:59.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.257+0000 7faf20000700 1 -- 172.21.15.27:0/2275709508 shutdown_connections 2024-04-26T22:57:59.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.257+0000 7faf20000700 1 -- 172.21.15.27:0/2275709508 wait complete. 2024-04-26T22:57:59.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.258+0000 7faf20000700 1 Processor -- start 2024-04-26T22:57:59.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.259+0000 7faf20000700 1 -- start start 2024-04-26T22:57:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.259+0000 7faf20000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf181056c0 0x7faf1819b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.259+0000 7faf20000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf181068c0 0x7faf1819be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.259+0000 7faf20000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faf18107260 0x7faf181a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.259+0000 7faf20000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faf18111180 con 0x7faf181056c0 2024-04-26T22:57:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.259+0000 7faf20000700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7faf18111000 con 0x7faf18107260 2024-04-26T22:57:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.259+0000 7faf20000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf18111300 con 0x7faf181068c0 2024-04-26T22:57:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.259+0000 7faf1d600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf181056c0 0x7faf1819b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.259+0000 7faf1d600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf181056c0 0x7faf1819b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59642/0 (socket says 172.21.15.27:59642) 2024-04-26T22:57:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.259+0000 7faf1d600700 1 -- 172.21.15.27:0/2148334383 learned_addr learned my addr 172.21.15.27:0/2148334383 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:57:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.259+0000 7faf1cc00700 1 --2- 172.21.15.27:0/2148334383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf181068c0 0x7faf1819be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.260+0000 7faf1e000700 1 --2- 172.21.15.27:0/2148334383 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faf18107260 0x7faf181a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.260+0000 7faf1d600700 1 -- 172.21.15.27:0/2148334383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf181068c0 msgr2=0x7faf1819be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.260+0000 7faf1d600700 1 --2- 172.21.15.27:0/2148334383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf181068c0 0x7faf1819be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.260+0000 7faf1d600700 1 -- 172.21.15.27:0/2148334383 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faf18107260 msgr2=0x7faf181a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.260+0000 7faf1d600700 1 --2- 172.21.15.27:0/2148334383 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faf18107260 0x7faf181a2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.260+0000 7faf1d600700 1 -- 172.21.15.27:0/2148334383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf181a34f0 con 0x7faf181056c0 2024-04-26T22:57:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.260+0000 7faf1cc00700 1 --2- 172.21.15.27:0/2148334383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf181068c0 0x7faf1819be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:57:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.260+0000 7faf1d600700 1 --2- 172.21.15.27:0/2148334383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf181056c0 0x7faf1819b8e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faf0c011040 tx=0x7faf0c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.261+0000 7faf16000700 1 -- 172.21.15.27:0/2148334383 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c027070 con 0x7faf181056c0 2024-04-26T22:57:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.261+0000 7faf16000700 1 -- 172.21.15.27:0/2148334383 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf0c00ba10 con 0x7faf181056c0 2024-04-26T22:57:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.261+0000 7faf20000700 1 -- 172.21.15.27:0/2148334383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf181a3780 con 0x7faf181056c0 2024-04-26T22:57:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.261+0000 7faf16000700 1 -- 172.21.15.27:0/2148334383 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c004910 con 0x7faf181056c0 2024-04-26T22:57:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.261+0000 7faf20000700 1 -- 172.21.15.27:0/2148334383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf181a3bd0 con 0x7faf181056c0 2024-04-26T22:57:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.262+0000 7faf20000700 1 -- 172.21.15.27:0/2148334383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf1806a110 con 0x7faf181056c0 2024-04-26T22:57:59.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.264+0000 7faf16000700 1 -- 172.21.15.27:0/2148334383 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faf0c02a070 con 0x7faf181056c0 2024-04-26T22:57:59.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.266+0000 7faf16000700 1 --2- 172.21.15.27:0/2148334383 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faf000a3d50 0x7faf000a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:57:59.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.266+0000 7faf16000700 1 -- 172.21.15.27:0/2148334383 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7faf0c0d1880 con 0x7faf181056c0 2024-04-26T22:57:59.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.266+0000 7faf1cc00700 1 --2- 172.21.15.27:0/2148334383 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faf000a3d50 0x7faf000a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:57:59.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.267+0000 7faf1cc00700 1 --2- 172.21.15.27:0/2148334383 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faf000a3d50 0x7faf000a6210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faf04011490 tx=0x7faf04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:57:59.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.268+0000 7faf16000700 1 -- 172.21.15.27:0/2148334383 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faf0c031020 con 0x7faf181056c0 2024-04-26T22:57:59.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.505+0000 7faf20000700 1 -- 172.21.15.27:0/2148334383 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7faf18063860 con 0x7faf000a3d50 2024-04-26T22:57:59.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.522+0000 7faf16000700 1 -- 172.21.15.27:0/2148334383 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7faf18063860 con 0x7faf000a3d50 2024-04-26T22:57:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T22:57:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:mds.all 12/12 92s ago 3m smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi094=c;smithi094=f;smithi094=i;smithi094=l;count:12 2024-04-26T22:57:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:mgr 3/3 92s ago 9m smithi027=x;smithi080=y;smithi094=z;count:3 2024-04-26T22:57:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:mon 3/3 92s ago 10m smithi027:172.21.15.27=a;smithi080:172.21.15.80=b;smithi094:172.21.15.94=c;count:3 2024-04-26T22:57:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:osd 12 92s ago - 2024-04-26T22:57:59.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.525+0000 7faf20000700 1 -- 172.21.15.27:0/2148334383 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faf000a3d50 msgr2=0x7faf000a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:59.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.525+0000 7faf20000700 1 --2- 172.21.15.27:0/2148334383 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faf000a3d50 0x7faf000a6210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faf04011490 tx=0x7faf04009260 comp rx=0 tx=0).stop 2024-04-26T22:57:59.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.525+0000 7faf20000700 1 -- 172.21.15.27:0/2148334383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf181056c0 msgr2=0x7faf1819b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:57:59.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.525+0000 7faf20000700 1 --2- 172.21.15.27:0/2148334383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf181056c0 0x7faf1819b8e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faf0c011040 tx=0x7faf0c005270 comp rx=0 tx=0).stop 2024-04-26T22:57:59.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.525+0000 7faf20000700 1 -- 172.21.15.27:0/2148334383 shutdown_connections 2024-04-26T22:57:59.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.525+0000 7faf20000700 1 --2- 172.21.15.27:0/2148334383 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faf000a3d50 0x7faf000a6210 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:59.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.525+0000 7faf20000700 1 --2- 172.21.15.27:0/2148334383 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faf18107260 0x7faf181a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:59.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.525+0000 7faf20000700 1 --2- 172.21.15.27:0/2148334383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf181056c0 0x7faf1819b8e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:59.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.525+0000 7faf20000700 1 --2- 172.21.15.27:0/2148334383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf181068c0 0x7faf1819be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:57:59.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.525+0000 7faf20000700 1 -- 172.21.15.27:0/2148334383 >> 172.21.15.27:0/2148334383 conn(0x7faf18100c80 msgr2=0x7faf18109ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:57:59.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.526+0000 7faf20000700 1 -- 172.21.15.27:0/2148334383 shutdown_connections 2024-04-26T22:57:59.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:57:59.526+0000 7faf20000700 1 -- 172.21.15.27:0/2148334383 wait complete. 2024-04-26T22:57:59.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:57:59 smithi027 ceph-mon[28034]: pgmap v411: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:00.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:57:59 smithi080 ceph-mon[32324]: pgmap v411: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:00.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:57:59 smithi094 ceph-mon[32889]: pgmap v411: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:00.561 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T22:58:00.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:00 smithi027 ceph-mon[28034]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:01.016 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:58:01.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:00 smithi080 ceph-mon[32324]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:01.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:00 smithi094 ceph-mon[32889]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:01.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:01 smithi027 ceph-mon[28034]: pgmap v412: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:58:02.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:01 smithi080 ceph-mon[32324]: pgmap v412: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:58:02.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:01 smithi094 ceph-mon[32889]: pgmap v412: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:58:03.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.505+0000 7eff40a00700 1 -- 172.21.15.27:0/2006891412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff3c1056c0 msgr2=0x7eff3c105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.505+0000 7eff40a00700 1 --2- 172.21.15.27:0/2006891412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff3c1056c0 0x7eff3c105ae0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0098f0 tx=0x7eff2c009c00 comp rx=0 tx=0).stop 2024-04-26T22:58:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.505+0000 7eff40a00700 1 -- 172.21.15.27:0/2006891412 shutdown_connections 2024-04-26T22:58:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.505+0000 7eff40a00700 1 --2- 172.21.15.27:0/2006891412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff3c107260 0x7eff3c10db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.505+0000 7eff40a00700 1 --2- 172.21.15.27:0/2006891412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff3c1056c0 0x7eff3c105ae0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.505+0000 7eff40a00700 1 --2- 172.21.15.27:0/2006891412 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff3c1068c0 0x7eff3c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.505+0000 7eff40a00700 1 -- 172.21.15.27:0/2006891412 >> 172.21.15.27:0/2006891412 conn(0x7eff3c100c80 msgr2=0x7eff3c1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.505+0000 7eff40a00700 1 -- 172.21.15.27:0/2006891412 shutdown_connections 2024-04-26T22:58:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.505+0000 7eff40a00700 1 -- 172.21.15.27:0/2006891412 wait complete. 2024-04-26T22:58:03.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.506+0000 7eff40a00700 1 Processor -- start 2024-04-26T22:58:03.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.507+0000 7eff40a00700 1 -- start start 2024-04-26T22:58:03.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.507+0000 7eff40a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff3c1056c0 0x7eff3c19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:03.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.507+0000 7eff40a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff3c1068c0 0x7eff3c19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.507+0000 7eff40a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff3c107260 0x7eff3c1a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.507+0000 7eff40a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c111140 con 0x7eff3c107260 2024-04-26T22:58:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.507+0000 7eff40a00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c110fc0 con 0x7eff3c1068c0 2024-04-26T22:58:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.507+0000 7eff40a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c1112c0 con 0x7eff3c1056c0 2024-04-26T22:58:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.508+0000 7eff39400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff3c1068c0 0x7eff3c19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.508+0000 7eff39e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff3c1056c0 0x7eff3c19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.508+0000 7eff39400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff3c1068c0 0x7eff3c19bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:44570/0 (socket says 172.21.15.27:44570) 2024-04-26T22:58:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.508+0000 7eff39400700 1 -- 172.21.15.27:0/947197584 learned_addr learned my addr 172.21.15.27:0/947197584 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.508+0000 7eff3a800700 1 --2- 172.21.15.27:0/947197584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff3c107260 0x7eff3c1a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.508+0000 7eff39e00700 1 -- 172.21.15.27:0/947197584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff3c1068c0 msgr2=0x7eff3c19bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.508+0000 7eff39e00700 1 --2- 172.21.15.27:0/947197584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff3c1068c0 0x7eff3c19bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.508+0000 7eff39e00700 1 -- 172.21.15.27:0/947197584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff3c107260 msgr2=0x7eff3c1a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.508+0000 7eff39e00700 1 --2- 172.21.15.27:0/947197584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff3c107260 0x7eff3c1a2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.508+0000 7eff39e00700 1 -- 172.21.15.27:0/947197584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3c1a3470 con 0x7eff3c1056c0 2024-04-26T22:58:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.508+0000 7eff39400700 1 --2- 172.21.15.27:0/947197584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff3c1068c0 0x7eff3c19bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.509+0000 7eff39e00700 1 --2- 172.21.15.27:0/947197584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff3c1056c0 0x7eff3c19b810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009230 tx=0x7eff2c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:03.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.509+0000 7eff32a00700 1 -- 172.21.15.27:0/947197584 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c005670 con 0x7eff3c1056c0 2024-04-26T22:58:03.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.509+0000 7eff32a00700 1 -- 172.21.15.27:0/947197584 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff2c00ba10 con 0x7eff3c1056c0 2024-04-26T22:58:03.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.509+0000 7eff40a00700 1 -- 172.21.15.27:0/947197584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff3c1a3700 con 0x7eff3c1056c0 2024-04-26T22:58:03.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.509+0000 7eff32a00700 1 -- 172.21.15.27:0/947197584 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c003d30 con 0x7eff3c1056c0 2024-04-26T22:58:03.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.509+0000 7eff40a00700 1 -- 172.21.15.27:0/947197584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff3c1a3bb0 con 0x7eff3c1056c0 2024-04-26T22:58:03.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.510+0000 7eff40a00700 1 -- 172.21.15.27:0/947197584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff3c06a110 con 0x7eff3c1056c0 2024-04-26T22:58:03.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.512+0000 7eff32a00700 1 -- 172.21.15.27:0/947197584 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7eff2c00b560 con 0x7eff3c1056c0 2024-04-26T22:58:03.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.514+0000 7eff32a00700 1 --2- 172.21.15.27:0/947197584 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7eff200a3d50 0x7eff200a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:03.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.514+0000 7eff32a00700 1 -- 172.21.15.27:0/947197584 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7eff2c0d1540 con 0x7eff3c1056c0 2024-04-26T22:58:03.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.514+0000 7eff39400700 1 --2- 172.21.15.27:0/947197584 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7eff200a3d50 0x7eff200a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:03.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.515+0000 7eff39400700 1 --2- 172.21.15.27:0/947197584 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7eff200a3d50 0x7eff200a6210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7eff28005950 tx=0x7eff28009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:03.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.516+0000 7eff32a00700 1 -- 172.21.15.27:0/947197584 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7eff2c013030 con 0x7eff3c1056c0 2024-04-26T22:58:03.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.763+0000 7eff40a00700 1 -- 172.21.15.27:0/947197584 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7eff3c063920 con 0x7eff200a3d50 2024-04-26T22:58:03.764 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:03 smithi027 ceph-mon[28034]: pgmap v413: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:58:03.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.767+0000 7eff32a00700 1 -- 172.21.15.27:0/947197584 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7eff3c063920 con 0x7eff200a3d50 2024-04-26T22:58:03.767 INFO:teuthology.orchestra.run.smithi027.stdout:HOST ADDR LABELS STATUS 2024-04-26T22:58:03.767 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 172.21.15.27 2024-04-26T22:58:03.768 INFO:teuthology.orchestra.run.smithi027.stdout:smithi080 172.21.15.80 2024-04-26T22:58:03.768 INFO:teuthology.orchestra.run.smithi027.stdout:smithi094 172.21.15.94 2024-04-26T22:58:03.768 INFO:teuthology.orchestra.run.smithi027.stdout:3 hosts in cluster 2024-04-26T22:58:03.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.770+0000 7eff40a00700 1 -- 172.21.15.27:0/947197584 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7eff200a3d50 msgr2=0x7eff200a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:03.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.770+0000 7eff40a00700 1 --2- 172.21.15.27:0/947197584 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7eff200a3d50 0x7eff200a6210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7eff28005950 tx=0x7eff28009310 comp rx=0 tx=0).stop 2024-04-26T22:58:03.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.770+0000 7eff40a00700 1 -- 172.21.15.27:0/947197584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff3c1056c0 msgr2=0x7eff3c19b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:03.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.770+0000 7eff40a00700 1 --2- 172.21.15.27:0/947197584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff3c1056c0 0x7eff3c19b810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009230 tx=0x7eff2c005270 comp rx=0 tx=0).stop 2024-04-26T22:58:03.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.770+0000 7eff40a00700 1 -- 172.21.15.27:0/947197584 shutdown_connections 2024-04-26T22:58:03.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.770+0000 7eff40a00700 1 --2- 172.21.15.27:0/947197584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff3c107260 0x7eff3c1a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:03.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.770+0000 7eff40a00700 1 --2- 172.21.15.27:0/947197584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff3c1056c0 0x7eff3c19b810 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:03.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.770+0000 7eff40a00700 1 --2- 172.21.15.27:0/947197584 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7eff200a3d50 0x7eff200a6210 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:03.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.770+0000 7eff40a00700 1 --2- 172.21.15.27:0/947197584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff3c1068c0 0x7eff3c19bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:03.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.770+0000 7eff40a00700 1 -- 172.21.15.27:0/947197584 >> 172.21.15.27:0/947197584 conn(0x7eff3c100c80 msgr2=0x7eff3c109c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:03.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.770+0000 7eff40a00700 1 -- 172.21.15.27:0/947197584 shutdown_connections 2024-04-26T22:58:03.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:03.770+0000 7eff40a00700 1 -- 172.21.15.27:0/947197584 wait complete. 2024-04-26T22:58:04.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:03 smithi080 ceph-mon[32324]: pgmap v413: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:58:04.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:03 smithi094 ceph-mon[32889]: pgmap v413: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:58:04.659 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T22:58:04.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:04 smithi027 ceph-mon[28034]: from='client.25246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:05.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:04 smithi080 ceph-mon[32324]: from='client.25246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:05.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:04 smithi094 ceph-mon[32889]: from='client.25246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:05.131 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:58:05.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:05 smithi027 ceph-mon[28034]: pgmap v414: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:06.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:05 smithi080 ceph-mon[32324]: pgmap v414: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:06.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:05 smithi094 ceph-mon[32889]: pgmap v414: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:07.727 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:07 smithi027 ceph-mon[28034]: pgmap v415: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:58:07.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.754+0000 7fe90da00700 1 -- 172.21.15.27:0/2133159154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9081056c0 msgr2=0x7fe908105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:07.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.754+0000 7fe90da00700 1 --2- 172.21.15.27:0/2133159154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9081056c0 0x7fe908105ae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009f90 tx=0x7fe8f800b330 comp rx=0 tx=0).stop 2024-04-26T22:58:07.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.754+0000 7fe90da00700 1 -- 172.21.15.27:0/2133159154 shutdown_connections 2024-04-26T22:58:07.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.754+0000 7fe90da00700 1 --2- 172.21.15.27:0/2133159154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe908107260 0x7fe90810db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:07.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.754+0000 7fe90da00700 1 --2- 172.21.15.27:0/2133159154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9081056c0 0x7fe908105ae0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:07.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.754+0000 7fe90da00700 1 --2- 172.21.15.27:0/2133159154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9081068c0 0x7fe908106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:07.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.754+0000 7fe90da00700 1 -- 172.21.15.27:0/2133159154 >> 172.21.15.27:0/2133159154 conn(0x7fe908100c80 msgr2=0x7fe9081030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:07.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.754+0000 7fe90da00700 1 -- 172.21.15.27:0/2133159154 shutdown_connections 2024-04-26T22:58:07.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.754+0000 7fe90da00700 1 -- 172.21.15.27:0/2133159154 wait complete. 2024-04-26T22:58:07.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.756+0000 7fe90da00700 1 Processor -- start 2024-04-26T22:58:07.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.756+0000 7fe90da00700 1 -- start start 2024-04-26T22:58:07.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.756+0000 7fe90da00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9081056c0 0x7fe90819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:07.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.757+0000 7fe90da00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9081068c0 0x7fe90819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:07.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.757+0000 7fe90da00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe908107260 0x7fe9081a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:07.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.757+0000 7fe90da00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe908111180 con 0x7fe908107260 2024-04-26T22:58:07.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.757+0000 7fe90da00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe908111000 con 0x7fe9081056c0 2024-04-26T22:58:07.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.757+0000 7fe90da00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe908111300 con 0x7fe9081068c0 2024-04-26T22:58:07.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.757+0000 7fe906a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9081056c0 0x7fe90819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:07.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.757+0000 7fe906a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9081056c0 0x7fe90819b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:44586/0 (socket says 172.21.15.27:44586) 2024-04-26T22:58:07.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.757+0000 7fe906a00700 1 -- 172.21.15.27:0/797845086 learned_addr learned my addr 172.21.15.27:0/797845086 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:07.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.757+0000 7fe8fe000700 1 --2- 172.21.15.27:0/797845086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9081068c0 0x7fe90819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:07.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.757+0000 7fe907400700 1 --2- 172.21.15.27:0/797845086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe908107260 0x7fe9081a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:07.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.757+0000 7fe906a00700 1 -- 172.21.15.27:0/797845086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9081068c0 msgr2=0x7fe90819bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:07.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.757+0000 7fe906a00700 1 --2- 172.21.15.27:0/797845086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9081068c0 0x7fe90819bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:07.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.757+0000 7fe906a00700 1 -- 172.21.15.27:0/797845086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe908107260 msgr2=0x7fe9081a2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:07.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.757+0000 7fe906a00700 1 --2- 172.21.15.27:0/797845086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe908107260 0x7fe9081a2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:07.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.757+0000 7fe906a00700 1 -- 172.21.15.27:0/797845086 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9081a3460 con 0x7fe9081056c0 2024-04-26T22:58:07.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.758+0000 7fe906a00700 1 --2- 172.21.15.27:0/797845086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9081056c0 0x7fe90819b800 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe8f800b6c0 tx=0x7fe8f8007960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:07.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.758+0000 7fe8ffe00700 1 -- 172.21.15.27:0/797845086 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8021070 con 0x7fe9081056c0 2024-04-26T22:58:07.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.758+0000 7fe8ffe00700 1 -- 172.21.15.27:0/797845086 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8f801db00 con 0x7fe9081056c0 2024-04-26T22:58:07.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.758+0000 7fe8ffe00700 1 -- 172.21.15.27:0/797845086 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f801bb80 con 0x7fe9081056c0 2024-04-26T22:58:07.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.758+0000 7fe90da00700 1 -- 172.21.15.27:0/797845086 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9081a36f0 con 0x7fe9081056c0 2024-04-26T22:58:07.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.758+0000 7fe90da00700 1 -- 172.21.15.27:0/797845086 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9081a3b50 con 0x7fe9081056c0 2024-04-26T22:58:07.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.760+0000 7fe90da00700 1 -- 172.21.15.27:0/797845086 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe90806a110 con 0x7fe9081056c0 2024-04-26T22:58:07.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.761+0000 7fe8ffe00700 1 -- 172.21.15.27:0/797845086 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe8f8024070 con 0x7fe9081056c0 2024-04-26T22:58:07.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.763+0000 7fe8ffe00700 1 --2- 172.21.15.27:0/797845086 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe8ec0a3d50 0x7fe8ec0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:07.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.763+0000 7fe8ffe00700 1 -- 172.21.15.27:0/797845086 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fe8f800f020 con 0x7fe9081056c0 2024-04-26T22:58:07.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.763+0000 7fe8fe000700 1 --2- 172.21.15.27:0/797845086 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe8ec0a3d50 0x7fe8ec0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:07.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.764+0000 7fe8fe000700 1 --2- 172.21.15.27:0/797845086 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe8ec0a3d50 0x7fe8ec0a6210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009680 tx=0x7fe8f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:07.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:07.767+0000 7fe8ffe00700 1 -- 172.21.15.27:0/797845086 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe8f80d6070 con 0x7fe9081056c0 2024-04-26T22:58:08.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:08.008+0000 7fe90da00700 1 -- 172.21.15.27:0/797845086 --> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe908063860 con 0x7fe8ec0a3d50 2024-04-26T22:58:08.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:08.013+0000 7fe8ffe00700 1 -- 172.21.15.27:0/797845086 <== mgr.14152 v2:172.21.15.27:6800/3636526714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe908063860 con 0x7fe8ec0a3d50 2024-04-26T22:58:08.013 INFO:teuthology.orchestra.run.smithi027.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T22:58:08.014 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001C400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T22:58:08.014 INFO:teuthology.orchestra.run.smithi027.stdout:smithi080 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204014T400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T22:58:08.014 INFO:teuthology.orchestra.run.smithi027.stdout:smithi094 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001L480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-26T22:58:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:08.015+0000 7fe90da00700 1 -- 172.21.15.27:0/797845086 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe8ec0a3d50 msgr2=0x7fe8ec0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:08.015+0000 7fe90da00700 1 --2- 172.21.15.27:0/797845086 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe8ec0a3d50 0x7fe8ec0a6210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009680 tx=0x7fe8f0009260 comp rx=0 tx=0).stop 2024-04-26T22:58:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:08.016+0000 7fe90da00700 1 -- 172.21.15.27:0/797845086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9081056c0 msgr2=0x7fe90819b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:08.016+0000 7fe90da00700 1 --2- 172.21.15.27:0/797845086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9081056c0 0x7fe90819b800 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe8f800b6c0 tx=0x7fe8f8007960 comp rx=0 tx=0).stop 2024-04-26T22:58:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:08.016+0000 7fe90da00700 1 -- 172.21.15.27:0/797845086 shutdown_connections 2024-04-26T22:58:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:08.016+0000 7fe90da00700 1 --2- 172.21.15.27:0/797845086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe908107260 0x7fe9081a2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:08.016+0000 7fe90da00700 1 --2- 172.21.15.27:0/797845086 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe8ec0a3d50 0x7fe8ec0a6210 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:08.016+0000 7fe90da00700 1 --2- 172.21.15.27:0/797845086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9081056c0 0x7fe90819b800 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:08.016+0000 7fe90da00700 1 --2- 172.21.15.27:0/797845086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9081068c0 0x7fe90819bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:08.016+0000 7fe90da00700 1 -- 172.21.15.27:0/797845086 >> 172.21.15.27:0/797845086 conn(0x7fe908100c80 msgr2=0x7fe908109c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:08.016+0000 7fe90da00700 1 -- 172.21.15.27:0/797845086 shutdown_connections 2024-04-26T22:58:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:08.017+0000 7fe90da00700 1 -- 172.21.15.27:0/797845086 wait complete. 2024-04-26T22:58:08.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:07 smithi080 ceph-mon[32324]: pgmap v415: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:58:08.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:07 smithi094 ceph-mon[32889]: pgmap v415: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:58:08.778 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:08 smithi027 ceph-mon[28034]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:08.951 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T22:58:08.962 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T22:58:08.962 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T22:58:09.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:08 smithi080 ceph-mon[32324]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:09.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:08 smithi094 ceph-mon[32889]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:09.416 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:58:09.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:09 smithi027 ceph-mon[28034]: pgmap v416: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:10.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:09 smithi080 ceph-mon[32324]: pgmap v416: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:10.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:09 smithi094 ceph-mon[32889]: pgmap v416: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:11.972 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:11 smithi027 ceph-mon[28034]: pgmap v417: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:58:12.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:11 smithi080 ceph-mon[32324]: pgmap v417: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:58:12.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:11 smithi094 ceph-mon[32889]: pgmap v417: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:58:12.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.038+0000 7fe21e200700 1 -- 172.21.15.27:0/1315732231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe21810acc0 msgr2=0x7fe21810b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:12.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.038+0000 7fe21e200700 1 --2- 172.21.15.27:0/1315732231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe21810acc0 0x7fe21810b120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe2040098f0 tx=0x7fe204009c00 comp rx=0 tx=0).stop 2024-04-26T22:58:12.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.038+0000 7fe21e200700 1 -- 172.21.15.27:0/1315732231 shutdown_connections 2024-04-26T22:58:12.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.038+0000 7fe21e200700 1 --2- 172.21.15.27:0/1315732231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe21810b660 0x7fe218111f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:12.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.038+0000 7fe21e200700 1 --2- 172.21.15.27:0/1315732231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe218109ac0 0x7fe218109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:12.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.038+0000 7fe21e200700 1 --2- 172.21.15.27:0/1315732231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe21810acc0 0x7fe21810b120 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:12.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.038+0000 7fe21e200700 1 -- 172.21.15.27:0/1315732231 >> 172.21.15.27:0/1315732231 conn(0x7fe218078a90 msgr2=0x7fe21807aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:12.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.038+0000 7fe21e200700 1 -- 172.21.15.27:0/1315732231 shutdown_connections 2024-04-26T22:58:12.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.038+0000 7fe21e200700 1 -- 172.21.15.27:0/1315732231 wait complete. 2024-04-26T22:58:12.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.040+0000 7fe21e200700 1 Processor -- start 2024-04-26T22:58:12.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.040+0000 7fe21e200700 1 -- start start 2024-04-26T22:58:12.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.040+0000 7fe21e200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe218109ac0 0x7fe21819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:12.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.040+0000 7fe21e200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe21810acc0 0x7fe2181a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:12.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.041+0000 7fe21e200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe21810b660 0x7fe2181a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:12.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.041+0000 7fe21e200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe218115560 con 0x7fe21810acc0 2024-04-26T22:58:12.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.041+0000 7fe21e200700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181153e0 con 0x7fe21810b660 2024-04-26T22:58:12.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.041+0000 7fe21e200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181156e0 con 0x7fe218109ac0 2024-04-26T22:58:12.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.041+0000 7fe216a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe21810acc0 0x7fe2181a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:12.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.041+0000 7fe217400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe218109ac0 0x7fe21819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:12.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.041+0000 7fe216a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe21810acc0 0x7fe2181a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51774/0 (socket says 172.21.15.27:51774) 2024-04-26T22:58:12.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.041+0000 7fe216a00700 1 -- 172.21.15.27:0/3374855515 learned_addr learned my addr 172.21.15.27:0/3374855515 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:12.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.041+0000 7fe217e00700 1 --2- 172.21.15.27:0/3374855515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe21810b660 0x7fe2181a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:12.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.041+0000 7fe217400700 1 -- 172.21.15.27:0/3374855515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe21810b660 msgr2=0x7fe2181a71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:12.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.041+0000 7fe217400700 1 --2- 172.21.15.27:0/3374855515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe21810b660 0x7fe2181a71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:12.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.041+0000 7fe217400700 1 -- 172.21.15.27:0/3374855515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe21810acc0 msgr2=0x7fe2181a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:12.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.041+0000 7fe217400700 1 --2- 172.21.15.27:0/3374855515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe21810acc0 0x7fe2181a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:12.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.041+0000 7fe217400700 1 -- 172.21.15.27:0/3374855515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2181a7870 con 0x7fe218109ac0 2024-04-26T22:58:12.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.042+0000 7fe217400700 1 --2- 172.21.15.27:0/3374855515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe218109ac0 0x7fe21819fc20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe208005fd0 tx=0x7fe20800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:12.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.042+0000 7fe20fe00700 1 -- 172.21.15.27:0/3374855515 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20800cbd0 con 0x7fe218109ac0 2024-04-26T22:58:12.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.042+0000 7fe20fe00700 1 -- 172.21.15.27:0/3374855515 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe20800cd70 con 0x7fe218109ac0 2024-04-26T22:58:12.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.042+0000 7fe21e200700 1 -- 172.21.15.27:0/3374855515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2181a7b60 con 0x7fe218109ac0 2024-04-26T22:58:12.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.042+0000 7fe20fe00700 1 -- 172.21.15.27:0/3374855515 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2080187a0 con 0x7fe218109ac0 2024-04-26T22:58:12.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.042+0000 7fe21e200700 1 -- 172.21.15.27:0/3374855515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2181a80f0 con 0x7fe218109ac0 2024-04-26T22:58:12.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.044+0000 7fe21e200700 1 -- 172.21.15.27:0/3374855515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe21806a110 con 0x7fe218109ac0 2024-04-26T22:58:12.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.046+0000 7fe20fe00700 1 -- 172.21.15.27:0/3374855515 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe20801f020 con 0x7fe218109ac0 2024-04-26T22:58:12.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.047+0000 7fe20fe00700 1 --2- 172.21.15.27:0/3374855515 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe1fc0a3d50 0x7fe1fc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:12.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.048+0000 7fe20fe00700 1 -- 172.21.15.27:0/3374855515 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fe2080d0f10 con 0x7fe218109ac0 2024-04-26T22:58:12.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.048+0000 7fe216a00700 1 --2- 172.21.15.27:0/3374855515 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe1fc0a3d50 0x7fe1fc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:12.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.049+0000 7fe216a00700 1 --2- 172.21.15.27:0/3374855515 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe1fc0a3d50 0x7fe1fc0a6210 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe204009680 tx=0x7fe204019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:12.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.049+0000 7fe20fe00700 1 -- 172.21.15.27:0/3374855515 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe20809d300 con 0x7fe218109ac0 2024-04-26T22:58:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.333+0000 7fe21e200700 1 -- 172.21.15.27:0/3374855515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe218064030 con 0x7fe218109ac0 2024-04-26T22:58:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.334+0000 7fe20fe00700 1 -- 172.21.15.27:0/3374855515 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3038 (secure 0 0 0) 0x7fe20809cc00 con 0x7fe218109ac0 2024-04-26T22:58:12.335 INFO:teuthology.orchestra.run.smithi027.stdout:e24 2024-04-26T22:58:12.335 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T22:58:12.335 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T22:58:12.335 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2024-04-26T22:58:12.335 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:58:12.336 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2024-04-26T22:58:12.336 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2024-04-26T22:58:12.336 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 24 2024-04-26T22:58:12.336 INFO:teuthology.orchestra.run.smithi027.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-26T22:58:12.336 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-04-26T22:55:25.265331+0000 2024-04-26T22:58:12.336 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2024-04-26T22:55:54.648644+0000 2024-04-26T22:58:12.336 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2024-04-26T22:58:12.336 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2024-04-26T22:58:12.336 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 300 2024-04-26T22:58:12.336 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2024-04-26T22:58:12.336 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2024-04-26T22:58:12.336 INFO:teuthology.orchestra.run.smithi027.stdout:max_xattr_size 65536 2024-04-26T22:58:12.336 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2024-04-26T22:58:12.336 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2024-04-26T22:58:12.337 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 0 2024-04-26T22:58:12.337 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T22:58:12.337 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 3 2024-04-26T22:58:12.337 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1,2 2024-04-26T22:58:12.337 INFO:teuthology.orchestra.run.smithi027.stdout:up {0=24578,1=24577,2=24560} 2024-04-26T22:58:12.337 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2024-04-26T22:58:12.337 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2024-04-26T22:58:12.337 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2024-04-26T22:58:12.337 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3,4] 2024-04-26T22:58:12.337 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2024-04-26T22:58:12.337 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2024-04-26T22:58:12.337 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2024-04-26T22:58:12.337 INFO:teuthology.orchestra.run.smithi027.stdout:bal_rank_mask -1 2024-04-26T22:58:12.337 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 0 2024-04-26T22:58:12.337 INFO:teuthology.orchestra.run.smithi027.stdout:qdb_cluster leader: 24578 members: 24560,24577,24578 2024-04-26T22:58:12.338 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b{0:24578} state up:active seq 3 addr [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:58:12.338 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.i{0:24559} state up:standby-replay seq 1 addr [v2:172.21.15.94:6838/1620677319,v1:172.21.15.94:6839/1620677319] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:58:12.338 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.g{1:24577} state up:active seq 12 addr [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:58:12.338 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c{1:24535} state up:standby-replay seq 1 addr [v2:172.21.15.94:6836/3367319222,v1:172.21.15.94:6837/3367319222] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:58:12.338 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a{2:24560} state up:active seq 11 addr [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:58:12.338 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.h{2:24518} state up:standby-replay seq 1 addr [v2:172.21.15.80:6836/3963089995,v1:172.21.15.80:6837/3963089995] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:58:12.338 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:58:12.338 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:58:12.338 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2024-04-26T22:58:12.338 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:58:12.338 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.j{-1:14385} state up:standby seq 1 addr [v2:172.21.15.27:6834/3122709125,v1:172.21.15.27:6835/3122709125] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:58:12.338 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.k{-1:24440} state up:standby seq 1 addr [v2:172.21.15.80:6832/899705990,v1:172.21.15.80:6833/899705990] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:58:12.338 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.l{-1:24451} state up:standby seq 1 addr [v2:172.21.15.94:6832/828002248,v1:172.21.15.94:6833/828002248] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:58:12.339 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d{-1:24476} state up:standby seq 1 addr [v2:172.21.15.27:6836/1639451555,v1:172.21.15.27:6837/1639451555] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:58:12.339 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.e{-1:24488} state up:standby seq 1 addr [v2:172.21.15.80:6834/3792288301,v1:172.21.15.80:6835/3792288301] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:58:12.339 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.f{-1:24517} state up:standby seq 1 addr [v2:172.21.15.94:6834/3513669668,v1:172.21.15.94:6835/3513669668] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:58:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.339+0000 7fe21e200700 1 -- 172.21.15.27:0/3374855515 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe1fc0a3d50 msgr2=0x7fe1fc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.339+0000 7fe21e200700 1 --2- 172.21.15.27:0/3374855515 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe1fc0a3d50 0x7fe1fc0a6210 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe204009680 tx=0x7fe204019040 comp rx=0 tx=0).stop 2024-04-26T22:58:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.339+0000 7fe21e200700 1 -- 172.21.15.27:0/3374855515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe218109ac0 msgr2=0x7fe21819fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.339+0000 7fe21e200700 1 --2- 172.21.15.27:0/3374855515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe218109ac0 0x7fe21819fc20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe208005fd0 tx=0x7fe20800ee70 comp rx=0 tx=0).stop 2024-04-26T22:58:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.339+0000 7fe21e200700 1 -- 172.21.15.27:0/3374855515 shutdown_connections 2024-04-26T22:58:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.339+0000 7fe21e200700 1 --2- 172.21.15.27:0/3374855515 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fe1fc0a3d50 0x7fe1fc0a6210 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.339+0000 7fe21e200700 1 --2- 172.21.15.27:0/3374855515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe21810b660 0x7fe2181a71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.339+0000 7fe21e200700 1 --2- 172.21.15.27:0/3374855515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe218109ac0 0x7fe21819fc20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.339+0000 7fe21e200700 1 --2- 172.21.15.27:0/3374855515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe21810acc0 0x7fe2181a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.339+0000 7fe21e200700 1 -- 172.21.15.27:0/3374855515 >> 172.21.15.27:0/3374855515 conn(0x7fe218078a90 msgr2=0x7fe21810dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.339+0000 7fe21e200700 1 -- 172.21.15.27:0/3374855515 shutdown_connections 2024-04-26T22:58:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:12.340+0000 7fe21e200700 1 -- 172.21.15.27:0/3374855515 wait complete. 2024-04-26T22:58:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-04-26T22:58:13.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:12 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3374855515' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T22:58:13.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:12 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3374855515' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T22:58:13.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:12 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3374855515' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T22:58:13.425 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-26T22:58:13.908 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:58:14.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:13 smithi080 ceph-mon[32324]: pgmap v418: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:58:14.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:13 smithi094 ceph-mon[32889]: pgmap v418: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:58:14.063 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:13 smithi027 ceph-mon[28034]: pgmap v418: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:58:15.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:15 smithi027 ceph-mon[28034]: pgmap v419: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:16.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:15 smithi080 ceph-mon[32324]: pgmap v419: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:16.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:15 smithi094 ceph-mon[32889]: pgmap v419: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:16.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.531+0000 7f27ca000700 1 -- 172.21.15.27:0/3940576301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c4109170 msgr2=0x7f27c410b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:16.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.531+0000 7f27ca000700 1 --2- 172.21.15.27:0/3940576301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c4109170 0x7f27c410b5e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00b0a0 tx=0x7f27bc00b3b0 comp rx=0 tx=0).stop 2024-04-26T22:58:16.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.532+0000 7f27ca000700 1 -- 172.21.15.27:0/3940576301 shutdown_connections 2024-04-26T22:58:16.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.532+0000 7f27ca000700 1 --2- 172.21.15.27:0/3940576301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c4109170 0x7f27c410b5e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:16.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.532+0000 7f27ca000700 1 --2- 172.21.15.27:0/3940576301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27c4103f10 0x7f27c4106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:16.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.532+0000 7f27ca000700 1 --2- 172.21.15.27:0/3940576301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27c4106840 0x7f27c4108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:16.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.532+0000 7f27ca000700 1 -- 172.21.15.27:0/3940576301 >> 172.21.15.27:0/3940576301 conn(0x7f27c40fd920 msgr2=0x7f27c40ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:16.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.532+0000 7f27ca000700 1 -- 172.21.15.27:0/3940576301 shutdown_connections 2024-04-26T22:58:16.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.532+0000 7f27ca000700 1 -- 172.21.15.27:0/3940576301 wait complete. 2024-04-26T22:58:16.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.533+0000 7f27ca000700 1 Processor -- start 2024-04-26T22:58:16.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.533+0000 7f27ca000700 1 -- start start 2024-04-26T22:58:16.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.533+0000 7f27ca000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c4103f10 0x7f27c4197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:16.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.533+0000 7f27ca000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27c4106840 0x7f27c4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:16.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.533+0000 7f27ca000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27c4109170 0x7f27c4194ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:16.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.533+0000 7f27ca000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f27c410ef40 con 0x7f27c4103f10 2024-04-26T22:58:16.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.533+0000 7f27ca000700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f27c410edc0 con 0x7f27c4109170 2024-04-26T22:58:16.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.533+0000 7f27ca000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f27c410f0c0 con 0x7f27c4106840 2024-04-26T22:58:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.534+0000 7f27c3400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c4103f10 0x7f27c4197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.534+0000 7f27bbe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27c4106840 0x7f27c4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.534+0000 7f27bbe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27c4106840 0x7f27c4198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:37152/0 (socket says 172.21.15.27:37152) 2024-04-26T22:58:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.534+0000 7f27bbe00700 1 -- 172.21.15.27:0/2937945400 learned_addr learned my addr 172.21.15.27:0/2937945400 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.534+0000 7f27c3400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c4103f10 0x7f27c4197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51794/0 (socket says 172.21.15.27:51794) 2024-04-26T22:58:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.534+0000 7f27c3e00700 1 --2- 172.21.15.27:0/2937945400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27c4109170 0x7f27c4194ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.534+0000 7f27bbe00700 1 -- 172.21.15.27:0/2937945400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27c4109170 msgr2=0x7f27c4194ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.534+0000 7f27bbe00700 1 --2- 172.21.15.27:0/2937945400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27c4109170 0x7f27c4194ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.534+0000 7f27bbe00700 1 -- 172.21.15.27:0/2937945400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c4103f10 msgr2=0x7f27c4197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.534+0000 7f27bbe00700 1 --2- 172.21.15.27:0/2937945400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c4103f10 0x7f27c4197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.534+0000 7f27bbe00700 1 -- 172.21.15.27:0/2937945400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c41953a0 con 0x7f27c4106840 2024-04-26T22:58:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.534+0000 7f27c3400700 1 --2- 172.21.15.27:0/2937945400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c4103f10 0x7f27c4197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:16.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.535+0000 7f27bbe00700 1 --2- 172.21.15.27:0/2937945400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27c4106840 0x7f27c4198490 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00e9a0 tx=0x7f27ac00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:16.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.535+0000 7f27c0c00700 1 -- 172.21.15.27:0/2937945400 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac00ca10 con 0x7f27c4106840 2024-04-26T22:58:16.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.535+0000 7f27c0c00700 1 -- 172.21.15.27:0/2937945400 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27ac00cbb0 con 0x7f27c4106840 2024-04-26T22:58:16.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.535+0000 7f27ca000700 1 -- 172.21.15.27:0/2937945400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c41a7c50 con 0x7f27c4106840 2024-04-26T22:58:16.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.535+0000 7f27c0c00700 1 -- 172.21.15.27:0/2937945400 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac018680 con 0x7f27c4106840 2024-04-26T22:58:16.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.535+0000 7f27ca000700 1 -- 172.21.15.27:0/2937945400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c41a80a0 con 0x7f27c4106840 2024-04-26T22:58:16.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.536+0000 7f27ca000700 1 -- 172.21.15.27:0/2937945400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c402e490 con 0x7f27c4106840 2024-04-26T22:58:16.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.538+0000 7f27c0c00700 1 -- 172.21.15.27:0/2937945400 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f27ac010430 con 0x7f27c4106840 2024-04-26T22:58:16.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.540+0000 7f27c0c00700 1 --2- 172.21.15.27:0/2937945400 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f27a80a3cd0 0x7f27a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:16.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.540+0000 7f27c0c00700 1 -- 172.21.15.27:0/2937945400 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f27ac0d0c50 con 0x7f27c4106840 2024-04-26T22:58:16.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.540+0000 7f27c3400700 1 --2- 172.21.15.27:0/2937945400 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f27a80a3cd0 0x7f27a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:16.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.541+0000 7f27c3400700 1 --2- 172.21.15.27:0/2937945400 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f27a80a3cd0 0x7f27a80a6190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f27b4011490 tx=0x7f27b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:16.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.542+0000 7f27c0c00700 1 -- 172.21.15.27:0/2937945400 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f27ac09d040 con 0x7f27c4106840 2024-04-26T22:58:16.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.777+0000 7f27ca000700 1 -- 172.21.15.27:0/2937945400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f27c402e820 con 0x7f27c4106840 2024-04-26T22:58:16.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.778+0000 7f27c0c00700 1 -- 172.21.15.27:0/2937945400 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5754 (secure 0 0 0) 0x7f27ac09c940 con 0x7f27c4106840 2024-04-26T22:58:16.779 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 94 2024-04-26T22:58:16.779 INFO:teuthology.orchestra.run.smithi027.stdout:fsid 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T22:58:16.779 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-04-26T22:45:52.404180+0000 2024-04-26T22:58:16.780 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2024-04-26T22:57:05.902371+0000 2024-04-26T22:58:16.780 INFO:teuthology.orchestra.run.smithi027.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T22:58:16.780 INFO:teuthology.orchestra.run.smithi027.stdout:crush_version 27 2024-04-26T22:58:16.780 INFO:teuthology.orchestra.run.smithi027.stdout:full_ratio 0.95 2024-04-26T22:58:16.780 INFO:teuthology.orchestra.run.smithi027.stdout:backfillfull_ratio 0.9 2024-04-26T22:58:16.780 INFO:teuthology.orchestra.run.smithi027.stdout:nearfull_ratio 0.85 2024-04-26T22:58:16.780 INFO:teuthology.orchestra.run.smithi027.stdout:require_min_compat_client luminous 2024-04-26T22:58:16.780 INFO:teuthology.orchestra.run.smithi027.stdout:min_compat_client luminous 2024-04-26T22:58:16.780 INFO:teuthology.orchestra.run.smithi027.stdout:require_osd_release squid 2024-04-26T22:58:16.780 INFO:teuthology.orchestra.run.smithi027.stdout:stretch_mode_enabled false 2024-04-26T22:58:16.780 INFO:teuthology.orchestra.run.smithi027.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-26T22:58:16.780 INFO:teuthology.orchestra.run.smithi027.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-26T22:58:16.780 INFO:teuthology.orchestra.run.smithi027.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-26T22:58:16.781 INFO:teuthology.orchestra.run.smithi027.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-26T22:58:16.781 INFO:teuthology.orchestra.run.smithi027.stdout:max_osd 12 2024-04-26T22:58:16.781 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6802/4133324880,v1:172.21.15.27:6803/4133324880] [v2:172.21.15.27:6804/4133324880,v1:172.21.15.27:6805/4133324880] exists,up 2f6f4675-af17-4d78-87cd-9b1ccef44def 2024-04-26T22:58:16.781 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.80:6800/691528245,v1:172.21.15.80:6801/691528245] [v2:172.21.15.80:6802/691528245,v1:172.21.15.80:6803/691528245] exists,up e980f676-0a81-4253-8a39-db33f37ed79d 2024-04-26T22:58:16.781 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 up in weight 1 up_from 18 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6800/3315198187,v1:172.21.15.94:6801/3315198187] [v2:172.21.15.94:6802/3315198187,v1:172.21.15.94:6803/3315198187] exists,up 57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0 2024-04-26T22:58:16.781 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 up in weight 1 up_from 25 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6810/506143849,v1:172.21.15.27:6811/506143849] [v2:172.21.15.27:6812/506143849,v1:172.21.15.27:6813/506143849] exists,up 95f248ab-e3e8-425c-a0f8-c26cfc6f1be8 2024-04-26T22:58:16.781 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 up in weight 1 up_from 33 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.80:6808/1483108588,v1:172.21.15.80:6809/1483108588] [v2:172.21.15.80:6810/1483108588,v1:172.21.15.80:6811/1483108588] exists,up 1d4f0826-5221-41cc-a658-fc518acf4c98 2024-04-26T22:58:16.781 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 up in weight 1 up_from 38 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6808/396894302,v1:172.21.15.94:6809/396894302] [v2:172.21.15.94:6810/396894302,v1:172.21.15.94:6811/396894302] exists,up 7369cc9a-e487-46bb-8f93-42a025150669 2024-04-26T22:58:16.781 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 up in weight 1 up_from 46 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6818/1730660655,v1:172.21.15.27:6819/1730660655] [v2:172.21.15.27:6820/1730660655,v1:172.21.15.27:6821/1730660655] exists,up 675deb21-5cf8-457d-acdb-188d3372dd8a 2024-04-26T22:58:16.781 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 up in weight 1 up_from 51 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.80:6816/534257127,v1:172.21.15.80:6817/534257127] [v2:172.21.15.80:6818/534257127,v1:172.21.15.80:6819/534257127] exists,up 5e9aa483-e206-4ba2-b39b-9f7547da1459 2024-04-26T22:58:16.781 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 up in weight 1 up_from 59 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6816/484486649,v1:172.21.15.94:6817/484486649] [v2:172.21.15.94:6818/484486649,v1:172.21.15.94:6819/484486649] exists,up effb8164-0b8e-425f-af66-624c2909eb96 2024-04-26T22:58:16.781 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 up in weight 1 up_from 67 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6826/2196150976,v1:172.21.15.27:6827/2196150976] [v2:172.21.15.27:6828/2196150976,v1:172.21.15.27:6829/2196150976] exists,up 6c52e6ee-8407-498e-a0fe-579588406f2f 2024-04-26T22:58:16.781 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 up in weight 1 up_from 72 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.80:6824/3521249391,v1:172.21.15.80:6825/3521249391] [v2:172.21.15.80:6826/3521249391,v1:172.21.15.80:6827/3521249391] exists,up 33af8d56-6def-4c77-9fb2-8d2837c18c95 2024-04-26T22:58:16.781 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 up in weight 1 up_from 80 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6824/2763583925,v1:172.21.15.94:6825/2763583925] [v2:172.21.15.94:6826/2763583925,v1:172.21.15.94:6827/2763583925] exists,up ffee2993-b4e0-4309-9cd8-83ac32df51c8 2024-04-26T22:58:16.781 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.b [2,4] 2024-04-26T22:58:16.782 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.16 [2,4,7,11] 2024-04-26T22:58:16.782 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.1a [8,1] 2024-04-26T22:58:16.782 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.1d [8,11] 2024-04-26T22:58:16.782 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.0 [4,8] 2024-04-26T22:58:16.782 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.11 [4,6] 2024-04-26T22:58:16.782 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.16 [1,8] 2024-04-26T22:58:16.782 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.19 [3,7] 2024-04-26T22:58:16.782 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.1a [5,7] 2024-04-26T22:58:16.782 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.21 [4,6,5,9] 2024-04-26T22:58:16.782 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.27 [3,9] 2024-04-26T22:58:16.782 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.39 [5,8] 2024-04-26T22:58:16.782 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.3c [4,9] 2024-04-26T22:58:16.782 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.b [9,10] 2024-04-26T22:58:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.16 [9,8] 2024-04-26T22:58:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.1a [5,8] 2024-04-26T22:58:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.1b [9,8] 2024-04-26T22:58:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/1661629783 expires 2024-04-27T22:47:04.292223+0000 2024-04-26T22:58:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/3136245138 expires 2024-04-27T22:47:04.292223+0000 2024-04-26T22:58:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/1661629783 expires 2024-04-27T22:47:04.292223+0000 2024-04-26T22:58:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/1521074971 expires 2024-04-27T22:46:28.107236+0000 2024-04-26T22:58:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/1521074971 expires 2024-04-27T22:46:28.107236+0000 2024-04-26T22:58:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/542026788 expires 2024-04-27T22:46:28.107236+0000 2024-04-26T22:58:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/1524274936 expires 2024-04-27T22:46:28.107236+0000 2024-04-26T22:58:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/3814309846 expires 2024-04-27T22:47:04.292223+0000 2024-04-26T22:58:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/2767985348 expires 2024-04-27T22:46:28.107236+0000 2024-04-26T22:58:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/3451701535 expires 2024-04-27T22:47:04.292223+0000 2024-04-26T22:58:16.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.782+0000 7f27ca000700 1 -- 172.21.15.27:0/2937945400 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f27a80a3cd0 msgr2=0x7f27a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:16.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.782+0000 7f27ca000700 1 --2- 172.21.15.27:0/2937945400 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f27a80a3cd0 0x7f27a80a6190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f27b4011490 tx=0x7f27b4009260 comp rx=0 tx=0).stop 2024-04-26T22:58:16.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.782+0000 7f27ca000700 1 -- 172.21.15.27:0/2937945400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27c4106840 msgr2=0x7f27c4198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:16.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.782+0000 7f27ca000700 1 --2- 172.21.15.27:0/2937945400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27c4106840 0x7f27c4198490 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00e9a0 tx=0x7f27ac00ecb0 comp rx=0 tx=0).stop 2024-04-26T22:58:16.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.783+0000 7f27ca000700 1 -- 172.21.15.27:0/2937945400 shutdown_connections 2024-04-26T22:58:16.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.783+0000 7f27ca000700 1 --2- 172.21.15.27:0/2937945400 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f27a80a3cd0 0x7f27a80a6190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:16.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.783+0000 7f27ca000700 1 --2- 172.21.15.27:0/2937945400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27c4109170 0x7f27c4194ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:16.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.783+0000 7f27ca000700 1 --2- 172.21.15.27:0/2937945400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c4103f10 0x7f27c4197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:16.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.783+0000 7f27ca000700 1 --2- 172.21.15.27:0/2937945400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27c4106840 0x7f27c4198490 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:16.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.783+0000 7f27ca000700 1 -- 172.21.15.27:0/2937945400 >> 172.21.15.27:0/2937945400 conn(0x7f27c40fd920 msgr2=0x7f27c41036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:16.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.783+0000 7f27ca000700 1 -- 172.21.15.27:0/2937945400 shutdown_connections 2024-04-26T22:58:16.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:16.783+0000 7f27ca000700 1 -- 172.21.15.27:0/2937945400 wait complete. 2024-04-26T22:58:17.761 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-26T22:58:17.772 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:58:18.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.002+0000 7f3911a00640 1 -- 172.21.15.27:0/157326564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c10a940 msgr2=0x7f390c1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:18.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.002+0000 7f3911a00640 1 --2- 172.21.15.27:0/157326564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c10a940 0x7f390c1111d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3904009ca0 tx=0x7f390402a8b0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.002+0000 7f3911a00640 1 -- 172.21.15.27:0/157326564 shutdown_connections 2024-04-26T22:58:18.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.002+0000 7f3911a00640 1 --2- 172.21.15.27:0/157326564 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f390c109f80 0x7f390c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.002+0000 7f3911a00640 1 --2- 172.21.15.27:0/157326564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c10a940 0x7f390c1111d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.002+0000 7f3911a00640 1 --2- 172.21.15.27:0/157326564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f390c108d90 0x7f390c1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.002+0000 7f3911a00640 1 -- 172.21.15.27:0/157326564 >> 172.21.15.27:0/157326564 conn(0x7f390c077d30 msgr2=0x7f390c07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:18.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.002+0000 7f3911a00640 1 -- 172.21.15.27:0/157326564 shutdown_connections 2024-04-26T22:58:18.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.002+0000 7f3911a00640 1 -- 172.21.15.27:0/157326564 wait complete. 2024-04-26T22:58:18.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.004+0000 7f3911a00640 1 Processor -- start 2024-04-26T22:58:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.004+0000 7f3911a00640 1 -- start start 2024-04-26T22:58:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.004+0000 7f3911a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f390c108d90 0x7f390c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.004+0000 7f3911a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c109f80 0x7f390c19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.004+0000 7f3911a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f390c10a940 0x7f390c1a6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.004+0000 7f3911a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f390c1148c0 con 0x7f390c109f80 2024-04-26T22:58:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.004+0000 7f3911a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f390c114740 con 0x7f390c108d90 2024-04-26T22:58:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.004+0000 7f3911a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f390c114a40 con 0x7f390c10a940 2024-04-26T22:58:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.005+0000 7f3903400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c109f80 0x7f390c19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.005+0000 7f3903400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c109f80 0x7f390c19f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48890/0 (socket says 172.21.15.27:48890) 2024-04-26T22:58:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.005+0000 7f3903400640 1 -- 172.21.15.27:0/891393657 learned_addr learned my addr 172.21.15.27:0/891393657 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.005+0000 7f390be00640 1 --2- 172.21.15.27:0/891393657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f390c108d90 0x7f390c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.005+0000 7f3911000640 1 --2- 172.21.15.27:0/891393657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f390c10a940 0x7f390c1a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.005+0000 7f3903400640 1 -- 172.21.15.27:0/891393657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f390c10a940 msgr2=0x7f390c1a6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.005+0000 7f3903400640 1 --2- 172.21.15.27:0/891393657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f390c10a940 0x7f390c1a6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.005+0000 7f3903400640 1 -- 172.21.15.27:0/891393657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f390c108d90 msgr2=0x7f390c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.005+0000 7f3903400640 1 --2- 172.21.15.27:0/891393657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f390c108d90 0x7f390c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.005+0000 7f3903400640 1 -- 172.21.15.27:0/891393657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c1a6ad0 con 0x7f390c109f80 2024-04-26T22:58:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.005+0000 7f390be00640 1 --2- 172.21.15.27:0/891393657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f390c108d90 0x7f390c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.006+0000 7f3903400640 1 --2- 172.21.15.27:0/891393657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c109f80 0x7f390c19f3f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f38f400ea60 tx=0x7f38f400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:18.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.006+0000 7f3909600640 1 -- 172.21.15.27:0/891393657 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f400cde0 con 0x7f390c109f80 2024-04-26T22:58:18.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.006+0000 7f3909600640 1 -- 172.21.15.27:0/891393657 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38f40035f0 con 0x7f390c109f80 2024-04-26T22:58:18.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.006+0000 7f3909600640 1 -- 172.21.15.27:0/891393657 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f4010690 con 0x7f390c109f80 2024-04-26T22:58:18.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.006+0000 7f3911a00640 1 -- 172.21.15.27:0/891393657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390c1a6dc0 con 0x7f390c109f80 2024-04-26T22:58:18.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.006+0000 7f3911a00640 1 -- 172.21.15.27:0/891393657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390c1a7290 con 0x7f390c109f80 2024-04-26T22:58:18.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.007+0000 7f3911a00640 1 -- 172.21.15.27:0/891393657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390c110050 con 0x7f390c109f80 2024-04-26T22:58:18.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.009+0000 7f3909600640 1 -- 172.21.15.27:0/891393657 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f38f40037a0 con 0x7f390c109f80 2024-04-26T22:58:18.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.011+0000 7f3909600640 1 --2- 172.21.15.27:0/891393657 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f38f00a3d70 0x7f38f00a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:18.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.011+0000 7f3909600640 1 -- 172.21.15.27:0/891393657 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f38f40d0c20 con 0x7f390c109f80 2024-04-26T22:58:18.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.011+0000 7f390be00640 1 --2- 172.21.15.27:0/891393657 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f38f00a3d70 0x7f38f00a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.012+0000 7f390be00640 1 --2- 172.21.15.27:0/891393657 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f38f00a3d70 0x7f38f00a6230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f390c109de0 tx=0x7f38fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.015+0000 7f3909600640 1 -- 172.21.15.27:0/891393657 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f38f409d050 con 0x7f390c109f80 2024-04-26T22:58:18.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:17 smithi080 ceph-mon[32324]: pgmap v420: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:58:18.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:17 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2937945400' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T22:58:18.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:17 smithi094 ceph-mon[32889]: pgmap v420: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:58:18.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:17 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2937945400' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T22:58:18.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:17 smithi027 ceph-mon[28034]: pgmap v420: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T22:58:18.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:17 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2937945400' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T22:58:18.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.229+0000 7f3911a00640 1 -- 172.21.15.27:0/891393657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f390c02dcd0 con 0x7f390c109f80 2024-04-26T22:58:18.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.232+0000 7f3909600640 1 -- 172.21.15.27:0/891393657 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24902 (secure 0 0 0) 0x7f38f409c950 con 0x7f390c109f80 2024-04-26T22:58:18.233 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:58:18.234 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":94,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:57:05.902371+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4},{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:58:18.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.235+0000 7f3911a00640 1 -- 172.21.15.27:0/891393657 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f38f00a3d70 msgr2=0x7f38f00a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:18.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.235+0000 7f3911a00640 1 --2- 172.21.15.27:0/891393657 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f38f00a3d70 0x7f38f00a6230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f390c109de0 tx=0x7f38fc005f90 comp rx=0 tx=0).stop 2024-04-26T22:58:18.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.235+0000 7f3911a00640 1 -- 172.21.15.27:0/891393657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c109f80 msgr2=0x7f390c19f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:18.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.235+0000 7f3911a00640 1 --2- 172.21.15.27:0/891393657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c109f80 0x7f390c19f3f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f38f400ea60 tx=0x7f38f400ef20 comp rx=0 tx=0).stop 2024-04-26T22:58:18.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.236+0000 7f3911a00640 1 -- 172.21.15.27:0/891393657 shutdown_connections 2024-04-26T22:58:18.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.236+0000 7f3911a00640 1 --2- 172.21.15.27:0/891393657 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f38f00a3d70 0x7f38f00a6230 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.236+0000 7f3911a00640 1 --2- 172.21.15.27:0/891393657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f390c10a940 0x7f390c1a6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.236+0000 7f3911a00640 1 --2- 172.21.15.27:0/891393657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c109f80 0x7f390c19f3f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.236+0000 7f3911a00640 1 --2- 172.21.15.27:0/891393657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f390c108d90 0x7f390c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.236+0000 7f3911a00640 1 -- 172.21.15.27:0/891393657 >> 172.21.15.27:0/891393657 conn(0x7f390c077d30 msgr2=0x7f390c10d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:18.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.236+0000 7f3911a00640 1 -- 172.21.15.27:0/891393657 shutdown_connections 2024-04-26T22:58:18.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.236+0000 7f3911a00640 1 -- 172.21.15.27:0/891393657 wait complete. 2024-04-26T22:58:18.258 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:49:54.482679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:55:22.899174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:55:24.218295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T22:55:28.394547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T22:58:18.258 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:58:18.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.474+0000 7fb315200640 1 -- 172.21.15.27:0/941212265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb31010a960 msgr2=0x7fb3101111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:18.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.474+0000 7fb315200640 1 --2- 172.21.15.27:0/941212265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb31010a960 0x7fb3101111f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb308009be0 tx=0x7fb30802a6c0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.474+0000 7fb315200640 1 -- 172.21.15.27:0/941212265 shutdown_connections 2024-04-26T22:58:18.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.474+0000 7fb315200640 1 --2- 172.21.15.27:0/941212265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb31010a960 0x7fb3101111f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.474+0000 7fb315200640 1 --2- 172.21.15.27:0/941212265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb310109fa0 0x7fb31010a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.474+0000 7fb315200640 1 --2- 172.21.15.27:0/941212265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb310108db0 0x7fb3101091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.474+0000 7fb315200640 1 -- 172.21.15.27:0/941212265 >> 172.21.15.27:0/941212265 conn(0x7fb310077d70 msgr2=0x7fb31007a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:18.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.475+0000 7fb315200640 1 -- 172.21.15.27:0/941212265 shutdown_connections 2024-04-26T22:58:18.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.475+0000 7fb315200640 1 -- 172.21.15.27:0/941212265 wait complete. 2024-04-26T22:58:18.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.476+0000 7fb315200640 1 Processor -- start 2024-04-26T22:58:18.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.476+0000 7fb315200640 1 -- start start 2024-04-26T22:58:18.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.477+0000 7fb315200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb310108db0 0x7fb31019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:18.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.477+0000 7fb315200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb310109fa0 0x7fb31019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:18.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.477+0000 7fb315200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb31010a960 0x7fb3101a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:18.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.477+0000 7fb315200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb3101148e0 con 0x7fb310108db0 2024-04-26T22:58:18.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.477+0000 7fb315200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb310114760 con 0x7fb310109fa0 2024-04-26T22:58:18.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.477+0000 7fb315200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb310114a60 con 0x7fb31010a960 2024-04-26T22:58:18.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.477+0000 7fb30f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb310108db0 0x7fb31019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.477+0000 7fb30f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb310108db0 0x7fb31019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48920/0 (socket says 172.21.15.27:48920) 2024-04-26T22:58:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.477+0000 7fb30f400640 1 -- 172.21.15.27:0/2085669167 learned_addr learned my addr 172.21.15.27:0/2085669167 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.477+0000 7fb30ea00640 1 --2- 172.21.15.27:0/2085669167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb310109fa0 0x7fb31019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.477+0000 7fb30fe00640 1 --2- 172.21.15.27:0/2085669167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb31010a960 0x7fb3101a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.478+0000 7fb30f400640 1 -- 172.21.15.27:0/2085669167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb31010a960 msgr2=0x7fb3101a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.478+0000 7fb30f400640 1 --2- 172.21.15.27:0/2085669167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb31010a960 0x7fb3101a6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.478+0000 7fb30f400640 1 -- 172.21.15.27:0/2085669167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb310109fa0 msgr2=0x7fb31019f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.478+0000 7fb30f400640 1 --2- 172.21.15.27:0/2085669167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb310109fa0 0x7fb31019f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.478+0000 7fb30f400640 1 -- 172.21.15.27:0/2085669167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3101a6b70 con 0x7fb310108db0 2024-04-26T22:58:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.478+0000 7fb30fe00640 1 --2- 172.21.15.27:0/2085669167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb31010a960 0x7fb3101a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.478+0000 7fb30f400640 1 --2- 172.21.15.27:0/2085669167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb310108db0 0x7fb31019ef50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb30000e9c0 tx=0x7fb30000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:18.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.478+0000 7fb307e00640 1 -- 172.21.15.27:0/2085669167 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30000cd80 con 0x7fb310108db0 2024-04-26T22:58:18.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.478+0000 7fb307e00640 1 -- 172.21.15.27:0/2085669167 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3000035f0 con 0x7fb310108db0 2024-04-26T22:58:18.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.478+0000 7fb315200640 1 -- 172.21.15.27:0/2085669167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3101a6e60 con 0x7fb310108db0 2024-04-26T22:58:18.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.478+0000 7fb307e00640 1 -- 172.21.15.27:0/2085669167 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300010690 con 0x7fb310108db0 2024-04-26T22:58:18.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.479+0000 7fb315200640 1 -- 172.21.15.27:0/2085669167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3101a72c0 con 0x7fb310108db0 2024-04-26T22:58:18.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.480+0000 7fb315200640 1 -- 172.21.15.27:0/2085669167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3101100d0 con 0x7fb310108db0 2024-04-26T22:58:18.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.482+0000 7fb307e00640 1 -- 172.21.15.27:0/2085669167 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb300004000 con 0x7fb310108db0 2024-04-26T22:58:18.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.483+0000 7fb307e00640 1 --2- 172.21.15.27:0/2085669167 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb2f40a3cc0 0x7fb2f40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:18.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.484+0000 7fb307e00640 1 -- 172.21.15.27:0/2085669167 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fb3000d0e80 con 0x7fb310108db0 2024-04-26T22:58:18.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.484+0000 7fb30ea00640 1 --2- 172.21.15.27:0/2085669167 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb2f40a3cc0 0x7fb2f40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:18.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.485+0000 7fb30ea00640 1 --2- 172.21.15.27:0/2085669167 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb2f40a3cc0 0x7fb2f40a6180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb3101a0460 tx=0x7fb2fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:18.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.488+0000 7fb307e00640 1 -- 172.21.15.27:0/2085669167 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb30009d270 con 0x7fb310108db0 2024-04-26T22:58:18.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.697+0000 7fb315200640 1 -- 172.21.15.27:0/2085669167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb31002dd30 con 0x7fb310108db0 2024-04-26T22:58:18.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.698+0000 7fb307e00640 1 -- 172.21.15.27:0/2085669167 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fb30009cb70 con 0x7fb310108db0 2024-04-26T22:58:18.699 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T22:58:18.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.701+0000 7fb315200640 1 -- 172.21.15.27:0/2085669167 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb2f40a3cc0 msgr2=0x7fb2f40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:18.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.701+0000 7fb315200640 1 --2- 172.21.15.27:0/2085669167 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb2f40a3cc0 0x7fb2f40a6180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb3101a0460 tx=0x7fb2fc009260 comp rx=0 tx=0).stop 2024-04-26T22:58:18.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.701+0000 7fb315200640 1 -- 172.21.15.27:0/2085669167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb310108db0 msgr2=0x7fb31019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:18.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.701+0000 7fb315200640 1 --2- 172.21.15.27:0/2085669167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb310108db0 0x7fb31019ef50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb30000e9c0 tx=0x7fb30000ee80 comp rx=0 tx=0).stop 2024-04-26T22:58:18.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.701+0000 7fb315200640 1 -- 172.21.15.27:0/2085669167 shutdown_connections 2024-04-26T22:58:18.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.701+0000 7fb315200640 1 --2- 172.21.15.27:0/2085669167 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fb2f40a3cc0 0x7fb2f40a6180 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.701+0000 7fb315200640 1 --2- 172.21.15.27:0/2085669167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb310109fa0 0x7fb31019f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.701+0000 7fb315200640 1 --2- 172.21.15.27:0/2085669167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb31010a960 0x7fb3101a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.701+0000 7fb315200640 1 --2- 172.21.15.27:0/2085669167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb310108db0 0x7fb31019ef50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.701+0000 7fb315200640 1 -- 172.21.15.27:0/2085669167 >> 172.21.15.27:0/2085669167 conn(0x7fb310077d70 msgr2=0x7fb31010d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:18.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.701+0000 7fb315200640 1 -- 172.21.15.27:0/2085669167 shutdown_connections 2024-04-26T22:58:18.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.701+0000 7fb315200640 1 -- 172.21.15.27:0/2085669167 wait complete. 2024-04-26T22:58:18.722 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:58:18.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.943+0000 7f3f12600640 1 -- 172.21.15.27:0/929789465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f0c10a960 msgr2=0x7f3f0c1111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:18.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.943+0000 7f3f12600640 1 --2- 172.21.15.27:0/929789465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f0c10a960 0x7f3f0c1111f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009ca0 tx=0x7f3f0402a8c0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.944+0000 7f3f12600640 1 -- 172.21.15.27:0/929789465 shutdown_connections 2024-04-26T22:58:18.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.944+0000 7f3f12600640 1 --2- 172.21.15.27:0/929789465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f0c10a960 0x7f3f0c1111f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.944+0000 7f3f12600640 1 --2- 172.21.15.27:0/929789465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f0c109fa0 0x7f3f0c10a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.944+0000 7f3f12600640 1 --2- 172.21.15.27:0/929789465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f0c108db0 0x7f3f0c1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.944+0000 7f3f12600640 1 -- 172.21.15.27:0/929789465 >> 172.21.15.27:0/929789465 conn(0x7f3f0c077d70 msgr2=0x7f3f0c07a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:18.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.944+0000 7f3f12600640 1 -- 172.21.15.27:0/929789465 shutdown_connections 2024-04-26T22:58:18.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.944+0000 7f3f12600640 1 -- 172.21.15.27:0/929789465 wait complete. 2024-04-26T22:58:18.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.945+0000 7f3f12600640 1 Processor -- start 2024-04-26T22:58:18.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.945+0000 7f3f12600640 1 -- start start 2024-04-26T22:58:18.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.945+0000 7f3f12600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f0c108db0 0x7f3f0c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:18.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.945+0000 7f3f12600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f0c109fa0 0x7f3f0c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:18.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.945+0000 7f3f12600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f0c10a960 0x7f3f0c1a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:18.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.945+0000 7f3f12600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0c114910 con 0x7f3f0c108db0 2024-04-26T22:58:18.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.945+0000 7f3f12600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0c114790 con 0x7f3f0c109fa0 2024-04-26T22:58:18.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.945+0000 7f3f12600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0c114a90 con 0x7f3f0c10a960 2024-04-26T22:58:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.946+0000 7f3f11200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f0c108db0 0x7f3f0c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.946+0000 7f3f11200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f0c108db0 0x7f3f0c19efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48940/0 (socket says 172.21.15.27:48940) 2024-04-26T22:58:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.946+0000 7f3f11200640 1 -- 172.21.15.27:0/4167820677 learned_addr learned my addr 172.21.15.27:0/4167820677 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.946+0000 7f3f11c00640 1 --2- 172.21.15.27:0/4167820677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f0c10a960 0x7f3f0c1a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.946+0000 7f3f0be00640 1 --2- 172.21.15.27:0/4167820677 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f0c109fa0 0x7f3f0c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.946+0000 7f3f11200640 1 -- 172.21.15.27:0/4167820677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f0c10a960 msgr2=0x7f3f0c1a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.946+0000 7f3f11200640 1 --2- 172.21.15.27:0/4167820677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f0c10a960 0x7f3f0c1a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.946+0000 7f3f11200640 1 -- 172.21.15.27:0/4167820677 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f0c109fa0 msgr2=0x7f3f0c19f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.946+0000 7f3f11200640 1 --2- 172.21.15.27:0/4167820677 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f0c109fa0 0x7f3f0c19f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.946+0000 7f3f11200640 1 -- 172.21.15.27:0/4167820677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0c1a6bb0 con 0x7f3f0c108db0 2024-04-26T22:58:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.946+0000 7f3f0be00640 1 --2- 172.21.15.27:0/4167820677 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f0c109fa0 0x7f3f0c19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.947+0000 7f3f11200640 1 --2- 172.21.15.27:0/4167820677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f0c108db0 0x7f3f0c19efe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000e9c0 tx=0x7f3f0000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.947+0000 7f3f09600640 1 -- 172.21.15.27:0/4167820677 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0000cd80 con 0x7f3f0c108db0 2024-04-26T22:58:18.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.947+0000 7f3f09600640 1 -- 172.21.15.27:0/4167820677 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f000035f0 con 0x7f3f0c108db0 2024-04-26T22:58:18.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.947+0000 7f3f12600640 1 -- 172.21.15.27:0/4167820677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f0c1a6ea0 con 0x7f3f0c108db0 2024-04-26T22:58:18.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.947+0000 7f3f09600640 1 -- 172.21.15.27:0/4167820677 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00010690 con 0x7f3f0c108db0 2024-04-26T22:58:18.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.947+0000 7f3f12600640 1 -- 172.21.15.27:0/4167820677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f0c1a7360 con 0x7f3f0c108db0 2024-04-26T22:58:18.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.948+0000 7f3f12600640 1 -- 172.21.15.27:0/4167820677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f0c1100d0 con 0x7f3f0c108db0 2024-04-26T22:58:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.951+0000 7f3f09600640 1 -- 172.21.15.27:0/4167820677 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3f000037a0 con 0x7f3f0c108db0 2024-04-26T22:58:18.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.952+0000 7f3f09600640 1 --2- 172.21.15.27:0/4167820677 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3ef40a3cc0 0x7f3ef40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:18.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.952+0000 7f3f09600640 1 -- 172.21.15.27:0/4167820677 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f3f000d0ca0 con 0x7f3f0c108db0 2024-04-26T22:58:18.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.952+0000 7f3f0be00640 1 --2- 172.21.15.27:0/4167820677 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3ef40a3cc0 0x7f3ef40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:18.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.953+0000 7f3f0be00640 1 --2- 172.21.15.27:0/4167820677 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3ef40a3cc0 0x7f3ef40a6180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c1a04a0 tx=0x7f3efc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:18.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:18.956+0000 7f3f09600640 1 -- 172.21.15.27:0/4167820677 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3f0009d0f0 con 0x7f3f0c108db0 2024-04-26T22:58:19.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:18 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/891393657' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:58:19.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:18 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/891393657' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:58:19.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:18 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/891393657' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:58:19.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.165+0000 7f3f12600640 1 -- 172.21.15.27:0/4167820677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3f0c02dce0 con 0x7f3f0c108db0 2024-04-26T22:58:19.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.166+0000 7f3f09600640 1 -- 172.21.15.27:0/4167820677 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f3f0009c9f0 con 0x7f3f0c108db0 2024-04-26T22:58:19.167 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T22:58:19.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.169+0000 7f3f12600640 1 -- 172.21.15.27:0/4167820677 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3ef40a3cc0 msgr2=0x7f3ef40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:19.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.169+0000 7f3f12600640 1 --2- 172.21.15.27:0/4167820677 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3ef40a3cc0 0x7f3ef40a6180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c1a04a0 tx=0x7f3efc005f90 comp rx=0 tx=0).stop 2024-04-26T22:58:19.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.169+0000 7f3f12600640 1 -- 172.21.15.27:0/4167820677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f0c108db0 msgr2=0x7f3f0c19efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:19.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.169+0000 7f3f12600640 1 --2- 172.21.15.27:0/4167820677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f0c108db0 0x7f3f0c19efe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000e9c0 tx=0x7f3f0000ee80 comp rx=0 tx=0).stop 2024-04-26T22:58:19.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.169+0000 7f3f12600640 1 -- 172.21.15.27:0/4167820677 shutdown_connections 2024-04-26T22:58:19.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.169+0000 7f3f12600640 1 --2- 172.21.15.27:0/4167820677 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3ef40a3cc0 0x7f3ef40a6180 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.169+0000 7f3f12600640 1 --2- 172.21.15.27:0/4167820677 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f0c109fa0 0x7f3f0c19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.169+0000 7f3f12600640 1 --2- 172.21.15.27:0/4167820677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f0c10a960 0x7f3f0c1a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.169+0000 7f3f12600640 1 --2- 172.21.15.27:0/4167820677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f0c108db0 0x7f3f0c19efe0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.169+0000 7f3f12600640 1 -- 172.21.15.27:0/4167820677 >> 172.21.15.27:0/4167820677 conn(0x7f3f0c077d70 msgr2=0x7f3f0c10d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:19.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.169+0000 7f3f12600640 1 -- 172.21.15.27:0/4167820677 shutdown_connections 2024-04-26T22:58:19.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.169+0000 7f3f12600640 1 -- 172.21.15.27:0/4167820677 wait complete. 2024-04-26T22:58:19.191 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:58:19.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.407+0000 7f124d000640 1 -- 172.21.15.27:0/2331832168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1248104490 msgr2=0x7f124810ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:19.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.407+0000 7f124d000640 1 --2- 172.21.15.27:0/2331832168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1248104490 0x7f124810ad20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1240009930 tx=0x7f124002a440 comp rx=0 tx=0).stop 2024-04-26T22:58:19.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.407+0000 7f124d000640 1 -- 172.21.15.27:0/2331832168 shutdown_connections 2024-04-26T22:58:19.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.407+0000 7f124d000640 1 --2- 172.21.15.27:0/2331832168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1248104490 0x7f124810ad20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.407+0000 7f124d000640 1 --2- 172.21.15.27:0/2331832168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1248103ad0 0x7f1248103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.407+0000 7f124d000640 1 --2- 172.21.15.27:0/2331832168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1248100a60 0x7f1248100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.407+0000 7f124d000640 1 -- 172.21.15.27:0/2331832168 >> 172.21.15.27:0/2331832168 conn(0x7f12480fc370 msgr2=0x7f12480fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:19.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.407+0000 7f124d000640 1 -- 172.21.15.27:0/2331832168 shutdown_connections 2024-04-26T22:58:19.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.408+0000 7f124d000640 1 -- 172.21.15.27:0/2331832168 wait complete. 2024-04-26T22:58:19.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.408+0000 7f124d000640 1 Processor -- start 2024-04-26T22:58:19.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.409+0000 7f124d000640 1 -- start start 2024-04-26T22:58:19.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.409+0000 7f124d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1248100a60 0x7f1248198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:19.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.409+0000 7f124d000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1248103ad0 0x7f1248198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:19.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.409+0000 7f124d000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1248104490 0x7f124819ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:19.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.409+0000 7f124d000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f124810e2b0 con 0x7f1248100a60 2024-04-26T22:58:19.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.409+0000 7f124d000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f124810e130 con 0x7f1248103ad0 2024-04-26T22:58:19.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.409+0000 7f124d000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f124810e430 con 0x7f1248104490 2024-04-26T22:58:19.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.409+0000 7f1247400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1248100a60 0x7f1248198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:19.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.409+0000 7f1247400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1248100a60 0x7f1248198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48948/0 (socket says 172.21.15.27:48948) 2024-04-26T22:58:19.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.409+0000 7f1247400640 1 -- 172.21.15.27:0/204380763 learned_addr learned my addr 172.21.15.27:0/204380763 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:19.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.410+0000 7f1246a00640 1 --2- 172.21.15.27:0/204380763 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1248103ad0 0x7f1248198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:19.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.410+0000 7f1247e00640 1 --2- 172.21.15.27:0/204380763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1248104490 0x7f124819ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:19.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.410+0000 7f1247400640 1 -- 172.21.15.27:0/204380763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1248104490 msgr2=0x7f124819ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:19.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.410+0000 7f1247400640 1 --2- 172.21.15.27:0/204380763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1248104490 0x7f124819ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.410+0000 7f1247400640 1 -- 172.21.15.27:0/204380763 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1248103ad0 msgr2=0x7f1248198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:19.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.410+0000 7f1247400640 1 --2- 172.21.15.27:0/204380763 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1248103ad0 0x7f1248198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.410+0000 7f1247400640 1 -- 172.21.15.27:0/204380763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12481a0580 con 0x7f1248100a60 2024-04-26T22:58:19.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.410+0000 7f1247400640 1 --2- 172.21.15.27:0/204380763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1248100a60 0x7f1248198960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f123800eb70 tx=0x7f123800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:19.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.411+0000 7f123fe00640 1 -- 172.21.15.27:0/204380763 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238003b70 con 0x7f1248100a60 2024-04-26T22:58:19.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.411+0000 7f124d000640 1 -- 172.21.15.27:0/204380763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12481a0870 con 0x7f1248100a60 2024-04-26T22:58:19.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.411+0000 7f123fe00640 1 -- 172.21.15.27:0/204380763 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f123800ee70 con 0x7f1248100a60 2024-04-26T22:58:19.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.411+0000 7f123fe00640 1 -- 172.21.15.27:0/204380763 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238010650 con 0x7f1248100a60 2024-04-26T22:58:19.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.411+0000 7f124d000640 1 -- 172.21.15.27:0/204380763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12481a0dc0 con 0x7f1248100a60 2024-04-26T22:58:19.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.412+0000 7f124d000640 1 -- 172.21.15.27:0/204380763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1248109d20 con 0x7f1248100a60 2024-04-26T22:58:19.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.414+0000 7f123fe00640 1 -- 172.21.15.27:0/204380763 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1238004020 con 0x7f1248100a60 2024-04-26T22:58:19.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.416+0000 7f123fe00640 1 --2- 172.21.15.27:0/204380763 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f122c0a3c30 0x7f122c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:19.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.417+0000 7f123fe00640 1 -- 172.21.15.27:0/204380763 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f12380d1080 con 0x7f1248100a60 2024-04-26T22:58:19.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.417+0000 7f1246a00640 1 --2- 172.21.15.27:0/204380763 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f122c0a3c30 0x7f122c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:19.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.417+0000 7f1246a00640 1 --2- 172.21.15.27:0/204380763 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f122c0a3c30 0x7f122c0a60f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1248199e70 tx=0x7f1234005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:19.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.421+0000 7f123fe00640 1 -- 172.21.15.27:0/204380763 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f123809d470 con 0x7f1248100a60 2024-04-26T22:58:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.629+0000 7f124d000640 1 -- 172.21.15.27:0/204380763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f124802dad0 con 0x7f1248100a60 2024-04-26T22:58:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.630+0000 7f123fe00640 1 -- 172.21.15.27:0/204380763 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f123809cd70 con 0x7f1248100a60 2024-04-26T22:58:19.631 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-26T22:58:19.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.633+0000 7f124d000640 1 -- 172.21.15.27:0/204380763 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f122c0a3c30 msgr2=0x7f122c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:19.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.633+0000 7f124d000640 1 --2- 172.21.15.27:0/204380763 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f122c0a3c30 0x7f122c0a60f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1248199e70 tx=0x7f1234005f90 comp rx=0 tx=0).stop 2024-04-26T22:58:19.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.633+0000 7f124d000640 1 -- 172.21.15.27:0/204380763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1248100a60 msgr2=0x7f1248198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:19.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.633+0000 7f124d000640 1 --2- 172.21.15.27:0/204380763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1248100a60 0x7f1248198960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f123800eb70 tx=0x7f123800c6a0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.634+0000 7f124d000640 1 -- 172.21.15.27:0/204380763 shutdown_connections 2024-04-26T22:58:19.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.634+0000 7f124d000640 1 --2- 172.21.15.27:0/204380763 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f122c0a3c30 0x7f122c0a60f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.634+0000 7f124d000640 1 --2- 172.21.15.27:0/204380763 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1248103ad0 0x7f1248198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.634+0000 7f124d000640 1 --2- 172.21.15.27:0/204380763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1248104490 0x7f124819ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.634+0000 7f124d000640 1 --2- 172.21.15.27:0/204380763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1248100a60 0x7f1248198960 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.634+0000 7f124d000640 1 -- 172.21.15.27:0/204380763 >> 172.21.15.27:0/204380763 conn(0x7f12480fc370 msgr2=0x7f1248106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:19.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.634+0000 7f124d000640 1 -- 172.21.15.27:0/204380763 shutdown_connections 2024-04-26T22:58:19.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.634+0000 7f124d000640 1 -- 172.21.15.27:0/204380763 wait complete. 2024-04-26T22:58:19.654 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T22:58:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.870+0000 7f61e4600640 1 -- 172.21.15.27:0/3374164395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc103b70 msgr2=0x7f61dc103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.870+0000 7f61e4600640 1 --2- 172.21.15.27:0/3374164395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc103b70 0x7f61dc103ff0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009900 tx=0x7f61c802a3f0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.870+0000 7f61e4600640 1 -- 172.21.15.27:0/3374164395 shutdown_connections 2024-04-26T22:58:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.870+0000 7f61e4600640 1 --2- 172.21.15.27:0/3374164395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc104530 0x7f61dc10adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.870+0000 7f61e4600640 1 --2- 172.21.15.27:0/3374164395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc103b70 0x7f61dc103ff0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.870+0000 7f61e4600640 1 --2- 172.21.15.27:0/3374164395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61dc102980 0x7f61dc102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.870+0000 7f61e4600640 1 -- 172.21.15.27:0/3374164395 >> 172.21.15.27:0/3374164395 conn(0x7f61dc0fdef0 msgr2=0x7f61dc100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.870+0000 7f61e4600640 1 -- 172.21.15.27:0/3374164395 shutdown_connections 2024-04-26T22:58:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.870+0000 7f61e4600640 1 -- 172.21.15.27:0/3374164395 wait complete. 2024-04-26T22:58:19.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.871+0000 7f61e4600640 1 Processor -- start 2024-04-26T22:58:19.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.871+0000 7f61e4600640 1 -- start start 2024-04-26T22:58:19.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.871+0000 7f61e4600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc102980 0x7f61dc19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:19.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.871+0000 7f61e4600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc103b70 0x7f61dc19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.871+0000 7f61e4600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61dc104530 0x7f61dc1a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.871+0000 7f61e4600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc10e4e0 con 0x7f61dc103b70 2024-04-26T22:58:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.871+0000 7f61e4600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc10e360 con 0x7f61dc104530 2024-04-26T22:58:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.871+0000 7f61e4600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc10e660 con 0x7f61dc102980 2024-04-26T22:58:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.872+0000 7f61e2600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61dc104530 0x7f61dc1a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.872+0000 7f61e1c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc102980 0x7f61dc19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.872+0000 7f61e2600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61dc104530 0x7f61dc1a23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33140/0 (socket says 172.21.15.27:33140) 2024-04-26T22:58:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.872+0000 7f61e2600640 1 -- 172.21.15.27:0/1049914489 learned_addr learned my addr 172.21.15.27:0/1049914489 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.872+0000 7f61e1200640 1 --2- 172.21.15.27:0/1049914489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc103b70 0x7f61dc19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.872+0000 7f61e2600640 1 -- 172.21.15.27:0/1049914489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc102980 msgr2=0x7f61dc19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.872+0000 7f61e2600640 1 --2- 172.21.15.27:0/1049914489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc102980 0x7f61dc19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.872+0000 7f61e2600640 1 -- 172.21.15.27:0/1049914489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc103b70 msgr2=0x7f61dc19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.872+0000 7f61e2600640 1 --2- 172.21.15.27:0/1049914489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc103b70 0x7f61dc19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.872+0000 7f61e2600640 1 -- 172.21.15.27:0/1049914489 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc1a2a10 con 0x7f61dc104530 2024-04-26T22:58:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.873+0000 7f61e2600640 1 --2- 172.21.15.27:0/1049914489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61dc104530 0x7f61dc1a23a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f61d400eb10 tx=0x7f61d400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.873+0000 7f61da000640 1 -- 172.21.15.27:0/1049914489 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4004b20 con 0x7f61dc104530 2024-04-26T22:58:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.873+0000 7f61da000640 1 -- 172.21.15.27:0/1049914489 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d4004cc0 con 0x7f61dc104530 2024-04-26T22:58:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.873+0000 7f61da000640 1 -- 172.21.15.27:0/1049914489 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4010670 con 0x7f61dc104530 2024-04-26T22:58:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.873+0000 7f61e4600640 1 -- 172.21.15.27:0/1049914489 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc1a2d00 con 0x7f61dc104530 2024-04-26T22:58:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.873+0000 7f61e4600640 1 -- 172.21.15.27:0/1049914489 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc1a31d0 con 0x7f61dc104530 2024-04-26T22:58:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.875+0000 7f61e4600640 1 -- 172.21.15.27:0/1049914489 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61dc1a3400 con 0x7f61dc104530 2024-04-26T22:58:19.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.877+0000 7f61da000640 1 -- 172.21.15.27:0/1049914489 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f61d4005020 con 0x7f61dc104530 2024-04-26T22:58:19.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.878+0000 7f61da000640 1 --2- 172.21.15.27:0/1049914489 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f61c40a3cc0 0x7f61c40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:19.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.878+0000 7f61da000640 1 -- 172.21.15.27:0/1049914489 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f61d40d0fb0 con 0x7f61dc104530 2024-04-26T22:58:19.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.879+0000 7f61e1c00640 1 --2- 172.21.15.27:0/1049914489 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f61c40a3cc0 0x7f61c40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:19.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.879+0000 7f61e1c00640 1 --2- 172.21.15.27:0/1049914489 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f61c40a3cc0 0x7f61c40a6180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f61dc1039d0 tx=0x7f61d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:19.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:19.883+0000 7f61da000640 1 -- 172.21.15.27:0/1049914489 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f61d409d3a0 con 0x7f61dc104530 2024-04-26T22:58:20.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:19 smithi080 ceph-mon[32324]: pgmap v421: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:20.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:19 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2085669167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:58:20.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:19 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/4167820677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:58:20.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:19 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/204380763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:58:20.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:19 smithi094 ceph-mon[32889]: pgmap v421: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:20.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:19 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2085669167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:58:20.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:19 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/4167820677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:58:20.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:19 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/204380763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:58:20.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.088+0000 7f61e4600640 1 -- 172.21.15.27:0/1049914489 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f61dc1a35d0 con 0x7f61dc104530 2024-04-26T22:58:20.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.089+0000 7f61da000640 1 -- 172.21.15.27:0/1049914489 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f61d409cca0 con 0x7f61dc104530 2024-04-26T22:58:20.089 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T22:58:20.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.092+0000 7f61e4600640 1 -- 172.21.15.27:0/1049914489 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f61c40a3cc0 msgr2=0x7f61c40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:20.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.092+0000 7f61e4600640 1 --2- 172.21.15.27:0/1049914489 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f61c40a3cc0 0x7f61c40a6180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f61dc1039d0 tx=0x7f61d0005f90 comp rx=0 tx=0).stop 2024-04-26T22:58:20.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.092+0000 7f61e4600640 1 -- 172.21.15.27:0/1049914489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61dc104530 msgr2=0x7f61dc1a23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:20.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.092+0000 7f61e4600640 1 --2- 172.21.15.27:0/1049914489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61dc104530 0x7f61dc1a23a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f61d400eb10 tx=0x7f61d400c2d0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.092+0000 7f61e4600640 1 -- 172.21.15.27:0/1049914489 shutdown_connections 2024-04-26T22:58:20.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.092+0000 7f61e4600640 1 --2- 172.21.15.27:0/1049914489 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f61c40a3cc0 0x7f61c40a6180 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.092+0000 7f61e4600640 1 --2- 172.21.15.27:0/1049914489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61dc104530 0x7f61dc1a23a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.092+0000 7f61e4600640 1 --2- 172.21.15.27:0/1049914489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc103b70 0x7f61dc19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.092+0000 7f61e4600640 1 --2- 172.21.15.27:0/1049914489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc102980 0x7f61dc19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.092+0000 7f61e4600640 1 -- 172.21.15.27:0/1049914489 >> 172.21.15.27:0/1049914489 conn(0x7f61dc0fdef0 msgr2=0x7f61dc109310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:20.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.092+0000 7f61e4600640 1 -- 172.21.15.27:0/1049914489 shutdown_connections 2024-04-26T22:58:20.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.092+0000 7f61e4600640 1 -- 172.21.15.27:0/1049914489 wait complete. 2024-04-26T22:58:20.109 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:19 smithi027 ceph-mon[28034]: pgmap v421: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:20.109 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:19 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2085669167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:58:20.109 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:19 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/4167820677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:58:20.109 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:19 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/204380763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:58:20.113 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:58:20.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.331+0000 7fa05ac00640 1 -- 172.21.15.27:0/4072971763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa054108650 msgr2=0x7fa05410aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:20.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.331+0000 7fa05ac00640 1 --2- 172.21.15.27:0/4072971763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa054108650 0x7fa05410aaa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009900 tx=0x7fa04c02a3f0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.331+0000 7fa05ac00640 1 -- 172.21.15.27:0/4072971763 shutdown_connections 2024-04-26T22:58:20.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.331+0000 7fa05ac00640 1 --2- 172.21.15.27:0/4072971763 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa054105d20 0x7fa054108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.331+0000 7fa05ac00640 1 --2- 172.21.15.27:0/4072971763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa054108650 0x7fa05410aaa0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.331+0000 7fa05ac00640 1 --2- 172.21.15.27:0/4072971763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0541033f0 0x7fa0541057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.331+0000 7fa05ac00640 1 -- 172.21.15.27:0/4072971763 >> 172.21.15.27:0/4072971763 conn(0x7fa0540fce00 msgr2=0x7fa0540ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.332+0000 7fa05ac00640 1 -- 172.21.15.27:0/4072971763 shutdown_connections 2024-04-26T22:58:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.332+0000 7fa05ac00640 1 -- 172.21.15.27:0/4072971763 wait complete. 2024-04-26T22:58:20.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.333+0000 7fa05ac00640 1 Processor -- start 2024-04-26T22:58:20.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.333+0000 7fa05ac00640 1 -- start start 2024-04-26T22:58:20.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.333+0000 7fa05ac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0541033f0 0x7fa05419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:20.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.333+0000 7fa05ac00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa054105d20 0x7fa05419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:20.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.333+0000 7fa05ac00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa054108650 0x7fa0541a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:20.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.333+0000 7fa05ac00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa05410e510 con 0x7fa054108650 2024-04-26T22:58:20.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.333+0000 7fa05ac00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa05410e390 con 0x7fa054105d20 2024-04-26T22:58:20.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.333+0000 7fa05ac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa05410e690 con 0x7fa0541033f0 2024-04-26T22:58:20.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.333+0000 7fa058c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa054108650 0x7fa0541a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:20.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.333+0000 7fa053e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0541033f0 0x7fa05419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.333+0000 7fa058c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa054108650 0x7fa0541a2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48990/0 (socket says 172.21.15.27:48990) 2024-04-26T22:58:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.333+0000 7fa058c00640 1 -- 172.21.15.27:0/489722775 learned_addr learned my addr 172.21.15.27:0/489722775 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.334+0000 7fa053400640 1 --2- 172.21.15.27:0/489722775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa054105d20 0x7fa05419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.334+0000 7fa053e00640 1 -- 172.21.15.27:0/489722775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa054105d20 msgr2=0x7fa05419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.334+0000 7fa053e00640 1 --2- 172.21.15.27:0/489722775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa054105d20 0x7fa05419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.334+0000 7fa053e00640 1 -- 172.21.15.27:0/489722775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa054108650 msgr2=0x7fa0541a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.334+0000 7fa053e00640 1 --2- 172.21.15.27:0/489722775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa054108650 0x7fa0541a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.334+0000 7fa053e00640 1 -- 172.21.15.27:0/489722775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0541a29d0 con 0x7fa0541033f0 2024-04-26T22:58:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.334+0000 7fa053400640 1 --2- 172.21.15.27:0/489722775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa054105d20 0x7fa05419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:58:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.334+0000 7fa053e00640 1 --2- 172.21.15.27:0/489722775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0541033f0 0x7fa05419adc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa04400dab0 tx=0x7fa04400df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.335+0000 7fa050c00640 1 -- 172.21.15.27:0/489722775 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04400be30 con 0x7fa0541033f0 2024-04-26T22:58:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.335+0000 7fa05ac00640 1 -- 172.21.15.27:0/489722775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0541a2cc0 con 0x7fa0541033f0 2024-04-26T22:58:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.335+0000 7fa050c00640 1 -- 172.21.15.27:0/489722775 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa044003e20 con 0x7fa0541033f0 2024-04-26T22:58:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.335+0000 7fa050c00640 1 -- 172.21.15.27:0/489722775 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0440106c0 con 0x7fa0541033f0 2024-04-26T22:58:20.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.335+0000 7fa05ac00640 1 -- 172.21.15.27:0/489722775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0541a31b0 con 0x7fa0541033f0 2024-04-26T22:58:20.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.336+0000 7fa05ac00640 1 -- 172.21.15.27:0/489722775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa05410a140 con 0x7fa0541033f0 2024-04-26T22:58:20.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.338+0000 7fa050c00640 1 -- 172.21.15.27:0/489722775 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa0440042e0 con 0x7fa0541033f0 2024-04-26T22:58:20.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.339+0000 7fa050c00640 1 --2- 172.21.15.27:0/489722775 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa0380a3c40 0x7fa0380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:20.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.340+0000 7fa050c00640 1 -- 172.21.15.27:0/489722775 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fa0440d0ef0 con 0x7fa0541033f0 2024-04-26T22:58:20.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.340+0000 7fa053400640 1 --2- 172.21.15.27:0/489722775 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa0380a3c40 0x7fa0380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:20.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.340+0000 7fa053400640 1 --2- 172.21.15.27:0/489722775 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa0380a3c40 0x7fa0380a6100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa05406b640 tx=0x7fa03c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:20.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.344+0000 7fa050c00640 1 -- 172.21.15.27:0/489722775 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa04409d2e0 con 0x7fa0541033f0 2024-04-26T22:58:20.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.593+0000 7fa05ac00640 1 -- 172.21.15.27:0/489722775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa05402dd10 con 0x7fa0541033f0 2024-04-26T22:58:20.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.595+0000 7fa050c00640 1 -- 172.21.15.27:0/489722775 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11850 (secure 0 0 0) 0x7fa04409cbe0 con 0x7fa0541033f0 2024-04-26T22:58:20.596 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:58:20.596 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T22:55:25.265331+0000","modified":"2024-04-26T22:55:54.648644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24578,"mds_1":24577,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/1620677319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1620677319},{"type":"v1","addr":"172.21.15.94:6839","nonce":1620677319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.27:6841/1571355196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1571355196},{"type":"v1","addr":"172.21.15.27:6841","nonce":1571355196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24577":{"gid":24577,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.27:6839/3193530521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3193530521},{"type":"v1","addr":"172.21.15.27:6839","nonce":3193530521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/95952000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":95952000},{"type":"v1","addr":"172.21.15.80:6839","nonce":95952000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24578,"qdb_cluster":[24560,24577,24578]},"id":1}]} 2024-04-26T22:58:20.596 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-04-26T22:58:20.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.597+0000 7fa05ac00640 1 -- 172.21.15.27:0/489722775 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa0380a3c40 msgr2=0x7fa0380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:20.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.598+0000 7fa05ac00640 1 --2- 172.21.15.27:0/489722775 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa0380a3c40 0x7fa0380a6100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa05406b640 tx=0x7fa03c005f90 comp rx=0 tx=0).stop 2024-04-26T22:58:20.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.598+0000 7fa05ac00640 1 -- 172.21.15.27:0/489722775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0541033f0 msgr2=0x7fa05419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:20.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.598+0000 7fa05ac00640 1 --2- 172.21.15.27:0/489722775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0541033f0 0x7fa05419adc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa04400dab0 tx=0x7fa04400df70 comp rx=0 tx=0).stop 2024-04-26T22:58:20.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.598+0000 7fa05ac00640 1 -- 172.21.15.27:0/489722775 shutdown_connections 2024-04-26T22:58:20.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.598+0000 7fa05ac00640 1 --2- 172.21.15.27:0/489722775 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa0380a3c40 0x7fa0380a6100 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.598+0000 7fa05ac00640 1 --2- 172.21.15.27:0/489722775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa054105d20 0x7fa05419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.598+0000 7fa05ac00640 1 --2- 172.21.15.27:0/489722775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa054108650 0x7fa0541a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.598+0000 7fa05ac00640 1 --2- 172.21.15.27:0/489722775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0541033f0 0x7fa05419adc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.598+0000 7fa05ac00640 1 -- 172.21.15.27:0/489722775 >> 172.21.15.27:0/489722775 conn(0x7fa0540fce00 msgr2=0x7fa054102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:20.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.598+0000 7fa05ac00640 1 -- 172.21.15.27:0/489722775 shutdown_connections 2024-04-26T22:58:20.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.598+0000 7fa05ac00640 1 -- 172.21.15.27:0/489722775 wait complete. 2024-04-26T22:58:20.618 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:58:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.842+0000 7f3921400640 1 -- 172.21.15.27:0/2432423782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f391c104320 msgr2=0x7f391c10abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.842+0000 7f3921400640 1 --2- 172.21.15.27:0/2432423782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f391c104320 0x7f391c10abb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3914009ca0 tx=0x7f391402a8c0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.842+0000 7f3921400640 1 -- 172.21.15.27:0/2432423782 shutdown_connections 2024-04-26T22:58:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.842+0000 7f3921400640 1 --2- 172.21.15.27:0/2432423782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f391c103960 0x7f391c103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.842+0000 7f3921400640 1 --2- 172.21.15.27:0/2432423782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f391c104320 0x7f391c10abb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.842+0000 7f3921400640 1 --2- 172.21.15.27:0/2432423782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f391c102770 0x7f391c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.842+0000 7f3921400640 1 -- 172.21.15.27:0/2432423782 >> 172.21.15.27:0/2432423782 conn(0x7f391c0fdce0 msgr2=0x7f391c100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.842+0000 7f3921400640 1 -- 172.21.15.27:0/2432423782 shutdown_connections 2024-04-26T22:58:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.842+0000 7f3921400640 1 -- 172.21.15.27:0/2432423782 wait complete. 2024-04-26T22:58:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.843+0000 7f3921400640 1 Processor -- start 2024-04-26T22:58:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.843+0000 7f3921400640 1 -- start start 2024-04-26T22:58:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.844+0000 7f3921400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f391c102770 0x7f391c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.844+0000 7f3921400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f391c103960 0x7f391c19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.844+0000 7f3921400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f391c104320 0x7f391c1a2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.844+0000 7f3921400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f391c10e2a0 con 0x7f391c102770 2024-04-26T22:58:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.844+0000 7f3921400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f391c10e120 con 0x7f391c104320 2024-04-26T22:58:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.844+0000 7f3921400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f391c10e420 con 0x7f391c103960 2024-04-26T22:58:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.844+0000 7f391be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f391c102770 0x7f391c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.844+0000 7f391be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f391c102770 0x7f391c19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49012/0 (socket says 172.21.15.27:49012) 2024-04-26T22:58:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.844+0000 7f391be00640 1 -- 172.21.15.27:0/2676098137 learned_addr learned my addr 172.21.15.27:0/2676098137 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.844+0000 7f391b400640 1 --2- 172.21.15.27:0/2676098137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f391c103960 0x7f391c19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.844+0000 7f3920a00640 1 --2- 172.21.15.27:0/2676098137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f391c104320 0x7f391c1a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.844+0000 7f391be00640 1 -- 172.21.15.27:0/2676098137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f391c103960 msgr2=0x7f391c19afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.844+0000 7f391be00640 1 --2- 172.21.15.27:0/2676098137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f391c103960 0x7f391c19afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.844+0000 7f391be00640 1 -- 172.21.15.27:0/2676098137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f391c104320 msgr2=0x7f391c1a2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.844+0000 7f391be00640 1 --2- 172.21.15.27:0/2676098137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f391c104320 0x7f391c1a2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.844+0000 7f391be00640 1 -- 172.21.15.27:0/2676098137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391c1a2690 con 0x7f391c102770 2024-04-26T22:58:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.845+0000 7f391be00640 1 --2- 172.21.15.27:0/2676098137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f391c102770 0x7f391c19aa70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f390c00e9c0 tx=0x7f390c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.845+0000 7f3918c00640 1 -- 172.21.15.27:0/2676098137 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c00cd80 con 0x7f391c102770 2024-04-26T22:58:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.845+0000 7f3921400640 1 -- 172.21.15.27:0/2676098137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391c1a2980 con 0x7f391c102770 2024-04-26T22:58:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.845+0000 7f3918c00640 1 -- 172.21.15.27:0/2676098137 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f390c0035f0 con 0x7f391c102770 2024-04-26T22:58:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.845+0000 7f3918c00640 1 -- 172.21.15.27:0/2676098137 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c010690 con 0x7f391c102770 2024-04-26T22:58:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.845+0000 7f3921400640 1 -- 172.21.15.27:0/2676098137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391c1a2ed0 con 0x7f391c102770 2024-04-26T22:58:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.847+0000 7f3921400640 1 -- 172.21.15.27:0/2676098137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391c1a3160 con 0x7f391c102770 2024-04-26T22:58:20.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.849+0000 7f3918c00640 1 -- 172.21.15.27:0/2676098137 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f390c0037a0 con 0x7f391c102770 2024-04-26T22:58:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.851+0000 7f3918c00640 1 --2- 172.21.15.27:0/2676098137 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f39000a3cf0 0x7f39000a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.851+0000 7f3918c00640 1 -- 172.21.15.27:0/2676098137 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f390c0d0d80 con 0x7f391c102770 2024-04-26T22:58:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.851+0000 7f391b400640 1 --2- 172.21.15.27:0/2676098137 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f39000a3cf0 0x7f39000a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.852+0000 7f391b400640 1 --2- 172.21.15.27:0/2676098137 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f39000a3cf0 0x7f39000a61b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f391c19bf80 tx=0x7f3908005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:20.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:20.855+0000 7f3918c00640 1 -- 172.21.15.27:0/2676098137 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f390c09d170 con 0x7f391c102770 2024-04-26T22:58:21.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:20 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1049914489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:58:21.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:20 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/489722775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:58:21.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:20 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1049914489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:58:21.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:20 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/489722775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:58:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.067+0000 7f3921400640 1 -- 172.21.15.27:0/2676098137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f391c1a3330 con 0x7f391c102770 2024-04-26T22:58:21.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.069+0000 7f3918c00640 1 -- 172.21.15.27:0/2676098137 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24902 (secure 0 0 0) 0x7f390c09ca70 con 0x7f391c102770 2024-04-26T22:58:21.070 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:58:21.070 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":94,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:57:05.902371+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4},{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:58:21.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.073+0000 7f3921400640 1 -- 172.21.15.27:0/2676098137 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f39000a3cf0 msgr2=0x7f39000a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:21.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.073+0000 7f3921400640 1 --2- 172.21.15.27:0/2676098137 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f39000a3cf0 0x7f39000a61b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f391c19bf80 tx=0x7f3908005f90 comp rx=0 tx=0).stop 2024-04-26T22:58:21.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.073+0000 7f3921400640 1 -- 172.21.15.27:0/2676098137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f391c102770 msgr2=0x7f391c19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:21.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.073+0000 7f3921400640 1 --2- 172.21.15.27:0/2676098137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f391c102770 0x7f391c19aa70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f390c00e9c0 tx=0x7f390c00ee80 comp rx=0 tx=0).stop 2024-04-26T22:58:21.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.074+0000 7f3921400640 1 -- 172.21.15.27:0/2676098137 shutdown_connections 2024-04-26T22:58:21.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.074+0000 7f3921400640 1 --2- 172.21.15.27:0/2676098137 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f39000a3cf0 0x7f39000a61b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.074+0000 7f3921400640 1 --2- 172.21.15.27:0/2676098137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f391c104320 0x7f391c1a2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.074+0000 7f3921400640 1 --2- 172.21.15.27:0/2676098137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f391c103960 0x7f391c19afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.074+0000 7f3921400640 1 --2- 172.21.15.27:0/2676098137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f391c102770 0x7f391c19aa70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.074+0000 7f3921400640 1 -- 172.21.15.27:0/2676098137 >> 172.21.15.27:0/2676098137 conn(0x7f391c0fdce0 msgr2=0x7f391c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:21.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.074+0000 7f3921400640 1 -- 172.21.15.27:0/2676098137 shutdown_connections 2024-04-26T22:58:21.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.074+0000 7f3921400640 1 -- 172.21.15.27:0/2676098137 wait complete. 2024-04-26T22:58:21.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:20 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1049914489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:58:21.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:20 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/489722775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:58:21.096 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:49:54.482679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:55:22.899174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:55:24.218295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T22:55:28.394547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T22:58:21.097 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:58:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.287+0000 7f91a9600640 1 -- 172.21.15.27:0/2948327845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91a4075330 msgr2=0x7f91a4075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.287+0000 7f91a9600640 1 --2- 172.21.15.27:0/2948327845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91a4075330 0x7f91a4075790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f918c009900 tx=0x7f918c02a480 comp rx=0 tx=0).stop 2024-04-26T22:58:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.287+0000 7f91a9600640 1 -- 172.21.15.27:0/2948327845 shutdown_connections 2024-04-26T22:58:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.287+0000 7f91a9600640 1 --2- 172.21.15.27:0/2948327845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91a4075330 0x7f91a4075790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.287+0000 7f91a9600640 1 --2- 172.21.15.27:0/2948327845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91a410a810 0x7f91a410ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.287+0000 7f91a9600640 1 --2- 172.21.15.27:0/2948327845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91a4076910 0x7f91a4074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.287+0000 7f91a9600640 1 -- 172.21.15.27:0/2948327845 >> 172.21.15.27:0/2948327845 conn(0x7f91a40fdef0 msgr2=0x7f91a4100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.287+0000 7f91a9600640 1 -- 172.21.15.27:0/2948327845 shutdown_connections 2024-04-26T22:58:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.287+0000 7f91a9600640 1 -- 172.21.15.27:0/2948327845 wait complete. 2024-04-26T22:58:21.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.288+0000 7f91a9600640 1 Processor -- start 2024-04-26T22:58:21.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.289+0000 7f91a9600640 1 -- start start 2024-04-26T22:58:21.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.289+0000 7f91a9600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91a4075330 0x7f91a419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:21.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.289+0000 7f91a9600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91a4076910 0x7f91a419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:21.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.289+0000 7f91a9600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91a410a810 0x7f91a41a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:21.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.289+0000 7f91a9600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f91a4110700 con 0x7f91a410a810 2024-04-26T22:58:21.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.289+0000 7f91a9600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f91a4110580 con 0x7f91a4075330 2024-04-26T22:58:21.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.289+0000 7f91a9600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91a4110880 con 0x7f91a4076910 2024-04-26T22:58:21.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.289+0000 7f919be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91a4076910 0x7f91a419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:21.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.289+0000 7f91a2a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91a4075330 0x7f91a419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:21.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.289+0000 7f919be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91a4076910 0x7f91a419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:44842/0 (socket says 172.21.15.27:44842) 2024-04-26T22:58:21.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.289+0000 7f91a2a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91a4075330 0x7f91a419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33194/0 (socket says 172.21.15.27:33194) 2024-04-26T22:58:21.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.289+0000 7f919be00640 1 -- 172.21.15.27:0/2080000554 learned_addr learned my addr 172.21.15.27:0/2080000554 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:21.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.290+0000 7f91a3400640 1 --2- 172.21.15.27:0/2080000554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91a410a810 0x7f91a41a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:21.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.290+0000 7f91a2a00640 1 -- 172.21.15.27:0/2080000554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91a4076910 msgr2=0x7f91a419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:21.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.290+0000 7f91a2a00640 1 --2- 172.21.15.27:0/2080000554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91a4076910 0x7f91a419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.290+0000 7f91a2a00640 1 -- 172.21.15.27:0/2080000554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91a410a810 msgr2=0x7f91a41a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:21.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.290+0000 7f91a2a00640 1 --2- 172.21.15.27:0/2080000554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91a410a810 0x7f91a41a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.290+0000 7f91a2a00640 1 -- 172.21.15.27:0/2080000554 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a41a29e0 con 0x7f91a4075330 2024-04-26T22:58:21.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.290+0000 7f91a2a00640 1 --2- 172.21.15.27:0/2080000554 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91a4075330 0x7f91a419adc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f919400e9f0 tx=0x7f919400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:21.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.290+0000 7f919b400640 1 -- 172.21.15.27:0/2080000554 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919400cd80 con 0x7f91a4075330 2024-04-26T22:58:21.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.290+0000 7f919b400640 1 -- 172.21.15.27:0/2080000554 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91940035f0 con 0x7f91a4075330 2024-04-26T22:58:21.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.290+0000 7f919b400640 1 -- 172.21.15.27:0/2080000554 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194010670 con 0x7f91a4075330 2024-04-26T22:58:21.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.290+0000 7f91a9600640 1 -- 172.21.15.27:0/2080000554 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a41a2cd0 con 0x7f91a4075330 2024-04-26T22:58:21.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.291+0000 7f91a9600640 1 -- 172.21.15.27:0/2080000554 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a41a31d0 con 0x7f91a4075330 2024-04-26T22:58:21.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.292+0000 7f91a9600640 1 -- 172.21.15.27:0/2080000554 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a41095d0 con 0x7f91a4075330 2024-04-26T22:58:21.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.294+0000 7f919b400640 1 -- 172.21.15.27:0/2080000554 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f91940037a0 con 0x7f91a4075330 2024-04-26T22:58:21.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.295+0000 7f919b400640 1 --2- 172.21.15.27:0/2080000554 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f91880a3c40 0x7f91880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:21.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.296+0000 7f919b400640 1 -- 172.21.15.27:0/2080000554 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f91940d0e50 con 0x7f91a4075330 2024-04-26T22:58:21.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.296+0000 7f919be00640 1 --2- 172.21.15.27:0/2080000554 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f91880a3c40 0x7f91880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:21.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.296+0000 7f919be00640 1 --2- 172.21.15.27:0/2080000554 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f91880a3c40 0x7f91880a6100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f91a419c2d0 tx=0x7f918c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:21.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.302+0000 7f919b400640 1 -- 172.21.15.27:0/2080000554 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f919409d240 con 0x7f91a4075330 2024-04-26T22:58:21.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.517+0000 7f91a9600640 1 -- 172.21.15.27:0/2080000554 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f91a402dd40 con 0x7f91a4075330 2024-04-26T22:58:21.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.518+0000 7f919b400640 1 -- 172.21.15.27:0/2080000554 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f919409cb40 con 0x7f91a4075330 2024-04-26T22:58:21.519 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T22:58:21.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.521+0000 7f91a9600640 1 -- 172.21.15.27:0/2080000554 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f91880a3c40 msgr2=0x7f91880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:21.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.521+0000 7f91a9600640 1 --2- 172.21.15.27:0/2080000554 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f91880a3c40 0x7f91880a6100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f91a419c2d0 tx=0x7f918c005a20 comp rx=0 tx=0).stop 2024-04-26T22:58:21.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.521+0000 7f91a9600640 1 -- 172.21.15.27:0/2080000554 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91a4075330 msgr2=0x7f91a419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:21.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.521+0000 7f91a9600640 1 --2- 172.21.15.27:0/2080000554 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91a4075330 0x7f91a419adc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f919400e9f0 tx=0x7f919400eeb0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.521+0000 7f91a9600640 1 -- 172.21.15.27:0/2080000554 shutdown_connections 2024-04-26T22:58:21.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.521+0000 7f91a9600640 1 --2- 172.21.15.27:0/2080000554 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f91880a3c40 0x7f91880a6100 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.521+0000 7f91a9600640 1 --2- 172.21.15.27:0/2080000554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91a410a810 0x7f91a41a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.521+0000 7f91a9600640 1 --2- 172.21.15.27:0/2080000554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91a4076910 0x7f91a419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.521+0000 7f91a9600640 1 --2- 172.21.15.27:0/2080000554 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91a4075330 0x7f91a419adc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.521+0000 7f91a9600640 1 -- 172.21.15.27:0/2080000554 >> 172.21.15.27:0/2080000554 conn(0x7f91a40fdef0 msgr2=0x7f91a41067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:21.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.521+0000 7f91a9600640 1 -- 172.21.15.27:0/2080000554 shutdown_connections 2024-04-26T22:58:21.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.521+0000 7f91a9600640 1 -- 172.21.15.27:0/2080000554 wait complete. 2024-04-26T22:58:21.541 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:58:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.734+0000 7f872a400640 1 -- 172.21.15.27:0/2108816815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8724105d70 msgr2=0x7f87241061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.734+0000 7f872a400640 1 --2- 172.21.15.27:0/2108816815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8724105d70 0x7f87241061f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8710009900 tx=0x7f871002a410 comp rx=0 tx=0).stop 2024-04-26T22:58:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.734+0000 7f872a400640 1 -- 172.21.15.27:0/2108816815 shutdown_connections 2024-04-26T22:58:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.734+0000 7f872a400640 1 --2- 172.21.15.27:0/2108816815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8724105d70 0x7f87241061f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.734+0000 7f872a400640 1 --2- 172.21.15.27:0/2108816815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724106730 0x7f872410cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.734+0000 7f872a400640 1 --2- 172.21.15.27:0/2108816815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8724104b80 0x7f8724104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.734+0000 7f872a400640 1 -- 172.21.15.27:0/2108816815 >> 172.21.15.27:0/2108816815 conn(0x7f87241000f0 msgr2=0x7f8724102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:21.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.735+0000 7f872a400640 1 -- 172.21.15.27:0/2108816815 shutdown_connections 2024-04-26T22:58:21.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.735+0000 7f872a400640 1 -- 172.21.15.27:0/2108816815 wait complete. 2024-04-26T22:58:21.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.735+0000 7f872a400640 1 Processor -- start 2024-04-26T22:58:21.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.736+0000 7f872a400640 1 -- start start 2024-04-26T22:58:21.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.736+0000 7f872a400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8724104b80 0x7f872419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:21.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.736+0000 7f872a400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724105d70 0x7f872419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:21.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.736+0000 7f872a400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8724106730 0x7f87241a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:21.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.736+0000 7f872a400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8724110700 con 0x7f8724106730 2024-04-26T22:58:21.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.736+0000 7f872a400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8724110580 con 0x7f8724104b80 2024-04-26T22:58:21.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.736+0000 7f872a400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8724110880 con 0x7f8724105d70 2024-04-26T22:58:21.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.736+0000 7f8723400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8724104b80 0x7f872419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:21.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.736+0000 7f8722a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724105d70 0x7f872419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:21.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.736+0000 7f8723400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8724104b80 0x7f872419add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33200/0 (socket says 172.21.15.27:33200) 2024-04-26T22:58:21.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.736+0000 7f8723400640 1 -- 172.21.15.27:0/938811422 learned_addr learned my addr 172.21.15.27:0/938811422 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:21.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.736+0000 7f8723e00640 1 --2- 172.21.15.27:0/938811422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8724106730 0x7f87241a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:21.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.736+0000 7f8722a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724105d70 0x7f872419b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:44848/0 (socket says 172.21.15.27:44848) 2024-04-26T22:58:21.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.737+0000 7f8723400640 1 -- 172.21.15.27:0/938811422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724105d70 msgr2=0x7f872419b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:21.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.737+0000 7f8723400640 1 --2- 172.21.15.27:0/938811422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724105d70 0x7f872419b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.737+0000 7f8723400640 1 -- 172.21.15.27:0/938811422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8724106730 msgr2=0x7f87241a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:21.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.737+0000 7f8723400640 1 --2- 172.21.15.27:0/938811422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8724106730 0x7f87241a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.737+0000 7f8723400640 1 -- 172.21.15.27:0/938811422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87241a29f0 con 0x7f8724104b80 2024-04-26T22:58:21.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.737+0000 7f8723400640 1 --2- 172.21.15.27:0/938811422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8724104b80 0x7f872419add0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f871400e9c0 tx=0x7f871400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:21.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.737+0000 7f871be00640 1 -- 172.21.15.27:0/938811422 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871400cd80 con 0x7f8724104b80 2024-04-26T22:58:21.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.737+0000 7f872a400640 1 -- 172.21.15.27:0/938811422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87241a2ce0 con 0x7f8724104b80 2024-04-26T22:58:21.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.737+0000 7f871be00640 1 -- 172.21.15.27:0/938811422 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87140035f0 con 0x7f8724104b80 2024-04-26T22:58:21.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.737+0000 7f871be00640 1 -- 172.21.15.27:0/938811422 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714010690 con 0x7f8724104b80 2024-04-26T22:58:21.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.737+0000 7f872a400640 1 -- 172.21.15.27:0/938811422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87241a31a0 con 0x7f8724104b80 2024-04-26T22:58:21.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.739+0000 7f872a400640 1 -- 172.21.15.27:0/938811422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f872410bef0 con 0x7f8724104b80 2024-04-26T22:58:21.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.741+0000 7f871be00640 1 -- 172.21.15.27:0/938811422 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f87140037a0 con 0x7f8724104b80 2024-04-26T22:58:21.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.742+0000 7f871be00640 1 --2- 172.21.15.27:0/938811422 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f87080a3cc0 0x7f87080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:21.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.742+0000 7f871be00640 1 -- 172.21.15.27:0/938811422 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f87140d0e80 con 0x7f8724104b80 2024-04-26T22:58:21.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.742+0000 7f8722a00640 1 --2- 172.21.15.27:0/938811422 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f87080a3cc0 0x7f87080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:21.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.743+0000 7f8722a00640 1 --2- 172.21.15.27:0/938811422 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f87080a3cc0 0x7f87080a6180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8710031500 tx=0x7f8710005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:21.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.748+0000 7f871be00640 1 -- 172.21.15.27:0/938811422 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f871409d270 con 0x7f8724104b80 2024-04-26T22:58:21.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:21 smithi027 ceph-mon[28034]: pgmap v422: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:58:21.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:58:21.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:21 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2676098137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:58:21.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:21.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:21 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2080000554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:58:21.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:21.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:21.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:21 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:21.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.971+0000 7f872a400640 1 -- 172.21.15.27:0/938811422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f872402dad0 con 0x7f8724104b80 2024-04-26T22:58:21.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.972+0000 7f871be00640 1 -- 172.21.15.27:0/938811422 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f872402dad0 con 0x7f8724104b80 2024-04-26T22:58:21.972 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T22:58:21.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.974+0000 7f872a400640 1 -- 172.21.15.27:0/938811422 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f87080a3cc0 msgr2=0x7f87080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:21.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.974+0000 7f872a400640 1 --2- 172.21.15.27:0/938811422 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f87080a3cc0 0x7f87080a6180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8710031500 tx=0x7f8710005a20 comp rx=0 tx=0).stop 2024-04-26T22:58:21.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.974+0000 7f872a400640 1 -- 172.21.15.27:0/938811422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8724104b80 msgr2=0x7f872419add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:21.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.974+0000 7f872a400640 1 --2- 172.21.15.27:0/938811422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8724104b80 0x7f872419add0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f871400e9c0 tx=0x7f871400ee80 comp rx=0 tx=0).stop 2024-04-26T22:58:21.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.974+0000 7f872a400640 1 -- 172.21.15.27:0/938811422 shutdown_connections 2024-04-26T22:58:21.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.974+0000 7f872a400640 1 --2- 172.21.15.27:0/938811422 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f87080a3cc0 0x7f87080a6180 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.974+0000 7f872a400640 1 --2- 172.21.15.27:0/938811422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8724106730 0x7f87241a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.974+0000 7f872a400640 1 --2- 172.21.15.27:0/938811422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724105d70 0x7f872419b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.974+0000 7f872a400640 1 --2- 172.21.15.27:0/938811422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8724104b80 0x7f872419add0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:21.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.974+0000 7f872a400640 1 -- 172.21.15.27:0/938811422 >> 172.21.15.27:0/938811422 conn(0x7f87241000f0 msgr2=0x7f87241090c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:21.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.975+0000 7f872a400640 1 -- 172.21.15.27:0/938811422 shutdown_connections 2024-04-26T22:58:21.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:21.975+0000 7f872a400640 1 -- 172.21.15.27:0/938811422 wait complete. 2024-04-26T22:58:21.992 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:58:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:21 smithi080 ceph-mon[32324]: pgmap v422: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:58:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:58:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:21 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2676098137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:58:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:21 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2080000554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:58:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:22.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:21 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:22.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:21 smithi094 ceph-mon[32889]: pgmap v422: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T22:58:22.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:58:22.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:21 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2676098137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:58:22.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:22.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:21 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2080000554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:58:22.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:22.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:22.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:21 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:22.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.182+0000 7f7bfca00640 1 -- 172.21.15.27:0/354267594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf81031f0 msgr2=0x7f7bf81055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:22.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.182+0000 7f7bfca00640 1 --2- 172.21.15.27:0/354267594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf81031f0 0x7f7bf81055e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009900 tx=0x7f7be802a410 comp rx=0 tx=0).stop 2024-04-26T22:58:22.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.182+0000 7f7bfca00640 1 -- 172.21.15.27:0/354267594 shutdown_connections 2024-04-26T22:58:22.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.182+0000 7f7bfca00640 1 --2- 172.21.15.27:0/354267594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8108450 0x7f7bf810a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.182+0000 7f7bfca00640 1 --2- 172.21.15.27:0/354267594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bf8105b20 0x7f7bf8107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.182+0000 7f7bfca00640 1 --2- 172.21.15.27:0/354267594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf81031f0 0x7f7bf81055e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.182+0000 7f7bfca00640 1 -- 172.21.15.27:0/354267594 >> 172.21.15.27:0/354267594 conn(0x7f7bf80fcc00 msgr2=0x7f7bf80ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:22.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.182+0000 7f7bfca00640 1 -- 172.21.15.27:0/354267594 shutdown_connections 2024-04-26T22:58:22.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.182+0000 7f7bfca00640 1 -- 172.21.15.27:0/354267594 wait complete. 2024-04-26T22:58:22.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.184+0000 7f7bfca00640 1 Processor -- start 2024-04-26T22:58:22.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.184+0000 7f7bfca00640 1 -- start start 2024-04-26T22:58:22.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.184+0000 7f7bfca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf81031f0 0x7f7bf81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:22.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.184+0000 7f7bfca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bf8105b20 0x7f7bf8198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:22.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.185+0000 7f7bfca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8108450 0x7f7bf819ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:22.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.185+0000 7f7bfca00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf810e320 con 0x7f7bf81031f0 2024-04-26T22:58:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.185+0000 7f7bfca00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf810e1a0 con 0x7f7bf8108450 2024-04-26T22:58:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.185+0000 7f7bfca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf810e4a0 con 0x7f7bf8105b20 2024-04-26T22:58:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.185+0000 7f7befe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bf8105b20 0x7f7bf8198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.185+0000 7f7befe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bf8105b20 0x7f7bf8198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:44856/0 (socket says 172.21.15.27:44856) 2024-04-26T22:58:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.185+0000 7f7befe00640 1 -- 172.21.15.27:0/4190977550 learned_addr learned my addr 172.21.15.27:0/4190977550 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.185+0000 7f7bf6800640 1 --2- 172.21.15.27:0/4190977550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf81031f0 0x7f7bf81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.185+0000 7f7bf7200640 1 --2- 172.21.15.27:0/4190977550 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8108450 0x7f7bf819ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.185+0000 7f7befe00640 1 -- 172.21.15.27:0/4190977550 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8108450 msgr2=0x7f7bf819ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.185+0000 7f7befe00640 1 --2- 172.21.15.27:0/4190977550 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8108450 0x7f7bf819ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.185+0000 7f7befe00640 1 -- 172.21.15.27:0/4190977550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf81031f0 msgr2=0x7f7bf81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.185+0000 7f7befe00640 1 --2- 172.21.15.27:0/4190977550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf81031f0 0x7f7bf81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.185+0000 7f7befe00640 1 -- 172.21.15.27:0/4190977550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf81a05c0 con 0x7f7bf8105b20 2024-04-26T22:58:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.185+0000 7f7bf6800640 1 --2- 172.21.15.27:0/4190977550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf81031f0 0x7f7bf81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.185+0000 7f7bf7200640 1 --2- 172.21.15.27:0/4190977550 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8108450 0x7f7bf819ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:22.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.186+0000 7f7befe00640 1 --2- 172.21.15.27:0/4190977550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bf8105b20 0x7f7bf8198ee0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7be000e9f0 tx=0x7f7be000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:22.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.186+0000 7f7bef400640 1 -- 172.21.15.27:0/4190977550 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be000ccc0 con 0x7f7bf8105b20 2024-04-26T22:58:22.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.186+0000 7f7bef400640 1 -- 172.21.15.27:0/4190977550 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be00054d0 con 0x7f7bf8105b20 2024-04-26T22:58:22.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.186+0000 7f7bfca00640 1 -- 172.21.15.27:0/4190977550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf81a08b0 con 0x7f7bf8105b20 2024-04-26T22:58:22.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.186+0000 7f7bef400640 1 -- 172.21.15.27:0/4190977550 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0010430 con 0x7f7bf8105b20 2024-04-26T22:58:22.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.186+0000 7f7bfca00640 1 -- 172.21.15.27:0/4190977550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf81a0db0 con 0x7f7bf8105b20 2024-04-26T22:58:22.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.187+0000 7f7bfca00640 1 -- 172.21.15.27:0/4190977550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf80fe6f0 con 0x7f7bf8105b20 2024-04-26T22:58:22.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.189+0000 7f7bef400640 1 -- 172.21.15.27:0/4190977550 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7be00046c0 con 0x7f7bf8105b20 2024-04-26T22:58:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.191+0000 7f7bef400640 1 --2- 172.21.15.27:0/4190977550 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7bdc0a3c30 0x7f7bdc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.191+0000 7f7bef400640 1 -- 172.21.15.27:0/4190977550 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f7be00d0d60 con 0x7f7bf8105b20 2024-04-26T22:58:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.191+0000 7f7bf6800640 1 --2- 172.21.15.27:0/4190977550 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7bdc0a3c30 0x7f7bdc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:22.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.192+0000 7f7bf6800640 1 --2- 172.21.15.27:0/4190977550 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7bdc0a3c30 0x7f7bdc0a60f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7be8031500 tx=0x7f7be8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.195+0000 7f7bef400640 1 -- 172.21.15.27:0/4190977550 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7be009d150 con 0x7f7bf8105b20 2024-04-26T22:58:22.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.394+0000 7f7bfca00640 1 -- 172.21.15.27:0/4190977550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7bf802dad0 con 0x7f7bf8105b20 2024-04-26T22:58:22.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.395+0000 7f7bef400640 1 -- 172.21.15.27:0/4190977550 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f7be009ca50 con 0x7f7bf8105b20 2024-04-26T22:58:22.395 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-26T22:58:22.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.397+0000 7f7bfca00640 1 -- 172.21.15.27:0/4190977550 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7bdc0a3c30 msgr2=0x7f7bdc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:22.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.397+0000 7f7bfca00640 1 --2- 172.21.15.27:0/4190977550 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7bdc0a3c30 0x7f7bdc0a60f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7be8031500 tx=0x7f7be8005a20 comp rx=0 tx=0).stop 2024-04-26T22:58:22.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.397+0000 7f7bfca00640 1 -- 172.21.15.27:0/4190977550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bf8105b20 msgr2=0x7f7bf8198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:22.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.397+0000 7f7bfca00640 1 --2- 172.21.15.27:0/4190977550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bf8105b20 0x7f7bf8198ee0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7be000e9f0 tx=0x7f7be000eeb0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.398+0000 7f7bfca00640 1 -- 172.21.15.27:0/4190977550 shutdown_connections 2024-04-26T22:58:22.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.398+0000 7f7bfca00640 1 --2- 172.21.15.27:0/4190977550 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7bdc0a3c30 0x7f7bdc0a60f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.398+0000 7f7bfca00640 1 --2- 172.21.15.27:0/4190977550 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8108450 0x7f7bf819ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.398+0000 7f7bfca00640 1 --2- 172.21.15.27:0/4190977550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bf8105b20 0x7f7bf8198ee0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.398+0000 7f7bfca00640 1 --2- 172.21.15.27:0/4190977550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf81031f0 0x7f7bf81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.398+0000 7f7bfca00640 1 -- 172.21.15.27:0/4190977550 >> 172.21.15.27:0/4190977550 conn(0x7f7bf80fcc00 msgr2=0x7f7bf8103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:22.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.398+0000 7f7bfca00640 1 -- 172.21.15.27:0/4190977550 shutdown_connections 2024-04-26T22:58:22.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.398+0000 7f7bfca00640 1 -- 172.21.15.27:0/4190977550 wait complete. 2024-04-26T22:58:22.417 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T22:58:22.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.622+0000 7f0ba6000640 1 -- 172.21.15.27:0/3077001896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba01039b0 msgr2=0x7f0ba0103e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:22.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.622+0000 7f0ba6000640 1 --2- 172.21.15.27:0/3077001896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba01039b0 0x7f0ba0103e30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009900 tx=0x7f0b8c02a3f0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.622+0000 7f0ba6000640 1 -- 172.21.15.27:0/3077001896 shutdown_connections 2024-04-26T22:58:22.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.622+0000 7f0ba6000640 1 --2- 172.21.15.27:0/3077001896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba01039b0 0x7f0ba0103e30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.622+0000 7f0ba6000640 1 --2- 172.21.15.27:0/3077001896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba0104370 0x7f0ba010ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.622+0000 7f0ba6000640 1 --2- 172.21.15.27:0/3077001896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ba01027c0 0x7f0ba0102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.622+0000 7f0ba6000640 1 -- 172.21.15.27:0/3077001896 >> 172.21.15.27:0/3077001896 conn(0x7f0ba00fdd30 msgr2=0x7f0ba0100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:22.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.622+0000 7f0ba6000640 1 -- 172.21.15.27:0/3077001896 shutdown_connections 2024-04-26T22:58:22.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.622+0000 7f0ba6000640 1 -- 172.21.15.27:0/3077001896 wait complete. 2024-04-26T22:58:22.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.623+0000 7f0ba6000640 1 Processor -- start 2024-04-26T22:58:22.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.624+0000 7f0ba6000640 1 -- start start 2024-04-26T22:58:22.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.624+0000 7f0ba6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba01027c0 0x7f0ba019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:22.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.624+0000 7f0ba6000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ba01039b0 0x7f0ba019b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:22.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.624+0000 7f0ba6000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba0104370 0x7f0ba01a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:22.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.624+0000 7f0ba6000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba010e320 con 0x7f0ba01039b0 2024-04-26T22:58:22.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.624+0000 7f0ba6000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba010e1a0 con 0x7f0ba0104370 2024-04-26T22:58:22.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.624+0000 7f0ba6000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba010e4a0 con 0x7f0ba01027c0 2024-04-26T22:58:22.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.624+0000 7f0ba4c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba01027c0 0x7f0ba019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:22.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.624+0000 7f0ba4c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba01027c0 0x7f0ba019ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:44866/0 (socket says 172.21.15.27:44866) 2024-04-26T22:58:22.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.624+0000 7f0b9fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ba01039b0 0x7f0ba019b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:22.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.624+0000 7f0ba4c00640 1 -- 172.21.15.27:0/2001718838 learned_addr learned my addr 172.21.15.27:0/2001718838 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:22.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.624+0000 7f0ba5600640 1 --2- 172.21.15.27:0/2001718838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba0104370 0x7f0ba01a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:22.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.625+0000 7f0ba4c00640 1 -- 172.21.15.27:0/2001718838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba0104370 msgr2=0x7f0ba01a21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:22.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.625+0000 7f0ba4c00640 1 --2- 172.21.15.27:0/2001718838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba0104370 0x7f0ba01a21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.625+0000 7f0ba4c00640 1 -- 172.21.15.27:0/2001718838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ba01039b0 msgr2=0x7f0ba019b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:22.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.625+0000 7f0ba4c00640 1 --2- 172.21.15.27:0/2001718838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ba01039b0 0x7f0ba019b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.625+0000 7f0ba4c00640 1 -- 172.21.15.27:0/2001718838 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba01a2810 con 0x7f0ba01027c0 2024-04-26T22:58:22.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.625+0000 7f0ba4c00640 1 --2- 172.21.15.27:0/2001718838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba01027c0 0x7f0ba019ac90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400e9c0 tx=0x7f0b9400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:22.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.625+0000 7f0b9d600640 1 -- 172.21.15.27:0/2001718838 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9400cd80 con 0x7f0ba01027c0 2024-04-26T22:58:22.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.625+0000 7f0ba6000640 1 -- 172.21.15.27:0/2001718838 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba01a2b00 con 0x7f0ba01027c0 2024-04-26T22:58:22.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.625+0000 7f0b9d600640 1 -- 172.21.15.27:0/2001718838 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b940035f0 con 0x7f0ba01027c0 2024-04-26T22:58:22.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.625+0000 7f0b9d600640 1 -- 172.21.15.27:0/2001718838 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94010690 con 0x7f0ba01027c0 2024-04-26T22:58:22.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.625+0000 7f0ba6000640 1 -- 172.21.15.27:0/2001718838 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba01a3070 con 0x7f0ba01027c0 2024-04-26T22:58:22.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.627+0000 7f0ba6000640 1 -- 172.21.15.27:0/2001718838 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba01a3240 con 0x7f0ba01027c0 2024-04-26T22:58:22.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.629+0000 7f0b9d600640 1 -- 172.21.15.27:0/2001718838 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0b940037a0 con 0x7f0ba01027c0 2024-04-26T22:58:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.630+0000 7f0b9d600640 1 --2- 172.21.15.27:0/2001718838 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0b880a3c40 0x7f0b880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.630+0000 7f0b9d600640 1 -- 172.21.15.27:0/2001718838 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f0b940d0d20 con 0x7f0ba01027c0 2024-04-26T22:58:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.631+0000 7f0b9fe00640 1 --2- 172.21.15.27:0/2001718838 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0b880a3c40 0x7f0b880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.631+0000 7f0b9fe00640 1 --2- 172.21.15.27:0/2001718838 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0b880a3c40 0x7f0b880a6100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c031500 tx=0x7f0b8c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:22.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.634+0000 7f0b9d600640 1 -- 172.21.15.27:0/2001718838 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0b9409d150 con 0x7f0ba01027c0 2024-04-26T22:58:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.853+0000 7f0ba6000640 1 -- 172.21.15.27:0/2001718838 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0ba01a3410 con 0x7f0ba01027c0 2024-04-26T22:58:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.854+0000 7f0b9d600640 1 -- 172.21.15.27:0/2001718838 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f0b9409ca50 con 0x7f0ba01027c0 2024-04-26T22:58:22.854 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T22:58:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.856+0000 7f0ba6000640 1 -- 172.21.15.27:0/2001718838 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0b880a3c40 msgr2=0x7f0b880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.856+0000 7f0ba6000640 1 --2- 172.21.15.27:0/2001718838 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0b880a3c40 0x7f0b880a6100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c031500 tx=0x7f0b8c005a20 comp rx=0 tx=0).stop 2024-04-26T22:58:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.856+0000 7f0ba6000640 1 -- 172.21.15.27:0/2001718838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba01027c0 msgr2=0x7f0ba019ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.856+0000 7f0ba6000640 1 --2- 172.21.15.27:0/2001718838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba01027c0 0x7f0ba019ac90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400e9c0 tx=0x7f0b9400ee80 comp rx=0 tx=0).stop 2024-04-26T22:58:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.856+0000 7f0ba6000640 1 -- 172.21.15.27:0/2001718838 shutdown_connections 2024-04-26T22:58:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.856+0000 7f0ba6000640 1 --2- 172.21.15.27:0/2001718838 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f0b880a3c40 0x7f0b880a6100 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.856+0000 7f0ba6000640 1 --2- 172.21.15.27:0/2001718838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba0104370 0x7f0ba01a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.856+0000 7f0ba6000640 1 --2- 172.21.15.27:0/2001718838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ba01039b0 0x7f0ba019b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.856+0000 7f0ba6000640 1 --2- 172.21.15.27:0/2001718838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba01027c0 0x7f0ba019ac90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.856+0000 7f0ba6000640 1 -- 172.21.15.27:0/2001718838 >> 172.21.15.27:0/2001718838 conn(0x7f0ba00fdd30 msgr2=0x7f0ba0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.857+0000 7f0ba6000640 1 -- 172.21.15.27:0/2001718838 shutdown_connections 2024-04-26T22:58:22.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:22.857+0000 7f0ba6000640 1 -- 172.21.15.27:0/2001718838 wait complete. 2024-04-26T22:58:22.877 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:58:23.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:23.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:22 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:23.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:22 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/938811422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:58:23.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:22 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/4190977550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:58:23.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.064+0000 7f2845a00640 1 -- 172.21.15.27:0/2665018145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f284010a940 msgr2=0x7f28401111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:23.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.064+0000 7f2845a00640 1 --2- 172.21.15.27:0/2665018145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f284010a940 0x7f28401111d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2838009ca0 tx=0x7f283802a8c0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.064+0000 7f2845a00640 1 -- 172.21.15.27:0/2665018145 shutdown_connections 2024-04-26T22:58:23.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.064+0000 7f2845a00640 1 --2- 172.21.15.27:0/2665018145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f284010a940 0x7f28401111d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.064+0000 7f2845a00640 1 --2- 172.21.15.27:0/2665018145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2840109f80 0x7f284010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.064+0000 7f2845a00640 1 --2- 172.21.15.27:0/2665018145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2840108d90 0x7f28401091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.064+0000 7f2845a00640 1 -- 172.21.15.27:0/2665018145 >> 172.21.15.27:0/2665018145 conn(0x7f2840077d30 msgr2=0x7f284007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:23.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.064+0000 7f2845a00640 1 -- 172.21.15.27:0/2665018145 shutdown_connections 2024-04-26T22:58:23.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.064+0000 7f2845a00640 1 -- 172.21.15.27:0/2665018145 wait complete. 2024-04-26T22:58:23.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.066+0000 7f2845a00640 1 Processor -- start 2024-04-26T22:58:23.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.066+0000 7f2845a00640 1 -- start start 2024-04-26T22:58:23.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.066+0000 7f2845a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2840108d90 0x7f284019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:23.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.066+0000 7f2845a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2840109f80 0x7f284019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:23.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.067+0000 7f2845a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f284010a940 0x7f28401a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:23.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.067+0000 7f2845a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f28401148f0 con 0x7f2840109f80 2024-04-26T22:58:23.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.067+0000 7f2845a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2840114770 con 0x7f2840108d90 2024-04-26T22:58:23.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.067+0000 7f2845a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2840114a70 con 0x7f284010a940 2024-04-26T22:58:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.067+0000 7f283fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2840108d90 0x7f284019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.067+0000 7f283f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2840109f80 0x7f284019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.067+0000 7f283fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2840108d90 0x7f284019ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33264/0 (socket says 172.21.15.27:33264) 2024-04-26T22:58:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.067+0000 7f283fe00640 1 -- 172.21.15.27:0/1807147895 learned_addr learned my addr 172.21.15.27:0/1807147895 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.067+0000 7f2845000640 1 --2- 172.21.15.27:0/1807147895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f284010a940 0x7f28401a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.067+0000 7f283fe00640 1 -- 172.21.15.27:0/1807147895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f284010a940 msgr2=0x7f28401a6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.067+0000 7f283fe00640 1 --2- 172.21.15.27:0/1807147895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f284010a940 0x7f28401a6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.067+0000 7f283fe00640 1 -- 172.21.15.27:0/1807147895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2840109f80 msgr2=0x7f284019f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.067+0000 7f283fe00640 1 --2- 172.21.15.27:0/1807147895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2840109f80 0x7f284019f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.067+0000 7f283fe00640 1 -- 172.21.15.27:0/1807147895 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28401a6a90 con 0x7f2840108d90 2024-04-26T22:58:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.068+0000 7f283fe00640 1 --2- 172.21.15.27:0/1807147895 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2840108d90 0x7f284019ee70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f283000d770 tx=0x7f283000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.068+0000 7f283cc00640 1 -- 172.21.15.27:0/1807147895 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2830017070 con 0x7f2840108d90 2024-04-26T22:58:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.068+0000 7f283cc00640 1 -- 172.21.15.27:0/1807147895 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2830009c30 con 0x7f2840108d90 2024-04-26T22:58:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.068+0000 7f283cc00640 1 -- 172.21.15.27:0/1807147895 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2830003cc0 con 0x7f2840108d90 2024-04-26T22:58:23.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.068+0000 7f2845a00640 1 -- 172.21.15.27:0/1807147895 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28401a6d80 con 0x7f2840108d90 2024-04-26T22:58:23.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.068+0000 7f2845a00640 1 -- 172.21.15.27:0/1807147895 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28401a71c0 con 0x7f2840108d90 2024-04-26T22:58:23.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.069+0000 7f2845a00640 1 -- 172.21.15.27:0/1807147895 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2840110050 con 0x7f2840108d90 2024-04-26T22:58:23.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.071+0000 7f283cc00640 1 -- 172.21.15.27:0/1807147895 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2830003e60 con 0x7f2840108d90 2024-04-26T22:58:23.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.073+0000 7f283cc00640 1 --2- 172.21.15.27:0/1807147895 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f28240a3cf0 0x7f28240a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:23.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.073+0000 7f283cc00640 1 -- 172.21.15.27:0/1807147895 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f28300d0320 con 0x7f2840108d90 2024-04-26T22:58:23.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.073+0000 7f283f400640 1 --2- 172.21.15.27:0/1807147895 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f28240a3cf0 0x7f28240a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:23.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.074+0000 7f283f400640 1 --2- 172.21.15.27:0/1807147895 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f28240a3cf0 0x7f28240a61b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f28401a0380 tx=0x7f282c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:23.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.077+0000 7f283cc00640 1 -- 172.21.15.27:0/1807147895 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2830009de0 con 0x7f2840108d90 2024-04-26T22:58:23.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:23.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:22 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:23.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:22 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/938811422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:58:23.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:22 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/4190977550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:58:23.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:23.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:22 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:23.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:22 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/938811422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:58:23.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:22 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/4190977550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:58:23.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.338+0000 7f2845a00640 1 -- 172.21.15.27:0/1807147895 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f284002dad0 con 0x7f2840108d90 2024-04-26T22:58:23.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.339+0000 7f283cc00640 1 -- 172.21.15.27:0/1807147895 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11850 (secure 0 0 0) 0x7f283009c0c0 con 0x7f2840108d90 2024-04-26T22:58:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:58:23.340 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T22:55:25.265331+0000","modified":"2024-04-26T22:55:54.648644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24578,"mds_1":24577,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/1620677319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1620677319},{"type":"v1","addr":"172.21.15.94:6839","nonce":1620677319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.27:6841/1571355196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1571355196},{"type":"v1","addr":"172.21.15.27:6841","nonce":1571355196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24577":{"gid":24577,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.27:6839/3193530521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3193530521},{"type":"v1","addr":"172.21.15.27:6839","nonce":3193530521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/95952000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":95952000},{"type":"v1","addr":"172.21.15.80:6839","nonce":95952000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24578,"qdb_cluster":[24560,24577,24578]},"id":1}]} 2024-04-26T22:58:23.340 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-04-26T22:58:23.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.342+0000 7f2845a00640 1 -- 172.21.15.27:0/1807147895 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f28240a3cf0 msgr2=0x7f28240a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:23.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.342+0000 7f2845a00640 1 --2- 172.21.15.27:0/1807147895 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f28240a3cf0 0x7f28240a61b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f28401a0380 tx=0x7f282c005f90 comp rx=0 tx=0).stop 2024-04-26T22:58:23.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.342+0000 7f2845a00640 1 -- 172.21.15.27:0/1807147895 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2840108d90 msgr2=0x7f284019ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:23.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.342+0000 7f2845a00640 1 --2- 172.21.15.27:0/1807147895 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2840108d90 0x7f284019ee70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f283000d770 tx=0x7f283000dc30 comp rx=0 tx=0).stop 2024-04-26T22:58:23.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.342+0000 7f2845a00640 1 -- 172.21.15.27:0/1807147895 shutdown_connections 2024-04-26T22:58:23.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.342+0000 7f2845a00640 1 --2- 172.21.15.27:0/1807147895 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f28240a3cf0 0x7f28240a61b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.342+0000 7f2845a00640 1 --2- 172.21.15.27:0/1807147895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f284010a940 0x7f28401a6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.342+0000 7f2845a00640 1 --2- 172.21.15.27:0/1807147895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2840109f80 0x7f284019f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.342+0000 7f2845a00640 1 --2- 172.21.15.27:0/1807147895 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2840108d90 0x7f284019ee70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.342+0000 7f2845a00640 1 -- 172.21.15.27:0/1807147895 >> 172.21.15.27:0/1807147895 conn(0x7f2840077d30 msgr2=0x7f284010d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.343+0000 7f2845a00640 1 -- 172.21.15.27:0/1807147895 shutdown_connections 2024-04-26T22:58:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.343+0000 7f2845a00640 1 -- 172.21.15.27:0/1807147895 wait complete. 2024-04-26T22:58:23.369 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:58:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.560+0000 7f113aa00640 1 -- 172.21.15.27:0/1546459423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1134103990 msgr2=0x7f1134103e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.560+0000 7f113aa00640 1 --2- 172.21.15.27:0/1546459423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1134103990 0x7f1134103e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1120009900 tx=0x7f112002a400 comp rx=0 tx=0).stop 2024-04-26T22:58:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.560+0000 7f113aa00640 1 -- 172.21.15.27:0/1546459423 shutdown_connections 2024-04-26T22:58:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.560+0000 7f113aa00640 1 --2- 172.21.15.27:0/1546459423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1134103990 0x7f1134103e10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.560+0000 7f113aa00640 1 --2- 172.21.15.27:0/1546459423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1134104350 0x7f113410abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.560+0000 7f113aa00640 1 --2- 172.21.15.27:0/1546459423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11341027a0 0x7f1134102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.560+0000 7f113aa00640 1 -- 172.21.15.27:0/1546459423 >> 172.21.15.27:0/1546459423 conn(0x7f11340fdd10 msgr2=0x7f1134100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.560+0000 7f113aa00640 1 -- 172.21.15.27:0/1546459423 shutdown_connections 2024-04-26T22:58:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.560+0000 7f113aa00640 1 -- 172.21.15.27:0/1546459423 wait complete. 2024-04-26T22:58:23.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.562+0000 7f113aa00640 1 Processor -- start 2024-04-26T22:58:23.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.562+0000 7f113aa00640 1 -- start start 2024-04-26T22:58:23.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.562+0000 7f113aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11341027a0 0x7f11341988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:23.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.562+0000 7f113aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1134103990 0x7f1134198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:23.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.562+0000 7f113aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1134104350 0x7f113419fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:23.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.562+0000 7f113aa00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f113410e300 con 0x7f1134104350 2024-04-26T22:58:23.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.562+0000 7f113aa00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f113410e180 con 0x7f11341027a0 2024-04-26T22:58:23.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.562+0000 7f113aa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f113410e480 con 0x7f1134103990 2024-04-26T22:58:23.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.563+0000 7f1139600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11341027a0 0x7f11341988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:23.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.563+0000 7f1138c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1134103990 0x7f1134198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:23.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.563+0000 7f1139600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11341027a0 0x7f11341988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33286/0 (socket says 172.21.15.27:33286) 2024-04-26T22:58:23.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.563+0000 7f1139600640 1 -- 172.21.15.27:0/2718288324 learned_addr learned my addr 172.21.15.27:0/2718288324 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:23.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.563+0000 7f113a000640 1 --2- 172.21.15.27:0/2718288324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1134104350 0x7f113419fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:23.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.563+0000 7f1139600640 1 -- 172.21.15.27:0/2718288324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1134103990 msgr2=0x7f1134198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:23.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.563+0000 7f1139600640 1 --2- 172.21.15.27:0/2718288324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1134103990 0x7f1134198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.563+0000 7f1139600640 1 -- 172.21.15.27:0/2718288324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1134104350 msgr2=0x7f113419fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:23.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.563+0000 7f1139600640 1 --2- 172.21.15.27:0/2718288324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1134104350 0x7f113419fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.563+0000 7f1139600640 1 -- 172.21.15.27:0/2718288324 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11341a0500 con 0x7f11341027a0 2024-04-26T22:58:23.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.563+0000 7f113a000640 1 --2- 172.21.15.27:0/2718288324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1134104350 0x7f113419fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:58:23.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.563+0000 7f1139600640 1 --2- 172.21.15.27:0/2718288324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11341027a0 0x7f11341988e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f112c00ebc0 tx=0x7f112c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:23.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.563+0000 7f1132000640 1 -- 172.21.15.27:0/2718288324 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c003b70 con 0x7f11341027a0 2024-04-26T22:58:23.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.564+0000 7f1132000640 1 -- 172.21.15.27:0/2718288324 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f112c003d10 con 0x7f11341027a0 2024-04-26T22:58:23.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.564+0000 7f113aa00640 1 -- 172.21.15.27:0/2718288324 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11341a07f0 con 0x7f11341027a0 2024-04-26T22:58:23.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.564+0000 7f1132000640 1 -- 172.21.15.27:0/2718288324 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c019800 con 0x7f11341027a0 2024-04-26T22:58:23.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.564+0000 7f113aa00640 1 -- 172.21.15.27:0/2718288324 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11341a0d10 con 0x7f11341027a0 2024-04-26T22:58:23.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.565+0000 7f113aa00640 1 -- 172.21.15.27:0/2718288324 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11341099f0 con 0x7f11341027a0 2024-04-26T22:58:23.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.567+0000 7f1132000640 1 -- 172.21.15.27:0/2718288324 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f112c0199a0 con 0x7f11341027a0 2024-04-26T22:58:23.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.568+0000 7f1132000640 1 --2- 172.21.15.27:0/2718288324 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f11180a3cf0 0x7f11180a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:23.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.569+0000 7f1132000640 1 -- 172.21.15.27:0/2718288324 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f112c0d1350 con 0x7f11341027a0 2024-04-26T22:58:23.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.569+0000 7f1138c00640 1 --2- 172.21.15.27:0/2718288324 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f11180a3cf0 0x7f11180a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:23.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.569+0000 7f1138c00640 1 --2- 172.21.15.27:0/2718288324 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f11180a3cf0 0x7f11180a61b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1120031500 tx=0x7f1120005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:23.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.572+0000 7f1132000640 1 -- 172.21.15.27:0/2718288324 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f112c09d740 con 0x7f11341027a0 2024-04-26T22:58:23.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.791+0000 7f113aa00640 1 -- 172.21.15.27:0/2718288324 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f113402dce0 con 0x7f11341027a0 2024-04-26T22:58:23.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.793+0000 7f1132000640 1 -- 172.21.15.27:0/2718288324 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24902 (secure 0 0 0) 0x7f112c09d040 con 0x7f11341027a0 2024-04-26T22:58:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T22:58:23.794 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":94,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T22:57:05.902371+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4},{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:58:23.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.796+0000 7f113aa00640 1 -- 172.21.15.27:0/2718288324 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f11180a3cf0 msgr2=0x7f11180a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:23.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.796+0000 7f113aa00640 1 --2- 172.21.15.27:0/2718288324 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f11180a3cf0 0x7f11180a61b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1120031500 tx=0x7f1120005a20 comp rx=0 tx=0).stop 2024-04-26T22:58:23.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.796+0000 7f113aa00640 1 -- 172.21.15.27:0/2718288324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11341027a0 msgr2=0x7f11341988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:23.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.796+0000 7f113aa00640 1 --2- 172.21.15.27:0/2718288324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11341027a0 0x7f11341988e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f112c00ebc0 tx=0x7f112c00c6a0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.796+0000 7f113aa00640 1 -- 172.21.15.27:0/2718288324 shutdown_connections 2024-04-26T22:58:23.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.796+0000 7f113aa00640 1 --2- 172.21.15.27:0/2718288324 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f11180a3cf0 0x7f11180a61b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.796+0000 7f113aa00640 1 --2- 172.21.15.27:0/2718288324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1134104350 0x7f113419fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.796+0000 7f113aa00640 1 --2- 172.21.15.27:0/2718288324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1134103990 0x7f1134198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.796+0000 7f113aa00640 1 --2- 172.21.15.27:0/2718288324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11341027a0 0x7f11341988e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:23.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.796+0000 7f113aa00640 1 -- 172.21.15.27:0/2718288324 >> 172.21.15.27:0/2718288324 conn(0x7f11340fdd10 msgr2=0x7f1134106bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:23.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.796+0000 7f113aa00640 1 -- 172.21.15.27:0/2718288324 shutdown_connections 2024-04-26T22:58:23.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:23.797+0000 7f113aa00640 1 -- 172.21.15.27:0/2718288324 wait complete. 2024-04-26T22:58:23.819 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T22:55:25.265331+0000', 'modified': '2024-04-26T22:55:54.648644+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24578, 'mds_1': 24577, 'mds_2': 24560}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.80:6837/3963089995', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6836', 'nonce': 3963089995}, {'type': 'v1', 'addr': '172.21.15.80:6837', 'nonce': 3963089995}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6837/3367319222', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6836', 'nonce': 3367319222}, {'type': 'v1', 'addr': '172.21.15.94:6837', 'nonce': 3367319222}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24559': {'gid': 24559, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6839/1620677319', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6838', 'nonce': 1620677319}, {'type': 'v1', 'addr': '172.21.15.94:6839', 'nonce': 1620677319}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24560': {'gid': 24560, 'name': 'a', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.27:6841/1571355196', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6840', 'nonce': 1571355196}, {'type': 'v1', 'addr': '172.21.15.27:6841', 'nonce': 1571355196}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24577': {'gid': 24577, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.27:6839/3193530521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6838', 'nonce': 3193530521}, {'type': 'v1', 'addr': '172.21.15.27:6839', 'nonce': 3193530521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6839/95952000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6838', 'nonce': 95952000}, {'type': 'v1', 'addr': '172.21.15.80:6839', 'nonce': 95952000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24578, 'qdb_cluster': [24560, 24577, 24578]} 2024-04-26T22:58:23.819 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T22:58:23.819 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T22:58:24.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.009+0000 7faef1a00640 1 -- 172.21.15.27:0/996629294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec106870 msgr2=0x7faeec10d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.009+0000 7faef1a00640 1 --2- 172.21.15.27:0/996629294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec106870 0x7faeec10d100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faee4009ca0 tx=0x7faee402a8c0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.010+0000 7faef1a00640 1 -- 172.21.15.27:0/996629294 shutdown_connections 2024-04-26T22:58:24.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.010+0000 7faef1a00640 1 --2- 172.21.15.27:0/996629294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec106870 0x7faeec10d100 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.010+0000 7faef1a00640 1 --2- 172.21.15.27:0/996629294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeec105eb0 0x7faeec106330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.010+0000 7faef1a00640 1 --2- 172.21.15.27:0/996629294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeec104cc0 0x7faeec1050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.010+0000 7faef1a00640 1 -- 172.21.15.27:0/996629294 >> 172.21.15.27:0/996629294 conn(0x7faeec100230 msgr2=0x7faeec102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.010+0000 7faef1a00640 1 -- 172.21.15.27:0/996629294 shutdown_connections 2024-04-26T22:58:24.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.010+0000 7faef1a00640 1 -- 172.21.15.27:0/996629294 wait complete. 2024-04-26T22:58:24.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.011+0000 7faef1a00640 1 Processor -- start 2024-04-26T22:58:24.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.011+0000 7faef1a00640 1 -- start start 2024-04-26T22:58:24.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.011+0000 7faef1a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec104cc0 0x7faeec19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.011+0000 7faef1a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeec105eb0 0x7faeec19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.011+0000 7faef1a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeec106870 0x7faeec1a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.011+0000 7faef1a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faeec1107d0 con 0x7faeec106870 2024-04-26T22:58:24.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.011+0000 7faef1a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7faeec110650 con 0x7faeec104cc0 2024-04-26T22:58:24.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.011+0000 7faef1a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faeec110950 con 0x7faeec105eb0 2024-04-26T22:58:24.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.011+0000 7faeeaa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec104cc0 0x7faeec19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.011+0000 7faeeaa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec104cc0 0x7faeec19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33298/0 (socket says 172.21.15.27:33298) 2024-04-26T22:58:24.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.011+0000 7faeeaa00640 1 -- 172.21.15.27:0/2925037160 learned_addr learned my addr 172.21.15.27:0/2925037160 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:24.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.011+0000 7faeeb400640 1 --2- 172.21.15.27:0/2925037160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeec106870 0x7faeec1a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.011+0000 7faeea000640 1 --2- 172.21.15.27:0/2925037160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeec105eb0 0x7faeec19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.012+0000 7faeeaa00640 1 -- 172.21.15.27:0/2925037160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeec105eb0 msgr2=0x7faeec19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.012+0000 7faeeaa00640 1 --2- 172.21.15.27:0/2925037160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeec105eb0 0x7faeec19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.012+0000 7faeeaa00640 1 -- 172.21.15.27:0/2925037160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeec106870 msgr2=0x7faeec1a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.012+0000 7faeeaa00640 1 --2- 172.21.15.27:0/2925037160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeec106870 0x7faeec1a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.012+0000 7faeeaa00640 1 -- 172.21.15.27:0/2925037160 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeec1a2a10 con 0x7faeec104cc0 2024-04-26T22:58:24.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.012+0000 7faeeaa00640 1 --2- 172.21.15.27:0/2925037160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec104cc0 0x7faeec19adf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faedc00e9c0 tx=0x7faedc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.012+0000 7faee3400640 1 -- 172.21.15.27:0/2925037160 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc00cd80 con 0x7faeec104cc0 2024-04-26T22:58:24.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.012+0000 7faee3400640 1 -- 172.21.15.27:0/2925037160 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faedc0035f0 con 0x7faeec104cc0 2024-04-26T22:58:24.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.012+0000 7faee3400640 1 -- 172.21.15.27:0/2925037160 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc010690 con 0x7faeec104cc0 2024-04-26T22:58:24.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.012+0000 7faef1a00640 1 -- 172.21.15.27:0/2925037160 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeec1a2d00 con 0x7faeec104cc0 2024-04-26T22:58:24.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.013+0000 7faef1a00640 1 -- 172.21.15.27:0/2925037160 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeec1a31a0 con 0x7faeec104cc0 2024-04-26T22:58:24.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.017+0000 7faef47d2740 1 Processor -- start 2024-04-26T22:58:24.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.017+0000 7faef47d2740 1 -- start start 2024-04-26T22:58:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.017+0000 7faef47d2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 0x5618c94e2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.017+0000 7faef47d2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e2ae0 0x5618c94e75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.017+0000 7faef47d2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e7c50 0x5618c94ea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.017+0000 7faef47d2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5618c9446600 con 0x5618c94e2ae0 2024-04-26T22:58:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.017+0000 7faef47d2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5618c93e3100 con 0x5618c94e20f0 2024-04-26T22:58:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.017+0000 7faef47d2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5618c94425c0 con 0x5618c94e7c50 2024-04-26T22:58:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.018+0000 7faee3400640 1 -- 172.21.15.27:0/2925037160 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faedc0037a0 con 0x7faeec104cc0 2024-04-26T22:58:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.018+0000 7faeeb400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 0x5618c94e2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.018+0000 7faeeb400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 0x5618c94e2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33304/0 (socket says 172.21.15.27:33304) 2024-04-26T22:58:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.018+0000 7faeeb400640 1 -- 172.21.15.27:0/630667616 learned_addr learned my addr 172.21.15.27:0/630667616 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.018+0000 7faeea000640 1 --2- 172.21.15.27:0/630667616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e7c50 0x5618c94ea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.018+0000 7faeeaa00640 1 --2- 172.21.15.27:0/630667616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e2ae0 0x5618c94e75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.018+0000 7faeeb400640 1 -- 172.21.15.27:0/630667616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e7c50 msgr2=0x5618c94ea0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.018+0000 7faeeb400640 1 --2- 172.21.15.27:0/630667616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e7c50 0x5618c94ea0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.018+0000 7faeeb400640 1 -- 172.21.15.27:0/630667616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e2ae0 msgr2=0x5618c94e75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.018+0000 7faeeb400640 1 --2- 172.21.15.27:0/630667616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e2ae0 0x5618c94e75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.018+0000 7faeeb400640 1 -- 172.21.15.27:0/630667616 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618c91582d0 con 0x5618c94e20f0 2024-04-26T22:58:24.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.018+0000 7faeeaa00640 1 --2- 172.21.15.27:0/630667616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e2ae0 0x5618c94e75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:58:24.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.018+0000 7faeeb400640 1 --2- 172.21.15.27:0/630667616 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 0x5618c94e2510 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faee4009ca0 tx=0x7faee40093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b795901f19fe4f76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.019+0000 7faed7e00640 1 -- 172.21.15.27:0/630667616 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee4031650 con 0x5618c94e20f0 2024-04-26T22:58:24.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.019+0000 7faed7e00640 1 -- 172.21.15.27:0/630667616 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faee4036030 con 0x5618c94e20f0 2024-04-26T22:58:24.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.019+0000 7faed7e00640 1 -- 172.21.15.27:0/630667616 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee4004680 con 0x5618c94e20f0 2024-04-26T22:58:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.020+0000 7faee3400640 1 --2- 172.21.15.27:0/2925037160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faed00a3c40 0x7faed00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.020+0000 7faef47d2740 1 -- 172.21.15.27:0/630667616 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 msgr2=0x5618c94e2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.020+0000 7faef47d2740 1 --2- 172.21.15.27:0/630667616 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 0x5618c94e2510 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faee4009ca0 tx=0x7faee40093c0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.020+0000 7faee3400640 1 -- 172.21.15.27:0/2925037160 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7faedc0d0d20 con 0x7faeec104cc0 2024-04-26T22:58:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.020+0000 7faef47d2740 1 -- 172.21.15.27:0/630667616 shutdown_connections 2024-04-26T22:58:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.020+0000 7faef47d2740 1 --2- 172.21.15.27:0/630667616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e7c50 0x5618c94ea0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.020+0000 7faef47d2740 1 --2- 172.21.15.27:0/630667616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e2ae0 0x5618c94e75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.020+0000 7faef47d2740 1 --2- 172.21.15.27:0/630667616 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 0x5618c94e2510 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.020+0000 7faeeaa00640 1 --2- 172.21.15.27:0/2925037160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faed00a3c40 0x7faed00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.020+0000 7faef47d2740 1 -- 172.21.15.27:0/630667616 >> 172.21.15.27:0/630667616 conn(0x5618c9476d10 msgr2=0x5618c94ddb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.020+0000 7faef47d2740 1 -- 172.21.15.27:0/630667616 shutdown_connections 2024-04-26T22:58:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.020+0000 7faef47d2740 1 -- 172.21.15.27:0/630667616 wait complete. 2024-04-26T22:58:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.020+0000 7faef47d2740 1 -- start start 2024-04-26T22:58:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.021+0000 7faeeaa00640 1 --2- 172.21.15.27:0/2925037160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faed00a3c40 0x7faed00a6100 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x5618c94e3550 tx=0x7faedc0046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.021+0000 7faef47d2740 1 Processor -- start 2024-04-26T22:58:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.021+0000 7faef47d2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 0x5618c94ec380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.021+0000 7faef47d2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e2ae0 0x5618c94ec8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.021+0000 7faef47d2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e7c50 0x5618c94f3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.021+0000 7faef47d2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5618c94eaa20 con 0x5618c94e7c50 2024-04-26T22:58:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.021+0000 7faef47d2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5618c93cdf70 con 0x5618c94e20f0 2024-04-26T22:58:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.021+0000 7faef47d2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5618c94eaba0 con 0x5618c94e2ae0 2024-04-26T22:58:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.021+0000 7faef47d2740 10 client.0 _ll_register_callbacks cb 0x5618c943c480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:58:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.022+0000 7faeea000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e2ae0 0x5618c94ec8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.022+0000 7faeea000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e2ae0 0x5618c94ec8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:44940/0 (socket says 172.21.15.27:44940) 2024-04-26T22:58:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.022+0000 7faeea000640 1 -- 172.21.15.27:0/3595838637 learned_addr learned my addr 172.21.15.27:0/3595838637 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.022+0000 7faeeb400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 0x5618c94ec380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.022+0000 7faeeb400640 1 --2- 172.21.15.27:0/3595838637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e7c50 0x5618c94f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.022+0000 7faeea000640 1 -- 172.21.15.27:0/3595838637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 msgr2=0x5618c94ec380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.022+0000 7faeea000640 1 --2- 172.21.15.27:0/3595838637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 0x5618c94ec380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.022+0000 7faeea000640 1 -- 172.21.15.27:0/3595838637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e7c50 msgr2=0x5618c94f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.022+0000 7faeea000640 1 --2- 172.21.15.27:0/3595838637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e7c50 0x5618c94f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.022+0000 7faeea000640 1 -- 172.21.15.27:0/3595838637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618c92226b0 con 0x5618c94e2ae0 2024-04-26T22:58:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.022+0000 7faeeb400640 1 --2- 172.21.15.27:0/3595838637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e7c50 0x5618c94f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:58:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.022+0000 7faeea000640 1 --2- 172.21.15.27:0/3595838637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e2ae0 0x5618c94ec8c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faed8012700 tx=0x7faed8012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c95b95a612109304 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.022+0000 7faed4c00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T22:58:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.022+0000 7faed4c00640 1 -- 172.21.15.27:0/3595838637 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed800f8c0 con 0x5618c94e2ae0 2024-04-26T22:58:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.022+0000 7faed4c00640 1 -- 172.21.15.27:0/3595838637 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7faed800fa60 con 0x5618c94e2ae0 2024-04-26T22:58:24.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.023+0000 7faef47d2740 1 -- 172.21.15.27:0/3595838637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faee4036030 con 0x5618c94e2ae0 2024-04-26T22:58:24.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.023+0000 7faed4c00640 1 -- 172.21.15.27:0/3595838637 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faed800fca0 con 0x5618c94e2ae0 2024-04-26T22:58:24.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.023+0000 7faef47d2740 10 client.25354 fetch_fsmap learned FSMap version 24 2024-04-26T22:58:24.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.023+0000 7faef47d2740 1 -- 172.21.15.27:0/3595838637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5618c94f6970 con 0x5618c94e2ae0 2024-04-26T22:58:24.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.023+0000 7faed4c00640 1 -- 172.21.15.27:0/3595838637 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faee4036030 con 0x5618c94e2ae0 2024-04-26T22:58:24.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.023+0000 7faed4c00640 1 -- 172.21.15.27:0/3595838637 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7faed8010430 con 0x5618c94e2ae0 2024-04-26T22:58:24.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.023+0000 7faef47d2740 10 client.25354 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T22:58:24.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.024+0000 7faef47d2740 10 client.25354 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T22:58:24.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.024+0000 7faef47d2740 20 client.25354 populate_metadata read hostname 'smithi027' 2024-04-26T22:58:24.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.024+0000 7faef47d2740 1 --2- 172.21.15.27:0/3595838637 >> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] conn(0x5618c94f9000 0x5618c94fa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.024+0000 7faef47d2740 4 client.25354 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:58:24.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.024+0000 7faef47d2740 1 -- 172.21.15.27:0/3595838637 --> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618c9454210 con 0x5618c94f9000 2024-04-26T22:58:24.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.024+0000 7faeeb400640 1 --2- 172.21.15.27:0/3595838637 >> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] conn(0x5618c94f9000 0x5618c94fa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.024+0000 7faeeb400640 1 --2- 172.21.15.27:0/3595838637 >> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] conn(0x5618c94f9000 0x5618c94fa470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=287df9cbab3e0428 server_cookie=cdde9aa3eb7e0311 in_seq=0 out_seq=0 2024-04-26T22:58:24.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.024+0000 7faed4c00640 10 client.25354 ms_handle_connect on v2:172.21.15.27:6840/1571355196 2024-04-26T22:58:24.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.025+0000 7faed4c00640 1 -- 172.21.15.27:0/3595838637 <== mds.2 v2:172.21.15.27:6840/1571355196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618c9454210 con 0x5618c94f9000 2024-04-26T22:58:24.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.025+0000 7faed4c00640 10 client.25354 handle_command_reply: tid=0 2024-04-26T22:58:24.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.025+0000 7faef47d2740 1 client.25354 shutdown 2024-04-26T22:58:24.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.025+0000 7faef47d2740 1 -- 172.21.15.27:0/3595838637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e2ae0 msgr2=0x5618c94ec8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.025+0000 7faef47d2740 1 --2- 172.21.15.27:0/3595838637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e2ae0 0x5618c94ec8c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faed8012700 tx=0x7faed8012bc0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.026+0000 7faef47d2740 1 -- 172.21.15.27:0/3595838637 shutdown_connections 2024-04-26T22:58:24.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.026+0000 7faef47d2740 1 --2- 172.21.15.27:0/3595838637 >> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] conn(0x5618c94f9000 0x5618c94fa470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.026+0000 7faef47d2740 1 --2- 172.21.15.27:0/3595838637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e7c50 0x5618c94f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.026+0000 7faef47d2740 1 --2- 172.21.15.27:0/3595838637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e2ae0 0x5618c94ec8c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.026+0000 7faef47d2740 1 --2- 172.21.15.27:0/3595838637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 0x5618c94ec380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.026+0000 7faed4c00640 0 client.25354 ms_handle_reset on v2:172.21.15.27:6840/1571355196 2024-04-26T22:58:24.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.026+0000 7faef47d2740 1 -- 172.21.15.27:0/3595838637 >> 172.21.15.27:0/3595838637 conn(0x5618c9476d10 msgr2=0x5618c94dceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.026+0000 7faef47d2740 1 -- 172.21.15.27:0/3595838637 shutdown_connections 2024-04-26T22:58:24.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.026+0000 7faef47d2740 1 -- 172.21.15.27:0/3595838637 wait complete. 2024-04-26T22:58:24.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.026+0000 7faef47d2740 20 client.25354 trim_cache size 0 max 16384 2024-04-26T22:58:24.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.045+0000 7faef47d2740 1 Processor -- start 2024-04-26T22:58:24.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.045+0000 7faef47d2740 1 -- start start 2024-04-26T22:58:24.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.045+0000 7faef47d2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 0x5618c94ec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.046+0000 7faef47d2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e2ae0 0x5618c94ec890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.046+0000 7faef47d2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e7c50 0x5618c94f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.046+0000 7faef47d2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5618c94f87b0 con 0x5618c94e7c50 2024-04-26T22:58:24.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.046+0000 7faef47d2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5618c94f8e80 con 0x5618c94e20f0 2024-04-26T22:58:24.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.046+0000 7faef47d2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5618c94f9550 con 0x5618c94e2ae0 2024-04-26T22:58:24.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.046+0000 7faeeb400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e7c50 0x5618c94f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.046+0000 7faeeb400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 0x5618c94ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.046+0000 7faeea000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e2ae0 0x5618c94ec890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.046+0000 7faeeb400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 0x5618c94ec350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33320/0 (socket says 172.21.15.27:33320) 2024-04-26T22:58:24.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.046+0000 7faeeb400640 1 -- 172.21.15.27:0/3446535517 learned_addr learned my addr 172.21.15.27:0/3446535517 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:24.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.046+0000 7faeeb400640 1 -- 172.21.15.27:0/3446535517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e2ae0 msgr2=0x5618c94ec890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.046+0000 7faeeb400640 1 --2- 172.21.15.27:0/3446535517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e2ae0 0x5618c94ec890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.046+0000 7faeeb400640 1 -- 172.21.15.27:0/3446535517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e7c50 msgr2=0x5618c94f38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.046+0000 7faeeb400640 1 --2- 172.21.15.27:0/3446535517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e7c50 0x5618c94f38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.046+0000 7faeeb400640 1 -- 172.21.15.27:0/3446535517 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618c94e2ec0 con 0x5618c94e20f0 2024-04-26T22:58:24.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.047+0000 7faeeb400640 1 --2- 172.21.15.27:0/3446535517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 0x5618c94ec350 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faee4057fd0 tx=0x7faee4031af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a573fdf1ae7cb336 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.047+0000 7faed7400640 1 -- 172.21.15.27:0/3446535517 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee404a040 con 0x5618c94e20f0 2024-04-26T22:58:24.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.047+0000 7faed7400640 1 -- 172.21.15.27:0/3446535517 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faee4036040 con 0x5618c94e20f0 2024-04-26T22:58:24.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.047+0000 7faed7400640 1 -- 172.21.15.27:0/3446535517 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee4004790 con 0x5618c94e20f0 2024-04-26T22:58:24.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.047+0000 7faef47d2740 1 -- 172.21.15.27:0/3446535517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 msgr2=0x5618c94ec350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.047+0000 7faef47d2740 1 --2- 172.21.15.27:0/3446535517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 0x5618c94ec350 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faee4057fd0 tx=0x7faee4031af0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.048+0000 7faef47d2740 1 -- 172.21.15.27:0/3446535517 shutdown_connections 2024-04-26T22:58:24.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.048+0000 7faef47d2740 1 --2- 172.21.15.27:0/3446535517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e7c50 0x5618c94f38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.048+0000 7faef47d2740 1 --2- 172.21.15.27:0/3446535517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e2ae0 0x5618c94ec890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.048+0000 7faef47d2740 1 --2- 172.21.15.27:0/3446535517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e20f0 0x5618c94ec350 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.048+0000 7faef47d2740 1 -- 172.21.15.27:0/3446535517 >> 172.21.15.27:0/3446535517 conn(0x5618c9476d10 msgr2=0x5618c94dd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.048+0000 7faef47d2740 1 -- 172.21.15.27:0/3446535517 shutdown_connections 2024-04-26T22:58:24.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.048+0000 7faef47d2740 1 -- 172.21.15.27:0/3446535517 wait complete. 2024-04-26T22:58:24.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.048+0000 7faef47d2740 1 -- start start 2024-04-26T22:58:24.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.048+0000 7faef47d2740 1 Processor -- start 2024-04-26T22:58:24.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faef47d2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e20f0 0x5618c94efb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faef47d2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e2ae0 0x5618c94f0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faef47d2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e7c50 0x5618c94f2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faef47d2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5618c94fa2f0 con 0x5618c94e20f0 2024-04-26T22:58:24.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faef47d2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5618c94f9c20 con 0x5618c94e2ae0 2024-04-26T22:58:24.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faef47d2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5618c94fa9c0 con 0x5618c94e7c50 2024-04-26T22:58:24.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faef47d2740 10 client.0 _ll_register_callbacks cb 0x7faed8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:58:24.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faeea000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e2ae0 0x5618c94f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faeea000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e2ae0 0x5618c94f0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33328/0 (socket says 172.21.15.27:33328) 2024-04-26T22:58:24.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faeeb400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e20f0 0x5618c94efb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faeea000640 1 -- 172.21.15.27:0/1004489502 learned_addr learned my addr 172.21.15.27:0/1004489502 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:24.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faeea000640 1 -- 172.21.15.27:0/1004489502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e7c50 msgr2=0x5618c94f2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faeeb400640 1 --2- 172.21.15.27:0/1004489502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e7c50 0x5618c94f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faeea000640 1 --2- 172.21.15.27:0/1004489502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e7c50 0x5618c94f2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faeea000640 1 -- 172.21.15.27:0/1004489502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e20f0 msgr2=0x5618c94efb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faeea000640 1 --2- 172.21.15.27:0/1004489502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e20f0 0x5618c94efb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.049+0000 7faeea000640 1 -- 172.21.15.27:0/1004489502 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618c94f2f90 con 0x5618c94e2ae0 2024-04-26T22:58:24.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.050+0000 7faeea000640 1 --2- 172.21.15.27:0/1004489502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e2ae0 0x5618c94f0050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faed8000c00 tx=0x7faed800fa40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94290098cf2adabf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.050+0000 7faed4c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T22:58:24.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.050+0000 7faed4c00640 1 -- 172.21.15.27:0/1004489502 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed800fe40 con 0x5618c94e2ae0 2024-04-26T22:58:24.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.050+0000 7faed4c00640 1 -- 172.21.15.27:0/1004489502 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7faed8010cd0 con 0x5618c94e2ae0 2024-04-26T22:58:24.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.050+0000 7faef47d2740 1 -- 172.21.15.27:0/1004489502 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faee4036040 con 0x5618c94e2ae0 2024-04-26T22:58:24.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.050+0000 7faed4c00640 1 -- 172.21.15.27:0/1004489502 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faed8005040 con 0x5618c94e2ae0 2024-04-26T22:58:24.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.050+0000 7faef47d2740 10 client.25334 fetch_fsmap learned FSMap version 24 2024-04-26T22:58:24.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.050+0000 7faef47d2740 1 -- 172.21.15.27:0/1004489502 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5618c955f450 con 0x5618c94e2ae0 2024-04-26T22:58:24.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.051+0000 7faed4c00640 1 -- 172.21.15.27:0/1004489502 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faee4036040 con 0x5618c94e2ae0 2024-04-26T22:58:24.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.051+0000 7faed4c00640 1 -- 172.21.15.27:0/1004489502 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7faed80039d0 con 0x5618c94e2ae0 2024-04-26T22:58:24.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.051+0000 7faef47d2740 10 client.25334 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T22:58:24.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.051+0000 7faef47d2740 10 client.25334 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T22:58:24.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.051+0000 7faef47d2740 20 client.25334 populate_metadata read hostname 'smithi027' 2024-04-26T22:58:24.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.051+0000 7faef47d2740 1 --2- 172.21.15.27:0/1004489502 >> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] conn(0x5618c9564060 0x5618c9562d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.051+0000 7faef47d2740 4 client.25334 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T22:58:24.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.051+0000 7faef47d2740 1 -- 172.21.15.27:0/1004489502 --> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] -- command(tid 0: {"prefix": "status"}) -- 0x5618c944e080 con 0x5618c9564060 2024-04-26T22:58:24.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.051+0000 7faeeb400640 1 --2- 172.21.15.27:0/1004489502 >> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] conn(0x5618c9564060 0x5618c9562d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.052+0000 7faeeb400640 1 --2- 172.21.15.27:0/1004489502 >> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] conn(0x5618c9564060 0x5618c9562d70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7adcb292732e5c1e server_cookie=dbf8037b7f7ca652 in_seq=0 out_seq=0 2024-04-26T22:58:24.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.052+0000 7faed4c00640 10 client.25334 ms_handle_connect on v2:172.21.15.27:6840/1571355196 2024-04-26T22:58:24.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.052+0000 7faed4c00640 1 -- 172.21.15.27:0/1004489502 <== mds.2 v2:172.21.15.27:6840/1571355196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5618c944e080 con 0x5618c9564060 2024-04-26T22:58:24.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.052+0000 7faed4c00640 10 client.25334 handle_command_reply: tid=0 2024-04-26T22:58:24.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.052+0000 7faef47d2740 1 client.25334 shutdown 2024-04-26T22:58:24.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.052+0000 7faef47d2740 1 -- 172.21.15.27:0/1004489502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e2ae0 msgr2=0x5618c94f0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.052+0000 7faef47d2740 1 --2- 172.21.15.27:0/1004489502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e2ae0 0x5618c94f0050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faed8000c00 tx=0x7faed800fa40 comp rx=0 tx=0).stop 2024-04-26T22:58:24.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.052+0000 7faef47d2740 1 -- 172.21.15.27:0/1004489502 shutdown_connections 2024-04-26T22:58:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.052+0000 7faef47d2740 1 --2- 172.21.15.27:0/1004489502 >> [v2:172.21.15.27:6840/1571355196,v1:172.21.15.27:6841/1571355196] conn(0x5618c9564060 0x5618c9562d70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.052+0000 7faef47d2740 1 --2- 172.21.15.27:0/1004489502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618c94e2ae0 0x5618c94f0050 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.052+0000 7faed4c00640 0 client.25334 ms_handle_reset on v2:172.21.15.27:6840/1571355196 2024-04-26T22:58:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.052+0000 7faef47d2740 1 --2- 172.21.15.27:0/1004489502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618c94e7c50 0x5618c94f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.052+0000 7faef47d2740 1 --2- 172.21.15.27:0/1004489502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618c94e20f0 0x5618c94efb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.052+0000 7faef47d2740 1 -- 172.21.15.27:0/1004489502 >> 172.21.15.27:0/1004489502 conn(0x5618c9476d10 msgr2=0x5618c94f7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.052+0000 7faef47d2740 1 -- 172.21.15.27:0/1004489502 shutdown_connections 2024-04-26T22:58:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.053+0000 7faef47d2740 1 -- 172.21.15.27:0/1004489502 wait complete. 2024-04-26T22:58:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.053+0000 7faef47d2740 20 client.25334 trim_cache size 0 max 16384 2024-04-26T22:58:24.057 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T22:58:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "95ebdda6-041e-11ef-bc93-c7b262605968", 2024-04-26T22:58:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-04-26T22:58:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24560, 2024-04-26T22:58:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T22:58:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T22:58:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T22:58:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 150.421577662, 2024-04-26T22:58:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 24, 2024-04-26T22:58:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 94, 2024-04-26T22:58:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T22:58:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 188.36938535499999 2024-04-26T22:58:24.058 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T22:58:24.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.053+0000 7faef1a00640 1 -- 172.21.15.27:0/2925037160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faed00a3c40 msgr2=0x7faed00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.053+0000 7faef1a00640 1 --2- 172.21.15.27:0/2925037160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faed00a3c40 0x7faed00a6100 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x5618c94e3550 tx=0x7faedc0046c0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.053+0000 7faef1a00640 1 -- 172.21.15.27:0/2925037160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec104cc0 msgr2=0x7faeec19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.053+0000 7faef1a00640 1 --2- 172.21.15.27:0/2925037160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec104cc0 0x7faeec19adf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faedc00e9c0 tx=0x7faedc00ee80 comp rx=0 tx=0).stop 2024-04-26T22:58:24.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.054+0000 7faef1a00640 1 -- 172.21.15.27:0/2925037160 shutdown_connections 2024-04-26T22:58:24.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.054+0000 7faef1a00640 1 --2- 172.21.15.27:0/2925037160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7faed00a3c40 0x7faed00a6100 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.054+0000 7faef1a00640 1 --2- 172.21.15.27:0/2925037160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeec106870 0x7faeec1a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.054+0000 7faef1a00640 1 --2- 172.21.15.27:0/2925037160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeec105eb0 0x7faeec19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.054+0000 7faef1a00640 1 --2- 172.21.15.27:0/2925037160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec104cc0 0x7faeec19adf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.054+0000 7faef1a00640 1 -- 172.21.15.27:0/2925037160 >> 172.21.15.27:0/2925037160 conn(0x7faeec100230 msgr2=0x7faeec1091b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.054+0000 7faef1a00640 1 -- 172.21.15.27:0/2925037160 shutdown_connections 2024-04-26T22:58:24.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.054+0000 7faef1a00640 1 -- 172.21.15.27:0/2925037160 wait complete. 2024-04-26T22:58:24.076 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:23 smithi027 ceph-mon[28034]: pgmap v423: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:58:24.076 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:23 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2001718838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:58:24.076 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:24.076 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:24.076 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:24.076 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:23 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:24.076 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:23 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1807147895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:58:24.076 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:23 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2718288324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:58:24.079 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-26T22:58:24.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:23 smithi080 ceph-mon[32324]: pgmap v423: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:58:24.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:23 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2001718838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:58:24.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:24.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:24.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:24.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:23 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:24.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:23 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1807147895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:58:24.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:23 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2718288324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:58:24.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.275+0000 7f472be00640 1 -- 172.21.15.27:0/247863397 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f472c104350 msgr2=0x7f472c10abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.275+0000 7f472be00640 1 --2- 172.21.15.27:0/247863397 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f472c104350 0x7f472c10abe0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4724009ca0 tx=0x7f472402a8c0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.275+0000 7f472be00640 1 -- 172.21.15.27:0/247863397 shutdown_connections 2024-04-26T22:58:24.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.275+0000 7f472be00640 1 --2- 172.21.15.27:0/247863397 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f472c104350 0x7f472c10abe0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.275+0000 7f472be00640 1 --2- 172.21.15.27:0/247863397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f472c103990 0x7f472c103e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.275+0000 7f472be00640 1 --2- 172.21.15.27:0/247863397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f472c1027a0 0x7f472c102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.275+0000 7f472be00640 1 -- 172.21.15.27:0/247863397 >> 172.21.15.27:0/247863397 conn(0x7f472c0fdd10 msgr2=0x7f472c100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.275+0000 7f472be00640 1 -- 172.21.15.27:0/247863397 shutdown_connections 2024-04-26T22:58:24.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.275+0000 7f472be00640 1 -- 172.21.15.27:0/247863397 wait complete. 2024-04-26T22:58:24.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.277+0000 7f472be00640 1 Processor -- start 2024-04-26T22:58:24.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.277+0000 7f472be00640 1 -- start start 2024-04-26T22:58:24.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.278+0000 7f472be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f472c1027a0 0x7f472c0766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.278+0000 7f472be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f472c103990 0x7f472c074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.278+0000 7f472be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f472c104350 0x7f472c075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.278+0000 7f472be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10e320 con 0x7f472c1027a0 2024-04-26T22:58:24.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.278+0000 7f472be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10e1a0 con 0x7f472c103990 2024-04-26T22:58:24.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.278+0000 7f472be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10e4a0 con 0x7f472c104350 2024-04-26T22:58:24.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.278+0000 7f472aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f472c1027a0 0x7f472c0766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.278+0000 7f472a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f472c103990 0x7f472c074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.278+0000 7f472a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f472c103990 0x7f472c074d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33354/0 (socket says 172.21.15.27:33354) 2024-04-26T22:58:24.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.278+0000 7f472b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f472c104350 0x7f472c075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.278+0000 7f472a000640 1 -- 172.21.15.27:0/2946489131 learned_addr learned my addr 172.21.15.27:0/2946489131 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:24.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.278+0000 7f472aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f472c1027a0 0x7f472c0766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49218/0 (socket says 172.21.15.27:49218) 2024-04-26T22:58:24.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.279+0000 7f472a000640 1 -- 172.21.15.27:0/2946489131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f472c104350 msgr2=0x7f472c075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.279+0000 7f472a000640 1 --2- 172.21.15.27:0/2946489131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f472c104350 0x7f472c075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.279+0000 7f472a000640 1 -- 172.21.15.27:0/2946489131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f472c1027a0 msgr2=0x7f472c0766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.279+0000 7f472a000640 1 --2- 172.21.15.27:0/2946489131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f472c1027a0 0x7f472c0766e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.279+0000 7f472a000640 1 -- 172.21.15.27:0/2946489131 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c075af0 con 0x7f472c103990 2024-04-26T22:58:24.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.279+0000 7f472a000640 1 --2- 172.21.15.27:0/2946489131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f472c103990 0x7f472c074d20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f471800e9f0 tx=0x7f471800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.279+0000 7f4730a00640 1 -- 172.21.15.27:0/2946489131 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471800cd80 con 0x7f472c103990 2024-04-26T22:58:24.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.279+0000 7f4730a00640 1 -- 172.21.15.27:0/2946489131 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47180108e0 con 0x7f472c103990 2024-04-26T22:58:24.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.279+0000 7f472be00640 1 -- 172.21.15.27:0/2946489131 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472c1a2a90 con 0x7f472c103990 2024-04-26T22:58:24.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.279+0000 7f4730a00640 1 -- 172.21.15.27:0/2946489131 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4718018980 con 0x7f472c103990 2024-04-26T22:58:24.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.279+0000 7f472be00640 1 -- 172.21.15.27:0/2946489131 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472c1a2fe0 con 0x7f472c103990 2024-04-26T22:58:24.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.282+0000 7f4734bcc740 1 Processor -- start 2024-04-26T22:58:24.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.283+0000 7f4734bcc740 1 -- start start 2024-04-26T22:58:24.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.283+0000 7f4734bcc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e8ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.283+0000 7f4730a00640 1 -- 172.21.15.27:0/2946489131 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4718018b20 con 0x7f472c103990 2024-04-26T22:58:24.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.283+0000 7f4734bcc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e9047b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.283+0000 7f4734bcc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e904e20 0x55bf2e9072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.283+0000 7f4734bcc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bf2e863600 con 0x55bf2e8ff2c0 2024-04-26T22:58:24.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.283+0000 7f4734bcc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55bf2e800100 con 0x55bf2e8ffcb0 2024-04-26T22:58:24.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.283+0000 7f4734bcc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bf2e85f5c0 con 0x55bf2e904e20 2024-04-26T22:58:24.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.283+0000 7f472b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e8ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.283+0000 7f472b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e8ff6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49230/0 (socket says 172.21.15.27:49230) 2024-04-26T22:58:24.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.283+0000 7f472b400640 1 -- 172.21.15.27:0/4005958750 learned_addr learned my addr 172.21.15.27:0/4005958750 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:24.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.283+0000 7f472aa00640 1 --2- 172.21.15.27:0/4005958750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e9047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.283+0000 7f472a000640 1 --2- 172.21.15.27:0/4005958750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e904e20 0x55bf2e9072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:23 smithi094 ceph-mon[32889]: pgmap v423: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T22:58:24.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:23 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2001718838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:58:24.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:24.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:24.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:24.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:23 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:24.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:23 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1807147895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:58:24.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:23 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2718288324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:58:24.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.284+0000 7f472b400640 1 -- 172.21.15.27:0/4005958750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e904e20 msgr2=0x55bf2e9072b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.284+0000 7f472b400640 1 --2- 172.21.15.27:0/4005958750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e904e20 0x55bf2e9072b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.284+0000 7f472b400640 1 -- 172.21.15.27:0/4005958750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e8ffcb0 msgr2=0x55bf2e9047b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.284+0000 7f472b400640 1 --2- 172.21.15.27:0/4005958750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e9047b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.284+0000 7f472b400640 1 -- 172.21.15.27:0/4005958750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf2e5752d0 con 0x55bf2e8ff2c0 2024-04-26T22:58:24.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.284+0000 7f472aa00640 1 --2- 172.21.15.27:0/4005958750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e9047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:24.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.284+0000 7f472b400640 1 --2- 172.21.15.27:0/4005958750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e8ff6e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4724009ca0 tx=0x7f47240093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fe3a34e1490720 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.285+0000 7f4717e00640 1 -- 172.21.15.27:0/4005958750 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724031650 con 0x55bf2e8ff2c0 2024-04-26T22:58:24.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.285+0000 7f4717e00640 1 -- 172.21.15.27:0/4005958750 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4724036030 con 0x55bf2e8ff2c0 2024-04-26T22:58:24.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.285+0000 7f4717e00640 1 -- 172.21.15.27:0/4005958750 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724004660 con 0x55bf2e8ff2c0 2024-04-26T22:58:24.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.285+0000 7f4734bcc740 1 -- 172.21.15.27:0/4005958750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 msgr2=0x55bf2e8ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.285+0000 7f4734bcc740 1 --2- 172.21.15.27:0/4005958750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e8ff6e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4724009ca0 tx=0x7f47240093c0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.285+0000 7f4734bcc740 1 -- 172.21.15.27:0/4005958750 shutdown_connections 2024-04-26T22:58:24.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.285+0000 7f4734bcc740 1 --2- 172.21.15.27:0/4005958750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e9047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.286+0000 7f4734bcc740 1 --2- 172.21.15.27:0/4005958750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e904e20 0x55bf2e9072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.286+0000 7f4734bcc740 1 --2- 172.21.15.27:0/4005958750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e8ff6e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.286+0000 7f4734bcc740 1 -- 172.21.15.27:0/4005958750 >> 172.21.15.27:0/4005958750 conn(0x55bf2e893d10 msgr2=0x55bf2e8fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.286+0000 7f4734bcc740 1 -- 172.21.15.27:0/4005958750 shutdown_connections 2024-04-26T22:58:24.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.286+0000 7f4734bcc740 1 -- 172.21.15.27:0/4005958750 wait complete. 2024-04-26T22:58:24.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.286+0000 7f4730a00640 1 --2- 172.21.15.27:0/2946489131 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f47100a3b40 0x7f47100a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.286+0000 7f4734bcc740 1 -- start start 2024-04-26T22:58:24.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.286+0000 7f4730a00640 1 -- 172.21.15.27:0/2946489131 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f47180d0d00 con 0x7f472c103990 2024-04-26T22:58:24.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.286+0000 7f472b400640 1 --2- 172.21.15.27:0/2946489131 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f47100a3b40 0x7f47100a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.286+0000 7f4734bcc740 1 Processor -- start 2024-04-26T22:58:24.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.287+0000 7f4734bcc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e9094c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.287+0000 7f4734bcc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e909a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.287+0000 7f4734bcc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e904e20 0x55bf2e910a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.287+0000 7f4734bcc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bf2e907bf0 con 0x55bf2e8ff2c0 2024-04-26T22:58:24.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.287+0000 7f4734bcc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55bf2e7eaf70 con 0x55bf2e904e20 2024-04-26T22:58:24.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.287+0000 7f4734bcc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bf2e907d70 con 0x55bf2e8ffcb0 2024-04-26T22:58:24.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.287+0000 7f4734bcc740 10 client.0 _ll_register_callbacks cb 0x55bf2e859480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:58:24.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.287+0000 7f472aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e9094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.287+0000 7f472b400640 1 --2- 172.21.15.27:0/2946489131 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f47100a3b40 0x7f47100a6000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f472402aeb0 tx=0x7f472402a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.287+0000 7f472aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e9094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49246/0 (socket says 172.21.15.27:49246) 2024-04-26T22:58:24.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.287+0000 7f472aa00640 1 -- 172.21.15.27:0/3660440227 learned_addr learned my addr 172.21.15.27:0/3660440227 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:24.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.287+0000 7f472b400640 1 --2- 172.21.15.27:0/3660440227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e904e20 0x55bf2e910a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.288+0000 7f472a000640 1 --2- 172.21.15.27:0/3660440227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e909a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.288+0000 7f472b400640 1 -- 172.21.15.27:0/3660440227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e8ffcb0 msgr2=0x55bf2e909a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.288+0000 7f472b400640 1 --2- 172.21.15.27:0/3660440227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e909a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.288+0000 7f472b400640 1 -- 172.21.15.27:0/3660440227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 msgr2=0x55bf2e9094c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.288+0000 7f472b400640 1 --2- 172.21.15.27:0/3660440227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e9094c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.288+0000 7f472b400640 1 -- 172.21.15.27:0/3660440227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf2e63f6b0 con 0x55bf2e904e20 2024-04-26T22:58:24.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.288+0000 7f472aa00640 1 --2- 172.21.15.27:0/3660440227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e9094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:24.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.288+0000 7f472b400640 1 --2- 172.21.15.27:0/3660440227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e904e20 0x55bf2e910a40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f472402ade0 tx=0x7f47240093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55cd666f68cc0957 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.288+0000 7f4714c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T22:58:24.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.288+0000 7f4714c00640 1 -- 172.21.15.27:0/3660440227 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724056040 con 0x55bf2e904e20 2024-04-26T22:58:24.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.289+0000 7f4714c00640 1 -- 172.21.15.27:0/3660440227 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f4724003cf0 con 0x55bf2e904e20 2024-04-26T22:58:24.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.289+0000 7f4734bcc740 1 -- 172.21.15.27:0/3660440227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4724036030 con 0x55bf2e904e20 2024-04-26T22:58:24.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.289+0000 7f4714c00640 1 -- 172.21.15.27:0/3660440227 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4724068050 con 0x55bf2e904e20 2024-04-26T22:58:24.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.289+0000 7f4734bcc740 10 client.25346 fetch_fsmap learned FSMap version 24 2024-04-26T22:58:24.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.289+0000 7f4734bcc740 1 -- 172.21.15.27:0/3660440227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bf2e907bf0 con 0x55bf2e904e20 2024-04-26T22:58:24.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.289+0000 7f4714c00640 1 -- 172.21.15.27:0/3660440227 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4724036030 con 0x55bf2e904e20 2024-04-26T22:58:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.289+0000 7f4714c00640 1 -- 172.21.15.27:0/3660440227 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4724059030 con 0x55bf2e904e20 2024-04-26T22:58:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.290+0000 7f4734bcc740 10 client.25346 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T22:58:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.290+0000 7f4734bcc740 10 client.25346 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-26T22:58:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.290+0000 7f4734bcc740 20 client.25346 populate_metadata read hostname 'smithi027' 2024-04-26T22:58:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.290+0000 7f4734bcc740 1 --2- 172.21.15.27:0/3660440227 >> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] conn(0x55bf2e915e70 0x55bf2e9172e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.290+0000 7f4734bcc740 4 client.25346 mds_command: new command op to 24577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:58:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.290+0000 7f4734bcc740 1 -- 172.21.15.27:0/3660440227 --> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf2e871210 con 0x55bf2e915e70 2024-04-26T22:58:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.290+0000 7f472aa00640 1 --2- 172.21.15.27:0/3660440227 >> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] conn(0x55bf2e915e70 0x55bf2e9172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.291+0000 7f472aa00640 1 --2- 172.21.15.27:0/3660440227 >> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] conn(0x55bf2e915e70 0x55bf2e9172e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43c5354401447dc5 server_cookie=db5c881dc26a242c in_seq=0 out_seq=0 2024-04-26T22:58:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.291+0000 7f4714c00640 10 client.25346 ms_handle_connect on v2:172.21.15.27:6838/3193530521 2024-04-26T22:58:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.291+0000 7f4714c00640 1 -- 172.21.15.27:0/3660440227 <== mds.1 v2:172.21.15.27:6838/3193530521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf2e871210 con 0x55bf2e915e70 2024-04-26T22:58:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.291+0000 7f4714c00640 10 client.25346 handle_command_reply: tid=0 2024-04-26T22:58:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.292+0000 7f4734bcc740 1 client.25346 shutdown 2024-04-26T22:58:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.292+0000 7f4734bcc740 1 -- 172.21.15.27:0/3660440227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e904e20 msgr2=0x55bf2e910a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.292+0000 7f4734bcc740 1 --2- 172.21.15.27:0/3660440227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e904e20 0x55bf2e910a40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f472402ade0 tx=0x7f47240093c0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.292+0000 7f4734bcc740 1 -- 172.21.15.27:0/3660440227 shutdown_connections 2024-04-26T22:58:24.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.292+0000 7f4734bcc740 1 --2- 172.21.15.27:0/3660440227 >> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] conn(0x55bf2e915e70 0x55bf2e9172e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.292+0000 7f4734bcc740 1 --2- 172.21.15.27:0/3660440227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e904e20 0x55bf2e910a40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.292+0000 7f4734bcc740 1 --2- 172.21.15.27:0/3660440227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e909a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.292+0000 7f4714c00640 0 client.25346 ms_handle_reset on v2:172.21.15.27:6838/3193530521 2024-04-26T22:58:24.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.292+0000 7f4734bcc740 1 --2- 172.21.15.27:0/3660440227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e9094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.292+0000 7f4734bcc740 1 -- 172.21.15.27:0/3660440227 >> 172.21.15.27:0/3660440227 conn(0x55bf2e893d10 msgr2=0x55bf2e8f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.292+0000 7f4734bcc740 1 -- 172.21.15.27:0/3660440227 shutdown_connections 2024-04-26T22:58:24.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.292+0000 7f4734bcc740 1 -- 172.21.15.27:0/3660440227 wait complete. 2024-04-26T22:58:24.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.292+0000 7f4734bcc740 20 client.25346 trim_cache size 0 max 16384 2024-04-26T22:58:24.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.313+0000 7f4734bcc740 1 Processor -- start 2024-04-26T22:58:24.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.313+0000 7f4734bcc740 1 -- start start 2024-04-26T22:58:24.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.314+0000 7f4734bcc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e8f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.314+0000 7f4734bcc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e910a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.314+0000 7f4734bcc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e904e20 0x55bf2e901070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.314+0000 7f4734bcc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bf2e915620 con 0x55bf2e8ff2c0 2024-04-26T22:58:24.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.314+0000 7f4734bcc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55bf2e915cf0 con 0x55bf2e904e20 2024-04-26T22:58:24.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.314+0000 7f4734bcc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bf2e9163c0 con 0x55bf2e8ffcb0 2024-04-26T22:58:24.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.314+0000 7f472aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e8f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.314+0000 7f472b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e904e20 0x55bf2e901070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.314+0000 7f472aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e8f5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49260/0 (socket says 172.21.15.27:49260) 2024-04-26T22:58:24.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.314+0000 7f472aa00640 1 -- 172.21.15.27:0/3932428248 learned_addr learned my addr 172.21.15.27:0/3932428248 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:24.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.314+0000 7f472a000640 1 --2- 172.21.15.27:0/3932428248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e910a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.314+0000 7f472b400640 1 -- 172.21.15.27:0/3932428248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e8ffcb0 msgr2=0x55bf2e910a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.314+0000 7f472b400640 1 --2- 172.21.15.27:0/3932428248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e910a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.314+0000 7f472b400640 1 -- 172.21.15.27:0/3932428248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 msgr2=0x55bf2e8f5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.314+0000 7f472b400640 1 --2- 172.21.15.27:0/3932428248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e8f5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.314+0000 7f472b400640 1 -- 172.21.15.27:0/3932428248 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf2e904340 con 0x55bf2e904e20 2024-04-26T22:58:24.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.315+0000 7f472b400640 1 --2- 172.21.15.27:0/3932428248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e904e20 0x55bf2e901070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4724005e30 tx=0x7f4724009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5527365aae27a540 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.315+0000 7f4717400640 1 -- 172.21.15.27:0/3932428248 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472404a040 con 0x55bf2e904e20 2024-04-26T22:58:24.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.315+0000 7f4717400640 1 -- 172.21.15.27:0/3932428248 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4724036030 con 0x55bf2e904e20 2024-04-26T22:58:24.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.315+0000 7f4717400640 1 -- 172.21.15.27:0/3932428248 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472402f610 con 0x55bf2e904e20 2024-04-26T22:58:24.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.316+0000 7f4734bcc740 1 -- 172.21.15.27:0/3932428248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e904e20 msgr2=0x55bf2e901070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.316+0000 7f4734bcc740 1 --2- 172.21.15.27:0/3932428248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e904e20 0x55bf2e901070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4724005e30 tx=0x7f4724009940 comp rx=0 tx=0).stop 2024-04-26T22:58:24.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.316+0000 7f4734bcc740 1 -- 172.21.15.27:0/3932428248 shutdown_connections 2024-04-26T22:58:24.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.316+0000 7f4734bcc740 1 --2- 172.21.15.27:0/3932428248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e904e20 0x55bf2e901070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.316+0000 7f4734bcc740 1 --2- 172.21.15.27:0/3932428248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e910a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.316+0000 7f4734bcc740 1 --2- 172.21.15.27:0/3932428248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e8f5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.316+0000 7f4734bcc740 1 -- 172.21.15.27:0/3932428248 >> 172.21.15.27:0/3932428248 conn(0x55bf2e893d10 msgr2=0x55bf2e900510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.316+0000 7f4734bcc740 1 -- 172.21.15.27:0/3932428248 shutdown_connections 2024-04-26T22:58:24.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.316+0000 7f4734bcc740 1 -- 172.21.15.27:0/3932428248 wait complete. 2024-04-26T22:58:24.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.316+0000 7f4734bcc740 1 -- start start 2024-04-26T22:58:24.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.316+0000 7f4734bcc740 1 Processor -- start 2024-04-26T22:58:24.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.317+0000 7f4734bcc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e90c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.317+0000 7f4734bcc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e90c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.317+0000 7f4734bcc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e904e20 0x55bf2e90f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.317+0000 7f4734bcc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bf2e917160 con 0x55bf2e904e20 2024-04-26T22:58:24.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.317+0000 7f4734bcc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55bf2e916a90 con 0x55bf2e8ff2c0 2024-04-26T22:58:24.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.317+0000 7f4734bcc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bf2e917830 con 0x55bf2e8ffcb0 2024-04-26T22:58:24.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.317+0000 7f4734bcc740 10 client.0 _ll_register_callbacks cb 0x7f471c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:58:24.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.317+0000 7f472b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e904e20 0x55bf2e90f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.317+0000 7f472aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e90c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.317+0000 7f472b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e904e20 0x55bf2e90f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49272/0 (socket says 172.21.15.27:49272) 2024-04-26T22:58:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.317+0000 7f472aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e90c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33400/0 (socket says 172.21.15.27:33400) 2024-04-26T22:58:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.317+0000 7f472b400640 1 -- 172.21.15.27:0/106628530 learned_addr learned my addr 172.21.15.27:0/106628530 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.317+0000 7f472a000640 1 --2- 172.21.15.27:0/106628530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e90c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.318+0000 7f472aa00640 1 -- 172.21.15.27:0/106628530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e8ffcb0 msgr2=0x55bf2e90c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.318+0000 7f472aa00640 1 --2- 172.21.15.27:0/106628530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e90c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.318+0000 7f472aa00640 1 -- 172.21.15.27:0/106628530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e904e20 msgr2=0x55bf2e90f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.318+0000 7f472aa00640 1 --2- 172.21.15.27:0/106628530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e904e20 0x55bf2e90f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.318+0000 7f472aa00640 1 -- 172.21.15.27:0/106628530 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf2e90fc20 con 0x55bf2e8ff2c0 2024-04-26T22:58:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.318+0000 7f472b400640 1 --2- 172.21.15.27:0/106628530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e904e20 0x55bf2e90f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.318+0000 7f472aa00640 1 --2- 172.21.15.27:0/106628530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e90c0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f471c004700 tx=0x7f471c012d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34e00c84ef2c66fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.318+0000 7f4714c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T22:58:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.318+0000 7f4714c00640 1 -- 172.21.15.27:0/106628530 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c005220 con 0x55bf2e8ff2c0 2024-04-26T22:58:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.318+0000 7f4714c00640 1 -- 172.21.15.27:0/106628530 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f471c0053c0 con 0x55bf2e8ff2c0 2024-04-26T22:58:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.318+0000 7f4734bcc740 1 -- 172.21.15.27:0/106628530 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4724036030 con 0x55bf2e8ff2c0 2024-04-26T22:58:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.319+0000 7f4714c00640 1 -- 172.21.15.27:0/106628530 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f471c031050 con 0x55bf2e8ff2c0 2024-04-26T22:58:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.319+0000 7f4734bcc740 10 client.25352 fetch_fsmap learned FSMap version 24 2024-04-26T22:58:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.319+0000 7f4734bcc740 1 -- 172.21.15.27:0/106628530 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bf2e97be20 con 0x55bf2e8ff2c0 2024-04-26T22:58:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.319+0000 7f4714c00640 1 -- 172.21.15.27:0/106628530 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4724036030 con 0x55bf2e8ff2c0 2024-04-26T22:58:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.319+0000 7f4714c00640 1 -- 172.21.15.27:0/106628530 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f471c0049d0 con 0x55bf2e8ff2c0 2024-04-26T22:58:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.319+0000 7f4734bcc740 10 client.25352 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T22:58:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.319+0000 7f4734bcc740 10 client.25352 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-26T22:58:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.319+0000 7f4734bcc740 20 client.25352 populate_metadata read hostname 'smithi027' 2024-04-26T22:58:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.319+0000 7f4734bcc740 1 --2- 172.21.15.27:0/106628530 >> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] conn(0x55bf2e980a00 0x55bf2e97f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.319+0000 7f4734bcc740 4 client.25352 mds_command: new command op to 24577 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T22:58:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.319+0000 7f4734bcc740 1 -- 172.21.15.27:0/106628530 --> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] -- command(tid 0: {"prefix": "status"}) -- 0x55bf2e86b080 con 0x55bf2e980a00 2024-04-26T22:58:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.320+0000 7f472a000640 1 --2- 172.21.15.27:0/106628530 >> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] conn(0x55bf2e980a00 0x55bf2e97f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.320+0000 7f4714c00640 10 client.25352 ms_handle_connect on v2:172.21.15.27:6838/3193530521 2024-04-26T22:58:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.320+0000 7f472a000640 1 --2- 172.21.15.27:0/106628530 >> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] conn(0x55bf2e980a00 0x55bf2e97f710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2d49a1a7c52dfc9 server_cookie=73ac1645ff260121 in_seq=0 out_seq=0 2024-04-26T22:58:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.320+0000 7f4714c00640 1 -- 172.21.15.27:0/106628530 <== mds.1 v2:172.21.15.27:6838/3193530521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bf2e86b080 con 0x55bf2e980a00 2024-04-26T22:58:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.320+0000 7f4714c00640 10 client.25352 handle_command_reply: tid=0 2024-04-26T22:58:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.320+0000 7f4734bcc740 1 client.25352 shutdown 2024-04-26T22:58:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.321+0000 7f4734bcc740 1 -- 172.21.15.27:0/106628530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e8ff2c0 msgr2=0x55bf2e90c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.321+0000 7f4734bcc740 1 --2- 172.21.15.27:0/106628530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e90c0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f471c004700 tx=0x7f471c012d30 comp rx=0 tx=0).stop 2024-04-26T22:58:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.321+0000 7f4734bcc740 1 -- 172.21.15.27:0/106628530 shutdown_connections 2024-04-26T22:58:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.321+0000 7f4734bcc740 1 --2- 172.21.15.27:0/106628530 >> [v2:172.21.15.27:6838/3193530521,v1:172.21.15.27:6839/3193530521] conn(0x55bf2e980a00 0x55bf2e97f710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.321+0000 7f4734bcc740 1 --2- 172.21.15.27:0/106628530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf2e904e20 0x55bf2e90f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.321+0000 7f4734bcc740 1 --2- 172.21.15.27:0/106628530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf2e8ffcb0 0x55bf2e90c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.321+0000 7f4714c00640 0 client.25352 ms_handle_reset on v2:172.21.15.27:6838/3193530521 2024-04-26T22:58:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.321+0000 7f4734bcc740 1 --2- 172.21.15.27:0/106628530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bf2e8ff2c0 0x55bf2e90c0d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.321+0000 7f4734bcc740 1 -- 172.21.15.27:0/106628530 >> 172.21.15.27:0/106628530 conn(0x55bf2e893d10 msgr2=0x55bf2e8f5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.321+0000 7f4734bcc740 1 -- 172.21.15.27:0/106628530 shutdown_connections 2024-04-26T22:58:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.321+0000 7f4734bcc740 1 -- 172.21.15.27:0/106628530 wait complete. 2024-04-26T22:58:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.321+0000 7f4734bcc740 20 client.25352 trim_cache size 0 max 16384 2024-04-26T22:58:24.326 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T22:58:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "95ebdda6-041e-11ef-bc93-c7b262605968", 2024-04-26T22:58:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-04-26T22:58:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24577, 2024-04-26T22:58:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T22:58:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T22:58:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T22:58:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 151.71388951099999, 2024-04-26T22:58:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 24, 2024-04-26T22:58:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 94, 2024-04-26T22:58:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T22:58:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 193.05063128399999 2024-04-26T22:58:24.327 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T22:58:24.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.322+0000 7f472be00640 1 -- 172.21.15.27:0/2946489131 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f47100a3b40 msgr2=0x7f47100a6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.322+0000 7f472be00640 1 --2- 172.21.15.27:0/2946489131 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f47100a3b40 0x7f47100a6000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f472402aeb0 tx=0x7f472402a8c0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.322+0000 7f472be00640 1 -- 172.21.15.27:0/2946489131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f472c103990 msgr2=0x7f472c074d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.322+0000 7f472be00640 1 --2- 172.21.15.27:0/2946489131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f472c103990 0x7f472c074d20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f471800e9f0 tx=0x7f471800eeb0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.322+0000 7f472be00640 1 -- 172.21.15.27:0/2946489131 shutdown_connections 2024-04-26T22:58:24.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.322+0000 7f472be00640 1 --2- 172.21.15.27:0/2946489131 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f47100a3b40 0x7f47100a6000 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.322+0000 7f472be00640 1 --2- 172.21.15.27:0/2946489131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f472c103990 0x7f472c074d20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.322+0000 7f472be00640 1 --2- 172.21.15.27:0/2946489131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f472c104350 0x7f472c075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.322+0000 7f472be00640 1 --2- 172.21.15.27:0/2946489131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f472c1027a0 0x7f472c0766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.322+0000 7f472be00640 1 -- 172.21.15.27:0/2946489131 >> 172.21.15.27:0/2946489131 conn(0x7f472c0fdd10 msgr2=0x7f472c106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.322+0000 7f472be00640 1 -- 172.21.15.27:0/2946489131 shutdown_connections 2024-04-26T22:58:24.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.322+0000 7f472be00640 1 -- 172.21.15.27:0/2946489131 wait complete. 2024-04-26T22:58:24.341 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T22:58:24.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.531+0000 7f2c07c00640 1 -- 172.21.15.27:0/1548066714 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c00101c60 msgr2=0x7f2c001020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.531+0000 7f2c07c00640 1 --2- 172.21.15.27:0/1548066714 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c00101c60 0x7f2c001020c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009860 tx=0x7f2bec02a340 comp rx=0 tx=0).stop 2024-04-26T22:58:24.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.532+0000 7f2c07c00640 1 -- 172.21.15.27:0/1548066714 shutdown_connections 2024-04-26T22:58:24.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.532+0000 7f2c07c00640 1 --2- 172.21.15.27:0/1548066714 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c00101c60 0x7f2c001020c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.532+0000 7f2c07c00640 1 --2- 172.21.15.27:0/1548066714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c00102600 0x7f2c00102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.532+0000 7f2c07c00640 1 --2- 172.21.15.27:0/1548066714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c00101300 0x7f2c00101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.532+0000 7f2c07c00640 1 -- 172.21.15.27:0/1548066714 >> 172.21.15.27:0/1548066714 conn(0x7f2c000fcf20 msgr2=0x7f2c000ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.532+0000 7f2c07c00640 1 -- 172.21.15.27:0/1548066714 shutdown_connections 2024-04-26T22:58:24.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.532+0000 7f2c07c00640 1 -- 172.21.15.27:0/1548066714 wait complete. 2024-04-26T22:58:24.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.533+0000 7f2c07c00640 1 Processor -- start 2024-04-26T22:58:24.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.533+0000 7f2c07c00640 1 -- start start 2024-04-26T22:58:24.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.533+0000 7f2c07c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c00101300 0x7f2c0019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.533+0000 7f2c07c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c00101c60 0x7f2c0019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.533+0000 7f2c07c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c00102600 0x7f2c001a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.533+0000 7f2c07c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00110710 con 0x7f2c00102600 2024-04-26T22:58:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.533+0000 7f2c07c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00110590 con 0x7f2c00101300 2024-04-26T22:58:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.533+0000 7f2c07c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00110890 con 0x7f2c00101c60 2024-04-26T22:58:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.533+0000 7f2c05200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c00101300 0x7f2c0019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.534+0000 7f2c05200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c00101300 0x7f2c0019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33404/0 (socket says 172.21.15.27:33404) 2024-04-26T22:58:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.534+0000 7f2c05200640 1 -- 172.21.15.27:0/3506630011 learned_addr learned my addr 172.21.15.27:0/3506630011 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.534+0000 7f2c05c00640 1 --2- 172.21.15.27:0/3506630011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c00102600 0x7f2c001a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.534+0000 7f2bffe00640 1 --2- 172.21.15.27:0/3506630011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c00101c60 0x7f2c0019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.534+0000 7f2c05200640 1 -- 172.21.15.27:0/3506630011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c00101c60 msgr2=0x7f2c0019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.534+0000 7f2c05200640 1 --2- 172.21.15.27:0/3506630011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c00101c60 0x7f2c0019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.534+0000 7f2c05200640 1 -- 172.21.15.27:0/3506630011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c00102600 msgr2=0x7f2c001a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.534+0000 7f2c05200640 1 --2- 172.21.15.27:0/3506630011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c00102600 0x7f2c001a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.534+0000 7f2c05200640 1 -- 172.21.15.27:0/3506630011 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c001a2a30 con 0x7f2c00101300 2024-04-26T22:58:24.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.534+0000 7f2c05200640 1 --2- 172.21.15.27:0/3506630011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c00101300 0x7f2c0019ae10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400e9f0 tx=0x7f2bf400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.535+0000 7f2bfd600640 1 -- 172.21.15.27:0/3506630011 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf400cd80 con 0x7f2c00101300 2024-04-26T22:58:24.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.535+0000 7f2bfd600640 1 -- 172.21.15.27:0/3506630011 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bf40108e0 con 0x7f2c00101300 2024-04-26T22:58:24.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.535+0000 7f2bfd600640 1 -- 172.21.15.27:0/3506630011 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4018980 con 0x7f2c00101300 2024-04-26T22:58:24.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.535+0000 7f2c07c00640 1 -- 172.21.15.27:0/3506630011 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c001a2d20 con 0x7f2c00101300 2024-04-26T22:58:24.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.535+0000 7f2c07c00640 1 -- 172.21.15.27:0/3506630011 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c001a31d0 con 0x7f2c00101300 2024-04-26T22:58:24.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.538+0000 7f2bfd600640 1 -- 172.21.15.27:0/3506630011 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2bf4010430 con 0x7f2c00101300 2024-04-26T22:58:24.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.538+0000 7f2c0ab9a740 1 Processor -- start 2024-04-26T22:58:24.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.539+0000 7f2c0ab9a740 1 -- start start 2024-04-26T22:58:24.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.539+0000 7f2c0ab9a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c800f0 0x562251c80510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.539+0000 7f2c0ab9a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c80ae0 0x562251c855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.539+0000 7f2c0ab9a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c85c50 0x562251c880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.539+0000 7f2c0ab9a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562251be4600 con 0x562251c800f0 2024-04-26T22:58:24.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.539+0000 7f2c0ab9a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x562251b81100 con 0x562251c80ae0 2024-04-26T22:58:24.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.539+0000 7f2c0ab9a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562251be05c0 con 0x562251c85c50 2024-04-26T22:58:24.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.539+0000 7f2c05c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c800f0 0x562251c80510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.539+0000 7f2c05200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c80ae0 0x562251c855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.539+0000 7f2c05c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c800f0 0x562251c80510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49294/0 (socket says 172.21.15.27:49294) 2024-04-26T22:58:24.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.539+0000 7f2c05c00640 1 -- 172.21.15.27:0/3284895133 learned_addr learned my addr 172.21.15.27:0/3284895133 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:24.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.539+0000 7f2bffe00640 1 --2- 172.21.15.27:0/3284895133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c85c50 0x562251c880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.540+0000 7f2bfd600640 1 --2- 172.21.15.27:0/3506630011 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2be80a3cc0 0x7f2be80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.540+0000 7f2bffe00640 1 -- 172.21.15.27:0/3284895133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c80ae0 msgr2=0x562251c855e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.540+0000 7f2bffe00640 1 --2- 172.21.15.27:0/3284895133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c80ae0 0x562251c855e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.540+0000 7f2bffe00640 1 -- 172.21.15.27:0/3284895133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c800f0 msgr2=0x562251c80510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.540+0000 7f2bffe00640 1 --2- 172.21.15.27:0/3284895133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c800f0 0x562251c80510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.540+0000 7f2bffe00640 1 -- 172.21.15.27:0/3284895133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622518f62d0 con 0x562251c85c50 2024-04-26T22:58:24.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.540+0000 7f2bfd600640 1 -- 172.21.15.27:0/3506630011 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f2bf40d0c80 con 0x7f2c00101300 2024-04-26T22:58:24.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.540+0000 7f2c05c00640 1 --2- 172.21.15.27:0/3506630011 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2be80a3cc0 0x7f2be80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.540+0000 7f2bffe00640 1 --2- 172.21.15.27:0/3284895133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c85c50 0x562251c880e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009830 tx=0x7f2bec005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=395ea95a0606c029 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.541+0000 7f2bf2000640 1 -- 172.21.15.27:0/3284895133 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec037040 con 0x562251c85c50 2024-04-26T22:58:24.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.541+0000 7f2bf2000640 1 -- 172.21.15.27:0/3284895133 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bec004030 con 0x562251c85c50 2024-04-26T22:58:24.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.541+0000 7f2bf2000640 1 -- 172.21.15.27:0/3284895133 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec003a80 con 0x562251c85c50 2024-04-26T22:58:24.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.541+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/3284895133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c85c50 msgr2=0x562251c880e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.541+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/3284895133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c85c50 0x562251c880e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009830 tx=0x7f2bec005b20 comp rx=0 tx=0).stop 2024-04-26T22:58:24.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.541+0000 7f2c05c00640 1 --2- 172.21.15.27:0/3506630011 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2be80a3cc0 0x7f2be80a6180 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009fd0 tx=0x7f2bf8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.541+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/3284895133 shutdown_connections 2024-04-26T22:58:24.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.541+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/3284895133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c80ae0 0x562251c855e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.541+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/3284895133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c85c50 0x562251c880e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.541+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/3284895133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c800f0 0x562251c80510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.541+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/3284895133 >> 172.21.15.27:0/3284895133 conn(0x562251c14d10 msgr2=0x562251c7bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.542+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/3284895133 shutdown_connections 2024-04-26T22:58:24.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.542+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/3284895133 wait complete. 2024-04-26T22:58:24.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.542+0000 7f2c0ab9a740 1 -- start start 2024-04-26T22:58:24.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.542+0000 7f2c0ab9a740 1 Processor -- start 2024-04-26T22:58:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.543+0000 7f2c0ab9a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c800f0 0x562251c8a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.543+0000 7f2c0ab9a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c80ae0 0x562251c8aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.543+0000 7f2c0ab9a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c85c50 0x562251c91aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.543+0000 7f2c0ab9a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562251c88ba0 con 0x562251c800f0 2024-04-26T22:58:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.543+0000 7f2c0ab9a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x562251b6bf70 con 0x562251c80ae0 2024-04-26T22:58:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.543+0000 7f2c0ab9a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562251c88d20 con 0x562251c85c50 2024-04-26T22:58:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.543+0000 7f2c0ab9a740 10 client.0 _ll_register_callbacks cb 0x562251bda480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:58:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.543+0000 7f2c05c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c85c50 0x562251c91aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.543+0000 7f2c05200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c800f0 0x562251c8a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.543+0000 7f2c05c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c85c50 0x562251c91aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:45040/0 (socket says 172.21.15.27:45040) 2024-04-26T22:58:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.544+0000 7f2c05200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c800f0 0x562251c8a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49300/0 (socket says 172.21.15.27:49300) 2024-04-26T22:58:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.544+0000 7f2c05c00640 1 -- 172.21.15.27:0/3894954646 learned_addr learned my addr 172.21.15.27:0/3894954646 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.544+0000 7f2bffe00640 1 --2- 172.21.15.27:0/3894954646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c80ae0 0x562251c8aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.544+0000 7f2c05200640 1 -- 172.21.15.27:0/3894954646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c85c50 msgr2=0x562251c91aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.544+0000 7f2c05200640 1 --2- 172.21.15.27:0/3894954646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c85c50 0x562251c91aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.544+0000 7f2c05200640 1 -- 172.21.15.27:0/3894954646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c80ae0 msgr2=0x562251c8aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.544+0000 7f2c05200640 1 --2- 172.21.15.27:0/3894954646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c80ae0 0x562251c8aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.544+0000 7f2c05200640 1 -- 172.21.15.27:0/3894954646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622519c06b0 con 0x562251c800f0 2024-04-26T22:58:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.544+0000 7f2c05c00640 1 --2- 172.21.15.27:0/3894954646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c85c50 0x562251c91aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.544+0000 7f2bffe00640 1 --2- 172.21.15.27:0/3894954646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c80ae0 0x562251c8aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.544+0000 7f2c05200640 1 --2- 172.21.15.27:0/3894954646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c800f0 0x562251c8a520 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40ce1a0 tx=0x7f2bf4018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9000acb5f8d0276 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.544+0000 7f2be6a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T22:58:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.545+0000 7f2be6a00640 1 -- 172.21.15.27:0/3894954646 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4010720 con 0x562251c800f0 2024-04-26T22:58:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.545+0000 7f2be6a00640 1 -- 172.21.15.27:0/3894954646 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f2bf40bbb30 con 0x562251c800f0 2024-04-26T22:58:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.545+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/3894954646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bec004030 con 0x562251c800f0 2024-04-26T22:58:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.545+0000 7f2be6a00640 1 -- 172.21.15.27:0/3894954646 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2bf409ca20 con 0x562251c800f0 2024-04-26T22:58:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.545+0000 7f2c0ab9a740 10 client.15261 fetch_fsmap learned FSMap version 24 2024-04-26T22:58:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.545+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/3894954646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562251c94b40 con 0x562251c800f0 2024-04-26T22:58:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.545+0000 7f2be6a00640 1 -- 172.21.15.27:0/3894954646 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bec004030 con 0x562251c800f0 2024-04-26T22:58:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.545+0000 7f2be6a00640 1 -- 172.21.15.27:0/3894954646 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2bf40d8040 con 0x562251c800f0 2024-04-26T22:58:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.545+0000 7f2c0ab9a740 10 client.15261 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T22:58:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.545+0000 7f2c0ab9a740 10 client.15261 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T22:58:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.545+0000 7f2c0ab9a740 20 client.15261 populate_metadata read hostname 'smithi027' 2024-04-26T22:58:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.545+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/3894954646 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x562251c970e0 0x562251c98550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.546+0000 7f2c0ab9a740 4 client.15261 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:58:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.546+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/3894954646 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562251bf2210 con 0x562251c970e0 2024-04-26T22:58:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.546+0000 7f2bffe00640 1 --2- 172.21.15.27:0/3894954646 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x562251c970e0 0x562251c98550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.546+0000 7f2bffe00640 1 --2- 172.21.15.27:0/3894954646 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x562251c970e0 0x562251c98550 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7397ce361894d65 server_cookie=510f6ef6543f9080 in_seq=0 out_seq=0 2024-04-26T22:58:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.546+0000 7f2be6a00640 10 client.15261 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:58:24.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.547+0000 7f2be6a00640 1 -- 172.21.15.27:0/3894954646 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562251bf2210 con 0x562251c970e0 2024-04-26T22:58:24.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.547+0000 7f2be6a00640 10 client.15261 handle_command_reply: tid=0 2024-04-26T22:58:24.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.547+0000 7f2c0ab9a740 1 client.15261 shutdown 2024-04-26T22:58:24.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.547+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/3894954646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c800f0 msgr2=0x562251c8a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.547+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/3894954646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c800f0 0x562251c8a520 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40ce1a0 tx=0x7f2bf4018420 comp rx=0 tx=0).stop 2024-04-26T22:58:24.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.547+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/3894954646 shutdown_connections 2024-04-26T22:58:24.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.547+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/3894954646 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x562251c970e0 0x562251c98550 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.547+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/3894954646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c80ae0 0x562251c8aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.547+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/3894954646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c85c50 0x562251c91aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.547+0000 7f2be6a00640 0 client.15261 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:58:24.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.547+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/3894954646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c800f0 0x562251c8a520 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.547+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/3894954646 >> 172.21.15.27:0/3894954646 conn(0x562251c14d10 msgr2=0x562251c7aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.548+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/3894954646 shutdown_connections 2024-04-26T22:58:24.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.548+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/3894954646 wait complete. 2024-04-26T22:58:24.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.548+0000 7f2c0ab9a740 20 client.15261 trim_cache size 0 max 16384 2024-04-26T22:58:24.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.567+0000 7f2c0ab9a740 1 Processor -- start 2024-04-26T22:58:24.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.568+0000 7f2c0ab9a740 1 -- start start 2024-04-26T22:58:24.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.568+0000 7f2c0ab9a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c80ae0 0x562251c8a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.568+0000 7f2c0ab9a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c85c50 0x562251c91a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.568+0000 7f2c0ab9a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c82650 0x562251c8a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.568+0000 7f2c0ab9a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562251c96890 con 0x562251c82650 2024-04-26T22:58:24.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.568+0000 7f2c0ab9a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x562251c96f60 con 0x562251c85c50 2024-04-26T22:58:24.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.568+0000 7f2c0ab9a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562251c97630 con 0x562251c80ae0 2024-04-26T22:58:24.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.568+0000 7f2c05c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c82650 0x562251c8a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.568+0000 7f2c05200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c80ae0 0x562251c8a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.568+0000 7f2c05200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c80ae0 0x562251c8a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:45044/0 (socket says 172.21.15.27:45044) 2024-04-26T22:58:24.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.568+0000 7f2c05200640 1 -- 172.21.15.27:0/388756849 learned_addr learned my addr 172.21.15.27:0/388756849 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:24.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.568+0000 7f2bffe00640 1 --2- 172.21.15.27:0/388756849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c85c50 0x562251c91a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.569+0000 7f2c05200640 1 -- 172.21.15.27:0/388756849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c85c50 msgr2=0x562251c91a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.569+0000 7f2c05200640 1 --2- 172.21.15.27:0/388756849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c85c50 0x562251c91a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.569+0000 7f2c05200640 1 -- 172.21.15.27:0/388756849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c82650 msgr2=0x562251c8a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.569+0000 7f2c05200640 1 --2- 172.21.15.27:0/388756849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c82650 0x562251c8a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.569+0000 7f2c05200640 1 -- 172.21.15.27:0/388756849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562251c80ec0 con 0x562251c80ae0 2024-04-26T22:58:24.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.569+0000 7f2bffe00640 1 --2- 172.21.15.27:0/388756849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c85c50 0x562251c91a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:24.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.569+0000 7f2c05200640 1 --2- 172.21.15.27:0/388756849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c80ae0 0x562251c8a130 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2bf409acd0 tx=0x7f2bf4009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e128e9df8584b233 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.569+0000 7f2bf1600640 1 -- 172.21.15.27:0/388756849 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf40a0d20 con 0x562251c80ae0 2024-04-26T22:58:24.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.569+0000 7f2bf1600640 1 -- 172.21.15.27:0/388756849 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bf40a55e0 con 0x562251c80ae0 2024-04-26T22:58:24.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.569+0000 7f2bf1600640 1 -- 172.21.15.27:0/388756849 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf40bbd90 con 0x562251c80ae0 2024-04-26T22:58:24.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.570+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/388756849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c80ae0 msgr2=0x562251c8a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.570+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/388756849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c80ae0 0x562251c8a130 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2bf409acd0 tx=0x7f2bf4009310 comp rx=0 tx=0).stop 2024-04-26T22:58:24.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.570+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/388756849 shutdown_connections 2024-04-26T22:58:24.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.570+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/388756849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c85c50 0x562251c91a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.570+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/388756849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c82650 0x562251c8a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.570+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/388756849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c80ae0 0x562251c8a130 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.570+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/388756849 >> 172.21.15.27:0/388756849 conn(0x562251c14d10 msgr2=0x562251c81f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.570+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/388756849 shutdown_connections 2024-04-26T22:58:24.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.571+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/388756849 wait complete. 2024-04-26T22:58:24.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.571+0000 7f2c0ab9a740 1 -- start start 2024-04-26T22:58:24.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.571+0000 7f2c0ab9a740 1 Processor -- start 2024-04-26T22:58:24.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.572+0000 7f2c0ab9a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c80ae0 0x562251c912d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.572+0000 7f2c0ab9a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c82650 0x562251c91810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.572+0000 7f2c0ab9a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c85c50 0x562251c8dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.572+0000 7f2c0ab9a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562251c983d0 con 0x562251c80ae0 2024-04-26T22:58:24.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.572+0000 7f2c0ab9a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x562251c97d00 con 0x562251c85c50 2024-04-26T22:58:24.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.572+0000 7f2c0ab9a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562251c98aa0 con 0x562251c82650 2024-04-26T22:58:24.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.572+0000 7f2c0ab9a740 10 client.0 _ll_register_callbacks cb 0x7f2bf4009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:58:24.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.572+0000 7f2c05200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c80ae0 0x562251c912d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.572+0000 7f2c05200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c80ae0 0x562251c912d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49318/0 (socket says 172.21.15.27:49318) 2024-04-26T22:58:24.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.572+0000 7f2c05200640 1 -- 172.21.15.27:0/689990132 learned_addr learned my addr 172.21.15.27:0/689990132 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:24.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.572+0000 7f2c05c00640 1 --2- 172.21.15.27:0/689990132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c85c50 0x562251c8dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.572+0000 7f2bffe00640 1 --2- 172.21.15.27:0/689990132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c82650 0x562251c91810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.573+0000 7f2bffe00640 1 -- 172.21.15.27:0/689990132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c85c50 msgr2=0x562251c8dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.573+0000 7f2bffe00640 1 --2- 172.21.15.27:0/689990132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c85c50 0x562251c8dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.573+0000 7f2bffe00640 1 -- 172.21.15.27:0/689990132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c80ae0 msgr2=0x562251c912d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:24.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.573+0000 7f2bffe00640 1 --2- 172.21.15.27:0/689990132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c80ae0 0x562251c912d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.573+0000 7f2bffe00640 1 -- 172.21.15.27:0/689990132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562251c8e480 con 0x562251c82650 2024-04-26T22:58:24.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.573+0000 7f2c05200640 1 --2- 172.21.15.27:0/689990132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c80ae0 0x562251c912d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:24.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.573+0000 7f2bffe00640 1 --2- 172.21.15.27:0/689990132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c82650 0x562251c91810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2bec02a860 tx=0x7f2bec02a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7df4f790c3808c5c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.573+0000 7f2be6a00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T22:58:24.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.573+0000 7f2be6a00640 1 -- 172.21.15.27:0/689990132 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec037040 con 0x562251c82650 2024-04-26T22:58:24.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.573+0000 7f2be6a00640 1 -- 172.21.15.27:0/689990132 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f2bec04db50 con 0x562251c82650 2024-04-26T22:58:24.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.573+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/689990132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bf40a55e0 con 0x562251c82650 2024-04-26T22:58:24.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.574+0000 7f2be6a00640 1 -- 172.21.15.27:0/689990132 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2bec05d050 con 0x562251c82650 2024-04-26T22:58:24.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.574+0000 7f2c0ab9a740 10 client.25393 fetch_fsmap learned FSMap version 24 2024-04-26T22:58:24.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.574+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/689990132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562251cfdc50 con 0x562251c82650 2024-04-26T22:58:24.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.574+0000 7f2be6a00640 1 -- 172.21.15.27:0/689990132 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bf40a55e0 con 0x562251c82650 2024-04-26T22:58:24.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.574+0000 7f2be6a00640 1 -- 172.21.15.27:0/689990132 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2bec005300 con 0x562251c82650 2024-04-26T22:58:24.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.574+0000 7f2c0ab9a740 10 client.25393 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T22:58:24.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.574+0000 7f2c0ab9a740 10 client.25393 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T22:58:24.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.574+0000 7f2c0ab9a740 20 client.25393 populate_metadata read hostname 'smithi027' 2024-04-26T22:58:24.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.574+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/689990132 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x562251d02480 0x562251cfef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.574+0000 7f2c0ab9a740 4 client.25393 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T22:58:24.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.574+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/689990132 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "status"}) -- 0x562251bec080 con 0x562251d02480 2024-04-26T22:58:24.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.575+0000 7f2c05200640 1 --2- 172.21.15.27:0/689990132 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x562251d02480 0x562251cfef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.575+0000 7f2c05200640 1 --2- 172.21.15.27:0/689990132 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x562251d02480 0x562251cfef90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7cebecf6d6b26ce server_cookie=95876b61851a539b in_seq=0 out_seq=0 2024-04-26T22:58:24.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.575+0000 7f2be6a00640 10 client.25393 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:58:24.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.575+0000 7f2be6a00640 1 -- 172.21.15.27:0/689990132 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562251bec080 con 0x562251d02480 2024-04-26T22:58:24.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.575+0000 7f2be6a00640 10 client.25393 handle_command_reply: tid=0 2024-04-26T22:58:24.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.575+0000 7f2c0ab9a740 1 client.25393 shutdown 2024-04-26T22:58:24.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.575+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/689990132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c82650 msgr2=0x562251c91810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.575+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/689990132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c82650 0x562251c91810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2bec02a860 tx=0x7f2bec02a340 comp rx=0 tx=0).stop 2024-04-26T22:58:24.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.575+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/689990132 shutdown_connections 2024-04-26T22:58:24.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.575+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/689990132 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x562251d02480 0x562251cfef90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.576+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/689990132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562251c85c50 0x562251c8dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.576+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/689990132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562251c82650 0x562251c91810 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.576+0000 7f2c0ab9a740 1 --2- 172.21.15.27:0/689990132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562251c80ae0 0x562251c912d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.576+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/689990132 >> 172.21.15.27:0/689990132 conn(0x562251c14d10 msgr2=0x562251c8a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.576+0000 7f2be6a00640 0 client.25393 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:58:24.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.576+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/689990132 shutdown_connections 2024-04-26T22:58:24.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.576+0000 7f2c0ab9a740 1 -- 172.21.15.27:0/689990132 wait complete. 2024-04-26T22:58:24.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.576+0000 7f2c0ab9a740 20 client.25393 trim_cache size 0 max 16384 2024-04-26T22:58:24.580 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T22:58:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "95ebdda6-041e-11ef-bc93-c7b262605968", 2024-04-26T22:58:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-04-26T22:58:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24578, 2024-04-26T22:58:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T22:58:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T22:58:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T22:58:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 178.331653007, 2024-04-26T22:58:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 24, 2024-04-26T22:58:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 94, 2024-04-26T22:58:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T22:58:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 186.05206043300001 2024-04-26T22:58:24.581 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T22:58:24.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.576+0000 7f2c07c00640 1 -- 172.21.15.27:0/3506630011 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2be80a3cc0 msgr2=0x7f2be80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.576+0000 7f2c07c00640 1 --2- 172.21.15.27:0/3506630011 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2be80a3cc0 0x7f2be80a6180 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009fd0 tx=0x7f2bf8004a40 comp rx=0 tx=0).stop 2024-04-26T22:58:24.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.576+0000 7f2c07c00640 1 -- 172.21.15.27:0/3506630011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c00101300 msgr2=0x7f2c0019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.576+0000 7f2c07c00640 1 --2- 172.21.15.27:0/3506630011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c00101300 0x7f2c0019ae10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400e9f0 tx=0x7f2bf400eeb0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.577+0000 7f2c07c00640 1 -- 172.21.15.27:0/3506630011 shutdown_connections 2024-04-26T22:58:24.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.577+0000 7f2c07c00640 1 --2- 172.21.15.27:0/3506630011 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f2be80a3cc0 0x7f2be80a6180 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.577+0000 7f2c07c00640 1 --2- 172.21.15.27:0/3506630011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c00102600 0x7f2c001a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.577+0000 7f2c07c00640 1 --2- 172.21.15.27:0/3506630011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c00101c60 0x7f2c0019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.577+0000 7f2c07c00640 1 --2- 172.21.15.27:0/3506630011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c00101300 0x7f2c0019ae10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.577+0000 7f2c07c00640 1 -- 172.21.15.27:0/3506630011 >> 172.21.15.27:0/3506630011 conn(0x7f2c000fcf20 msgr2=0x7f2c000fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.577+0000 7f2c07c00640 1 -- 172.21.15.27:0/3506630011 shutdown_connections 2024-04-26T22:58:24.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.577+0000 7f2c07c00640 1 -- 172.21.15.27:0/3506630011 wait complete. 2024-04-26T22:58:24.595 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T22:58:24.605 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T22:58:24.605 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T22:58:24.605 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T22:58:24.605 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-26T22:58:24.606 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T22:58:24.606 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T22:58:24.606 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-26T22:58:24.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.784+0000 7f7addc00640 1 -- 172.21.15.27:0/1150188853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad8106790 msgr2=0x7f7ad810d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.784+0000 7f7addc00640 1 --2- 172.21.15.27:0/1150188853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad8106790 0x7f7ad810d020 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009ca0 tx=0x7f7ad002a8c0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.784+0000 7f7addc00640 1 -- 172.21.15.27:0/1150188853 shutdown_connections 2024-04-26T22:58:24.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.784+0000 7f7addc00640 1 --2- 172.21.15.27:0/1150188853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8105dd0 0x7f7ad8106250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.784+0000 7f7addc00640 1 --2- 172.21.15.27:0/1150188853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad8106790 0x7f7ad810d020 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.784+0000 7f7addc00640 1 --2- 172.21.15.27:0/1150188853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ad8104be0 0x7f7ad8105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.784+0000 7f7addc00640 1 -- 172.21.15.27:0/1150188853 >> 172.21.15.27:0/1150188853 conn(0x7f7ad8100150 msgr2=0x7f7ad8102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:24.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.784+0000 7f7addc00640 1 -- 172.21.15.27:0/1150188853 shutdown_connections 2024-04-26T22:58:24.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.784+0000 7f7addc00640 1 -- 172.21.15.27:0/1150188853 wait complete. 2024-04-26T22:58:24.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.785+0000 7f7addc00640 1 Processor -- start 2024-04-26T22:58:24.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.785+0000 7f7addc00640 1 -- start start 2024-04-26T22:58:24.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7addc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8104be0 0x7f7ad819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7addc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ad8105dd0 0x7f7ad819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7addc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad8106790 0x7f7ad81a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7addc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad8110760 con 0x7f7ad8105dd0 2024-04-26T22:58:24.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7addc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad81105e0 con 0x7f7ad8104be0 2024-04-26T22:58:24.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7addc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad81108e0 con 0x7f7ad8106790 2024-04-26T22:58:24.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7ad6a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8104be0 0x7f7ad819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7ad6a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8104be0 0x7f7ad819adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33442/0 (socket says 172.21.15.27:33442) 2024-04-26T22:58:24.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7ad6a00640 1 -- 172.21.15.27:0/550486575 learned_addr learned my addr 172.21.15.27:0/550486575 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:24.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7ad6000640 1 --2- 172.21.15.27:0/550486575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ad8105dd0 0x7f7ad819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7ad7400640 1 --2- 172.21.15.27:0/550486575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad8106790 0x7f7ad81a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7ad6a00640 1 -- 172.21.15.27:0/550486575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad8106790 msgr2=0x7f7ad81a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7ad6a00640 1 --2- 172.21.15.27:0/550486575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad8106790 0x7f7ad81a2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7ad6a00640 1 -- 172.21.15.27:0/550486575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ad8105dd0 msgr2=0x7f7ad819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:24.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7ad6a00640 1 --2- 172.21.15.27:0/550486575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ad8105dd0 0x7f7ad819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:24.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7ad6a00640 1 -- 172.21.15.27:0/550486575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad81a29d0 con 0x7f7ad8104be0 2024-04-26T22:58:24.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.786+0000 7f7ad6000640 1 --2- 172.21.15.27:0/550486575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ad8105dd0 0x7f7ad819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:58:24.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.787+0000 7f7ad6a00640 1 --2- 172.21.15.27:0/550486575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8104be0 0x7f7ad819adb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7ac800e9c0 tx=0x7f7ac800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.787+0000 7f7acf400640 1 -- 172.21.15.27:0/550486575 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac800cd80 con 0x7f7ad8104be0 2024-04-26T22:58:24.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.787+0000 7f7acf400640 1 -- 172.21.15.27:0/550486575 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ac80035f0 con 0x7f7ad8104be0 2024-04-26T22:58:24.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.787+0000 7f7addc00640 1 -- 172.21.15.27:0/550486575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad81a2cc0 con 0x7f7ad8104be0 2024-04-26T22:58:24.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.787+0000 7f7acf400640 1 -- 172.21.15.27:0/550486575 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8010690 con 0x7f7ad8104be0 2024-04-26T22:58:24.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.787+0000 7f7addc00640 1 -- 172.21.15.27:0/550486575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad81a31c0 con 0x7f7ad8104be0 2024-04-26T22:58:24.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.789+0000 7f7addc00640 1 -- 172.21.15.27:0/550486575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad810bf00 con 0x7f7ad8104be0 2024-04-26T22:58:24.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.790+0000 7f7acf400640 1 -- 172.21.15.27:0/550486575 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7ac80037a0 con 0x7f7ad8104be0 2024-04-26T22:58:24.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.792+0000 7f7acf400640 1 --2- 172.21.15.27:0/550486575 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7abc0a3c30 0x7f7abc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:24.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.792+0000 7f7acf400640 1 -- 172.21.15.27:0/550486575 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f7ac80d0ef0 con 0x7f7ad8104be0 2024-04-26T22:58:24.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.792+0000 7f7ad6000640 1 --2- 172.21.15.27:0/550486575 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7abc0a3c30 0x7f7abc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:24.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.793+0000 7f7ad6000640 1 --2- 172.21.15.27:0/550486575 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7abc0a3c30 0x7f7abc0a60f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7ad819c2c0 tx=0x7f7ac4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:24.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:24.795+0000 7f7acf400640 1 -- 172.21.15.27:0/550486575 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7ac809d2e0 con 0x7f7ad8104be0 2024-04-26T22:58:25.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:25.060+0000 7f7addc00640 1 -- 172.21.15.27:0/550486575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f7ad802dad0 con 0x7f7ad8104be0 2024-04-26T22:58:25.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:25.866+0000 7f7acf400640 1 -- 172.21.15.27:0/550486575 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mgrmap(e 18) ==== 138714+0+0 (secure 0 0 0) 0x7f7ac8099250 con 0x7f7ad8104be0 2024-04-26T22:58:25.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:25.868+0000 7f7acf400640 1 -- 172.21.15.27:0/550486575 <== mon.2 v2:172.21.15.80:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) ==== 92+0+0 (secure 0 0 0) 0x7f7ac8077ae0 con 0x7f7ad8104be0 2024-04-26T22:58:25.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:25.872+0000 7f7addc00640 1 -- 172.21.15.27:0/550486575 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7abc0a3c30 msgr2=0x7f7abc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:25.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:25.872+0000 7f7addc00640 1 --2- 172.21.15.27:0/550486575 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7abc0a3c30 0x7f7abc0a60f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7ad819c2c0 tx=0x7f7ac4005f90 comp rx=0 tx=0).stop 2024-04-26T22:58:25.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:25.872+0000 7f7addc00640 1 -- 172.21.15.27:0/550486575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8104be0 msgr2=0x7f7ad819adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:25.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:25.872+0000 7f7addc00640 1 --2- 172.21.15.27:0/550486575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8104be0 0x7f7ad819adb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7ac800e9c0 tx=0x7f7ac800ee80 comp rx=0 tx=0).stop 2024-04-26T22:58:25.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:25.872+0000 7f7addc00640 1 -- 172.21.15.27:0/550486575 shutdown_connections 2024-04-26T22:58:25.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:25.872+0000 7f7addc00640 1 --2- 172.21.15.27:0/550486575 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f7abc0a3c30 0x7f7abc0a60f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:25.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:25.872+0000 7f7addc00640 1 --2- 172.21.15.27:0/550486575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad8106790 0x7f7ad81a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:25.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:25.872+0000 7f7addc00640 1 --2- 172.21.15.27:0/550486575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ad8105dd0 0x7f7ad819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:25.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:25.872+0000 7f7addc00640 1 --2- 172.21.15.27:0/550486575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8104be0 0x7f7ad819adb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:25.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:25.872+0000 7f7addc00640 1 -- 172.21.15.27:0/550486575 >> 172.21.15.27:0/550486575 conn(0x7f7ad8100150 msgr2=0x7f7ad81090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:25.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:25.872+0000 7f7addc00640 1 -- 172.21.15.27:0/550486575 shutdown_connections 2024-04-26T22:58:25.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:25.872+0000 7f7addc00640 1 -- 172.21.15.27:0/550486575 wait complete. 2024-04-26T22:58:25.895 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-26T22:58:25.981 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:25 smithi094 ceph-mon[32889]: pgmap v424: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:25.981 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:25.981 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:25 smithi094 ceph-mon[32889]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:25.981 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:25 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/550486575' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T22:58:25.981 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:25 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T22:58:25.982 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:25 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: ignoring --setuser ceph since I am not root 2024-04-26T22:58:25.982 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:25 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: ignoring --setgroup ceph since I am not root 2024-04-26T22:58:25.982 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:25 smithi094 ceph-mgr[34748]: -- 172.21.15.94:0/2968457895 <== mon.1 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5572ae7705a0 con 0x5572ae744800 2024-04-26T22:58:25.982 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:25 smithi080 ceph-mon[32324]: pgmap v424: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:25.982 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:25.982 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:25 smithi080 ceph-mon[32324]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:25.982 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:25 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/550486575' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T22:58:25.983 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:25 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T22:58:25.983 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:25 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: ignoring --setuser ceph since I am not root 2024-04-26T22:58:25.983 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:25 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: ignoring --setgroup ceph since I am not root 2024-04-26T22:58:25.983 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:25 smithi080 ceph-mgr[33714]: -- 172.21.15.80:0/1959635599 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4c8a5e5a0 con 0x55a4c8a32c00 2024-04-26T22:58:25.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:25 smithi027 ceph-mon[28034]: pgmap v424: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T22:58:25.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:25.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:25 smithi027 ceph-mon[28034]: from='mgr.14152 172.21.15.27:0/4097648403' entity='mgr.x' 2024-04-26T22:58:25.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:25 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/550486575' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T22:58:25.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:25 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T22:58:26.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.113+0000 7fbe12200640 1 -- 172.21.15.27:0/2286648694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe0c103b90 msgr2=0x7fbe0c104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.113+0000 7fbe12200640 1 --2- 172.21.15.27:0/2286648694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe0c103b90 0x7fbe0c104010 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009900 tx=0x7fbdf802a400 comp rx=0 tx=0).stop 2024-04-26T22:58:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.113+0000 7fbe12200640 1 -- 172.21.15.27:0/2286648694 shutdown_connections 2024-04-26T22:58:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.113+0000 7fbe12200640 1 --2- 172.21.15.27:0/2286648694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe0c104550 0x7fbe0c10ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.113+0000 7fbe12200640 1 --2- 172.21.15.27:0/2286648694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe0c103b90 0x7fbe0c104010 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.113+0000 7fbe12200640 1 --2- 172.21.15.27:0/2286648694 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe0c1029a0 0x7fbe0c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.113+0000 7fbe12200640 1 -- 172.21.15.27:0/2286648694 >> 172.21.15.27:0/2286648694 conn(0x7fbe0c0fdf10 msgr2=0x7fbe0c100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.113+0000 7fbe12200640 1 -- 172.21.15.27:0/2286648694 shutdown_connections 2024-04-26T22:58:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.113+0000 7fbe12200640 1 -- 172.21.15.27:0/2286648694 wait complete. 2024-04-26T22:58:26.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.114+0000 7fbe12200640 1 Processor -- start 2024-04-26T22:58:26.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.114+0000 7fbe12200640 1 -- start start 2024-04-26T22:58:26.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.115+0000 7fbe12200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe0c1029a0 0x7fbe0c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:26.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.115+0000 7fbe12200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe0c103b90 0x7fbe0c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.115+0000 7fbe12200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe0c104550 0x7fbe0c1a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.115+0000 7fbe12200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c10e4d0 con 0x7fbe0c1029a0 2024-04-26T22:58:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.115+0000 7fbe12200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c10e350 con 0x7fbe0c103b90 2024-04-26T22:58:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.115+0000 7fbe12200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c10e650 con 0x7fbe0c104550 2024-04-26T22:58:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.115+0000 7fbe0aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe0c103b90 0x7fbe0c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.115+0000 7fbe0b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe0c1029a0 0x7fbe0c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.115+0000 7fbe0aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe0c103b90 0x7fbe0c19b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33452/0 (socket says 172.21.15.27:33452) 2024-04-26T22:58:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.115+0000 7fbe0aa00640 1 -- 172.21.15.27:0/4101491861 learned_addr learned my addr 172.21.15.27:0/4101491861 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.115+0000 7fbe0be00640 1 --2- 172.21.15.27:0/4101491861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe0c104550 0x7fbe0c1a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.115+0000 7fbe0aa00640 1 -- 172.21.15.27:0/4101491861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe0c104550 msgr2=0x7fbe0c1a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.115+0000 7fbe0aa00640 1 --2- 172.21.15.27:0/4101491861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe0c104550 0x7fbe0c1a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.115+0000 7fbe0aa00640 1 -- 172.21.15.27:0/4101491861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe0c1029a0 msgr2=0x7fbe0c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.115+0000 7fbe0aa00640 1 --2- 172.21.15.27:0/4101491861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe0c1029a0 0x7fbe0c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.115+0000 7fbe0aa00640 1 -- 172.21.15.27:0/4101491861 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c1a29d0 con 0x7fbe0c103b90 2024-04-26T22:58:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.116+0000 7fbe0aa00640 1 --2- 172.21.15.27:0/4101491861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe0c103b90 0x7fbe0c19b300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8005940 tx=0x7fbdf80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.116+0000 7fbe03e00640 1 -- 172.21.15.27:0/4101491861 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf803f030 con 0x7fbe0c103b90 2024-04-26T22:58:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.116+0000 7fbe12200640 1 -- 172.21.15.27:0/4101491861 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0c1a2c60 con 0x7fbe0c103b90 2024-04-26T22:58:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.116+0000 7fbe03e00640 1 -- 172.21.15.27:0/4101491861 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdf8003cb0 con 0x7fbe0c103b90 2024-04-26T22:58:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.116+0000 7fbe03e00640 1 -- 172.21.15.27:0/4101491861 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8035070 con 0x7fbe0c103b90 2024-04-26T22:58:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.116+0000 7fbe12200640 1 -- 172.21.15.27:0/4101491861 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe0c1a31a0 con 0x7fbe0c103b90 2024-04-26T22:58:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.117+0000 7fbe12200640 1 -- 172.21.15.27:0/4101491861 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe0c1a3370 con 0x7fbe0c103b90 2024-04-26T22:58:26.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.119+0000 7fbe03e00640 1 -- 172.21.15.27:0/4101491861 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 138714+0+0 (secure 0 0 0) 0x7fbdf8042030 con 0x7fbe0c103b90 2024-04-26T22:58:26.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.121+0000 7fbe03e00640 1 --2- 172.21.15.27:0/4101491861 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbdf00a3d00 0x7fbdf00a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:26.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.121+0000 7fbe0b400640 1 -- 172.21.15.27:0/4101491861 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbdf00a3d00 msgr2=0x7fbdf00a61c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3636526714 2024-04-26T22:58:26.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.121+0000 7fbe03e00640 1 -- 172.21.15.27:0/4101491861 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fbdf80f0e40 con 0x7fbe0c103b90 2024-04-26T22:58:26.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.121+0000 7fbe0b400640 1 --2- 172.21.15.27:0/4101491861 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbdf00a3d00 0x7fbdf00a61c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T22:58:26.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.129+0000 7fbe03e00640 1 -- 172.21.15.27:0/4101491861 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbdf8003e60 con 0x7fbe0c103b90 2024-04-26T22:58:26.271 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:26 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:26.034+0000 7fb7e07ed200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T22:58:26.271 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:26 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:26.221+0000 7fb7e07ed200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:58:26.284 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:26 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:26.032+0000 7f98835ed200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T22:58:26.284 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:26 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:26.215+0000 7f98835ed200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:58:26.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.322+0000 7fbe0b400640 1 -- 172.21.15.27:0/4101491861 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbdf00a3d00 msgr2=0x7fbdf00a61c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3636526714 2024-04-26T22:58:26.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.322+0000 7fbe0b400640 1 --2- 172.21.15.27:0/4101491861 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbdf00a3d00 0x7fbdf00a61c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T22:58:26.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.339+0000 7fbe12200640 1 -- 172.21.15.27:0/4101491861 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fbe0c1a3540 con 0x7fbe0c103b90 2024-04-26T22:58:26.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.349+0000 7fbe03e00640 1 -- 172.21.15.27:0/4101491861 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7fbdf80bcbe0 con 0x7fbe0c103b90 2024-04-26T22:58:26.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.351+0000 7fbe12200640 1 -- 172.21.15.27:0/4101491861 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbdf00a3d00 msgr2=0x7fbdf00a61c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:58:26.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.351+0000 7fbe12200640 1 --2- 172.21.15.27:0/4101491861 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbdf00a3d00 0x7fbdf00a61c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.351+0000 7fbe12200640 1 -- 172.21.15.27:0/4101491861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe0c103b90 msgr2=0x7fbe0c19b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:26.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.351+0000 7fbe12200640 1 --2- 172.21.15.27:0/4101491861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe0c103b90 0x7fbe0c19b300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8005940 tx=0x7fbdf80042f0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.351+0000 7fbe12200640 1 -- 172.21.15.27:0/4101491861 shutdown_connections 2024-04-26T22:58:26.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.351+0000 7fbe12200640 1 --2- 172.21.15.27:0/4101491861 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fbdf00a3d00 0x7fbdf00a61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.351+0000 7fbe12200640 1 --2- 172.21.15.27:0/4101491861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe0c103b90 0x7fbe0c19b300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.351+0000 7fbe12200640 1 --2- 172.21.15.27:0/4101491861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe0c104550 0x7fbe0c1a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.351+0000 7fbe12200640 1 --2- 172.21.15.27:0/4101491861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe0c1029a0 0x7fbe0c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.352+0000 7fbe12200640 1 -- 172.21.15.27:0/4101491861 >> 172.21.15.27:0/4101491861 conn(0x7fbe0c0fdf10 msgr2=0x7fbe0c109330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:26.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.352+0000 7fbe12200640 1 -- 172.21.15.27:0/4101491861 shutdown_connections 2024-04-26T22:58:26.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.352+0000 7fbe12200640 1 -- 172.21.15.27:0/4101491861 wait complete. 2024-04-26T22:58:26.364 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:26 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: ignoring --setuser ceph since I am not root 2024-04-26T22:58:26.364 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:26 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: ignoring --setgroup ceph since I am not root 2024-04-26T22:58:26.365 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:26 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:26.233+0000 7fc9394c7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T22:58:26.371 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-26T22:58:26.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.568+0000 7f3b1b000640 1 -- 172.21.15.27:0/3699548152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b14076970 msgr2=0x7f3b14074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:26.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.568+0000 7f3b1b000640 1 --2- 172.21.15.27:0/3699548152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b14076970 0x7f3b14074dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009900 tx=0x7f3b0402a3f0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.568+0000 7f3b1b000640 1 -- 172.21.15.27:0/3699548152 shutdown_connections 2024-04-26T22:58:26.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.568+0000 7f3b1b000640 1 --2- 172.21.15.27:0/3699548152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b1410a880 0x7f3b1410cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.568+0000 7f3b1b000640 1 --2- 172.21.15.27:0/3699548152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b14075390 0x7f3b14075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.568+0000 7f3b1b000640 1 --2- 172.21.15.27:0/3699548152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b14076970 0x7f3b14074dc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.568+0000 7f3b1b000640 1 -- 172.21.15.27:0/3699548152 >> 172.21.15.27:0/3699548152 conn(0x7f3b140fdf50 msgr2=0x7f3b14100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:26.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.568+0000 7f3b1b000640 1 -- 172.21.15.27:0/3699548152 shutdown_connections 2024-04-26T22:58:26.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.568+0000 7f3b1b000640 1 -- 172.21.15.27:0/3699548152 wait complete. 2024-04-26T22:58:26.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.569+0000 7f3b1b000640 1 Processor -- start 2024-04-26T22:58:26.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.569+0000 7f3b1b000640 1 -- start start 2024-04-26T22:58:26.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.570+0000 7f3b1b000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b14075390 0x7f3b1419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:26.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.570+0000 7f3b1b000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b14076970 0x7f3b1419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:26.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.570+0000 7f3b1b000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b1410a880 0x7f3b141a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:26.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.570+0000 7f3b1b000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3b14110720 con 0x7f3b14076970 2024-04-26T22:58:26.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.570+0000 7f3b1b000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3b141105a0 con 0x7f3b14075390 2024-04-26T22:58:26.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.570+0000 7f3b1b000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3b141108a0 con 0x7f3b1410a880 2024-04-26T22:58:26.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.570+0000 7f3b13e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b14075390 0x7f3b1419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:26.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.570+0000 7f3b13e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b14075390 0x7f3b1419ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33474/0 (socket says 172.21.15.27:33474) 2024-04-26T22:58:26.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.570+0000 7f3b13e00640 1 -- 172.21.15.27:0/1839940624 learned_addr learned my addr 172.21.15.27:0/1839940624 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:26.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.570+0000 7f3b0be00640 1 --2- 172.21.15.27:0/1839940624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b14076970 0x7f3b1419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:26.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.570+0000 7f3b19000640 1 --2- 172.21.15.27:0/1839940624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b1410a880 0x7f3b141a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:26.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.571+0000 7f3b13e00640 1 -- 172.21.15.27:0/1839940624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b1410a880 msgr2=0x7f3b141a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:26.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.571+0000 7f3b13e00640 1 --2- 172.21.15.27:0/1839940624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b1410a880 0x7f3b141a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.571+0000 7f3b13e00640 1 -- 172.21.15.27:0/1839940624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b14076970 msgr2=0x7f3b1419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:26.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.571+0000 7f3b13e00640 1 --2- 172.21.15.27:0/1839940624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b14076970 0x7f3b1419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.571+0000 7f3b13e00640 1 -- 172.21.15.27:0/1839940624 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b141a2a50 con 0x7f3b14075390 2024-04-26T22:58:26.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.571+0000 7f3b0be00640 1 --2- 172.21.15.27:0/1839940624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b14076970 0x7f3b1419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:58:26.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.571+0000 7f3b13e00640 1 --2- 172.21.15.27:0/1839940624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b14075390 0x7f3b1419ae80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3b04005ab0 tx=0x7f3b04003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:26.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.571+0000 7f3b11600640 1 -- 172.21.15.27:0/1839940624 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0403f070 con 0x7f3b14075390 2024-04-26T22:58:26.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.571+0000 7f3b11600640 1 -- 172.21.15.27:0/1839940624 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b04031c50 con 0x7f3b14075390 2024-04-26T22:58:26.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.571+0000 7f3b1b000640 1 -- 172.21.15.27:0/1839940624 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b141a2ce0 con 0x7f3b14075390 2024-04-26T22:58:26.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.571+0000 7f3b11600640 1 -- 172.21.15.27:0/1839940624 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04035030 con 0x7f3b14075390 2024-04-26T22:58:26.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.571+0000 7f3b1b000640 1 -- 172.21.15.27:0/1839940624 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b141a31d0 con 0x7f3b14075390 2024-04-26T22:58:26.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.573+0000 7f3b1b000640 1 -- 172.21.15.27:0/1839940624 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b141095b0 con 0x7f3b14075390 2024-04-26T22:58:26.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.574+0000 7f3b11600640 1 -- 172.21.15.27:0/1839940624 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 138714+0+0 (secure 0 0 0) 0x7f3b04042070 con 0x7f3b14075390 2024-04-26T22:58:26.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.576+0000 7f3b11600640 1 --2- 172.21.15.27:0/1839940624 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3af80a3d00 0x7f3af80a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:26.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.576+0000 7f3b0be00640 1 -- 172.21.15.27:0/1839940624 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3af80a3d00 msgr2=0x7f3af80a61c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3636526714 2024-04-26T22:58:26.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.576+0000 7f3b11600640 1 -- 172.21.15.27:0/1839940624 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7f3b040f12b0 con 0x7f3b14075390 2024-04-26T22:58:26.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.576+0000 7f3b0be00640 1 --2- 172.21.15.27:0/1839940624 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3af80a3d00 0x7f3af80a61c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T22:58:26.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.584+0000 7f3b11600640 1 -- 172.21.15.27:0/1839940624 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3b040bd6a0 con 0x7f3b14075390 2024-04-26T22:58:26.627 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:26 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:26.316+0000 7f98835ed200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T22:58:26.638 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:26 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:26.497+0000 7fc9394c7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:58:26.640 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:26 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:26.325+0000 7fb7e07ed200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T22:58:26.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.777+0000 7f3b0be00640 1 -- 172.21.15.27:0/1839940624 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3af80a3d00 msgr2=0x7f3af80a61c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3636526714 2024-04-26T22:58:26.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.777+0000 7f3b0be00640 1 --2- 172.21.15.27:0/1839940624 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3af80a3d00 0x7f3af80a61c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T22:58:26.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.790+0000 7f3b1b000640 1 -- 172.21.15.27:0/1839940624 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f3b140635d0 con 0x7f3b14075390 2024-04-26T22:58:26.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.799+0000 7f3b11600640 1 -- 172.21.15.27:0/1839940624 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7f3b040bcfa0 con 0x7f3b14075390 2024-04-26T22:58:26.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.802+0000 7f3b1b000640 1 -- 172.21.15.27:0/1839940624 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3af80a3d00 msgr2=0x7f3af80a61c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:58:26.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.802+0000 7f3b1b000640 1 --2- 172.21.15.27:0/1839940624 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3af80a3d00 0x7f3af80a61c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.802+0000 7f3b1b000640 1 -- 172.21.15.27:0/1839940624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b14075390 msgr2=0x7f3b1419ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:26.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.802+0000 7f3b1b000640 1 --2- 172.21.15.27:0/1839940624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b14075390 0x7f3b1419ae80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3b04005ab0 tx=0x7f3b04003b90 comp rx=0 tx=0).stop 2024-04-26T22:58:26.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.802+0000 7f3b1b000640 1 -- 172.21.15.27:0/1839940624 shutdown_connections 2024-04-26T22:58:26.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.802+0000 7f3b1b000640 1 --2- 172.21.15.27:0/1839940624 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7f3af80a3d00 0x7f3af80a61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.802+0000 7f3b1b000640 1 --2- 172.21.15.27:0/1839940624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b1410a880 0x7f3b141a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.802+0000 7f3b1b000640 1 --2- 172.21.15.27:0/1839940624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b14076970 0x7f3b1419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.802+0000 7f3b1b000640 1 --2- 172.21.15.27:0/1839940624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b14075390 0x7f3b1419ae80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:26.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.802+0000 7f3b1b000640 1 -- 172.21.15.27:0/1839940624 >> 172.21.15.27:0/1839940624 conn(0x7f3b140fdf50 msgr2=0x7f3b14106780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:26.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.802+0000 7f3b1b000640 1 -- 172.21.15.27:0/1839940624 shutdown_connections 2024-04-26T22:58:26.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:26.802+0000 7f3b1b000640 1 -- 172.21.15.27:0/1839940624 wait complete. 2024-04-26T22:58:26.821 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T22:58:26.842 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T22:58:26.842 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-26T22:58:26.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:58:26.843 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-04-26T22:58:26.897 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-26T22:58:26.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:58:26.898 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T22:58:26.966 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:26 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:26.637+0000 7fc9394c7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T22:58:26.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:26 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T22:58:26.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:26 smithi027 ceph-mon[28034]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-04-26T22:58:26.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:26 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' 2024-04-26T22:58:26.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:26 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' 2024-04-26T22:58:26.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:58:26.982 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-04-26T22:58:27.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:26 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T22:58:27.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:26 smithi080 ceph-mon[32324]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-04-26T22:58:27.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:26 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' 2024-04-26T22:58:27.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:26 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' 2024-04-26T22:58:27.022 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:26 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:26.640+0000 7fb7e07ed200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T22:58:27.022 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:26 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:26.725+0000 7fb7e07ed200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T22:58:27.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:26 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T22:58:27.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:26 smithi094 ceph-mon[32889]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-04-26T22:58:27.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:26 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' 2024-04-26T22:58:27.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:26 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' 2024-04-26T22:58:27.035 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:26 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:26.626+0000 7f98835ed200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T22:58:27.035 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:26 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:26.711+0000 7f98835ed200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T22:58:27.095 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-26T22:58:27.095 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T22:58:27.095 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T22:58:27.095 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T22:58:27.364 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:27 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:27.060+0000 7fc9394c7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T22:58:27.364 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:27 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:27.184+0000 7fc9394c7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T22:58:27.502 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:58:28.222 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:27 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:27.884+0000 7f98835ed200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T22:58:28.222 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:28 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:28.053+0000 7f98835ed200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T22:58:28.237 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:27 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:27.900+0000 7fb7e07ed200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T22:58:28.237 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:28 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:28.068+0000 7fb7e07ed200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T22:58:28.237 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:28 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:28.237+0000 7fb7e07ed200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T22:58:28.510 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:28 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:28.221+0000 7f98835ed200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T22:58:28.784 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:28 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:28.508+0000 7f98835ed200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T22:58:28.784 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:28 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:28.602+0000 7f98835ed200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T22:58:28.820 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:28 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:28.524+0000 7fb7e07ed200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T22:58:28.820 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:28 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:28.617+0000 7fb7e07ed200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T22:58:28.821 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:28 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:28.820+0000 7fb7e07ed200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T22:58:29.271 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:29 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:29.013+0000 7fb7e07ed200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:58:29.284 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:28 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:28.804+0000 7f98835ed200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T22:58:29.284 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:28 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:28.997+0000 7f98835ed200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:58:29.359 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:28 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:28.867+0000 7fc9394c7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T22:58:29.359 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:29 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:29.116+0000 7fc9394c7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T22:58:29.614 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:29 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:29.358+0000 7fc9394c7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T22:58:29.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.908+0000 7fdfa6400700 1 -- 172.21.15.27:0/3276906660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfa010fe80 msgr2=0x7fdfa0112270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:29.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.908+0000 7fdfa6400700 1 --2- 172.21.15.27:0/3276906660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfa010fe80 0x7fdfa0112270 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009f90 tx=0x7fdf9800b330 comp rx=0 tx=0).stop 2024-04-26T22:58:29.910 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:29 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:29.780+0000 7fc9394c7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T22:58:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.908+0000 7fdfa6400700 1 -- 172.21.15.27:0/3276906660 shutdown_connections 2024-04-26T22:58:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.908+0000 7fdfa6400700 1 --2- 172.21.15.27:0/3276906660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfa010fe80 0x7fdfa0112270 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.908+0000 7fdfa6400700 1 --2- 172.21.15.27:0/3276906660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfa0075650 0x7fdfa0075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.908+0000 7fdfa6400700 1 --2- 172.21.15.27:0/3276906660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa0075f70 0x7fdfa010f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.908+0000 7fdfa6400700 1 -- 172.21.15.27:0/3276906660 >> 172.21.15.27:0/3276906660 conn(0x7fdfa007a580 msgr2=0x7fdfa007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.908+0000 7fdfa6400700 1 -- 172.21.15.27:0/3276906660 shutdown_connections 2024-04-26T22:58:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.908+0000 7fdfa6400700 1 -- 172.21.15.27:0/3276906660 wait complete. 2024-04-26T22:58:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.910+0000 7fdfa6400700 1 Processor -- start 2024-04-26T22:58:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.910+0000 7fdfa6400700 1 -- start start 2024-04-26T22:58:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.910+0000 7fdfa6400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfa0075650 0x7fdfa01a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.910+0000 7fdfa6400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfa0075f70 0x7fdfa01a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.910+0000 7fdfa6400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa010fe80 0x7fdfa019ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.910+0000 7fdfa6400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa0115840 con 0x7fdfa0075650 2024-04-26T22:58:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.910+0000 7fdfa6400700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa01156c0 con 0x7fdfa0075f70 2024-04-26T22:58:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.910+0000 7fdfa6400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa01159c0 con 0x7fdfa010fe80 2024-04-26T22:58:29.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.911+0000 7fdf9f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfa0075650 0x7fdfa01a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:29.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.911+0000 7fdf9ea00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfa0075f70 0x7fdfa01a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:29.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.911+0000 7fdf9ea00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfa0075f70 0x7fdfa01a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:53836/0 (socket says 172.21.15.27:53836) 2024-04-26T22:58:29.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.911+0000 7fdf9ea00700 1 -- 172.21.15.27:0/688680099 learned_addr learned my addr 172.21.15.27:0/688680099 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.911+0000 7fdf9fe00700 1 --2- 172.21.15.27:0/688680099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa010fe80 0x7fdfa019ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.911+0000 7fdf9ea00700 1 -- 172.21.15.27:0/688680099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa010fe80 msgr2=0x7fdfa019ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.911+0000 7fdf9ea00700 1 --2- 172.21.15.27:0/688680099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa010fe80 0x7fdfa019ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.911+0000 7fdf9ea00700 1 -- 172.21.15.27:0/688680099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfa0075650 msgr2=0x7fdfa01a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.911+0000 7fdf9ea00700 1 --2- 172.21.15.27:0/688680099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfa0075650 0x7fdfa01a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.911+0000 7fdf9ea00700 1 -- 172.21.15.27:0/688680099 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa019c1e0 con 0x7fdfa0075f70 2024-04-26T22:58:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.911+0000 7fdf9fe00700 1 --2- 172.21.15.27:0/688680099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa010fe80 0x7fdfa019ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.912+0000 7fdf9ea00700 1 --2- 172.21.15.27:0/688680099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfa0075f70 0x7fdfa01a1e00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00ec10 tx=0x7fdf8c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.912+0000 7fdf97e00700 1 -- 172.21.15.27:0/688680099 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c00cc90 con 0x7fdfa0075f70 2024-04-26T22:58:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.912+0000 7fdf97e00700 1 -- 172.21.15.27:0/688680099 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf8c00ce30 con 0x7fdfa0075f70 2024-04-26T22:58:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.912+0000 7fdfa6400700 1 -- 172.21.15.27:0/688680099 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa019c4d0 con 0x7fdfa0075f70 2024-04-26T22:58:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.912+0000 7fdf97e00700 1 -- 172.21.15.27:0/688680099 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c010430 con 0x7fdfa0075f70 2024-04-26T22:58:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.912+0000 7fdfa6400700 1 -- 172.21.15.27:0/688680099 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa01a8790 con 0x7fdfa0075f70 2024-04-26T22:58:29.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.914+0000 7fdfa6400700 1 -- 172.21.15.27:0/688680099 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa006a110 con 0x7fdfa0075f70 2024-04-26T22:58:29.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.916+0000 7fdf97e00700 1 -- 172.21.15.27:0/688680099 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 138714+0+0 (secure 0 0 0) 0x7fdf8c004000 con 0x7fdfa0075f70 2024-04-26T22:58:29.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.918+0000 7fdf97e00700 1 --2- 172.21.15.27:0/688680099 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdf840a3da0 0x7fdf840a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:29.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.918+0000 7fdf9f400700 1 -- 172.21.15.27:0/688680099 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdf840a3da0 msgr2=0x7fdf840a6260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3636526714 2024-04-26T22:58:29.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.918+0000 7fdf97e00700 1 -- 172.21.15.27:0/688680099 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fdf8c0d0c60 con 0x7fdfa0075f70 2024-04-26T22:58:29.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.918+0000 7fdf9f400700 1 --2- 172.21.15.27:0/688680099 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdf840a3da0 0x7fdf840a6260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T22:58:29.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:29.926+0000 7fdf97e00700 1 -- 172.21.15.27:0/688680099 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdf8c09d050 con 0x7fdfa0075f70 2024-04-26T22:58:30.019 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:29 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:29.595+0000 7f98835ed200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T22:58:30.021 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:29 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:29.632+0000 7fb7e07ed200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T22:58:30.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:30.119+0000 7fdf9f400700 1 -- 172.21.15.27:0/688680099 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdf840a3da0 msgr2=0x7fdf840a6260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3636526714 2024-04-26T22:58:30.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:30.119+0000 7fdf9f400700 1 --2- 172.21.15.27:0/688680099 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdf840a3da0 0x7fdf840a6260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T22:58:30.184 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:29 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:29.917+0000 7fc9394c7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T22:58:30.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:30.199+0000 7fdfa6400700 1 -- 172.21.15.27:0/688680099 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdfa002e490 con 0x7fdfa0075f70 2024-04-26T22:58:30.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:30.200+0000 7fdf97e00700 1 -- 172.21.15.27:0/688680099 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7fdf8c09c950 con 0x7fdfa0075f70 2024-04-26T22:58:30.201 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T22:58:30.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:30.203+0000 7fdfa6400700 1 -- 172.21.15.27:0/688680099 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdf840a3da0 msgr2=0x7fdf840a6260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:58:30.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:30.203+0000 7fdfa6400700 1 --2- 172.21.15.27:0/688680099 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdf840a3da0 0x7fdf840a6260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:30.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:30.203+0000 7fdfa6400700 1 -- 172.21.15.27:0/688680099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfa0075f70 msgr2=0x7fdfa01a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:30.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:30.203+0000 7fdfa6400700 1 --2- 172.21.15.27:0/688680099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfa0075f70 0x7fdfa01a1e00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00ec10 tx=0x7fdf8c00ef20 comp rx=0 tx=0).stop 2024-04-26T22:58:30.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:30.203+0000 7fdfa6400700 1 -- 172.21.15.27:0/688680099 shutdown_connections 2024-04-26T22:58:30.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:30.203+0000 7fdfa6400700 1 --2- 172.21.15.27:0/688680099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa010fe80 0x7fdfa019ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:30.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:30.203+0000 7fdfa6400700 1 --2- 172.21.15.27:0/688680099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfa0075650 0x7fdfa01a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:30.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:30.203+0000 7fdfa6400700 1 --2- 172.21.15.27:0/688680099 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fdf840a3da0 0x7fdf840a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:30.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:30.203+0000 7fdfa6400700 1 --2- 172.21.15.27:0/688680099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfa0075f70 0x7fdfa01a1e00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:30.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:30.203+0000 7fdfa6400700 1 -- 172.21.15.27:0/688680099 >> 172.21.15.27:0/688680099 conn(0x7fdfa007a580 msgr2=0x7fdfa01006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:30.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:30.203+0000 7fdfa6400700 1 -- 172.21.15.27:0/688680099 shutdown_connections 2024-04-26T22:58:30.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:30.203+0000 7fdfa6400700 1 -- 172.21.15.27:0/688680099 wait complete. 2024-04-26T22:58:30.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:30 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/688680099' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T22:58:30.285 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:30 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:30.018+0000 7f98835ed200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T22:58:30.285 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:30 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:30.115+0000 7f98835ed200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T22:58:30.387 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:30 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/688680099' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T22:58:30.387 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:30 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:30.077+0000 7fb7e07ed200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T22:58:30.387 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:30 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:30.177+0000 7fb7e07ed200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T22:58:30.455 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:30 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/688680099' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T22:58:30.455 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:30 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:30.183+0000 7fc9394c7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T22:58:30.771 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:30 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:30.386+0000 7fb7e07ed200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T22:58:30.784 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:30 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:30.318+0000 7f98835ed200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T22:58:30.864 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:30 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:30.454+0000 7fc9394c7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:58:31.169 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:30 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:30.835+0000 7f98835ed200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T22:58:31.177 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:30 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:30.927+0000 7f98835ed200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T22:58:31.244 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:30 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:30.909+0000 7fb7e07ed200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T22:58:31.244 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:31.001+0000 7fb7e07ed200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T22:58:31.249 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T22:58:31.249 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T22:58:31.249 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-04-26T22:58:31.249 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T22:58:31.249 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T22:58:31.249 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T22:58:31.249 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T22:58:31.249 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T22:58:31.249 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T22:58:31.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:58:31.250 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-04-26T22:58:31.307 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:58:31.307 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:58:31.307 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:58:31.307 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:58:31.307 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-04-26T22:58:31.307 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:58:31.307 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:58:31.307 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:58:31.307 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-04-26T22:58:31.308 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:58:31.308 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:58:31.308 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-04-26T22:58:31.308 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T22:58:31.308 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-04-26T22:58:31.308 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-04-26T22:58:31.308 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T22:58:31.308 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41871sec preferred_lft 41871sec 2024-04-26T22:58:31.308 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2024-04-26T22:58:31.308 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:58:31.308 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:58:31.308 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-04-26T22:58:31.308 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-04-26T22:58:31.309 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T22:58:31.309 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:58:31.309 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-26T22:58:31.310 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2024-04-26T22:58:31.310 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2024-04-26T22:58:31.310 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2024-04-26T22:58:31.310 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T22:58:31.310 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-26T22:58:31.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:58:31.467 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:31 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:31.301+0000 7fc9394c7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T22:58:31.511 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:31.167+0000 7f98835ed200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T22:58:31.511 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:31.300+0000 7f98835ed200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T22:58:31.511 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:31.395+0000 7f98835ed200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T22:58:31.511 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:31.509+0000 7f98835ed200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T22:58:31.521 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:31.244+0000 7fb7e07ed200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T22:58:31.521 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:31.376+0000 7fb7e07ed200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T22:58:31.521 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:31.471+0000 7fb7e07ed200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T22:58:31.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:58:31.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:58:31.541 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T22:58:31.601 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-04-26T22:58:31.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:58:31.604 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-04-26T22:58:31.647 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-04-26T22:58:31.647 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T22:58:31.647 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T22:58:31.648 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T22:58:31.648 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T22:58:31.649 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:58:31.649 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-26T22:58:31.649 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T22:58:31.649 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T22:58:31.649 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T22:58:31.649 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-26T22:58:31.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:58:31.885 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:31.585+0000 7fb7e07ed200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T22:58:31.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:58:31.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:58:31.897 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-04-26T22:58:31.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:58:31.943 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2024-04-26T22:58:32.030 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:58:32.030 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-26T22:58:32.030 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T22:58:32.030 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T22:58:32.030 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-26T22:58:32.114 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:31 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:31.894+0000 7fc9394c7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T22:58:32.114 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:32 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:32.035+0000 7fc9394c7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T22:58:32.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:58:32.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:58:32.257 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T22:58:32.257 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:58:32.257 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-26T22:58:32.257 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T22:58:32.258 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T22:58:32.258 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T22:58:32.258 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T22:58:32.258 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T22:58:32.258 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-26T22:58:32.271 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:31.884+0000 7fb7e07ed200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T22:58:32.284 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:31.806+0000 7f98835ed200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T22:58:32.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:58:32.457 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:32 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:32.321+0000 7fc9394c7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T22:58:32.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:58:32.569 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:58:32.570 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-26T22:58:32.570 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-04-26T22:58:32.570 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T22:58:32.570 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-26T22:58:32.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:58:32.693 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:32 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:32.405+0000 7f98835ed200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T22:58:32.693 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 22:58:32 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z[34744]: 2024-04-26T22:58:32.623+0000 7f98835ed200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T22:58:32.723 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:32 smithi027 ceph-mon[28034]: Standby manager daemon z restarted 2024-04-26T22:58:32.723 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:32 smithi027 ceph-mon[28034]: Standby manager daemon z started 2024-04-26T22:58:32.723 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:32 smithi027 ceph-mon[28034]: from='mgr.? 172.21.15.94:0/1947873219' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T22:58:32.723 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:32 smithi027 ceph-mon[28034]: from='mgr.? 172.21.15.94:0/1947873219' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:58:32.724 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:32 smithi027 ceph-mon[28034]: from='mgr.? 172.21.15.94:0/1947873219' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T22:58:32.724 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:32 smithi027 ceph-mon[28034]: from='mgr.? 172.21.15.94:0/1947873219' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:58:32.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:58:32.771 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:32 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:32.484+0000 7fb7e07ed200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T22:58:32.771 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 22:58:32 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y[33710]: 2024-04-26T22:58:32.701+0000 7fb7e07ed200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T22:58:32.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:32 smithi080 ceph-mon[32324]: Standby manager daemon z restarted 2024-04-26T22:58:32.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:32 smithi080 ceph-mon[32324]: Standby manager daemon z started 2024-04-26T22:58:32.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:32 smithi080 ceph-mon[32324]: from='mgr.? 172.21.15.94:0/1947873219' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T22:58:32.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:32 smithi080 ceph-mon[32324]: from='mgr.? 172.21.15.94:0/1947873219' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:58:32.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:32 smithi080 ceph-mon[32324]: from='mgr.? 172.21.15.94:0/1947873219' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T22:58:32.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:32 smithi080 ceph-mon[32324]: from='mgr.? 172.21.15.94:0/1947873219' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:58:32.775 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:58:32.775 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T22:58:32.818 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T22:58:32.818 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:58:32.819 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T22:58:32.902 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-26T22:58:32.902 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-26T22:58:32.902 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/432cadfa-e601-4549-861d-e29e1f305178 2024-04-26T22:58:32.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:58:32.902 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/432cadfa-e601-4549-861d-e29e1f305178 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-26T22:58:33.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:32 smithi094 ceph-mon[32889]: Standby manager daemon z restarted 2024-04-26T22:58:33.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:32 smithi094 ceph-mon[32889]: Standby manager daemon z started 2024-04-26T22:58:33.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:32 smithi094 ceph-mon[32889]: from='mgr.? 172.21.15.94:0/1947873219' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T22:58:33.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:32 smithi094 ceph-mon[32889]: from='mgr.? 172.21.15.94:0/1947873219' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:58:33.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:32 smithi094 ceph-mon[32889]: from='mgr.? 172.21.15.94:0/1947873219' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T22:58:33.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:32 smithi094 ceph-mon[32889]: from='mgr.? 172.21.15.94:0/1947873219' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:58:33.070 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-26T22:58:33.070 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-04-26T22:58:33.364 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:33 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:33.037+0000 7fc9394c7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T22:58:33.365 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:33 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:33.174+0000 7fc9394c7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T22:58:33.698 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:33 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:33.527+0000 7fc9394c7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T22:58:33.910 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-26T22:58:33.910 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-04-26T22:58:33.910 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.80:3300,172.21.15.94:3300" 2024-04-26T22:58:33.910 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@95ebdda6-041e-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/432cadfa-e601-4549-861d-e29e1f305178 2024-04-26T22:58:33.910 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.27:3300/172.21.15.80:3300/172.21.15.94:3300" will pass to kernel 2024-04-26T22:58:33.913 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T22:58:33.913 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-04-26T22:58:33.913 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T22:58:33.913 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T22:58:33.913 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T22:58:33.918 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T22:58:33.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:58:33.918 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T22:58:33.978 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:33 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:33.724+0000 7fc9394c7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T22:58:33.979 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:33 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:33.859+0000 7fc9394c7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T22:58:33.979 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:33 smithi027 ceph-mon[28034]: mgrmap e19: x(active, since 11m), standbys: y, z 2024-04-26T22:58:33.979 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:33 smithi027 ceph-mon[28034]: Standby manager daemon y restarted 2024-04-26T22:58:33.979 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:33 smithi027 ceph-mon[28034]: Standby manager daemon y started 2024-04-26T22:58:33.979 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:33 smithi027 ceph-mon[28034]: from='mgr.? 172.21.15.80:0/1885522169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T22:58:33.979 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:33 smithi027 ceph-mon[28034]: from='mgr.? 172.21.15.80:0/1885522169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:58:33.979 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:33 smithi027 ceph-mon[28034]: from='mgr.? 172.21.15.80:0/1885522169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T22:58:33.979 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:33 smithi027 ceph-mon[28034]: from='mgr.? 172.21.15.80:0/1885522169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:58:33.991 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:58:34.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:33 smithi080 ceph-mon[32324]: mgrmap e19: x(active, since 11m), standbys: y, z 2024-04-26T22:58:34.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:33 smithi080 ceph-mon[32324]: Standby manager daemon y restarted 2024-04-26T22:58:34.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:33 smithi080 ceph-mon[32324]: Standby manager daemon y started 2024-04-26T22:58:34.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:33 smithi080 ceph-mon[32324]: from='mgr.? 172.21.15.80:0/1885522169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T22:58:34.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:33 smithi080 ceph-mon[32324]: from='mgr.? 172.21.15.80:0/1885522169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:58:34.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:33 smithi080 ceph-mon[32324]: from='mgr.? 172.21.15.80:0/1885522169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T22:58:34.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:33 smithi080 ceph-mon[32324]: from='mgr.? 172.21.15.80:0/1885522169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:58:34.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:33 smithi094 ceph-mon[32889]: mgrmap e19: x(active, since 11m), standbys: y, z 2024-04-26T22:58:34.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:33 smithi094 ceph-mon[32889]: Standby manager daemon y restarted 2024-04-26T22:58:34.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:33 smithi094 ceph-mon[32889]: Standby manager daemon y started 2024-04-26T22:58:34.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:33 smithi094 ceph-mon[32889]: from='mgr.? 172.21.15.80:0/1885522169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T22:58:34.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:33 smithi094 ceph-mon[32889]: from='mgr.? 172.21.15.80:0/1885522169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:58:34.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:33 smithi094 ceph-mon[32889]: from='mgr.? 172.21.15.80:0/1885522169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T22:58:34.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:33 smithi094 ceph-mon[32889]: from='mgr.? 172.21.15.80:0/1885522169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:58:34.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:58:34.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:58:34.108 INFO:teuthology.orchestra.run.smithi027.stdout:client25435 2024-04-26T22:58:34.108 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T22:58:34.108 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:58:34.157 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:58:34.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:58:34.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:58:34.327 INFO:teuthology.orchestra.run.smithi027.stdout:client25435 2024-04-26T22:58:34.327 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:58:34.364 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:34.023+0000 7fc9394c7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T22:58:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:58:34.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:58:34.450 INFO:teuthology.orchestra.run.smithi027.stdout:client25435 2024-04-26T22:58:34.450 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/95ebdda6-041e-11ef-bc93-c7b262605968.client25435/status') 2024-04-26T22:58:34.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:58:34.625 INFO:teuthology.orchestra.run.smithi027.stdout:instance: client.25435 (3)172.21.15.27:0/2204837443 2024-04-26T22:58:34.625 INFO:teuthology.orchestra.run.smithi027.stdout:blocklisted: false 2024-04-26T22:58:34.625 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records in 2024-04-26T22:58:34.625 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records out 2024-04-26T22:58:34.625 INFO:teuthology.orchestra.run.smithi027.stderr:71 bytes copied, 8.2804e-05 s, 857 kB/s 2024-04-26T22:58:34.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:58:34.638 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T22:58:34.648 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T22:58:34.648 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T22:58:34.648 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-26T22:58:34.709 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:34.417+0000 7fc9394c7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T22:58:34.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.840+0000 7fa239600640 1 -- 172.21.15.27:0/1718131235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2341028d0 msgr2=0x7fa234102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:34.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.840+0000 7fa239600640 1 --2- 172.21.15.27:0/1718131235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2341028d0 0x7fa234102d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa220009900 tx=0x7fa22002a3f0 comp rx=0 tx=0).stop 2024-04-26T22:58:34.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.840+0000 7fa239600640 1 -- 172.21.15.27:0/1718131235 shutdown_connections 2024-04-26T22:58:34.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.840+0000 7fa239600640 1 --2- 172.21.15.27:0/1718131235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa234103290 0x7fa234109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:34.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.840+0000 7fa239600640 1 --2- 172.21.15.27:0/1718131235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2341028d0 0x7fa234102d50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:34.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.840+0000 7fa239600640 1 --2- 172.21.15.27:0/1718131235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2341016e0 0x7fa234101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:34.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.840+0000 7fa239600640 1 -- 172.21.15.27:0/1718131235 >> 172.21.15.27:0/1718131235 conn(0x7fa2340fcc10 msgr2=0x7fa2340ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:34.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.840+0000 7fa239600640 1 -- 172.21.15.27:0/1718131235 shutdown_connections 2024-04-26T22:58:34.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.840+0000 7fa239600640 1 -- 172.21.15.27:0/1718131235 wait complete. 2024-04-26T22:58:34.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.841+0000 7fa239600640 1 Processor -- start 2024-04-26T22:58:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.841+0000 7fa239600640 1 -- start start 2024-04-26T22:58:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.842+0000 7fa239600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2341016e0 0x7fa23419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.842+0000 7fa239600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2341028d0 0x7fa23419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.842+0000 7fa239600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa234103290 0x7fa2341a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.842+0000 7fa239600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa23410e390 con 0x7fa2341028d0 2024-04-26T22:58:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.842+0000 7fa239600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa23410e210 con 0x7fa234103290 2024-04-26T22:58:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.842+0000 7fa239600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa23410e510 con 0x7fa2341016e0 2024-04-26T22:58:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.842+0000 7fa233400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2341028d0 0x7fa23419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.842+0000 7fa233e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2341016e0 0x7fa23419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.842+0000 7fa233400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2341028d0 0x7fa23419b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46010/0 (socket says 172.21.15.27:46010) 2024-04-26T22:58:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.842+0000 7fa233400640 1 -- 172.21.15.27:0/1550631160 learned_addr learned my addr 172.21.15.27:0/1550631160 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.842+0000 7fa238c00640 1 --2- 172.21.15.27:0/1550631160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa234103290 0x7fa2341a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.843+0000 7fa238c00640 1 -- 172.21.15.27:0/1550631160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2341016e0 msgr2=0x7fa23419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.843+0000 7fa238c00640 1 --2- 172.21.15.27:0/1550631160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2341016e0 0x7fa23419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.843+0000 7fa238c00640 1 -- 172.21.15.27:0/1550631160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2341028d0 msgr2=0x7fa23419b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:34.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.843+0000 7fa238c00640 1 --2- 172.21.15.27:0/1550631160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2341028d0 0x7fa23419b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:34.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.843+0000 7fa238c00640 1 -- 172.21.15.27:0/1550631160 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2341a27f0 con 0x7fa234103290 2024-04-26T22:58:34.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.843+0000 7fa233400640 1 --2- 172.21.15.27:0/1550631160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2341028d0 0x7fa23419b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:34.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.843+0000 7fa238c00640 1 --2- 172.21.15.27:0/1550631160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa234103290 0x7fa2341a2180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009fd0 tx=0x7fa22c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:34.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.843+0000 7fa230c00640 1 -- 172.21.15.27:0/1550631160 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c00ce20 con 0x7fa234103290 2024-04-26T22:58:34.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.843+0000 7fa239600640 1 -- 172.21.15.27:0/1550631160 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2341a2ae0 con 0x7fa234103290 2024-04-26T22:58:34.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.843+0000 7fa230c00640 1 -- 172.21.15.27:0/1550631160 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa22c005020 con 0x7fa234103290 2024-04-26T22:58:34.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.843+0000 7fa230c00640 1 -- 172.21.15.27:0/1550631160 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c010690 con 0x7fa234103290 2024-04-26T22:58:34.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.844+0000 7fa239600640 1 -- 172.21.15.27:0/1550631160 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2341a2fe0 con 0x7fa234103290 2024-04-26T22:58:34.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.845+0000 7fa239600640 1 -- 172.21.15.27:0/1550631160 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2341a3270 con 0x7fa234103290 2024-04-26T22:58:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.847+0000 7fa230c00640 1 -- 172.21.15.27:0/1550631160 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 20) ==== 138714+0+0 (secure 0 0 0) 0x7fa22c0054d0 con 0x7fa234103290 2024-04-26T22:58:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.848+0000 7fa230c00640 1 --2- 172.21.15.27:0/1550631160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2180a3c70 0x7fa2180a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.848+0000 7fa233e00640 1 -- 172.21.15.27:0/1550631160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2180a3c70 msgr2=0x7fa2180a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3636526714 2024-04-26T22:58:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.848+0000 7fa230c00640 1 -- 172.21.15.27:0/1550631160 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9311+0+0 (secure 0 0 0) 0x7fa22c0d1140 con 0x7fa234103290 2024-04-26T22:58:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.848+0000 7fa233e00640 1 --2- 172.21.15.27:0/1550631160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2180a3c70 0x7fa2180a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T22:58:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:34.852+0000 7fa230c00640 1 -- 172.21.15.27:0/1550631160 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa22c09d530 con 0x7fa234103290 2024-04-26T22:58:35.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:34 smithi080 ceph-mon[32324]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-04-26T22:58:35.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:34 smithi094 ceph-mon[32889]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-04-26T22:58:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.049+0000 7fa233e00640 1 -- 172.21.15.27:0/1550631160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2180a3c70 msgr2=0x7fa2180a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3636526714 2024-04-26T22:58:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.049+0000 7fa233e00640 1 --2- 172.21.15.27:0/1550631160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2180a3c70 0x7fa2180a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T22:58:35.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.096+0000 7fa239600640 1 -- 172.21.15.27:0/1550631160 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fa2341a3660 con 0x7fa234103290 2024-04-26T22:58:35.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:34 smithi027 ceph-mon[28034]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-04-26T22:58:35.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.450+0000 7fa233e00640 1 -- 172.21.15.27:0/1550631160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2180a3c70 msgr2=0x7fa2180a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3636526714 2024-04-26T22:58:35.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.450+0000 7fa233e00640 1 --2- 172.21.15.27:0/1550631160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2180a3c70 0x7fa2180a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T22:58:35.538 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:35.233+0000 7fc9394c7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T22:58:35.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.565+0000 7fa230c00640 1 -- 172.21.15.27:0/1550631160 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7fa22c0994a0 con 0x7fa234103290 2024-04-26T22:58:35.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.565+0000 7fa230c00640 1 -- 172.21.15.27:0/1550631160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2180a3c70 msgr2=0x7fa2180a6130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:58:35.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.565+0000 7fa230c00640 1 --2- 172.21.15.27:0/1550631160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2180a3c70 0x7fa2180a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.724+0000 7fa230c00640 1 -- 172.21.15.27:0/1550631160 <== mon.2 v2:172.21.15.80:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7fa22c065f80 con 0x7fa234103290 2024-04-26T22:58:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.728+0000 7fa239600640 1 -- 172.21.15.27:0/1550631160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa234103290 msgr2=0x7fa2341a2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.728+0000 7fa239600640 1 --2- 172.21.15.27:0/1550631160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa234103290 0x7fa2341a2180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009fd0 tx=0x7fa22c00ef10 comp rx=0 tx=0).stop 2024-04-26T22:58:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.728+0000 7fa239600640 1 -- 172.21.15.27:0/1550631160 shutdown_connections 2024-04-26T22:58:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.728+0000 7fa239600640 1 --2- 172.21.15.27:0/1550631160 >> [v2:172.21.15.27:6800/3636526714,v1:172.21.15.27:6801/3636526714] conn(0x7fa2180a3c70 0x7fa2180a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:35.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.728+0000 7fa239600640 1 --2- 172.21.15.27:0/1550631160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa234103290 0x7fa2341a2180 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:35.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.728+0000 7fa239600640 1 --2- 172.21.15.27:0/1550631160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2341028d0 0x7fa23419b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:35.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.728+0000 7fa239600640 1 --2- 172.21.15.27:0/1550631160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2341016e0 0x7fa23419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:35.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.728+0000 7fa239600640 1 -- 172.21.15.27:0/1550631160 >> 172.21.15.27:0/1550631160 conn(0x7fa2340fcc10 msgr2=0x7fa23410a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:35.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.728+0000 7fa239600640 1 -- 172.21.15.27:0/1550631160 shutdown_connections 2024-04-26T22:58:35.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:35.729+0000 7fa239600640 1 -- 172.21.15.27:0/1550631160 wait complete. 2024-04-26T22:58:35.750 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-26T22:58:35.761 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T22:58:35.802 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T22:58:35.802 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T22:58:35.802 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-26T22:58:35.804 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1550631160' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T22:58:35.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T22:58:35.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: Active manager daemon x restarted 2024-04-26T22:58:35.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: Activating manager daemon x 2024-04-26T22:58:35.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T22:58:35.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: mgrmap e21: x(active, starting, since 0.0290131s), standbys: y, z 2024-04-26T22:58:35.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:58:35.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:58:35.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:58:35.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T22:58:35.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T22:58:35.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T22:58:35.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T22:58:35.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T22:58:35.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T22:58:35.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T22:58:35.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T22:58:35.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T22:58:35.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T22:58:35.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T22:58:35.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T22:58:35.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T22:58:35.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T22:58:35.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T22:58:35.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:58:35.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:58:35.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:58:35.807 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:58:35.807 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:58:35.807 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:58:35.807 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:58:35.807 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:58:35.807 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:58:35.807 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:58:35.807 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:58:35.807 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:58:35.807 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T22:58:35.807 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T22:58:35.807 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T22:58:35.807 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: Manager daemon x is now available 2024-04-26T22:58:35.807 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:58:35.808 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:35.537+0000 7fc9394c7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T22:58:36.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.015+0000 7f981d800640 1 -- 172.21.15.27:0/846424535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98181034e0 msgr2=0x7f98181058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:36.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.015+0000 7f981d800640 1 --2- 172.21.15.27:0/846424535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98181034e0 0x7f98181058d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9808009900 tx=0x7f980802a3f0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.015+0000 7f981d800640 1 -- 172.21.15.27:0/846424535 shutdown_connections 2024-04-26T22:58:36.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.015+0000 7f981d800640 1 --2- 172.21.15.27:0/846424535 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9818108740 0x7f981810ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.015+0000 7f981d800640 1 --2- 172.21.15.27:0/846424535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9818105e10 0x7f9818108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.015+0000 7f981d800640 1 --2- 172.21.15.27:0/846424535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98181034e0 0x7f98181058d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.015+0000 7f981d800640 1 -- 172.21.15.27:0/846424535 >> 172.21.15.27:0/846424535 conn(0x7f98180fce50 msgr2=0x7f98180ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:36.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.016+0000 7f981d800640 1 -- 172.21.15.27:0/846424535 shutdown_connections 2024-04-26T22:58:36.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.016+0000 7f981d800640 1 -- 172.21.15.27:0/846424535 wait complete. 2024-04-26T22:58:36.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.017+0000 7f981d800640 1 Processor -- start 2024-04-26T22:58:36.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.017+0000 7f981d800640 1 -- start start 2024-04-26T22:58:36.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.017+0000 7f981d800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98181034e0 0x7f98181969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:36.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.018+0000 7f981d800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9818105e10 0x7f9818196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:36.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.018+0000 7f981d800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9818108740 0x7f981819df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:36.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.018+0000 7f981d800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f981810c350 con 0x7f9818105e10 2024-04-26T22:58:36.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.018+0000 7f981d800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f981810c1d0 con 0x7f9818108740 2024-04-26T22:58:36.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.018+0000 7f981d800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f981810c4d0 con 0x7f98181034e0 2024-04-26T22:58:36.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.018+0000 7f9816a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98181034e0 0x7f98181969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:36.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.018+0000 7f9816a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98181034e0 0x7f98181969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:53444/0 (socket says 172.21.15.27:53444) 2024-04-26T22:58:36.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.018+0000 7f9816a00640 1 -- 172.21.15.27:0/988828768 learned_addr learned my addr 172.21.15.27:0/988828768 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:36.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.018+0000 7f9816000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9818105e10 0x7f9818196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:36.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.018+0000 7f9817400640 1 --2- 172.21.15.27:0/988828768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9818108740 0x7f981819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:36.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.018+0000 7f9816a00640 1 -- 172.21.15.27:0/988828768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9818108740 msgr2=0x7f981819df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:36.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.018+0000 7f9816a00640 1 --2- 172.21.15.27:0/988828768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9818108740 0x7f981819df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.018+0000 7f9816a00640 1 -- 172.21.15.27:0/988828768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9818105e10 msgr2=0x7f9818196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:36.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.018+0000 7f9816a00640 1 --2- 172.21.15.27:0/988828768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9818105e10 0x7f9818196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.018+0000 7f9816a00640 1 -- 172.21.15.27:0/988828768 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f981819e5e0 con 0x7f98181034e0 2024-04-26T22:58:36.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.019+0000 7f9816a00640 1 --2- 172.21.15.27:0/988828768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98181034e0 0x7f98181969c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9808005940 tx=0x7f98080042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:36.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.019+0000 7f980f400640 1 -- 172.21.15.27:0/988828768 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980803f030 con 0x7f98181034e0 2024-04-26T22:58:36.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.019+0000 7f981d800640 1 -- 172.21.15.27:0/988828768 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f981819e870 con 0x7f98181034e0 2024-04-26T22:58:36.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.019+0000 7f980f400640 1 -- 172.21.15.27:0/988828768 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9808003cb0 con 0x7f98181034e0 2024-04-26T22:58:36.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.019+0000 7f980f400640 1 -- 172.21.15.27:0/988828768 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808035070 con 0x7f98181034e0 2024-04-26T22:58:36.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.019+0000 7f981d800640 1 -- 172.21.15.27:0/988828768 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f981819edb0 con 0x7f98181034e0 2024-04-26T22:58:36.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.020+0000 7f981d800640 1 -- 172.21.15.27:0/988828768 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98180fe940 con 0x7f98181034e0 2024-04-26T22:58:36.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1550631160' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T22:58:36.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T22:58:36.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: Active manager daemon x restarted 2024-04-26T22:58:36.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: Activating manager daemon x 2024-04-26T22:58:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T22:58:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: mgrmap e21: x(active, starting, since 0.0290131s), standbys: y, z 2024-04-26T22:58:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:58:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:58:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:58:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T22:58:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T22:58:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T22:58:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T22:58:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T22:58:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T22:58:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T22:58:36.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T22:58:36.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T22:58:36.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T22:58:36.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T22:58:36.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T22:58:36.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T22:58:36.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T22:58:36.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T22:58:36.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:58:36.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:58:36.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:58:36.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:58:36.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:58:36.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:58:36.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:58:36.023 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:58:36.024 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:58:36.024 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:58:36.024 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:58:36.024 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:58:36.024 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T22:58:36.024 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T22:58:36.024 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T22:58:36.024 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: Manager daemon x is now available 2024-04-26T22:58:36.024 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:35 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:58:36.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.024+0000 7f980f400640 1 -- 172.21.15.27:0/988828768 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f9808042030 con 0x7f98181034e0 2024-04-26T22:58:36.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.025+0000 7f980f400640 1 -- 172.21.15.27:0/988828768 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f98080f0740 con 0x7f98181034e0 2024-04-26T22:58:36.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.029+0000 7f980f400640 1 -- 172.21.15.27:0/988828768 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f98080bc910 con 0x7f98181034e0 2024-04-26T22:58:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1550631160' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T22:58:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T22:58:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: Active manager daemon x restarted 2024-04-26T22:58:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: Activating manager daemon x 2024-04-26T22:58:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T22:58:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: mgrmap e21: x(active, starting, since 0.0290131s), standbys: y, z 2024-04-26T22:58:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:58:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:58:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:58:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T22:58:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T22:58:36.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T22:58:36.036 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T22:58:36.036 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T22:58:36.036 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T22:58:36.036 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T22:58:36.036 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T22:58:36.036 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T22:58:36.036 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T22:58:36.036 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T22:58:36.036 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T22:58:36.036 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T22:58:36.036 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T22:58:36.036 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T22:58:36.036 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:58:36.036 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:58:36.036 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:58:36.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:58:36.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:58:36.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:58:36.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:58:36.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:58:36.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:58:36.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:58:36.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:58:36.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:58:36.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T22:58:36.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T22:58:36.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T22:58:36.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: Manager daemon x is now available 2024-04-26T22:58:36.037 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:35 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:58:36.115 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:35.924+0000 7fc88ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:58:36.116 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:35.924+0000 7fc88ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:58:36.116 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:35.924+0000 7fc88ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:58:36.116 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:35.924+0000 7fc88ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:58:36.116 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:35.924+0000 7fc88ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:58:36.116 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:35.926+0000 7fc88b200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:58:36.116 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:35.926+0000 7fc88b200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:58:36.116 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:35.926+0000 7fc88b200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:58:36.116 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:35.926+0000 7fc88b200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:58:36.116 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:58:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:58:35.926+0000 7fc88b200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:58:36.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.579+0000 7f980f400640 1 -- 172.21.15.27:0/988828768 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f98080b8660 con 0x7f98181034e0 2024-04-26T22:58:36.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.580+0000 7f980f400640 1 --2- 172.21.15.27:0/988828768 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f97fc0ac310 0x7f97fc0ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:36.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.580+0000 7f980f400640 1 -- 172.21.15.27:0/988828768 --> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f9808041e80 con 0x7f97fc0ac310 2024-04-26T22:58:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.589+0000 7f9816000640 1 --2- 172.21.15.27:0/988828768 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f97fc0ac310 0x7f97fc0ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:36.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.600+0000 7f9816000640 1 --2- 172.21.15.27:0/988828768 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f97fc0ac310 0x7f97fc0ae700 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9818197ed0 tx=0x7f9804005f90 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:36.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.718+0000 7f980f400640 1 -- 172.21.15.27:0/988828768 <== mgr.25391 v2:172.21.15.27:6800/3058395889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9808041e80 con 0x7f97fc0ac310 2024-04-26T22:58:36.719 INFO:teuthology.orchestra.run.smithi027.stdout:Schedule set for path / 2024-04-26T22:58:36.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.721+0000 7f981d800640 1 -- 172.21.15.27:0/988828768 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f97fc0ac310 msgr2=0x7f97fc0ae700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:36.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.721+0000 7f981d800640 1 --2- 172.21.15.27:0/988828768 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f97fc0ac310 0x7f97fc0ae700 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9818197ed0 tx=0x7f9804005f90 comp rx=0 tx=0).stop 2024-04-26T22:58:36.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.721+0000 7f981d800640 1 -- 172.21.15.27:0/988828768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98181034e0 msgr2=0x7f98181969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.721+0000 7f981d800640 1 --2- 172.21.15.27:0/988828768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98181034e0 0x7f98181969c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9808005940 tx=0x7f98080042f0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.721+0000 7f981d800640 1 -- 172.21.15.27:0/988828768 shutdown_connections 2024-04-26T22:58:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.721+0000 7f981d800640 1 --2- 172.21.15.27:0/988828768 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f97fc0ac310 0x7f97fc0ae700 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.721+0000 7f981d800640 1 --2- 172.21.15.27:0/988828768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9818108740 0x7f981819df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.721+0000 7f981d800640 1 --2- 172.21.15.27:0/988828768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9818105e10 0x7f9818196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.721+0000 7f981d800640 1 --2- 172.21.15.27:0/988828768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98181034e0 0x7f98181969c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.721+0000 7f981d800640 1 -- 172.21.15.27:0/988828768 >> 172.21.15.27:0/988828768 conn(0x7f98180fce50 msgr2=0x7f9818103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.721+0000 7f981d800640 1 -- 172.21.15.27:0/988828768 shutdown_connections 2024-04-26T22:58:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.721+0000 7f981d800640 1 -- 172.21.15.27:0/988828768 wait complete. 2024-04-26T22:58:36.740 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-26T22:58:36.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.931+0000 7f6ed2000640 1 -- 172.21.15.27:0/2500464627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ecc10ef70 msgr2=0x7f6ecc111360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:36.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.931+0000 7f6ed2000640 1 --2- 172.21.15.27:0/2500464627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ecc10ef70 0x7f6ecc111360 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009cd0 tx=0x7f6ec402a8e0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.931+0000 7f6ed2000640 1 -- 172.21.15.27:0/2500464627 shutdown_connections 2024-04-26T22:58:36.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.931+0000 7f6ed2000640 1 --2- 172.21.15.27:0/2500464627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ecc10ef70 0x7f6ecc111360 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.931+0000 7f6ed2000640 1 --2- 172.21.15.27:0/2500464627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ecc075390 0x7f6ecc075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.931+0000 7f6ed2000640 1 --2- 172.21.15.27:0/2500464627 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc076970 0x7f6ecc074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.931+0000 7f6ed2000640 1 -- 172.21.15.27:0/2500464627 >> 172.21.15.27:0/2500464627 conn(0x7f6ecc0fdf10 msgr2=0x7f6ecc100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:36.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.931+0000 7f6ed2000640 1 -- 172.21.15.27:0/2500464627 shutdown_connections 2024-04-26T22:58:36.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.931+0000 7f6ed2000640 1 -- 172.21.15.27:0/2500464627 wait complete. 2024-04-26T22:58:36.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.932+0000 7f6ed2000640 1 Processor -- start 2024-04-26T22:58:36.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.932+0000 7f6ed2000640 1 -- start start 2024-04-26T22:58:36.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.933+0000 7f6ed2000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ecc075390 0x7f6ecc19f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.933+0000 7f6ed2000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc076970 0x7f6ecc19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.933+0000 7f6ed2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ecc10ef70 0x7f6ecc1a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.933+0000 7f6ed2000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc114b30 con 0x7f6ecc10ef70 2024-04-26T22:58:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.933+0000 7f6ed2000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1149b0 con 0x7f6ecc076970 2024-04-26T22:58:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.933+0000 7f6ed2000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc114cb0 con 0x7f6ecc075390 2024-04-26T22:58:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.933+0000 7f6ecaa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc076970 0x7f6ecc19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.933+0000 7f6ecbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ecc10ef70 0x7f6ecc1a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.933+0000 7f6ecaa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc076970 0x7f6ecc19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:53976/0 (socket says 172.21.15.27:53976) 2024-04-26T22:58:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.933+0000 7f6ecaa00640 1 -- 172.21.15.27:0/3838710238 learned_addr learned my addr 172.21.15.27:0/3838710238 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.933+0000 7f6ecb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ecc075390 0x7f6ecc19f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.933+0000 7f6ecaa00640 1 -- 172.21.15.27:0/3838710238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ecc075390 msgr2=0x7f6ecc19f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.933+0000 7f6ecaa00640 1 --2- 172.21.15.27:0/3838710238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ecc075390 0x7f6ecc19f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.933+0000 7f6ecaa00640 1 -- 172.21.15.27:0/3838710238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ecc10ef70 msgr2=0x7f6ecc1a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.933+0000 7f6ecaa00640 1 --2- 172.21.15.27:0/3838710238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ecc10ef70 0x7f6ecc1a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:36.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.934+0000 7f6ecaa00640 1 -- 172.21.15.27:0/3838710238 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc1a6e60 con 0x7f6ecc076970 2024-04-26T22:58:36.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.934+0000 7f6ecaa00640 1 --2- 172.21.15.27:0/3838710238 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc076970 0x7f6ecc19f7d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800e9f0 tx=0x7f6eb800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:36.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.934+0000 7f6ec3e00640 1 -- 172.21.15.27:0/3838710238 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb800cd80 con 0x7f6ecc076970 2024-04-26T22:58:36.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.934+0000 7f6ed2000640 1 -- 172.21.15.27:0/3838710238 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ecc1a7150 con 0x7f6ecc076970 2024-04-26T22:58:36.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.934+0000 7f6ec3e00640 1 -- 172.21.15.27:0/3838710238 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb80054d0 con 0x7f6ecc076970 2024-04-26T22:58:36.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.934+0000 7f6ec3e00640 1 -- 172.21.15.27:0/3838710238 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8010690 con 0x7f6ecc076970 2024-04-26T22:58:36.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.934+0000 7f6ed2000640 1 -- 172.21.15.27:0/3838710238 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ecc1a76a0 con 0x7f6ecc076970 2024-04-26T22:58:36.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.936+0000 7f6ed2000640 1 -- 172.21.15.27:0/3838710238 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ecc105140 con 0x7f6ecc076970 2024-04-26T22:58:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.938+0000 7f6ec3e00640 1 -- 172.21.15.27:0/3838710238 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f6eb8004e90 con 0x7f6ecc076970 2024-04-26T22:58:36.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.940+0000 7f6ec3e00640 1 --2- 172.21.15.27:0/3838710238 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6eb00a3a10 0x7f6eb00a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:36.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.940+0000 7f6ec3e00640 1 -- 172.21.15.27:0/3838710238 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f6eb80d0930 con 0x7f6ecc076970 2024-04-26T22:58:36.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.940+0000 7f6ecb400640 1 --2- 172.21.15.27:0/3838710238 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6eb00a3a10 0x7f6eb00a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:36.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.941+0000 7f6ecb400640 1 --2- 172.21.15.27:0/3838710238 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6eb00a3a10 0x7f6eb00a5ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0095f0 tx=0x7f6ebc005f90 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:36.944+0000 7f6ec3e00640 1 -- 172.21.15.27:0/3838710238 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6eb809d060 con 0x7f6ecc076970 2024-04-26T22:58:36.992 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:36 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T22:58:36.992 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:36 smithi094 ceph-mon[32889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T22:58:36.992 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:36 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T22:58:36.992 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:36 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T22:58:36.992 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:36 smithi094 ceph-mon[32889]: mgrmap e22: x(active, since 1.03787s), standbys: y, z 2024-04-26T22:58:37.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:36 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T22:58:37.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:36 smithi080 ceph-mon[32324]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T22:58:37.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:36 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T22:58:37.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:36 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T22:58:37.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:36 smithi080 ceph-mon[32324]: mgrmap e22: x(active, since 1.03787s), standbys: y, z 2024-04-26T22:58:37.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:36 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T22:58:37.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:36 smithi027 ceph-mon[28034]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T22:58:37.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:36 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T22:58:37.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:36 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T22:58:37.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:36 smithi027 ceph-mon[28034]: mgrmap e22: x(active, since 1.03787s), standbys: y, z 2024-04-26T22:58:37.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.193+0000 7f6ed2000640 1 -- 172.21.15.27:0/3838710238 --> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f6ecc062e10 con 0x7f6eb00a3a10 2024-04-26T22:58:37.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.335+0000 7f6ec3e00640 1 -- 172.21.15.27:0/3838710238 <== mgr.25391 v2:172.21.15.27:6800/3058395889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f6ecc062e10 con 0x7f6eb00a3a10 2024-04-26T22:58:37.336 INFO:teuthology.orchestra.run.smithi027.stdout:Retention added to path / 2024-04-26T22:58:37.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.338+0000 7f6ed2000640 1 -- 172.21.15.27:0/3838710238 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6eb00a3a10 msgr2=0x7f6eb00a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:37.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.338+0000 7f6ed2000640 1 --2- 172.21.15.27:0/3838710238 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6eb00a3a10 0x7f6eb00a5ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0095f0 tx=0x7f6ebc005f90 comp rx=0 tx=0).stop 2024-04-26T22:58:37.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.338+0000 7f6ed2000640 1 -- 172.21.15.27:0/3838710238 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc076970 msgr2=0x7f6ecc19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:37.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.338+0000 7f6ed2000640 1 --2- 172.21.15.27:0/3838710238 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc076970 0x7f6ecc19f7d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800e9f0 tx=0x7f6eb800edf0 comp rx=0 tx=0).stop 2024-04-26T22:58:37.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.338+0000 7f6ed2000640 1 -- 172.21.15.27:0/3838710238 shutdown_connections 2024-04-26T22:58:37.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.338+0000 7f6ed2000640 1 --2- 172.21.15.27:0/3838710238 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6eb00a3a10 0x7f6eb00a5ed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:37.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.338+0000 7f6ed2000640 1 --2- 172.21.15.27:0/3838710238 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc076970 0x7f6ecc19f7d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:37.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.338+0000 7f6ed2000640 1 --2- 172.21.15.27:0/3838710238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ecc10ef70 0x7f6ecc1a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:37.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.338+0000 7f6ed2000640 1 --2- 172.21.15.27:0/3838710238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ecc075390 0x7f6ecc19f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:37.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.338+0000 7f6ed2000640 1 -- 172.21.15.27:0/3838710238 >> 172.21.15.27:0/3838710238 conn(0x7f6ecc0fdf10 msgr2=0x7f6ecc102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:37.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.338+0000 7f6ed2000640 1 -- 172.21.15.27:0/3838710238 shutdown_connections 2024-04-26T22:58:37.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.339+0000 7f6ed2000640 1 -- 172.21.15.27:0/3838710238 wait complete. 2024-04-26T22:58:37.356 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T22:58:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.552+0000 7f3f85600640 1 -- 172.21.15.27:0/3542655750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80075390 msgr2=0x7f3f80075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.552+0000 7f3f85600640 1 --2- 172.21.15.27:0/3542655750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80075390 0x7f3f80075810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009900 tx=0x7f3f6802a3f0 comp rx=0 tx=0).stop 2024-04-26T22:58:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.552+0000 7f3f85600640 1 -- 172.21.15.27:0/3542655750 shutdown_connections 2024-04-26T22:58:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.552+0000 7f3f85600640 1 --2- 172.21.15.27:0/3542655750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f8010edc0 0x7f3f801111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.552+0000 7f3f85600640 1 --2- 172.21.15.27:0/3542655750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80075390 0x7f3f80075810 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.552+0000 7f3f85600640 1 --2- 172.21.15.27:0/3542655750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f80076970 0x7f3f80074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.552+0000 7f3f85600640 1 -- 172.21.15.27:0/3542655750 >> 172.21.15.27:0/3542655750 conn(0x7f3f800fdd60 msgr2=0x7f3f801001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:37.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.552+0000 7f3f85600640 1 -- 172.21.15.27:0/3542655750 shutdown_connections 2024-04-26T22:58:37.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.552+0000 7f3f85600640 1 -- 172.21.15.27:0/3542655750 wait complete. 2024-04-26T22:58:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.553+0000 7f3f85600640 1 Processor -- start 2024-04-26T22:58:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.554+0000 7f3f85600640 1 -- start start 2024-04-26T22:58:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.554+0000 7f3f85600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f80075390 0x7f3f8019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.554+0000 7f3f85600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80076970 0x7f3f8019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.554+0000 7f3f85600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f8010edc0 0x7f3f801a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.554+0000 7f3f85600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80114940 con 0x7f3f80075390 2024-04-26T22:58:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.554+0000 7f3f85600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801147c0 con 0x7f3f8010edc0 2024-04-26T22:58:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.554+0000 7f3f85600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80114ac0 con 0x7f3f80076970 2024-04-26T22:58:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.554+0000 7f3f7fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f80075390 0x7f3f8019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.554+0000 7f3f7fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f80075390 0x7f3f8019f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46166/0 (socket says 172.21.15.27:46166) 2024-04-26T22:58:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.554+0000 7f3f7fe00640 1 -- 172.21.15.27:0/676783620 learned_addr learned my addr 172.21.15.27:0/676783620 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.554+0000 7f3f77e00640 1 --2- 172.21.15.27:0/676783620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80076970 0x7f3f8019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.554+0000 7f3f84c00640 1 --2- 172.21.15.27:0/676783620 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f8010edc0 0x7f3f801a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.555+0000 7f3f7fe00640 1 -- 172.21.15.27:0/676783620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80076970 msgr2=0x7f3f8019f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.555+0000 7f3f7fe00640 1 --2- 172.21.15.27:0/676783620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80076970 0x7f3f8019f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.555+0000 7f3f7fe00640 1 -- 172.21.15.27:0/676783620 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f8010edc0 msgr2=0x7f3f801a65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.555+0000 7f3f7fe00640 1 --2- 172.21.15.27:0/676783620 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f8010edc0 0x7f3f801a65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.555+0000 7f3f7fe00640 1 -- 172.21.15.27:0/676783620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f801a6c30 con 0x7f3f80075390 2024-04-26T22:58:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.555+0000 7f3f7fe00640 1 --2- 172.21.15.27:0/676783620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f80075390 0x7f3f8019f060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000e9f0 tx=0x7f3f7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.555+0000 7f3f7d600640 1 -- 172.21.15.27:0/676783620 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7000cd80 con 0x7f3f80075390 2024-04-26T22:58:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.555+0000 7f3f7d600640 1 -- 172.21.15.27:0/676783620 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f700108e0 con 0x7f3f80075390 2024-04-26T22:58:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.556+0000 7f3f85600640 1 -- 172.21.15.27:0/676783620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f801a6f20 con 0x7f3f80075390 2024-04-26T22:58:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.556+0000 7f3f7d600640 1 -- 172.21.15.27:0/676783620 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70018980 con 0x7f3f80075390 2024-04-26T22:58:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.556+0000 7f3f85600640 1 -- 172.21.15.27:0/676783620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f801a7380 con 0x7f3f80075390 2024-04-26T22:58:37.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.557+0000 7f3f85600640 1 -- 172.21.15.27:0/676783620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f80104f90 con 0x7f3f80075390 2024-04-26T22:58:37.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.559+0000 7f3f7d600640 1 -- 172.21.15.27:0/676783620 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f3f70010430 con 0x7f3f80075390 2024-04-26T22:58:37.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.560+0000 7f3f7d600640 1 --2- 172.21.15.27:0/676783620 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f3f640a3990 0x7f3f640a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:37.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.561+0000 7f3f7d600640 1 -- 172.21.15.27:0/676783620 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f3f700d0850 con 0x7f3f80075390 2024-04-26T22:58:37.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.561+0000 7f3f77e00640 1 --2- 172.21.15.27:0/676783620 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f3f640a3990 0x7f3f640a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:37.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.562+0000 7f3f77e00640 1 --2- 172.21.15.27:0/676783620 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f3f640a3990 0x7f3f640a5e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3f68005ab0 tx=0x7f3f68005a20 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:37.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.565+0000 7f3f7d600640 1 -- 172.21.15.27:0/676783620 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3f7009c930 con 0x7f3f80075390 2024-04-26T22:58:37.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.751+0000 7f3f7d600640 1 -- 172.21.15.27:0/676783620 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f3f70098fa0 con 0x7f3f80075390 2024-04-26T22:58:37.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.818+0000 7f3f85600640 1 -- 172.21.15.27:0/676783620 --> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f3f80062d70 con 0x7f3f640a3990 2024-04-26T22:58:37.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.841+0000 7f3f7d600640 1 -- 172.21.15.27:0/676783620 <== mgr.25391 v2:172.21.15.27:6800/3058395889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f3f80062d70 con 0x7f3f640a3990 2024-04-26T22:58:37.842 INFO:teuthology.orchestra.run.smithi027.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T22:58:36", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-26T22:58:37.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.844+0000 7f3f85600640 1 -- 172.21.15.27:0/676783620 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f3f640a3990 msgr2=0x7f3f640a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:37.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.844+0000 7f3f85600640 1 --2- 172.21.15.27:0/676783620 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f3f640a3990 0x7f3f640a5e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3f68005ab0 tx=0x7f3f68005a20 comp rx=0 tx=0).stop 2024-04-26T22:58:37.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.844+0000 7f3f85600640 1 -- 172.21.15.27:0/676783620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f80075390 msgr2=0x7f3f8019f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:37.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.844+0000 7f3f85600640 1 --2- 172.21.15.27:0/676783620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f80075390 0x7f3f8019f060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000e9f0 tx=0x7f3f7000eeb0 comp rx=0 tx=0).stop 2024-04-26T22:58:37.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.845+0000 7f3f85600640 1 -- 172.21.15.27:0/676783620 shutdown_connections 2024-04-26T22:58:37.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.845+0000 7f3f85600640 1 --2- 172.21.15.27:0/676783620 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f3f640a3990 0x7f3f640a5e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:37.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.845+0000 7f3f85600640 1 --2- 172.21.15.27:0/676783620 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f8010edc0 0x7f3f801a65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:37.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.845+0000 7f3f85600640 1 --2- 172.21.15.27:0/676783620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80076970 0x7f3f8019f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:37.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.845+0000 7f3f85600640 1 --2- 172.21.15.27:0/676783620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f80075390 0x7f3f8019f060 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:37.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.845+0000 7f3f85600640 1 -- 172.21.15.27:0/676783620 >> 172.21.15.27:0/676783620 conn(0x7f3f800fdd60 msgr2=0x7f3f80102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:37.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.845+0000 7f3f85600640 1 -- 172.21.15.27:0/676783620 shutdown_connections 2024-04-26T22:58:37.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:37.845+0000 7f3f85600640 1 -- 172.21.15.27:0/676783620 wait complete. 2024-04-26T22:58:37.864 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T22:58:38.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:37 smithi080 ceph-mon[32324]: [26/Apr/2024:22:58:36] ENGINE Bus STARTING 2024-04-26T22:58:38.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:37 smithi080 ceph-mon[32324]: [26/Apr/2024:22:58:36] ENGINE Serving on http://172.21.15.27:8765 2024-04-26T22:58:38.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:37 smithi080 ceph-mon[32324]: [26/Apr/2024:22:58:36] ENGINE Serving on https://172.21.15.27:7150 2024-04-26T22:58:38.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:37 smithi080 ceph-mon[32324]: [26/Apr/2024:22:58:36] ENGINE Bus STARTED 2024-04-26T22:58:38.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:37 smithi080 ceph-mon[32324]: from='client.25483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:38.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:37 smithi080 ceph-mon[32324]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:58:38.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:37 smithi094 ceph-mon[32889]: [26/Apr/2024:22:58:36] ENGINE Bus STARTING 2024-04-26T22:58:38.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:37 smithi094 ceph-mon[32889]: [26/Apr/2024:22:58:36] ENGINE Serving on http://172.21.15.27:8765 2024-04-26T22:58:38.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:37 smithi094 ceph-mon[32889]: [26/Apr/2024:22:58:36] ENGINE Serving on https://172.21.15.27:7150 2024-04-26T22:58:38.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:37 smithi094 ceph-mon[32889]: [26/Apr/2024:22:58:36] ENGINE Bus STARTED 2024-04-26T22:58:38.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:37 smithi094 ceph-mon[32889]: from='client.25483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:38.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:37 smithi094 ceph-mon[32889]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:58:38.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.073+0000 7f7491200640 1 -- 172.21.15.27:0/3766462750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f748c076970 msgr2=0x7f748c074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:38.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.073+0000 7f7491200640 1 --2- 172.21.15.27:0/3766462750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f748c076970 0x7f748c074dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f747c009900 tx=0x7f747c02a400 comp rx=0 tx=0).stop 2024-04-26T22:58:38.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.073+0000 7f7491200640 1 -- 172.21.15.27:0/3766462750 shutdown_connections 2024-04-26T22:58:38.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.073+0000 7f7491200640 1 --2- 172.21.15.27:0/3766462750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f748c10edc0 0x7f748c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:38.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.073+0000 7f7491200640 1 --2- 172.21.15.27:0/3766462750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f748c075390 0x7f748c075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:38.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.073+0000 7f7491200640 1 --2- 172.21.15.27:0/3766462750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f748c076970 0x7f748c074dc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:38.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.073+0000 7f7491200640 1 -- 172.21.15.27:0/3766462750 >> 172.21.15.27:0/3766462750 conn(0x7f748c0fdd60 msgr2=0x7f748c1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:38.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.074+0000 7f7491200640 1 -- 172.21.15.27:0/3766462750 shutdown_connections 2024-04-26T22:58:38.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.074+0000 7f7491200640 1 -- 172.21.15.27:0/3766462750 wait complete. 2024-04-26T22:58:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.075+0000 7f7491200640 1 Processor -- start 2024-04-26T22:58:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.075+0000 7f7491200640 1 -- start start 2024-04-26T22:58:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.075+0000 7f7491200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f748c075390 0x7f748c19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.075+0000 7f7491200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f748c076970 0x7f748c19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.075+0000 7f7491200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f748c10edc0 0x7f748c1a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.075+0000 7f7491200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f748c114940 con 0x7f748c10edc0 2024-04-26T22:58:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.075+0000 7f7491200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f748c1147c0 con 0x7f748c075390 2024-04-26T22:58:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.075+0000 7f7491200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f748c114ac0 con 0x7f748c076970 2024-04-26T22:58:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.076+0000 7f748b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f748c075390 0x7f748c19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.076+0000 7f748aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f748c076970 0x7f748c19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.076+0000 7f748aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f748c076970 0x7f748c19f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:34852/0 (socket says 172.21.15.27:34852) 2024-04-26T22:58:38.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.076+0000 7f748b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f748c075390 0x7f748c19f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:46448/0 (socket says 172.21.15.27:46448) 2024-04-26T22:58:38.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.076+0000 7f748aa00640 1 -- 172.21.15.27:0/3564083556 learned_addr learned my addr 172.21.15.27:0/3564083556 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:38.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.076+0000 7f748be00640 1 --2- 172.21.15.27:0/3564083556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f748c10edc0 0x7f748c1a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.076+0000 7f748aa00640 1 -- 172.21.15.27:0/3564083556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f748c075390 msgr2=0x7f748c19f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.076+0000 7f748aa00640 1 --2- 172.21.15.27:0/3564083556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f748c075390 0x7f748c19f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.076+0000 7f748aa00640 1 -- 172.21.15.27:0/3564083556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f748c10edc0 msgr2=0x7f748c1a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.076+0000 7f748aa00640 1 --2- 172.21.15.27:0/3564083556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f748c10edc0 0x7f748c1a65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.076+0000 7f748aa00640 1 -- 172.21.15.27:0/3564083556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748c1a6c60 con 0x7f748c076970 2024-04-26T22:58:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.076+0000 7f748aa00640 1 --2- 172.21.15.27:0/3564083556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f748c076970 0x7f748c19f580 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f747800d950 tx=0x7f747800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.077+0000 7f7483e00640 1 -- 172.21.15.27:0/3564083556 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478017070 con 0x7f748c076970 2024-04-26T22:58:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.077+0000 7f7483e00640 1 -- 172.21.15.27:0/3564083556 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7478009c30 con 0x7f748c076970 2024-04-26T22:58:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.077+0000 7f7483e00640 1 -- 172.21.15.27:0/3564083556 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747800f620 con 0x7f748c076970 2024-04-26T22:58:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.077+0000 7f7491200640 1 -- 172.21.15.27:0/3564083556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748c1a6f50 con 0x7f748c076970 2024-04-26T22:58:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.077+0000 7f7491200640 1 -- 172.21.15.27:0/3564083556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f748c1a73c0 con 0x7f748c076970 2024-04-26T22:58:38.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.078+0000 7f7491200640 1 -- 172.21.15.27:0/3564083556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f748c104f90 con 0x7f748c076970 2024-04-26T22:58:38.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.081+0000 7f7483e00640 1 -- 172.21.15.27:0/3564083556 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f7478010460 con 0x7f748c076970 2024-04-26T22:58:38.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.082+0000 7f7483e00640 1 --2- 172.21.15.27:0/3564083556 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f74700a3d80 0x7f74700a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:38.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.082+0000 7f7483e00640 1 -- 172.21.15.27:0/3564083556 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f74780d0780 con 0x7f748c076970 2024-04-26T22:58:38.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.082+0000 7f748b400640 1 --2- 172.21.15.27:0/3564083556 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f74700a3d80 0x7f74700a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:38.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.083+0000 7f748b400640 1 --2- 172.21.15.27:0/3564083556 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f74700a3d80 0x7f74700a6240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f747c005ab0 tx=0x7f747c005a20 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:38.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.087+0000 7f7483e00640 1 -- 172.21.15.27:0/3564083556 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7478009de0 con 0x7f748c076970 2024-04-26T22:58:38.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:37 smithi027 ceph-mon[28034]: [26/Apr/2024:22:58:36] ENGINE Bus STARTING 2024-04-26T22:58:38.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:37 smithi027 ceph-mon[28034]: [26/Apr/2024:22:58:36] ENGINE Serving on http://172.21.15.27:8765 2024-04-26T22:58:38.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:37 smithi027 ceph-mon[28034]: [26/Apr/2024:22:58:36] ENGINE Serving on https://172.21.15.27:7150 2024-04-26T22:58:38.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:37 smithi027 ceph-mon[28034]: [26/Apr/2024:22:58:36] ENGINE Bus STARTED 2024-04-26T22:58:38.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:37 smithi027 ceph-mon[28034]: from='client.25483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:38.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:37 smithi027 ceph-mon[28034]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:58:38.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.327+0000 7f7491200640 1 -- 172.21.15.27:0/3564083556 --> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f748c062d70 con 0x7f74700a3d80 2024-04-26T22:58:38.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.337+0000 7f7483e00640 1 -- 172.21.15.27:0/3564083556 <== mgr.25391 v2:172.21.15.27:6800/3058395889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f748c062d70 con 0x7f74700a3d80 2024-04-26T22:58:38.336 INFO:teuthology.orchestra.run.smithi027.stdout:/ 1m 6m3h 2024-04-26T22:58:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.339+0000 7f7491200640 1 -- 172.21.15.27:0/3564083556 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f74700a3d80 msgr2=0x7f74700a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.339+0000 7f7491200640 1 --2- 172.21.15.27:0/3564083556 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f74700a3d80 0x7f74700a6240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f747c005ab0 tx=0x7f747c005a20 comp rx=0 tx=0).stop 2024-04-26T22:58:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.339+0000 7f7491200640 1 -- 172.21.15.27:0/3564083556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f748c076970 msgr2=0x7f748c19f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.339+0000 7f7491200640 1 --2- 172.21.15.27:0/3564083556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f748c076970 0x7f748c19f580 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f747800d950 tx=0x7f747800de10 comp rx=0 tx=0).stop 2024-04-26T22:58:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.339+0000 7f7491200640 1 -- 172.21.15.27:0/3564083556 shutdown_connections 2024-04-26T22:58:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.339+0000 7f7491200640 1 --2- 172.21.15.27:0/3564083556 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f74700a3d80 0x7f74700a6240 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.339+0000 7f7491200640 1 --2- 172.21.15.27:0/3564083556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f748c10edc0 0x7f748c1a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.339+0000 7f7491200640 1 --2- 172.21.15.27:0/3564083556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f748c076970 0x7f748c19f580 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.339+0000 7f7491200640 1 --2- 172.21.15.27:0/3564083556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f748c075390 0x7f748c19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.339+0000 7f7491200640 1 -- 172.21.15.27:0/3564083556 >> 172.21.15.27:0/3564083556 conn(0x7f748c0fdd60 msgr2=0x7f748c102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.339+0000 7f7491200640 1 -- 172.21.15.27:0/3564083556 shutdown_connections 2024-04-26T22:58:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T22:58:38.339+0000 7f7491200640 1 -- 172.21.15.27:0/3564083556 wait complete. 2024-04-26T22:58:38.357 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-26T22:58:38.428 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-26T22:58:38.438 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-26T22:58:38.480 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T22:58:38.481 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-26T22:58:38.489 INFO:teuthology.run_tasks:Running task workunit... 2024-04-26T22:58:38.515 INFO:tasks.workunit:Pulling workunits from ref bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:58:38.516 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T22:58:38.516 INFO:tasks.workunit:timeout=6h 2024-04-26T22:58:38.516 INFO:tasks.workunit:cleanup=True 2024-04-26T22:58:38.516 DEBUG:teuthology.orchestra.run.smithi027:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T22:58:38.565 INFO:teuthology.orchestra.run.smithi027.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T22:58:38.565 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-26T22:58:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-26T22:58:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:58:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-26T22:58:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-26 22:55:56.683139244 +0000 2024-04-26T22:58:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-26 22:55:56.683139244 +0000 2024-04-26T22:58:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-26 22:58:33.985218654 +0000 2024-04-26T22:58:38.565 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-26 22:55:56.683139244 +0000 2024-04-26T22:58:38.566 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T22:58:38.566 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T22:58:38.645 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:58:38.691 INFO:tasks.workunit.client.0.smithi027.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T22:58:39.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:38 smithi080 ceph-mon[32324]: from='client.25463 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:39.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:38 smithi080 ceph-mon[32324]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:58:39.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:38 smithi080 ceph-mon[32324]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-26T22:58:39.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:38 smithi080 ceph-mon[32324]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:39.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:38 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:39.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:38 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:39.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:38 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:39.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:38 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:39.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:38 smithi094 ceph-mon[32889]: from='client.25463 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:39.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:38 smithi094 ceph-mon[32889]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:58:39.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:38 smithi094 ceph-mon[32889]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-26T22:58:39.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:38 smithi094 ceph-mon[32889]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:39.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:38 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:39.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:38 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:39.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:38 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:39.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:38 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:39.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:38 smithi027 ceph-mon[28034]: from='client.25463 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:39.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:38 smithi027 ceph-mon[28034]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:58:39.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:38 smithi027 ceph-mon[28034]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-26T22:58:39.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:38 smithi027 ceph-mon[28034]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:39.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:38 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:39.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:38 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:39.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:38 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:39.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:38 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:39 smithi080 ceph-mon[32324]: from='client.25507 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:40.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:39 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:39 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:39 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:39 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:39 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:39 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:39 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:39 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:39 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:39 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:39 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:39 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:39 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:39 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:39 smithi094 ceph-mon[32889]: from='client.25507 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:40.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:39 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:39 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:39 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:39 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:39 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:39 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:39 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:39 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:39 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:39 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:39 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:39 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:39 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:39 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:39 smithi027 ceph-mon[28034]: from='client.25507 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:58:40.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:39 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:39 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:39 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:39 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:39 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:39 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:39 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:39 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:39 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:39 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:40.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:39 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:39 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:39 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:40.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:39 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:41.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:40 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-26T22:58:41.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:40 smithi080 ceph-mon[32324]: Unable to set osd_memory_target on smithi094 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:58:41.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:40 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi080 to 176.2M 2024-04-26T22:58:41.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:40 smithi080 ceph-mon[32324]: Unable to set osd_memory_target on smithi080 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:58:41.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:40 smithi080 ceph-mon[32324]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:58:41.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:40 smithi080 ceph-mon[32324]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-26T22:58:41.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:40 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-26T22:58:41.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:40 smithi094 ceph-mon[32889]: Unable to set osd_memory_target on smithi094 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:58:41.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:40 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi080 to 176.2M 2024-04-26T22:58:41.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:40 smithi094 ceph-mon[32889]: Unable to set osd_memory_target on smithi080 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:58:41.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:40 smithi094 ceph-mon[32889]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:58:41.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:40 smithi094 ceph-mon[32889]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-26T22:58:41.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:40 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-26T22:58:41.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:40 smithi027 ceph-mon[28034]: Unable to set osd_memory_target on smithi094 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:58:41.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:40 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi080 to 176.2M 2024-04-26T22:58:41.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:40 smithi027 ceph-mon[28034]: Unable to set osd_memory_target on smithi080 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:58:41.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:40 smithi027 ceph-mon[28034]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:58:41.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:40 smithi027 ceph-mon[28034]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-26T22:58:42.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:42.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:42.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:42.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:42.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:42.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:42.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-04-26T22:58:42.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: Unable to set osd_memory_target on smithi027 to 185039769: error parsing value: Value '185039769' is below minimum 939524096 2024-04-26T22:58:42.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:58:42.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:58:42.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-26T22:58:42.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: Updating smithi080:/etc/ceph/ceph.conf 2024-04-26T22:58:42.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T22:58:42.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: Updating smithi080:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:58:42.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:58:42.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: Updating smithi094:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:58:42.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: Updating smithi080:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:58:42.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:42 smithi080 ceph-mon[32324]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:58:42.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:42.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:42.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:42.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:42.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:42.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:42.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-04-26T22:58:42.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: Unable to set osd_memory_target on smithi027 to 185039769: error parsing value: Value '185039769' is below minimum 939524096 2024-04-26T22:58:42.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:58:42.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:58:42.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-26T22:58:42.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: Updating smithi080:/etc/ceph/ceph.conf 2024-04-26T22:58:42.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T22:58:42.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: Updating smithi080:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:58:42.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:58:42.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: Updating smithi094:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:58:42.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: Updating smithi080:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:58:42.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:42 smithi094 ceph-mon[32889]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:58:42.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:42.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:42.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:42.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:42.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:42.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:42.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-04-26T22:58:42.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: Unable to set osd_memory_target on smithi027 to 185039769: error parsing value: Value '185039769' is below minimum 939524096 2024-04-26T22:58:42.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:58:42.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:58:42.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-26T22:58:42.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: Updating smithi080:/etc/ceph/ceph.conf 2024-04-26T22:58:42.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T22:58:42.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: Updating smithi080:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:58:42.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:58:42.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: Updating smithi094:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:58:42.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: Updating smithi080:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:58:42.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:42 smithi027 ceph-mon[28034]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:58:44.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:43 smithi080 ceph-mon[32324]: Updating smithi027:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:58:44.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:43 smithi080 ceph-mon[32324]: Updating smithi080:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:58:44.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:43 smithi080 ceph-mon[32324]: Updating smithi094:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:58:44.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:43 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:44.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:43 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:44.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:43 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:44.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:43 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:44.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:43 smithi080 ceph-mon[32324]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:58:44.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:43 smithi094 ceph-mon[32889]: Updating smithi027:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:58:44.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:43 smithi094 ceph-mon[32889]: Updating smithi080:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:58:44.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:43 smithi094 ceph-mon[32889]: Updating smithi094:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:58:44.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:43 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:44.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:43 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:44.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:43 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:44.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:43 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:44.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:43 smithi094 ceph-mon[32889]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:58:44.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:43 smithi027 ceph-mon[28034]: Updating smithi027:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:58:44.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:43 smithi027 ceph-mon[28034]: Updating smithi080:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:58:44.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:43 smithi027 ceph-mon[28034]: Updating smithi094:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:58:44.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:43 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:44.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:43 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:44.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:43 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:44.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:43 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:44.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:43 smithi027 ceph-mon[28034]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:58:45.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:44 smithi080 ceph-mon[32324]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-04-26T22:58:45.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:44 smithi080 ceph-mon[32324]: Updating smithi027:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:58:45.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:44 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:45.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:44 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:45.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:44 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:45.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:44 smithi094 ceph-mon[32889]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-04-26T22:58:45.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:44 smithi094 ceph-mon[32889]: Updating smithi027:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:58:45.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:44 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:45.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:44 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:45.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:44 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:45.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:44 smithi027 ceph-mon[28034]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-04-26T22:58:45.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:44 smithi027 ceph-mon[28034]: Updating smithi027:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:58:45.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:44 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:45.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:44 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:45.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:44 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:46.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:45 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:58:46.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:45 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:58:46.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:45 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:58:46.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:45 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:46.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:45 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:58:46.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:45 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:58:46.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:45 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:58:46.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:45 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:46.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:45 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:58:46.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:45 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:58:46.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:45 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:58:46.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:45 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:58:47.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:46 smithi080 ceph-mon[32324]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-04-26T22:58:47.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:46 smithi094 ceph-mon[32889]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-04-26T22:58:47.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:46 smithi027 ceph-mon[28034]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-04-26T22:58:49.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:48 smithi080 ceph-mon[32324]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-26T22:58:49.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:48 smithi094 ceph-mon[32889]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-26T22:58:49.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:48 smithi027 ceph-mon[28034]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-26T22:58:51.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:50 smithi080 ceph-mon[32324]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-04-26T22:58:51.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:50 smithi094 ceph-mon[32889]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-04-26T22:58:51.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:50 smithi027 ceph-mon[28034]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-04-26T22:58:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.753+0000 7efbffe00640 1 -- 172.21.15.27:0/2114989216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc00101050 msgr2=0x7efc001014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.753+0000 7efbffe00640 1 --2- 172.21.15.27:0/2114989216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc00101050 0x7efc001014d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009890 tx=0x7efbe802a370 comp rx=0 tx=0).stop 2024-04-26T22:58:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.753+0000 7efbffe00640 1 -- 172.21.15.27:0/2114989216 shutdown_connections 2024-04-26T22:58:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.753+0000 7efbffe00640 1 --2- 172.21.15.27:0/2114989216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc00101a10 0x7efc0010f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.753+0000 7efbffe00640 1 --2- 172.21.15.27:0/2114989216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc00101050 0x7efc001014d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.753+0000 7efbffe00640 1 --2- 172.21.15.27:0/2114989216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc001088e0 0x7efc00108cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.753+0000 7efbffe00640 1 -- 172.21.15.27:0/2114989216 >> 172.21.15.27:0/2114989216 conn(0x7efc000769a0 msgr2=0x7efc00076db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.753+0000 7efbffe00640 1 -- 172.21.15.27:0/2114989216 shutdown_connections 2024-04-26T22:58:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.753+0000 7efbffe00640 1 -- 172.21.15.27:0/2114989216 wait complete. 2024-04-26T22:58:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.755+0000 7efbffe00640 1 Processor -- start 2024-04-26T22:58:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.755+0000 7efbffe00640 1 -- start start 2024-04-26T22:58:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.755+0000 7efbffe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc00101050 0x7efc00198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.755+0000 7efbffe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc00101a10 0x7efc001994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.755+0000 7efbffe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc001088e0 0x7efc0019d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.755+0000 7efbffe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc00112a50 con 0x7efc001088e0 2024-04-26T22:58:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.755+0000 7efbffe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efc001128d0 con 0x7efc00101a10 2024-04-26T22:58:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.755+0000 7efbffe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc00112bd0 con 0x7efc00101050 2024-04-26T22:58:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.756+0000 7efbfea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc00101050 0x7efc00198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.756+0000 7efbff400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc001088e0 0x7efc0019d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.756+0000 7efbfea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc00101050 0x7efc00198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:49048/0 (socket says 172.21.15.27:49048) 2024-04-26T22:58:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.756+0000 7efbfea00640 1 -- 172.21.15.27:0/3566156212 learned_addr learned my addr 172.21.15.27:0/3566156212 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.756+0000 7efbf7e00640 1 --2- 172.21.15.27:0/3566156212 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc00101a10 0x7efc001994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.756+0000 7efbfea00640 1 -- 172.21.15.27:0/3566156212 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc00101a10 msgr2=0x7efc001994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.756+0000 7efbfea00640 1 --2- 172.21.15.27:0/3566156212 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc00101a10 0x7efc001994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.756+0000 7efbfea00640 1 -- 172.21.15.27:0/3566156212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc001088e0 msgr2=0x7efc0019d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.756+0000 7efbfea00640 1 --2- 172.21.15.27:0/3566156212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc001088e0 0x7efc0019d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.756+0000 7efbfea00640 1 -- 172.21.15.27:0/3566156212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0019dfa0 con 0x7efc00101050 2024-04-26T22:58:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.757+0000 7efbfea00640 1 --2- 172.21.15.27:0/3566156212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc00101050 0x7efc00198fb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efbf000e9a0 tx=0x7efbf000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.757+0000 7efbf7400640 1 -- 172.21.15.27:0/3566156212 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf000cd80 con 0x7efc00101050 2024-04-26T22:58:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.757+0000 7efbf7400640 1 -- 172.21.15.27:0/3566156212 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf00108e0 con 0x7efc00101050 2024-04-26T22:58:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.757+0000 7efbf7400640 1 -- 172.21.15.27:0/3566156212 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf0018960 con 0x7efc00101050 2024-04-26T22:58:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.757+0000 7efbffe00640 1 -- 172.21.15.27:0/3566156212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0019e290 con 0x7efc00101050 2024-04-26T22:58:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.757+0000 7efbffe00640 1 -- 172.21.15.27:0/3566156212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc001a5b20 con 0x7efc00101050 2024-04-26T22:58:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.760+0000 7efc0836c740 1 Processor -- start 2024-04-26T22:58:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.761+0000 7efbf7400640 1 -- 172.21.15.27:0/3566156212 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7efbf0010a90 con 0x7efc00101050 2024-04-26T22:58:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.761+0000 7efc0836c740 1 -- start start 2024-04-26T22:58:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.761+0000 7efc0836c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722bdfe0 0x5599722be3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.761+0000 7efc0836c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722be900 0x5599722c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.761+0000 7efc0836c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x5599722cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.761+0000 7efc0836c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559972223380 con 0x5599722bdfe0 2024-04-26T22:58:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.761+0000 7efc0836c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5599721bfc30 con 0x5599722c91c0 2024-04-26T22:58:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.761+0000 7efc0836c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55997221f340 con 0x5599722be900 2024-04-26T22:58:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.761+0000 7efbff400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722bdfe0 0x5599722be3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.761+0000 7efbfea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722be900 0x5599722c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.761+0000 7efbf7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x5599722cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.761+0000 7efbff400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722bdfe0 0x5599722be3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59788/0 (socket says 172.21.15.27:59788) 2024-04-26T22:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.761+0000 7efbff400640 1 -- 172.21.15.27:0/3597577178 learned_addr learned my addr 172.21.15.27:0/3597577178 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.762+0000 7efbf7e00640 1 -- 172.21.15.27:0/3597577178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722be900 msgr2=0x5599722c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.762+0000 7efbf7e00640 1 --2- 172.21.15.27:0/3597577178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722be900 0x5599722c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.762+0000 7efbf7e00640 1 -- 172.21.15.27:0/3597577178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722bdfe0 msgr2=0x5599722be3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.762+0000 7efbf7e00640 1 --2- 172.21.15.27:0/3597577178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722bdfe0 0x5599722be3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.762+0000 7efbf7e00640 1 -- 172.21.15.27:0/3597577178 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599721e1940 con 0x5599722c91c0 2024-04-26T22:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.762+0000 7efbf7e00640 1 --2- 172.21.15.27:0/3597577178 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x5599722cb5b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efbe8035040 tx=0x7efbe8009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a54194ebd42c4546 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.762+0000 7efbef400640 1 -- 172.21.15.27:0/3597577178 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8042020 con 0x5599722c91c0 2024-04-26T22:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.762+0000 7efbef400640 1 -- 172.21.15.27:0/3597577178 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbe8038040 con 0x5599722c91c0 2024-04-26T22:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.762+0000 7efbef400640 1 -- 172.21.15.27:0/3597577178 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8037040 con 0x5599722c91c0 2024-04-26T22:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.763+0000 7efbf7400640 1 --2- 172.21.15.27:0/3566156212 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7efbe40a3d90 0x7efbe40a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.763+0000 7efc0836c740 1 -- 172.21.15.27:0/3597577178 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 msgr2=0x5599722cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.763+0000 7efc0836c740 1 --2- 172.21.15.27:0/3597577178 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x5599722cb5b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efbe8035040 tx=0x7efbe8009560 comp rx=0 tx=0).stop 2024-04-26T22:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.763+0000 7efbf7400640 1 -- 172.21.15.27:0/3566156212 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7efbf00d1160 con 0x7efc00101050 2024-04-26T22:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.763+0000 7efbff400640 1 --2- 172.21.15.27:0/3566156212 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7efbe40a3d90 0x7efbe40a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.763+0000 7efc0836c740 1 -- 172.21.15.27:0/3597577178 shutdown_connections 2024-04-26T22:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.763+0000 7efc0836c740 1 --2- 172.21.15.27:0/3597577178 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x5599722cb5b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.763+0000 7efc0836c740 1 --2- 172.21.15.27:0/3597577178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722be900 0x5599722c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.763+0000 7efc0836c740 1 --2- 172.21.15.27:0/3597577178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722bdfe0 0x5599722be3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.763+0000 7efc0836c740 1 -- 172.21.15.27:0/3597577178 >> 172.21.15.27:0/3597577178 conn(0x5599722ba4f0 msgr2=0x5599722ba8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.763+0000 7efc0836c740 1 -- 172.21.15.27:0/3597577178 shutdown_connections 2024-04-26T22:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.763+0000 7efc0836c740 1 -- 172.21.15.27:0/3597577178 wait complete. 2024-04-26T22:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.764+0000 7efc0836c740 1 -- start start 2024-04-26T22:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.764+0000 7efbff400640 1 --2- 172.21.15.27:0/3566156212 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7efbe40a3d90 0x7efbe40a6250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efbf8005fd0 tx=0x7efbf8009500 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.764+0000 7efc0836c740 1 Processor -- start 2024-04-26T22:58:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efc0836c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 0x5599722c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efc0836c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599722c6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efc0836c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x5599722c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efc0836c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5599722cbd90 con 0x5599722be900 2024-04-26T22:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efc0836c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5599721aac60 con 0x5599722c91c0 2024-04-26T22:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efc0836c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5599722cbf10 con 0x5599722bdfe0 2024-04-26T22:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efc0836c740 10 client.0 _ll_register_callbacks cb 0x5599721d6e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efbf7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599722c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efbf7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599722c6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59800/0 (socket says 172.21.15.27:59800) 2024-04-26T22:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efbf7e00640 1 -- 172.21.15.27:0/1319635691 learned_addr learned my addr 172.21.15.27:0/1319635691 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efbfea00640 1 --2- 172.21.15.27:0/1319635691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 0x5599722c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efbff400640 1 --2- 172.21.15.27:0/1319635691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x5599722c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efbf7e00640 1 -- 172.21.15.27:0/1319635691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 msgr2=0x5599722c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efbf7e00640 1 --2- 172.21.15.27:0/1319635691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 0x5599722c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efbf7e00640 1 -- 172.21.15.27:0/1319635691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 msgr2=0x5599722c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efbf7e00640 1 --2- 172.21.15.27:0/1319635691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x5599722c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.765+0000 7efbf7e00640 1 -- 172.21.15.27:0/1319635691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599721b9510 con 0x5599722be900 2024-04-26T22:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.766+0000 7efbf7e00640 1 --2- 172.21.15.27:0/1319635691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599722c6600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efbe8005940 tx=0x7efbe80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bb75688b471fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.766+0000 7efbe3e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T22:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.766+0000 7efbe3e00640 1 -- 172.21.15.27:0/1319635691 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8042030 con 0x5599722be900 2024-04-26T22:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.766+0000 7efbe3e00640 1 -- 172.21.15.27:0/1319635691 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7efbe8030650 con 0x5599722be900 2024-04-26T22:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.766+0000 7efc0836c740 1 -- 172.21.15.27:0/1319635691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbe8038040 con 0x5599722be900 2024-04-26T22:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.766+0000 7efbe3e00640 1 -- 172.21.15.27:0/1319635691 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efbe8037040 con 0x5599722be900 2024-04-26T22:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.766+0000 7efc0836c740 10 client.15375 fetch_fsmap learned FSMap version 25 2024-04-26T22:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.766+0000 7efc0836c740 1 -- 172.21.15.27:0/1319635691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559971f35210 con 0x5599722be900 2024-04-26T22:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.767+0000 7efbe3e00640 1 -- 172.21.15.27:0/1319635691 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbe8038040 con 0x5599722be900 2024-04-26T22:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.767+0000 7efbe3e00640 1 -- 172.21.15.27:0/1319635691 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efbe8004a20 con 0x5599722be900 2024-04-26T22:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.767+0000 7efc0836c740 10 client.15375 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.767+0000 7efc0836c740 10 client.15375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.767+0000 7efc0836c740 20 client.15375 populate_metadata read hostname 'smithi027' 2024-04-26T22:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.767+0000 7efc0836c740 1 --2- 172.21.15.27:0/1319635691 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5599722d8cb0 0x5599722db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.767+0000 7efc0836c740 4 client.15375 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.767+0000 7efc0836c740 1 -- 172.21.15.27:0/1319635691 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559972230f90 con 0x5599722d8cb0 2024-04-26T22:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.768+0000 7efbfea00640 1 --2- 172.21.15.27:0/1319635691 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5599722d8cb0 0x5599722db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.768+0000 7efbfea00640 1 --2- 172.21.15.27:0/1319635691 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5599722d8cb0 0x5599722db0a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c55a64244d62b954 server_cookie=d45b8a4cba3ecb93 in_seq=0 out_seq=0 2024-04-26T22:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.768+0000 7efbe3e00640 10 client.15375 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.769+0000 7efbe3e00640 1 -- 172.21.15.27:0/1319635691 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559972230f90 con 0x5599722d8cb0 2024-04-26T22:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.769+0000 7efbe3e00640 10 client.15375 handle_command_reply: tid=0 2024-04-26T22:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.769+0000 7efc0836c740 1 client.15375 shutdown 2024-04-26T22:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.769+0000 7efc0836c740 1 -- 172.21.15.27:0/1319635691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 msgr2=0x5599722c6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.769+0000 7efc0836c740 1 --2- 172.21.15.27:0/1319635691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599722c6600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efbe8005940 tx=0x7efbe80042f0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.769+0000 7efc0836c740 1 -- 172.21.15.27:0/1319635691 shutdown_connections 2024-04-26T22:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.769+0000 7efc0836c740 1 --2- 172.21.15.27:0/1319635691 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5599722d8cb0 0x5599722db0a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.769+0000 7efc0836c740 1 --2- 172.21.15.27:0/1319635691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x5599722c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.769+0000 7efc0836c740 1 --2- 172.21.15.27:0/1319635691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599722c6600 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.769+0000 7efbe3e00640 0 client.15375 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:58:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.769+0000 7efc0836c740 1 --2- 172.21.15.27:0/1319635691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 0x5599722c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.769+0000 7efc0836c740 1 -- 172.21.15.27:0/1319635691 >> 172.21.15.27:0/1319635691 conn(0x5599722ba4f0 msgr2=0x5599722c0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.770+0000 7efc0836c740 1 -- 172.21.15.27:0/1319635691 shutdown_connections 2024-04-26T22:58:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.770+0000 7efc0836c740 1 -- 172.21.15.27:0/1319635691 wait complete. 2024-04-26T22:58:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.770+0000 7efc0836c740 20 client.15375 trim_cache size 0 max 16384 2024-04-26T22:58:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.786+0000 7efc0836c740 1 Processor -- start 2024-04-26T22:58:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.786+0000 7efc0836c740 1 -- start start 2024-04-26T22:58:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.786+0000 7efc0836c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 0x5599722d4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.786+0000 7efc0836c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599722d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.787+0000 7efc0836c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x5599722c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.787+0000 7efc0836c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc001b2660 con 0x5599722be900 2024-04-26T22:58:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.787+0000 7efc0836c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efc001aff30 con 0x5599722c91c0 2024-04-26T22:58:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.787+0000 7efc0836c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc001add80 con 0x5599722bdfe0 2024-04-26T22:58:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.787+0000 7efbf7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599722d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.787+0000 7efbf7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599722d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59804/0 (socket says 172.21.15.27:59804) 2024-04-26T22:58:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.787+0000 7efbf7e00640 1 -- 172.21.15.27:0/309324574 learned_addr learned my addr 172.21.15.27:0/309324574 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.787+0000 7efbfea00640 1 --2- 172.21.15.27:0/309324574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 0x5599722d4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.787+0000 7efbff400640 1 --2- 172.21.15.27:0/309324574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x5599722c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.787+0000 7efbfea00640 1 -- 172.21.15.27:0/309324574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 msgr2=0x5599722c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.787+0000 7efbfea00640 1 --2- 172.21.15.27:0/309324574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x5599722c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.787+0000 7efbfea00640 1 -- 172.21.15.27:0/309324574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 msgr2=0x5599722d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.787+0000 7efbfea00640 1 --2- 172.21.15.27:0/309324574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599722d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.787+0000 7efbfea00640 1 -- 172.21.15.27:0/309324574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599722be730 con 0x5599722bdfe0 2024-04-26T22:58:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.787+0000 7efbf7e00640 1 --2- 172.21.15.27:0/309324574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599722d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:58:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.788+0000 7efbfea00640 1 --2- 172.21.15.27:0/309324574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 0x5599722d4c50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efbf00ce350 tx=0x7efbf00bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6815817d177c1824 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.788+0000 7efbeea00640 1 -- 172.21.15.27:0/309324574 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf009c630 con 0x5599722bdfe0 2024-04-26T22:58:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.788+0000 7efbeea00640 1 -- 172.21.15.27:0/309324574 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf009d1f0 con 0x5599722bdfe0 2024-04-26T22:58:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.788+0000 7efbeea00640 1 -- 172.21.15.27:0/309324574 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf009c450 con 0x5599722bdfe0 2024-04-26T22:58:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.788+0000 7efc0836c740 1 -- 172.21.15.27:0/309324574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 msgr2=0x5599722d4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.789+0000 7efc0836c740 1 --2- 172.21.15.27:0/309324574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 0x5599722d4c50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efbf00ce350 tx=0x7efbf00bd040 comp rx=0 tx=0).stop 2024-04-26T22:58:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.789+0000 7efc0836c740 1 -- 172.21.15.27:0/309324574 shutdown_connections 2024-04-26T22:58:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.789+0000 7efc0836c740 1 --2- 172.21.15.27:0/309324574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x5599722c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.789+0000 7efc0836c740 1 --2- 172.21.15.27:0/309324574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599722d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.789+0000 7efc0836c740 1 --2- 172.21.15.27:0/309324574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 0x5599722d4c50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.789+0000 7efc0836c740 1 -- 172.21.15.27:0/309324574 >> 172.21.15.27:0/309324574 conn(0x5599722ba4f0 msgr2=0x5599722c77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.789+0000 7efc0836c740 1 -- 172.21.15.27:0/309324574 shutdown_connections 2024-04-26T22:58:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.789+0000 7efc0836c740 1 -- 172.21.15.27:0/309324574 wait complete. 2024-04-26T22:58:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.789+0000 7efc0836c740 1 -- start start 2024-04-26T22:58:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.789+0000 7efc0836c740 1 Processor -- start 2024-04-26T22:58:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.790+0000 7efc0836c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 0x559972336ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.790+0000 7efc0836c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599723373e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.790+0000 7efc0836c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x559972341f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.790+0000 7efc0836c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc001ae3f0 con 0x5599722be900 2024-04-26T22:58:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.790+0000 7efc0836c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efc001af860 con 0x5599722c91c0 2024-04-26T22:58:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.790+0000 7efc0836c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc001aeac0 con 0x5599722bdfe0 2024-04-26T22:58:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.790+0000 7efc0836c740 10 client.0 _ll_register_callbacks cb 0x7efbf8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:58:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.790+0000 7efbf7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599723373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.790+0000 7efbfea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 0x559972336ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.790+0000 7efbff400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x559972341f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.790+0000 7efbfea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 0x559972336ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:49070/0 (socket says 172.21.15.27:49070) 2024-04-26T22:58:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.790+0000 7efbf7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599723373e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59806/0 (socket says 172.21.15.27:59806) 2024-04-26T22:58:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.790+0000 7efbf7e00640 1 -- 172.21.15.27:0/4008234140 learned_addr learned my addr 172.21.15.27:0/4008234140 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:58:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.791+0000 7efbf7e00640 1 -- 172.21.15.27:0/4008234140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 msgr2=0x559972336ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.791+0000 7efbf7e00640 1 --2- 172.21.15.27:0/4008234140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 0x559972336ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.791+0000 7efbf7e00640 1 -- 172.21.15.27:0/4008234140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 msgr2=0x559972341f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:58:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.791+0000 7efbf7e00640 1 --2- 172.21.15.27:0/4008234140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x559972341f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.791+0000 7efbf7e00640 1 -- 172.21.15.27:0/4008234140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559972001620 con 0x5599722be900 2024-04-26T22:58:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.791+0000 7efbf7e00640 1 --2- 172.21.15.27:0/4008234140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599723373e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efbe8005b60 tx=0x7efbe8030970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63135da38c8569a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:58:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.791+0000 7efbe3e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T22:58:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.791+0000 7efbe3e00640 1 -- 172.21.15.27:0/4008234140 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8042040 con 0x5599722be900 2024-04-26T22:58:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.791+0000 7efc0836c740 1 -- 172.21.15.27:0/4008234140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf009d1f0 con 0x5599722be900 2024-04-26T22:58:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.791+0000 7efbe3e00640 1 -- 172.21.15.27:0/4008234140 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7efbe80048c0 con 0x5599722be900 2024-04-26T22:58:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.792+0000 7efbe3e00640 1 -- 172.21.15.27:0/4008234140 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efbe8005c90 con 0x5599722be900 2024-04-26T22:58:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.792+0000 7efc0836c740 10 client.15381 fetch_fsmap learned FSMap version 25 2024-04-26T22:58:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.792+0000 7efc0836c740 1 -- 172.21.15.27:0/4008234140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5599721ddcb0 con 0x5599722be900 2024-04-26T22:58:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.792+0000 7efbe3e00640 1 -- 172.21.15.27:0/4008234140 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf009d1f0 con 0x5599722be900 2024-04-26T22:58:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.792+0000 7efbe3e00640 1 -- 172.21.15.27:0/4008234140 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efbe8003800 con 0x5599722be900 2024-04-26T22:58:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.792+0000 7efc0836c740 10 client.15381 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:58:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.792+0000 7efc0836c740 10 client.15381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:58:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.792+0000 7efc0836c740 20 client.15381 populate_metadata read hostname 'smithi027' 2024-04-26T22:58:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.792+0000 7efc0836c740 1 --2- 172.21.15.27:0/4008234140 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5599723436f0 0x559972345b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:58:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.792+0000 7efc0836c740 4 client.15381 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:58:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.792+0000 7efc0836c740 1 -- 172.21.15.27:0/4008234140 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55997222ae00 con 0x5599723436f0 2024-04-26T22:58:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.793+0000 7efbfea00640 1 --2- 172.21.15.27:0/4008234140 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5599723436f0 0x559972345b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:58:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.793+0000 7efbfea00640 1 --2- 172.21.15.27:0/4008234140 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5599723436f0 0x559972345b10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72db05d5c74defeb server_cookie=39ab8e33755dff7 in_seq=0 out_seq=0 2024-04-26T22:58:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.793+0000 7efbe3e00640 10 client.15381 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:58:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.806+0000 7efbe3e00640 1 -- 172.21.15.27:0/4008234140 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55997222ae00 con 0x5599723436f0 2024-04-26T22:58:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.807+0000 7efbe3e00640 10 client.15381 handle_command_reply: tid=0 2024-04-26T22:58:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.807+0000 7efc0836c740 1 client.15381 shutdown 2024-04-26T22:58:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.807+0000 7efc0836c740 1 -- 172.21.15.27:0/4008234140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 msgr2=0x5599723373e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.807+0000 7efc0836c740 1 --2- 172.21.15.27:0/4008234140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599723373e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efbe8005b60 tx=0x7efbe8030970 comp rx=0 tx=0).stop 2024-04-26T22:58:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.807+0000 7efc0836c740 1 -- 172.21.15.27:0/4008234140 shutdown_connections 2024-04-26T22:58:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.807+0000 7efc0836c740 1 --2- 172.21.15.27:0/4008234140 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5599723436f0 0x559972345b10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.807+0000 7efc0836c740 1 --2- 172.21.15.27:0/4008234140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599722c91c0 0x559972341f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.807+0000 7efc0836c740 1 --2- 172.21.15.27:0/4008234140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599722be900 0x5599723373e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.807+0000 7efbe3e00640 0 client.15381 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:58:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.807+0000 7efc0836c740 1 --2- 172.21.15.27:0/4008234140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599722bdfe0 0x559972336ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.807+0000 7efc0836c740 1 -- 172.21.15.27:0/4008234140 >> 172.21.15.27:0/4008234140 conn(0x5599722ba4f0 msgr2=0x5599722c68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.807+0000 7efc0836c740 1 -- 172.21.15.27:0/4008234140 shutdown_connections 2024-04-26T22:58:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.807+0000 7efc0836c740 1 -- 172.21.15.27:0/4008234140 wait complete. 2024-04-26T22:58:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.807+0000 7efc0836c740 20 client.15381 trim_cache size 0 max 16384 2024-04-26T22:58:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T22:58:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T22:58:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T22:58:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T22:58:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.808+0000 7efbffe00640 1 -- 172.21.15.27:0/3566156212 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7efbe40a3d90 msgr2=0x7efbe40a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.808+0000 7efbffe00640 1 --2- 172.21.15.27:0/3566156212 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7efbe40a3d90 0x7efbe40a6250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efbf8005fd0 tx=0x7efbf8009500 comp rx=0 tx=0).stop 2024-04-26T22:58:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.808+0000 7efbffe00640 1 -- 172.21.15.27:0/3566156212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc00101050 msgr2=0x7efc00198fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:58:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.808+0000 7efbffe00640 1 --2- 172.21.15.27:0/3566156212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc00101050 0x7efc00198fb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efbf000e9a0 tx=0x7efbf000ee60 comp rx=0 tx=0).stop 2024-04-26T22:58:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.808+0000 7efbffe00640 1 -- 172.21.15.27:0/3566156212 shutdown_connections 2024-04-26T22:58:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.808+0000 7efbffe00640 1 --2- 172.21.15.27:0/3566156212 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7efbe40a3d90 0x7efbe40a6250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.808+0000 7efbffe00640 1 --2- 172.21.15.27:0/3566156212 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc00101a10 0x7efc001994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.808+0000 7efbffe00640 1 --2- 172.21.15.27:0/3566156212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc001088e0 0x7efc0019d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.808+0000 7efbffe00640 1 --2- 172.21.15.27:0/3566156212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc00101050 0x7efc00198fb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:58:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.808+0000 7efbffe00640 1 -- 172.21.15.27:0/3566156212 >> 172.21.15.27:0/3566156212 conn(0x7efc000769a0 msgr2=0x7efc000ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:58:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.808+0000 7efbffe00640 1 -- 172.21.15.27:0/3566156212 shutdown_connections 2024-04-26T22:58:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:58:51.808+0000 7efbffe00640 1 -- 172.21.15.27:0/3566156212 wait complete. 2024-04-26T22:58:53.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:52 smithi080 ceph-mon[32324]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T22:58:53.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:52 smithi094 ceph-mon[32889]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T22:58:53.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:52 smithi027 ceph-mon[28034]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T22:58:55.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:54 smithi080 ceph-mon[32324]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-26T22:58:55.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:54 smithi094 ceph-mon[32889]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-26T22:58:55.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:54 smithi027 ceph-mon[28034]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-26T22:58:57.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:56 smithi080 ceph-mon[32324]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:58:57.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:56 smithi094 ceph-mon[32889]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:58:57.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:56 smithi027 ceph-mon[28034]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:58:59.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:58:58 smithi080 ceph-mon[32324]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-26T22:58:59.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:58:58 smithi094 ceph-mon[32889]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-26T22:58:59.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:58:58 smithi027 ceph-mon[28034]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-26T22:59:00.865 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:59:00 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:59:00.362+0000 7fc864600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:59:00.865 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:59:00 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:59:00.362+0000 7fc864600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:59:00.865 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:59:00 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:59:00.362+0000 7fc864600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:59:00.865 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:59:00 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:59:00.362+0000 7fc864600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:59:00.865 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 22:59:00 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T22:59:00.362+0000 7fc864600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:59:01.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:00 smithi080 ceph-mon[32324]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-26T22:59:01.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:00 smithi094 ceph-mon[32889]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-26T22:59:01.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:00 smithi027 ceph-mon[28034]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-26T22:59:03.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:02 smithi080 ceph-mon[32324]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-26T22:59:03.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:02 smithi080 ceph-mon[32324]: mgrmap e25: x(active, since 26s), standbys: y, z 2024-04-26T22:59:03.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:02 smithi094 ceph-mon[32889]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-26T22:59:03.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:02 smithi094 ceph-mon[32889]: mgrmap e25: x(active, since 26s), standbys: y, z 2024-04-26T22:59:03.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:02 smithi027 ceph-mon[28034]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-26T22:59:03.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:02 smithi027 ceph-mon[28034]: mgrmap e25: x(active, since 26s), standbys: y, z 2024-04-26T22:59:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.957+0000 7f8136e00640 1 -- 172.21.15.27:0/266430771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f813010ee50 msgr2=0x7f813010f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.957+0000 7f8136e00640 1 --2- 172.21.15.27:0/266430771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f813010ee50 0x7f813010f230 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8124009900 tx=0x7f812402a480 comp rx=0 tx=0).stop 2024-04-26T22:59:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.958+0000 7f8136e00640 1 -- 172.21.15.27:0/266430771 shutdown_connections 2024-04-26T22:59:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.958+0000 7f8136e00640 1 --2- 172.21.15.27:0/266430771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130109730 0x7f8130115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.958+0000 7f8136e00640 1 --2- 172.21.15.27:0/266430771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8130108d90 0x7f81301091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.958+0000 7f8136e00640 1 --2- 172.21.15.27:0/266430771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f813010ee50 0x7f813010f230 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.958+0000 7f8136e00640 1 -- 172.21.15.27:0/266430771 >> 172.21.15.27:0/266430771 conn(0x7f81300784e0 msgr2=0x7f813007a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.958+0000 7f8136e00640 1 -- 172.21.15.27:0/266430771 shutdown_connections 2024-04-26T22:59:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.958+0000 7f8136e00640 1 -- 172.21.15.27:0/266430771 wait complete. 2024-04-26T22:59:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.959+0000 7f8136e00640 1 Processor -- start 2024-04-26T22:59:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.959+0000 7f8136e00640 1 -- start start 2024-04-26T22:59:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.959+0000 7f8136e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8130108d90 0x7f813019f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.959+0000 7f8136e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130109730 0x7f813019fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.959+0000 7f8136e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f813010ee50 0x7f81301a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.959+0000 7f8136e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8130119010 con 0x7f8130108d90 2024-04-26T22:59:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.959+0000 7f8136e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8130118e90 con 0x7f8130109730 2024-04-26T22:59:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.959+0000 7f8136e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8130119190 con 0x7f813010ee50 2024-04-26T22:59:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.960+0000 7f8135a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8130108d90 0x7f813019f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.960+0000 7f8135000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130109730 0x7f813019fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.960+0000 7f8135a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8130108d90 0x7f813019f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33332/0 (socket says 172.21.15.27:33332) 2024-04-26T22:59:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.960+0000 7f8135a00640 1 -- 172.21.15.27:0/791397050 learned_addr learned my addr 172.21.15.27:0/791397050 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.960+0000 7f8136400640 1 --2- 172.21.15.27:0/791397050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f813010ee50 0x7f81301a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.960+0000 7f8135000640 1 -- 172.21.15.27:0/791397050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f813010ee50 msgr2=0x7f81301a3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.960+0000 7f8135000640 1 --2- 172.21.15.27:0/791397050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f813010ee50 0x7f81301a3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.960+0000 7f8135000640 1 -- 172.21.15.27:0/791397050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8130108d90 msgr2=0x7f813019f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.960+0000 7f8135000640 1 --2- 172.21.15.27:0/791397050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8130108d90 0x7f813019f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.960+0000 7f8135000640 1 -- 172.21.15.27:0/791397050 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81301a4520 con 0x7f8130109730 2024-04-26T22:59:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.960+0000 7f8135000640 1 --2- 172.21.15.27:0/791397050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130109730 0x7f813019fa70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f811c00e9f0 tx=0x7f811c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.961+0000 7f812e000640 1 -- 172.21.15.27:0/791397050 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c00cd80 con 0x7f8130109730 2024-04-26T22:59:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.961+0000 7f812e000640 1 -- 172.21.15.27:0/791397050 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f811c0035f0 con 0x7f8130109730 2024-04-26T22:59:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.961+0000 7f8136e00640 1 -- 172.21.15.27:0/791397050 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81301a4810 con 0x7f8130109730 2024-04-26T22:59:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.961+0000 7f812e000640 1 -- 172.21.15.27:0/791397050 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c010670 con 0x7f8130109730 2024-04-26T22:59:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.961+0000 7f8136e00640 1 -- 172.21.15.27:0/791397050 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81301ac0a0 con 0x7f8130109730 2024-04-26T22:59:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.965+0000 7f813b3a1740 1 Processor -- start 2024-04-26T22:59:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.966+0000 7f813b3a1740 1 -- start start 2024-04-26T22:59:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.966+0000 7f812e000640 1 -- 172.21.15.27:0/791397050 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f811c0037a0 con 0x7f8130109730 2024-04-26T22:59:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.966+0000 7f813b3a1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba2fe0 0x555ce5ba33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.966+0000 7f813b3a1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba3900 0x555ce5badc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.966+0000 7f813b3a1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5bae1c0 0x555ce5bb05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.966+0000 7f813b3a1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555ce5b08380 con 0x555ce5bae1c0 2024-04-26T22:59:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.966+0000 7f813b3a1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x555ce5aa4c30 con 0x555ce5ba2fe0 2024-04-26T22:59:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.966+0000 7f813b3a1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555ce5b04340 con 0x555ce5ba3900 2024-04-26T22:59:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.967+0000 7f8135a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba3900 0x555ce5badc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.967+0000 7f8135000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5bae1c0 0x555ce5bb05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.967+0000 7f8135a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba3900 0x555ce5badc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:52082/0 (socket says 172.21.15.27:52082) 2024-04-26T22:59:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.967+0000 7f8135a00640 1 -- 172.21.15.27:0/3483295937 learned_addr learned my addr 172.21.15.27:0/3483295937 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.967+0000 7f8136400640 1 --2- 172.21.15.27:0/3483295937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba2fe0 0x555ce5ba33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.967+0000 7f8135a00640 1 -- 172.21.15.27:0/3483295937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba2fe0 msgr2=0x555ce5ba33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.967+0000 7f8135a00640 1 --2- 172.21.15.27:0/3483295937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba2fe0 0x555ce5ba33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.967+0000 7f8135a00640 1 -- 172.21.15.27:0/3483295937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5bae1c0 msgr2=0x555ce5bb05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.967+0000 7f8135a00640 1 --2- 172.21.15.27:0/3483295937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5bae1c0 0x555ce5bb05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.967+0000 7f8135a00640 1 -- 172.21.15.27:0/3483295937 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ce5ac6940 con 0x555ce5ba3900 2024-04-26T22:59:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.967+0000 7f8136400640 1 --2- 172.21.15.27:0/3483295937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba2fe0 0x555ce5ba33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:59:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.967+0000 7f8135a00640 1 --2- 172.21.15.27:0/3483295937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba3900 0x555ce5badc80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8124006010 tx=0x7f8124005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0243556f1c6b89c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.968+0000 7f8122a00640 1 -- 172.21.15.27:0/3483295937 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124040070 con 0x555ce5ba3900 2024-04-26T22:59:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.968+0000 7f8122a00640 1 -- 172.21.15.27:0/3483295937 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81240054d0 con 0x555ce5ba3900 2024-04-26T22:59:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.968+0000 7f8122a00640 1 -- 172.21.15.27:0/3483295937 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124035040 con 0x555ce5ba3900 2024-04-26T22:59:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.968+0000 7f812e000640 1 --2- 172.21.15.27:0/791397050 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f81180a3ea0 0x7f81180a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.968+0000 7f813b3a1740 1 -- 172.21.15.27:0/3483295937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba3900 msgr2=0x555ce5badc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.968+0000 7f813b3a1740 1 --2- 172.21.15.27:0/3483295937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba3900 0x555ce5badc80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8124006010 tx=0x7f8124005c00 comp rx=0 tx=0).stop 2024-04-26T22:59:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.968+0000 7f812e000640 1 -- 172.21.15.27:0/791397050 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f811c0d10c0 con 0x7f8130109730 2024-04-26T22:59:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.968+0000 7f8136400640 1 --2- 172.21.15.27:0/791397050 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f81180a3ea0 0x7f81180a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.969+0000 7f813b3a1740 1 -- 172.21.15.27:0/3483295937 shutdown_connections 2024-04-26T22:59:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.969+0000 7f813b3a1740 1 --2- 172.21.15.27:0/3483295937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5bae1c0 0x555ce5bb05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.969+0000 7f813b3a1740 1 --2- 172.21.15.27:0/3483295937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba3900 0x555ce5badc80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.969+0000 7f813b3a1740 1 --2- 172.21.15.27:0/3483295937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba2fe0 0x555ce5ba33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.969+0000 7f813b3a1740 1 -- 172.21.15.27:0/3483295937 >> 172.21.15.27:0/3483295937 conn(0x555ce5b9f4f0 msgr2=0x555ce5b9f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.969+0000 7f813b3a1740 1 -- 172.21.15.27:0/3483295937 shutdown_connections 2024-04-26T22:59:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.969+0000 7f813b3a1740 1 -- 172.21.15.27:0/3483295937 wait complete. 2024-04-26T22:59:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.969+0000 7f813b3a1740 1 -- start start 2024-04-26T22:59:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.969+0000 7f8136400640 1 --2- 172.21.15.27:0/791397050 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f81180a3ea0 0x7f81180a6360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8128005fd0 tx=0x7f81280096a0 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.969+0000 7f813b3a1740 1 Processor -- start 2024-04-26T22:59:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.970+0000 7f813b3a1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba2fe0 0x555ce5bab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.970+0000 7f813b3a1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba3900 0x555ce5bab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.970+0000 7f813b3a1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5bae1c0 0x555ce5babae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.970+0000 7f813b3a1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555ce5bb0d90 con 0x555ce5bae1c0 2024-04-26T22:59:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.970+0000 7f813b3a1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x555ce5a8fc60 con 0x555ce5ba3900 2024-04-26T22:59:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.970+0000 7f813b3a1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555ce5bb0f10 con 0x555ce5ba2fe0 2024-04-26T22:59:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.970+0000 7f813b3a1740 10 client.0 _ll_register_callbacks cb 0x555ce5abbe50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:59:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.970+0000 7f8135a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba2fe0 0x555ce5bab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.970+0000 7f8135a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba2fe0 0x555ce5bab060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:52092/0 (socket says 172.21.15.27:52092) 2024-04-26T22:59:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.970+0000 7f8135a00640 1 -- 172.21.15.27:0/1477146647 learned_addr learned my addr 172.21.15.27:0/1477146647 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.970+0000 7f8136400640 1 --2- 172.21.15.27:0/1477146647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5bae1c0 0x555ce5babae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.971+0000 7f8135000640 1 --2- 172.21.15.27:0/1477146647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba3900 0x555ce5bab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.971+0000 7f8135a00640 1 -- 172.21.15.27:0/1477146647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba3900 msgr2=0x555ce5bab5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.971+0000 7f8135a00640 1 --2- 172.21.15.27:0/1477146647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba3900 0x555ce5bab5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.971+0000 7f8135a00640 1 -- 172.21.15.27:0/1477146647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5bae1c0 msgr2=0x555ce5babae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.971+0000 7f8135a00640 1 --2- 172.21.15.27:0/1477146647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5bae1c0 0x555ce5babae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.971+0000 7f8135a00640 1 -- 172.21.15.27:0/1477146647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ce5a9e510 con 0x555ce5ba2fe0 2024-04-26T22:59:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.971+0000 7f8135a00640 1 --2- 172.21.15.27:0/1477146647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba2fe0 0x555ce5bab060 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8124000c00 tx=0x7f8124009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=847a3d13d898f591 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.971+0000 7f8117400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T22:59:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.971+0000 7f8117400640 1 -- 172.21.15.27:0/1477146647 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124040070 con 0x555ce5ba2fe0 2024-04-26T22:59:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.971+0000 7f813b3a1740 1 -- 172.21.15.27:0/1477146647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81240054d0 con 0x555ce5ba2fe0 2024-04-26T22:59:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.971+0000 7f8117400640 1 -- 172.21.15.27:0/1477146647 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f81240051a0 con 0x555ce5ba2fe0 2024-04-26T22:59:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.972+0000 7f8117400640 1 -- 172.21.15.27:0/1477146647 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8124056050 con 0x555ce5ba2fe0 2024-04-26T22:59:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.972+0000 7f813b3a1740 10 client.25543 fetch_fsmap learned FSMap version 25 2024-04-26T22:59:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.972+0000 7f813b3a1740 1 -- 172.21.15.27:0/1477146647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555ce581a210 con 0x555ce5ba2fe0 2024-04-26T22:59:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.972+0000 7f8117400640 1 -- 172.21.15.27:0/1477146647 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81240054d0 con 0x555ce5ba2fe0 2024-04-26T22:59:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.972+0000 7f8117400640 1 -- 172.21.15.27:0/1477146647 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f81240314a0 con 0x555ce5ba2fe0 2024-04-26T22:59:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.972+0000 7f813b3a1740 10 client.25543 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:59:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.972+0000 7f813b3a1740 10 client.25543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:59:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.972+0000 7f813b3a1740 20 client.25543 populate_metadata read hostname 'smithi027' 2024-04-26T22:59:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.972+0000 7f813b3a1740 1 --2- 172.21.15.27:0/1477146647 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x555ce5bbdc50 0x555ce5bc0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.972+0000 7f813b3a1740 4 client.25543 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:59:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.972+0000 7f813b3a1740 1 -- 172.21.15.27:0/1477146647 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ce5b15f90 con 0x555ce5bbdc50 2024-04-26T22:59:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.973+0000 7f8135000640 1 --2- 172.21.15.27:0/1477146647 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x555ce5bbdc50 0x555ce5bc0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.973+0000 7f8135000640 1 --2- 172.21.15.27:0/1477146647 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x555ce5bbdc50 0x555ce5bc0040 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7811421909d1a9a1 server_cookie=f3e2044f278d7822 in_seq=0 out_seq=0 2024-04-26T22:59:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.973+0000 7f8117400640 10 client.25543 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.974+0000 7f8117400640 1 -- 172.21.15.27:0/1477146647 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ce5b15f90 con 0x555ce5bbdc50 2024-04-26T22:59:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.974+0000 7f8117400640 10 client.25543 handle_command_reply: tid=0 2024-04-26T22:59:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.974+0000 7f813b3a1740 1 client.25543 shutdown 2024-04-26T22:59:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.974+0000 7f813b3a1740 1 -- 172.21.15.27:0/1477146647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba2fe0 msgr2=0x555ce5bab060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.974+0000 7f813b3a1740 1 --2- 172.21.15.27:0/1477146647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba2fe0 0x555ce5bab060 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8124000c00 tx=0x7f8124009260 comp rx=0 tx=0).stop 2024-04-26T22:59:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.974+0000 7f813b3a1740 1 -- 172.21.15.27:0/1477146647 shutdown_connections 2024-04-26T22:59:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.974+0000 7f813b3a1740 1 --2- 172.21.15.27:0/1477146647 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x555ce5bbdc50 0x555ce5bc0040 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.974+0000 7f813b3a1740 1 --2- 172.21.15.27:0/1477146647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba3900 0x555ce5bab5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.974+0000 7f813b3a1740 1 --2- 172.21.15.27:0/1477146647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5bae1c0 0x555ce5babae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.974+0000 7f813b3a1740 1 --2- 172.21.15.27:0/1477146647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba2fe0 0x555ce5bab060 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.974+0000 7f813b3a1740 1 -- 172.21.15.27:0/1477146647 >> 172.21.15.27:0/1477146647 conn(0x555ce5b9f4f0 msgr2=0x555ce5ba5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.974+0000 7f8117400640 0 client.25543 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.975+0000 7f813b3a1740 1 -- 172.21.15.27:0/1477146647 shutdown_connections 2024-04-26T22:59:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.975+0000 7f813b3a1740 1 -- 172.21.15.27:0/1477146647 wait complete. 2024-04-26T22:59:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.975+0000 7f813b3a1740 20 client.25543 trim_cache size 0 max 16384 2024-04-26T22:59:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.991+0000 7f813b3a1740 1 Processor -- start 2024-04-26T22:59:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.991+0000 7f813b3a1740 1 -- start start 2024-04-26T22:59:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.992+0000 7f813b3a1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5ba2fe0 0x555ce5baa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.992+0000 7f813b3a1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba3900 0x555ce5bace90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.992+0000 7f813b3a1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5bae1c0 0x555ce5bc3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.992+0000 7f813b3a1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b8b30 con 0x555ce5ba2fe0 2024-04-26T22:59:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.992+0000 7f813b3a1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b6400 con 0x555ce5bae1c0 2024-04-26T22:59:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.992+0000 7f813b3a1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b4250 con 0x555ce5ba3900 2024-04-26T22:59:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.992+0000 7f8135a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5ba2fe0 0x555ce5baa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.992+0000 7f8135a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5ba2fe0 0x555ce5baa960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33350/0 (socket says 172.21.15.27:33350) 2024-04-26T22:59:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.992+0000 7f8135a00640 1 -- 172.21.15.27:0/3440518086 learned_addr learned my addr 172.21.15.27:0/3440518086 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.992+0000 7f8136400640 1 --2- 172.21.15.27:0/3440518086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5bae1c0 0x555ce5bc3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.992+0000 7f8135000640 1 --2- 172.21.15.27:0/3440518086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba3900 0x555ce5bace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.992+0000 7f8135a00640 1 -- 172.21.15.27:0/3440518086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba3900 msgr2=0x555ce5bace90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.992+0000 7f8135a00640 1 --2- 172.21.15.27:0/3440518086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba3900 0x555ce5bace90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.992+0000 7f8135a00640 1 -- 172.21.15.27:0/3440518086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5bae1c0 msgr2=0x555ce5bc3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.992+0000 7f8135a00640 1 --2- 172.21.15.27:0/3440518086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5bae1c0 0x555ce5bc3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.992+0000 7f8135a00640 1 -- 172.21.15.27:0/3440518086 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ce58e6620 con 0x555ce5ba2fe0 2024-04-26T22:59:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.993+0000 7f8135a00640 1 --2- 172.21.15.27:0/3440518086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5ba2fe0 0x555ce5baa960 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8124004100 tx=0x7f8124009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f099e3804da882 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.993+0000 7f8122000640 1 -- 172.21.15.27:0/3440518086 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124040040 con 0x555ce5ba2fe0 2024-04-26T22:59:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.993+0000 7f8122000640 1 -- 172.21.15.27:0/3440518086 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f812404d070 con 0x555ce5ba2fe0 2024-04-26T22:59:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.993+0000 7f8122000640 1 -- 172.21.15.27:0/3440518086 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124035040 con 0x555ce5ba2fe0 2024-04-26T22:59:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.994+0000 7f813b3a1740 1 -- 172.21.15.27:0/3440518086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5ba2fe0 msgr2=0x555ce5baa960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.994+0000 7f813b3a1740 1 --2- 172.21.15.27:0/3440518086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5ba2fe0 0x555ce5baa960 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8124004100 tx=0x7f8124009780 comp rx=0 tx=0).stop 2024-04-26T22:59:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.994+0000 7f813b3a1740 1 -- 172.21.15.27:0/3440518086 shutdown_connections 2024-04-26T22:59:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.994+0000 7f813b3a1740 1 --2- 172.21.15.27:0/3440518086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5bae1c0 0x555ce5bc3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.994+0000 7f813b3a1740 1 --2- 172.21.15.27:0/3440518086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5ba3900 0x555ce5bace90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.994+0000 7f813b3a1740 1 --2- 172.21.15.27:0/3440518086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5ba2fe0 0x555ce5baa960 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.994+0000 7f813b3a1740 1 -- 172.21.15.27:0/3440518086 >> 172.21.15.27:0/3440518086 conn(0x555ce5b9f4f0 msgr2=0x555ce5bac780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.994+0000 7f813b3a1740 1 -- 172.21.15.27:0/3440518086 shutdown_connections 2024-04-26T22:59:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.994+0000 7f813b3a1740 1 -- 172.21.15.27:0/3440518086 wait complete. 2024-04-26T22:59:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.994+0000 7f813b3a1740 1 -- start start 2024-04-26T22:59:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.995+0000 7f813b3a1740 1 Processor -- start 2024-04-26T22:59:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.995+0000 7f813b3a1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba2fe0 0x555ce5bc10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.995+0000 7f813b3a1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5ba3900 0x555ce5bc1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.995+0000 7f813b3a1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5bae1c0 0x555ce5c22e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.995+0000 7f813b3a1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b48c0 con 0x555ce5ba3900 2024-04-26T22:59:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.995+0000 7f813b3a1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b5d30 con 0x555ce5ba2fe0 2024-04-26T22:59:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.995+0000 7f813b3a1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b4f90 con 0x555ce5bae1c0 2024-04-26T22:59:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.995+0000 7f813b3a1740 10 client.0 _ll_register_callbacks cb 0x7f811c0d4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:59:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.995+0000 7f8135a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba2fe0 0x555ce5bc10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.995+0000 7f8135000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5ba3900 0x555ce5bc1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.995+0000 7f8135a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba2fe0 0x555ce5bc10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:38636/0 (socket says 172.21.15.27:38636) 2024-04-26T22:59:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.996+0000 7f8135a00640 1 -- 172.21.15.27:0/1289846937 learned_addr learned my addr 172.21.15.27:0/1289846937 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.996+0000 7f8136400640 1 --2- 172.21.15.27:0/1289846937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5bae1c0 0x555ce5c22e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.996+0000 7f8135a00640 1 -- 172.21.15.27:0/1289846937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5bae1c0 msgr2=0x555ce5c22e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.996+0000 7f8135a00640 1 --2- 172.21.15.27:0/1289846937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5bae1c0 0x555ce5c22e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.996+0000 7f8135a00640 1 -- 172.21.15.27:0/1289846937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5ba3900 msgr2=0x555ce5bc1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.996+0000 7f8135a00640 1 --2- 172.21.15.27:0/1289846937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5ba3900 0x555ce5bc1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.996+0000 7f8135a00640 1 -- 172.21.15.27:0/1289846937 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ce5ac2cb0 con 0x555ce5ba2fe0 2024-04-26T22:59:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.996+0000 7f8135000640 1 --2- 172.21.15.27:0/1289846937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5ba3900 0x555ce5bc1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:59:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.996+0000 7f8135a00640 1 --2- 172.21.15.27:0/1289846937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba2fe0 0x555ce5bc10e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8124004650 tx=0x7f8124004a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec7517e645175f2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.996+0000 7f8117400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T22:59:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.996+0000 7f8117400640 1 -- 172.21.15.27:0/1289846937 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124035030 con 0x555ce5ba2fe0 2024-04-26T22:59:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.996+0000 7f813b3a1740 1 -- 172.21.15.27:0/1289846937 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f812404d070 con 0x555ce5ba2fe0 2024-04-26T22:59:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.997+0000 7f8117400640 1 -- 172.21.15.27:0/1289846937 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f8124031da0 con 0x555ce5ba2fe0 2024-04-26T22:59:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.997+0000 7f8117400640 1 -- 172.21.15.27:0/1289846937 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8124040070 con 0x555ce5ba2fe0 2024-04-26T22:59:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.997+0000 7f813b3a1740 10 client.25520 fetch_fsmap learned FSMap version 25 2024-04-26T22:59:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.997+0000 7f813b3a1740 1 -- 172.21.15.27:0/1289846937 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555ce5c243b0 con 0x555ce5ba2fe0 2024-04-26T22:59:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.997+0000 7f8117400640 1 -- 172.21.15.27:0/1289846937 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f812404d070 con 0x555ce5ba2fe0 2024-04-26T22:59:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.997+0000 7f8117400640 1 -- 172.21.15.27:0/1289846937 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8124003890 con 0x555ce5ba2fe0 2024-04-26T22:59:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.997+0000 7f813b3a1740 10 client.25520 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:59:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.997+0000 7f813b3a1740 10 client.25520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:59:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.997+0000 7f813b3a1740 20 client.25520 populate_metadata read hostname 'smithi027' 2024-04-26T22:59:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.997+0000 7f813b3a1740 1 --2- 172.21.15.27:0/1289846937 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x555ce5c249c0 0x555ce5c26db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.997+0000 7f813b3a1740 4 client.25520 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:59:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.997+0000 7f813b3a1740 1 -- 172.21.15.27:0/1289846937 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ce5b0fe00 con 0x555ce5c249c0 2024-04-26T22:59:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.998+0000 7f8135000640 1 --2- 172.21.15.27:0/1289846937 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x555ce5c249c0 0x555ce5c26db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.998+0000 7f8135000640 1 --2- 172.21.15.27:0/1289846937 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x555ce5c249c0 0x555ce5c26db0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ad92eb805b05870 server_cookie=562d6b56270f67da in_seq=0 out_seq=0 2024-04-26T22:59:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:04.998+0000 7f8117400640 10 client.25520 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.005+0000 7f8117400640 1 -- 172.21.15.27:0/1289846937 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555ce5b0fe00 con 0x555ce5c249c0 2024-04-26T22:59:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.005+0000 7f8117400640 10 client.25520 handle_command_reply: tid=0 2024-04-26T22:59:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.005+0000 7f813b3a1740 1 client.25520 shutdown 2024-04-26T22:59:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.005+0000 7f813b3a1740 1 -- 172.21.15.27:0/1289846937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba2fe0 msgr2=0x555ce5bc10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.005+0000 7f813b3a1740 1 --2- 172.21.15.27:0/1289846937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba2fe0 0x555ce5bc10e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8124004650 tx=0x7f8124004a70 comp rx=0 tx=0).stop 2024-04-26T22:59:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.006+0000 7f813b3a1740 1 -- 172.21.15.27:0/1289846937 shutdown_connections 2024-04-26T22:59:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.006+0000 7f813b3a1740 1 --2- 172.21.15.27:0/1289846937 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x555ce5c249c0 0x555ce5c26db0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.006+0000 7f813b3a1740 1 --2- 172.21.15.27:0/1289846937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ce5bae1c0 0x555ce5c22e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.006+0000 7f813b3a1740 1 --2- 172.21.15.27:0/1289846937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ce5ba3900 0x555ce5bc1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.006+0000 7f8117400640 0 client.25520 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.006+0000 7f813b3a1740 1 --2- 172.21.15.27:0/1289846937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555ce5ba2fe0 0x555ce5bc10e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.006+0000 7f813b3a1740 1 -- 172.21.15.27:0/1289846937 >> 172.21.15.27:0/1289846937 conn(0x555ce5b9f4f0 msgr2=0x555ce5bb99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.006+0000 7f813b3a1740 1 -- 172.21.15.27:0/1289846937 shutdown_connections 2024-04-26T22:59:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.006+0000 7f813b3a1740 1 -- 172.21.15.27:0/1289846937 wait complete. 2024-04-26T22:59:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.006+0000 7f813b3a1740 20 client.25520 trim_cache size 0 max 16384 2024-04-26T22:59:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T22:59:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T22:59:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T22:59:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T22:59:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.007+0000 7f8136e00640 1 -- 172.21.15.27:0/791397050 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f81180a3ea0 msgr2=0x7f81180a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.007+0000 7f8136e00640 1 --2- 172.21.15.27:0/791397050 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f81180a3ea0 0x7f81180a6360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8128005fd0 tx=0x7f81280096a0 comp rx=0 tx=0).stop 2024-04-26T22:59:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.007+0000 7f8136e00640 1 -- 172.21.15.27:0/791397050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130109730 msgr2=0x7f813019fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.007+0000 7f8136e00640 1 --2- 172.21.15.27:0/791397050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130109730 0x7f813019fa70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f811c00e9f0 tx=0x7f811c00eeb0 comp rx=0 tx=0).stop 2024-04-26T22:59:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.007+0000 7f8136e00640 1 -- 172.21.15.27:0/791397050 shutdown_connections 2024-04-26T22:59:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.007+0000 7f8136e00640 1 --2- 172.21.15.27:0/791397050 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f81180a3ea0 0x7f81180a6360 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.007+0000 7f8136e00640 1 --2- 172.21.15.27:0/791397050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130109730 0x7f813019fa70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.007+0000 7f8136e00640 1 --2- 172.21.15.27:0/791397050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f813010ee50 0x7f81301a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.007+0000 7f8136e00640 1 --2- 172.21.15.27:0/791397050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8130108d90 0x7f813019f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.007+0000 7f8136e00640 1 -- 172.21.15.27:0/791397050 >> 172.21.15.27:0/791397050 conn(0x7f81300784e0 msgr2=0x7f8130079ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.007+0000 7f8136e00640 1 -- 172.21.15.27:0/791397050 shutdown_connections 2024-04-26T22:59:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:05.007+0000 7f8136e00640 1 -- 172.21.15.27:0/791397050 wait complete. 2024-04-26T22:59:05.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:04 smithi080 ceph-mon[32324]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-26T22:59:05.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:04 smithi094 ceph-mon[32889]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-26T22:59:05.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:04 smithi027 ceph-mon[28034]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-26T22:59:07.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:06 smithi080 ceph-mon[32324]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-26T22:59:07.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:06 smithi094 ceph-mon[32889]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-26T22:59:07.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:06 smithi027 ceph-mon[28034]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-26T22:59:09.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:08 smithi080 ceph-mon[32324]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 29 op/s 2024-04-26T22:59:09.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:08 smithi094 ceph-mon[32889]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 29 op/s 2024-04-26T22:59:09.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:08 smithi027 ceph-mon[28034]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 29 op/s 2024-04-26T22:59:11.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:10 smithi080 ceph-mon[32324]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T22:59:11.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:10 smithi094 ceph-mon[32889]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T22:59:11.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:10 smithi027 ceph-mon[28034]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T22:59:13.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:12 smithi080 ceph-mon[32324]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-26T22:59:13.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:12 smithi094 ceph-mon[32889]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-26T22:59:13.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:12 smithi027 ceph-mon[28034]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-26T22:59:15.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:14 smithi080 ceph-mon[32324]: pgmap v22: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 22 op/s 2024-04-26T22:59:15.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:14 smithi094 ceph-mon[32889]: pgmap v22: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 22 op/s 2024-04-26T22:59:15.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:14 smithi027 ceph-mon[28034]: pgmap v22: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 22 op/s 2024-04-26T22:59:17.273 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:16 smithi080 ceph-mon[32324]: pgmap v23: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-04-26T22:59:17.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:16 smithi094 ceph-mon[32889]: pgmap v23: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-04-26T22:59:17.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:16 smithi027 ceph-mon[28034]: pgmap v23: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-04-26T22:59:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.158+0000 7fef42000640 1 -- 172.21.15.27:0/629565500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef3c10ee50 msgr2=0x7fef3c10f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.158+0000 7fef42000640 1 --2- 172.21.15.27:0/629565500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef3c10ee50 0x7fef3c10f230 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009ca0 tx=0x7fef2c02a950 comp rx=0 tx=0).stop 2024-04-26T22:59:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.158+0000 7fef42000640 1 -- 172.21.15.27:0/629565500 shutdown_connections 2024-04-26T22:59:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.158+0000 7fef42000640 1 --2- 172.21.15.27:0/629565500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef3c109730 0x7fef3c115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.158+0000 7fef42000640 1 --2- 172.21.15.27:0/629565500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef3c108d90 0x7fef3c1091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.158+0000 7fef42000640 1 --2- 172.21.15.27:0/629565500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef3c10ee50 0x7fef3c10f230 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.158+0000 7fef42000640 1 -- 172.21.15.27:0/629565500 >> 172.21.15.27:0/629565500 conn(0x7fef3c0784e0 msgr2=0x7fef3c07a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.158+0000 7fef42000640 1 -- 172.21.15.27:0/629565500 shutdown_connections 2024-04-26T22:59:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.158+0000 7fef42000640 1 -- 172.21.15.27:0/629565500 wait complete. 2024-04-26T22:59:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.160+0000 7fef42000640 1 Processor -- start 2024-04-26T22:59:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.160+0000 7fef42000640 1 -- start start 2024-04-26T22:59:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.161+0000 7fef42000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef3c108d90 0x7fef3c19f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.161+0000 7fef42000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef3c109730 0x7fef3c19fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.161+0000 7fef42000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef3c10ee50 0x7fef3c1a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.161+0000 7fef42000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c119010 con 0x7fef3c108d90 2024-04-26T22:59:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.161+0000 7fef42000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c118e90 con 0x7fef3c109730 2024-04-26T22:59:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.161+0000 7fef42000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c119190 con 0x7fef3c10ee50 2024-04-26T22:59:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.161+0000 7fef40c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef3c108d90 0x7fef3c19f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.161+0000 7fef40c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef3c108d90 0x7fef3c19f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49720/0 (socket says 172.21.15.27:49720) 2024-04-26T22:59:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.161+0000 7fef40c00640 1 -- 172.21.15.27:0/1390512848 learned_addr learned my addr 172.21.15.27:0/1390512848 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.161+0000 7fef33e00640 1 --2- 172.21.15.27:0/1390512848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef3c109730 0x7fef3c19fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.161+0000 7fef41600640 1 --2- 172.21.15.27:0/1390512848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef3c10ee50 0x7fef3c1a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.161+0000 7fef40c00640 1 -- 172.21.15.27:0/1390512848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef3c10ee50 msgr2=0x7fef3c1a3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.161+0000 7fef40c00640 1 --2- 172.21.15.27:0/1390512848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef3c10ee50 0x7fef3c1a3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.161+0000 7fef40c00640 1 -- 172.21.15.27:0/1390512848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef3c109730 msgr2=0x7fef3c19fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.161+0000 7fef40c00640 1 --2- 172.21.15.27:0/1390512848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef3c109730 0x7fef3c19fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.161+0000 7fef40c00640 1 -- 172.21.15.27:0/1390512848 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3c1a4520 con 0x7fef3c108d90 2024-04-26T22:59:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.162+0000 7fef40c00640 1 --2- 172.21.15.27:0/1390512848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef3c108d90 0x7fef3c19f530 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fef2c005e30 tx=0x7fef2c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.162+0000 7fef3a000640 1 -- 172.21.15.27:0/1390512848 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c034040 con 0x7fef3c108d90 2024-04-26T22:59:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.162+0000 7fef42000640 1 -- 172.21.15.27:0/1390512848 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef3c1a47b0 con 0x7fef3c108d90 2024-04-26T22:59:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.162+0000 7fef3a000640 1 -- 172.21.15.27:0/1390512848 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef2c030850 con 0x7fef3c108d90 2024-04-26T22:59:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.162+0000 7fef3a000640 1 -- 172.21.15.27:0/1390512848 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c004790 con 0x7fef3c108d90 2024-04-26T22:59:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.162+0000 7fef42000640 1 -- 172.21.15.27:0/1390512848 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef3c1ac0a0 con 0x7fef3c108d90 2024-04-26T22:59:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.166+0000 7fef46425740 1 Processor -- start 2024-04-26T22:59:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.166+0000 7fef46425740 1 -- start start 2024-04-26T22:59:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.166+0000 7fef3a000640 1 -- 172.21.15.27:0/1390512848 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fef2c037020 con 0x7fef3c108d90 2024-04-26T22:59:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.166+0000 7fef46425740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dcefe0 0x5555d2dcf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.166+0000 7fef46425740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcf900 0x5555d2dd9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.166+0000 7fef46425740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dda1c0 0x5555d2ddc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.166+0000 7fef46425740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5555d2d34380 con 0x5555d2dda1c0 2024-04-26T22:59:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.166+0000 7fef46425740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5555d2cd0c30 con 0x5555d2dcf900 2024-04-26T22:59:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.166+0000 7fef46425740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5555d2d30340 con 0x5555d2dcefe0 2024-04-26T22:59:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.167+0000 7fef41600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dcefe0 0x5555d2dcf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.167+0000 7fef40c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcf900 0x5555d2dd9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.167+0000 7fef33e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dda1c0 0x5555d2ddc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.167+0000 7fef41600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dcefe0 0x5555d2dcf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:44536/0 (socket says 172.21.15.27:44536) 2024-04-26T22:59:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.167+0000 7fef33e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dda1c0 0x5555d2ddc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49734/0 (socket says 172.21.15.27:49734) 2024-04-26T22:59:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.167+0000 7fef41600640 1 -- 172.21.15.27:0/2010708890 learned_addr learned my addr 172.21.15.27:0/2010708890 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.167+0000 7fef40c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcf900 0x5555d2dd9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:34044/0 (socket says 172.21.15.27:34044) 2024-04-26T22:59:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.167+0000 7fef41600640 1 -- 172.21.15.27:0/2010708890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcf900 msgr2=0x5555d2dd9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.167+0000 7fef41600640 1 --2- 172.21.15.27:0/2010708890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcf900 0x5555d2dd9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.167+0000 7fef41600640 1 -- 172.21.15.27:0/2010708890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dda1c0 msgr2=0x5555d2ddc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.167+0000 7fef41600640 1 --2- 172.21.15.27:0/2010708890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dda1c0 0x5555d2ddc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.167+0000 7fef41600640 1 -- 172.21.15.27:0/2010708890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d2cf2940 con 0x5555d2dcefe0 2024-04-26T22:59:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.167+0000 7fef40c00640 1 --2- 172.21.15.27:0/2010708890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcf900 0x5555d2dd9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:59:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.168+0000 7fef41600640 1 --2- 172.21.15.27:0/2010708890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dcefe0 0x5555d2dcf3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fef3400b6b0 tx=0x7fef3400bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6fabdb1338f72b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.168+0000 7fef3a000640 1 --2- 172.21.15.27:0/1390512848 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fef200a3ea0 0x7fef200a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.168+0000 7fef31c00640 1 -- 172.21.15.27:0/2010708890 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34003bd0 con 0x5555d2dcefe0 2024-04-26T22:59:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.168+0000 7fef3a000640 1 -- 172.21.15.27:0/1390512848 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fef2c0f1ca0 con 0x7fef3c108d90 2024-04-26T22:59:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.168+0000 7fef31c00640 1 -- 172.21.15.27:0/2010708890 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef34015070 con 0x5555d2dcefe0 2024-04-26T22:59:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.168+0000 7fef31c00640 1 -- 172.21.15.27:0/2010708890 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34011640 con 0x5555d2dcefe0 2024-04-26T22:59:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.168+0000 7fef40c00640 1 --2- 172.21.15.27:0/1390512848 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fef200a3ea0 0x7fef200a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.169+0000 7fef46425740 1 -- 172.21.15.27:0/2010708890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dcefe0 msgr2=0x5555d2dcf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.169+0000 7fef40c00640 1 --2- 172.21.15.27:0/1390512848 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fef200a3ea0 0x7fef200a6360 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0eef40 tx=0x7fef2c0f5730 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.169+0000 7fef46425740 1 --2- 172.21.15.27:0/2010708890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dcefe0 0x5555d2dcf3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fef3400b6b0 tx=0x7fef3400bb70 comp rx=0 tx=0).stop 2024-04-26T22:59:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.169+0000 7fef46425740 1 -- 172.21.15.27:0/2010708890 shutdown_connections 2024-04-26T22:59:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.169+0000 7fef46425740 1 --2- 172.21.15.27:0/2010708890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcf900 0x5555d2dd9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.169+0000 7fef46425740 1 --2- 172.21.15.27:0/2010708890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dda1c0 0x5555d2ddc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.169+0000 7fef46425740 1 --2- 172.21.15.27:0/2010708890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dcefe0 0x5555d2dcf3c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.169+0000 7fef46425740 1 -- 172.21.15.27:0/2010708890 >> 172.21.15.27:0/2010708890 conn(0x5555d2dcb4f0 msgr2=0x5555d2dcb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.169+0000 7fef46425740 1 -- 172.21.15.27:0/2010708890 shutdown_connections 2024-04-26T22:59:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.169+0000 7fef46425740 1 -- 172.21.15.27:0/2010708890 wait complete. 2024-04-26T22:59:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.170+0000 7fef46425740 1 -- start start 2024-04-26T22:59:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.170+0000 7fef46425740 1 Processor -- start 2024-04-26T22:59:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.170+0000 7fef46425740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcefe0 0x5555d2dd7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.171+0000 7fef46425740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcf900 0x5555d2dd75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.171+0000 7fef46425740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dda1c0 0x5555d2dd7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.171+0000 7fef46425740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5555d2ddcdf0 con 0x5555d2dcf900 2024-04-26T22:59:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.171+0000 7fef46425740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5555d2cbbc60 con 0x5555d2dcefe0 2024-04-26T22:59:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.171+0000 7fef46425740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5555d2ddcf70 con 0x5555d2dda1c0 2024-04-26T22:59:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.171+0000 7fef46425740 10 client.0 _ll_register_callbacks cb 0x5555d2ce7e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:59:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.171+0000 7fef33e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcf900 0x5555d2dd75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.171+0000 7fef41600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcefe0 0x5555d2dd7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.171+0000 7fef33e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcf900 0x5555d2dd75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49746/0 (socket says 172.21.15.27:49746) 2024-04-26T22:59:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.171+0000 7fef33e00640 1 -- 172.21.15.27:0/338565795 learned_addr learned my addr 172.21.15.27:0/338565795 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.171+0000 7fef41600640 1 --2- 172.21.15.27:0/338565795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dda1c0 0x5555d2dd7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.171+0000 7fef33e00640 1 -- 172.21.15.27:0/338565795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dda1c0 msgr2=0x5555d2dd7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.171+0000 7fef33e00640 1 --2- 172.21.15.27:0/338565795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dda1c0 0x5555d2dd7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.171+0000 7fef33e00640 1 -- 172.21.15.27:0/338565795 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcefe0 msgr2=0x5555d2dd7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.171+0000 7fef33e00640 1 --2- 172.21.15.27:0/338565795 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcefe0 0x5555d2dd7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.171+0000 7fef33e00640 1 -- 172.21.15.27:0/338565795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d2cca510 con 0x5555d2dcf900 2024-04-26T22:59:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.172+0000 7fef33e00640 1 --2- 172.21.15.27:0/338565795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcf900 0x5555d2dd75d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fef24012700 tx=0x7fef24012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240803548405dc2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.172+0000 7fef2a000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T22:59:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.172+0000 7fef2a000640 1 -- 172.21.15.27:0/338565795 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2400f9a0 con 0x5555d2dcf900 2024-04-26T22:59:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.172+0000 7fef2a000640 1 -- 172.21.15.27:0/338565795 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fef2400fb40 con 0x5555d2dcf900 2024-04-26T22:59:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.172+0000 7fef46425740 1 -- 172.21.15.27:0/338565795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef34015070 con 0x5555d2dcf900 2024-04-26T22:59:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.172+0000 7fef2a000640 1 -- 172.21.15.27:0/338565795 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fef24025050 con 0x5555d2dcf900 2024-04-26T22:59:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.172+0000 7fef46425740 10 client.15417 fetch_fsmap learned FSMap version 25 2024-04-26T22:59:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.172+0000 7fef46425740 1 -- 172.21.15.27:0/338565795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5555d2a46210 con 0x5555d2dcf900 2024-04-26T22:59:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.173+0000 7fef2a000640 1 -- 172.21.15.27:0/338565795 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef34015070 con 0x5555d2dcf900 2024-04-26T22:59:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.173+0000 7fef2a000640 1 -- 172.21.15.27:0/338565795 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fef24010430 con 0x5555d2dcf900 2024-04-26T22:59:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.173+0000 7fef46425740 10 client.15417 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:59:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.173+0000 7fef46425740 10 client.15417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:59:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.173+0000 7fef46425740 20 client.15417 populate_metadata read hostname 'smithi027' 2024-04-26T22:59:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.173+0000 7fef46425740 1 --2- 172.21.15.27:0/338565795 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5555d2de9ee0 0x5555d2dec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.173+0000 7fef46425740 4 client.15417 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:59:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.173+0000 7fef46425740 1 -- 172.21.15.27:0/338565795 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555d2d41f90 con 0x5555d2de9ee0 2024-04-26T22:59:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.174+0000 7fef41600640 1 --2- 172.21.15.27:0/338565795 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5555d2de9ee0 0x5555d2dec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.174+0000 7fef41600640 1 --2- 172.21.15.27:0/338565795 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5555d2de9ee0 0x5555d2dec340 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35089409b83ca297 server_cookie=cda4d7624be10d67 in_seq=0 out_seq=0 2024-04-26T22:59:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.174+0000 7fef2a000640 10 client.15417 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.175+0000 7fef2a000640 1 -- 172.21.15.27:0/338565795 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555d2d41f90 con 0x5555d2de9ee0 2024-04-26T22:59:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.175+0000 7fef2a000640 10 client.15417 handle_command_reply: tid=0 2024-04-26T22:59:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.175+0000 7fef46425740 1 client.15417 shutdown 2024-04-26T22:59:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.175+0000 7fef46425740 1 -- 172.21.15.27:0/338565795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcf900 msgr2=0x5555d2dd75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.175+0000 7fef46425740 1 --2- 172.21.15.27:0/338565795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcf900 0x5555d2dd75d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fef24012700 tx=0x7fef24012bc0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.175+0000 7fef46425740 1 -- 172.21.15.27:0/338565795 shutdown_connections 2024-04-26T22:59:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.175+0000 7fef46425740 1 --2- 172.21.15.27:0/338565795 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5555d2de9ee0 0x5555d2dec340 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.175+0000 7fef46425740 1 --2- 172.21.15.27:0/338565795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dda1c0 0x5555d2dd7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.175+0000 7fef46425740 1 --2- 172.21.15.27:0/338565795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcf900 0x5555d2dd75d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.175+0000 7fef46425740 1 --2- 172.21.15.27:0/338565795 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcefe0 0x5555d2dd7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.175+0000 7fef2a000640 0 client.15417 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.175+0000 7fef46425740 1 -- 172.21.15.27:0/338565795 >> 172.21.15.27:0/338565795 conn(0x5555d2dcb4f0 msgr2=0x5555d2dd1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.175+0000 7fef46425740 1 -- 172.21.15.27:0/338565795 shutdown_connections 2024-04-26T22:59:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.175+0000 7fef46425740 1 -- 172.21.15.27:0/338565795 wait complete. 2024-04-26T22:59:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.176+0000 7fef46425740 20 client.15417 trim_cache size 0 max 16384 2024-04-26T22:59:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.192+0000 7fef46425740 1 Processor -- start 2024-04-26T22:59:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.192+0000 7fef46425740 1 -- start start 2024-04-26T22:59:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.193+0000 7fef46425740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcefe0 0x5555d2de5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.193+0000 7fef46425740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcf900 0x5555d2dd8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.193+0000 7fef46425740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dda1c0 0x5555d2df1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.193+0000 7fef46425740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c1b8be0 con 0x5555d2dcefe0 2024-04-26T22:59:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.193+0000 7fef46425740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c1b64b0 con 0x5555d2dcf900 2024-04-26T22:59:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.193+0000 7fef46425740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c1b4300 con 0x5555d2dda1c0 2024-04-26T22:59:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.193+0000 7fef41600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcefe0 0x5555d2de5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.193+0000 7fef41600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcefe0 0x5555d2de5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49756/0 (socket says 172.21.15.27:49756) 2024-04-26T22:59:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.193+0000 7fef41600640 1 -- 172.21.15.27:0/127000792 learned_addr learned my addr 172.21.15.27:0/127000792 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.193+0000 7fef33e00640 1 --2- 172.21.15.27:0/127000792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcf900 0x5555d2dd8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.193+0000 7fef41600640 1 --2- 172.21.15.27:0/127000792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dda1c0 0x5555d2df1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.193+0000 7fef41600640 1 -- 172.21.15.27:0/127000792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dda1c0 msgr2=0x5555d2df1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.193+0000 7fef41600640 1 --2- 172.21.15.27:0/127000792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dda1c0 0x5555d2df1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.193+0000 7fef41600640 1 -- 172.21.15.27:0/127000792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcf900 msgr2=0x5555d2dd8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.193+0000 7fef41600640 1 --2- 172.21.15.27:0/127000792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcf900 0x5555d2dd8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.193+0000 7fef41600640 1 -- 172.21.15.27:0/127000792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d2b12620 con 0x5555d2dcefe0 2024-04-26T22:59:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.194+0000 7fef41600640 1 --2- 172.21.15.27:0/127000792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcefe0 0x5555d2de5cb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fef3400b680 tx=0x7fef3401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a3bdff8192776e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.194+0000 7fef31200640 1 -- 172.21.15.27:0/127000792 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34023040 con 0x5555d2dcefe0 2024-04-26T22:59:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.194+0000 7fef31200640 1 -- 172.21.15.27:0/127000792 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef34015070 con 0x5555d2dcefe0 2024-04-26T22:59:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.194+0000 7fef31200640 1 -- 172.21.15.27:0/127000792 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34004680 con 0x5555d2dcefe0 2024-04-26T22:59:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.195+0000 7fef46425740 1 -- 172.21.15.27:0/127000792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcefe0 msgr2=0x5555d2de5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.195+0000 7fef46425740 1 --2- 172.21.15.27:0/127000792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcefe0 0x5555d2de5cb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fef3400b680 tx=0x7fef3401ad30 comp rx=0 tx=0).stop 2024-04-26T22:59:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.195+0000 7fef46425740 1 -- 172.21.15.27:0/127000792 shutdown_connections 2024-04-26T22:59:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.195+0000 7fef46425740 1 --2- 172.21.15.27:0/127000792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcf900 0x5555d2dd8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.195+0000 7fef46425740 1 --2- 172.21.15.27:0/127000792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dda1c0 0x5555d2df1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.195+0000 7fef46425740 1 --2- 172.21.15.27:0/127000792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcefe0 0x5555d2de5cb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.195+0000 7fef46425740 1 -- 172.21.15.27:0/127000792 >> 172.21.15.27:0/127000792 conn(0x5555d2dcb4f0 msgr2=0x5555d2dd87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.195+0000 7fef46425740 1 -- 172.21.15.27:0/127000792 shutdown_connections 2024-04-26T22:59:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.195+0000 7fef46425740 1 -- 172.21.15.27:0/127000792 wait complete. 2024-04-26T22:59:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.195+0000 7fef46425740 1 -- start start 2024-04-26T22:59:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.195+0000 7fef46425740 1 Processor -- start 2024-04-26T22:59:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.196+0000 7fef46425740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcefe0 0x5555d2dd3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.196+0000 7fef46425740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcf900 0x5555d2dd4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.196+0000 7fef46425740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dda1c0 0x5555d2dd47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.196+0000 7fef46425740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c1b4970 con 0x5555d2dcf900 2024-04-26T22:59:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.196+0000 7fef46425740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c1b5de0 con 0x5555d2dcefe0 2024-04-26T22:59:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.196+0000 7fef46425740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c1b5040 con 0x5555d2dda1c0 2024-04-26T22:59:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.196+0000 7fef46425740 10 client.0 _ll_register_callbacks cb 0x7fef24009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:59:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.196+0000 7fef33e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcf900 0x5555d2dd4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.196+0000 7fef41600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcefe0 0x5555d2dd3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.196+0000 7fef33e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcf900 0x5555d2dd4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49770/0 (socket says 172.21.15.27:49770) 2024-04-26T22:59:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.196+0000 7fef33e00640 1 -- 172.21.15.27:0/1722563586 learned_addr learned my addr 172.21.15.27:0/1722563586 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.196+0000 7fef41600640 1 --2- 172.21.15.27:0/1722563586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dda1c0 0x5555d2dd47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.196+0000 7fef33e00640 1 -- 172.21.15.27:0/1722563586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dda1c0 msgr2=0x5555d2dd47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.197+0000 7fef33e00640 1 --2- 172.21.15.27:0/1722563586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dda1c0 0x5555d2dd47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.197+0000 7fef33e00640 1 -- 172.21.15.27:0/1722563586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcefe0 msgr2=0x5555d2dd3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.197+0000 7fef33e00640 1 --2- 172.21.15.27:0/1722563586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcefe0 0x5555d2dd3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.197+0000 7fef33e00640 1 -- 172.21.15.27:0/1722563586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d2ceecb0 con 0x5555d2dcf900 2024-04-26T22:59:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.197+0000 7fef33e00640 1 --2- 172.21.15.27:0/1722563586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcf900 0x5555d2dd4270 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5555d2dd8660 tx=0x7fef24009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6257e89f69744814 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.197+0000 7fef2a000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T22:59:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.197+0000 7fef2a000640 1 -- 172.21.15.27:0/1722563586 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef24022040 con 0x5555d2dcf900 2024-04-26T22:59:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.197+0000 7fef2a000640 1 -- 172.21.15.27:0/1722563586 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fef24010a70 con 0x5555d2dcf900 2024-04-26T22:59:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.197+0000 7fef46425740 1 -- 172.21.15.27:0/1722563586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef34015070 con 0x5555d2dcf900 2024-04-26T22:59:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.198+0000 7fef2a000640 1 -- 172.21.15.27:0/1722563586 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fef24010c90 con 0x5555d2dcf900 2024-04-26T22:59:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.198+0000 7fef46425740 10 client.15423 fetch_fsmap learned FSMap version 25 2024-04-26T22:59:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.198+0000 7fef46425740 1 -- 172.21.15.27:0/1722563586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5555d2e505d0 con 0x5555d2dcf900 2024-04-26T22:59:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.198+0000 7fef2a000640 1 -- 172.21.15.27:0/1722563586 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef34015070 con 0x5555d2dcf900 2024-04-26T22:59:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.198+0000 7fef2a000640 1 -- 172.21.15.27:0/1722563586 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fef240054c0 con 0x5555d2dcf900 2024-04-26T22:59:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.198+0000 7fef46425740 10 client.15423 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:59:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.198+0000 7fef46425740 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:59:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.198+0000 7fef46425740 20 client.15423 populate_metadata read hostname 'smithi027' 2024-04-26T22:59:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.198+0000 7fef46425740 1 --2- 172.21.15.27:0/1722563586 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5555d2e50d60 0x5555d2e53180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.198+0000 7fef46425740 4 client.15423 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:59:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.198+0000 7fef46425740 1 -- 172.21.15.27:0/1722563586 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555d2d3be00 con 0x5555d2e50d60 2024-04-26T22:59:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.199+0000 7fef41600640 1 --2- 172.21.15.27:0/1722563586 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5555d2e50d60 0x5555d2e53180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.199+0000 7fef41600640 1 --2- 172.21.15.27:0/1722563586 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5555d2e50d60 0x5555d2e53180 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ab179a8e890ffa9 server_cookie=77d359a4c9f91bda in_seq=0 out_seq=0 2024-04-26T22:59:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.199+0000 7fef2a000640 10 client.15423 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.207+0000 7fef2a000640 1 -- 172.21.15.27:0/1722563586 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5555d2d3be00 con 0x5555d2e50d60 2024-04-26T22:59:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.207+0000 7fef2a000640 10 client.15423 handle_command_reply: tid=0 2024-04-26T22:59:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.207+0000 7fef46425740 1 client.15423 shutdown 2024-04-26T22:59:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.207+0000 7fef46425740 1 -- 172.21.15.27:0/1722563586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcf900 msgr2=0x5555d2dd4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.207+0000 7fef46425740 1 --2- 172.21.15.27:0/1722563586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcf900 0x5555d2dd4270 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5555d2dd8660 tx=0x7fef24009fd0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.207+0000 7fef46425740 1 -- 172.21.15.27:0/1722563586 shutdown_connections 2024-04-26T22:59:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.207+0000 7fef46425740 1 --2- 172.21.15.27:0/1722563586 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5555d2e50d60 0x5555d2e53180 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.207+0000 7fef46425740 1 --2- 172.21.15.27:0/1722563586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555d2dda1c0 0x5555d2dd47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.207+0000 7fef46425740 1 --2- 172.21.15.27:0/1722563586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555d2dcf900 0x5555d2dd4270 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.207+0000 7fef2a000640 0 client.15423 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.207+0000 7fef46425740 1 --2- 172.21.15.27:0/1722563586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5555d2dcefe0 0x5555d2dd3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.207+0000 7fef46425740 1 -- 172.21.15.27:0/1722563586 >> 172.21.15.27:0/1722563586 conn(0x5555d2dcb4f0 msgr2=0x5555d2dd1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.207+0000 7fef46425740 1 -- 172.21.15.27:0/1722563586 shutdown_connections 2024-04-26T22:59:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.207+0000 7fef46425740 1 -- 172.21.15.27:0/1722563586 wait complete. 2024-04-26T22:59:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.208+0000 7fef46425740 20 client.15423 trim_cache size 0 max 16384 2024-04-26T22:59:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T22:59:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T22:59:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T22:59:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T22:59:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.208+0000 7fef42000640 1 -- 172.21.15.27:0/1390512848 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fef200a3ea0 msgr2=0x7fef200a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.208+0000 7fef42000640 1 --2- 172.21.15.27:0/1390512848 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fef200a3ea0 0x7fef200a6360 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0eef40 tx=0x7fef2c0f5730 comp rx=0 tx=0).stop 2024-04-26T22:59:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.208+0000 7fef42000640 1 -- 172.21.15.27:0/1390512848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef3c108d90 msgr2=0x7fef3c19f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.208+0000 7fef42000640 1 --2- 172.21.15.27:0/1390512848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef3c108d90 0x7fef3c19f530 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fef2c005e30 tx=0x7fef2c003f90 comp rx=0 tx=0).stop 2024-04-26T22:59:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.208+0000 7fef42000640 1 -- 172.21.15.27:0/1390512848 shutdown_connections 2024-04-26T22:59:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.209+0000 7fef42000640 1 --2- 172.21.15.27:0/1390512848 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fef200a3ea0 0x7fef200a6360 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.209+0000 7fef42000640 1 --2- 172.21.15.27:0/1390512848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef3c109730 0x7fef3c19fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.209+0000 7fef42000640 1 --2- 172.21.15.27:0/1390512848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef3c10ee50 0x7fef3c1a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.209+0000 7fef42000640 1 --2- 172.21.15.27:0/1390512848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef3c108d90 0x7fef3c19f530 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.209+0000 7fef42000640 1 -- 172.21.15.27:0/1390512848 >> 172.21.15.27:0/1390512848 conn(0x7fef3c0784e0 msgr2=0x7fef3c079ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.209+0000 7fef42000640 1 -- 172.21.15.27:0/1390512848 shutdown_connections 2024-04-26T22:59:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:18.209+0000 7fef42000640 1 -- 172.21.15.27:0/1390512848 wait complete. 2024-04-26T22:59:19.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:18 smithi080 ceph-mon[32324]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-04-26T22:59:19.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:18 smithi094 ceph-mon[32889]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-04-26T22:59:19.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:18 smithi027 ceph-mon[28034]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-04-26T22:59:21.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:20 smithi080 ceph-mon[32324]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-26T22:59:21.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:20 smithi094 ceph-mon[32889]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-26T22:59:21.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:20 smithi027 ceph-mon[28034]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-26T22:59:23.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:22 smithi080 ceph-mon[32324]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T22:59:23.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:22 smithi094 ceph-mon[32889]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T22:59:23.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:22 smithi027 ceph-mon[28034]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T22:59:25.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:24 smithi080 ceph-mon[32324]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-26T22:59:25.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:24 smithi094 ceph-mon[32889]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-26T22:59:25.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:24 smithi027 ceph-mon[28034]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-26T22:59:27.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:26 smithi080 ceph-mon[32324]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T22:59:27.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:26 smithi094 ceph-mon[32889]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T22:59:27.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:26 smithi027 ceph-mon[28034]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T22:59:29.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:28 smithi080 ceph-mon[32324]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T22:59:29.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:28 smithi094 ceph-mon[32889]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T22:59:29.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:28 smithi027 ceph-mon[28034]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T22:59:31.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:31 smithi080 ceph-mon[32324]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T22:59:31.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:31 smithi094 ceph-mon[32889]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T22:59:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.351+0000 7f62f5200640 1 -- 172.21.15.27:0/57959466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f0101ea0 msgr2=0x7f62f010eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.351+0000 7f62f5200640 1 --2- 172.21.15.27:0/57959466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f0101ea0 0x7f62f010eb60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009900 tx=0x7f62dc02a3f0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.351+0000 7f62f5200640 1 -- 172.21.15.27:0/57959466 shutdown_connections 2024-04-26T22:59:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.351+0000 7f62f5200640 1 --2- 172.21.15.27:0/57959466 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62f010f0a0 0x7f62f0111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.351+0000 7f62f5200640 1 --2- 172.21.15.27:0/57959466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f0101ea0 0x7f62f010eb60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.351+0000 7f62f5200640 1 --2- 172.21.15.27:0/57959466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62f01014f0 0x7f62f01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.351+0000 7f62f5200640 1 -- 172.21.15.27:0/57959466 >> 172.21.15.27:0/57959466 conn(0x7f62f00fd110 msgr2=0x7f62f00ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.351+0000 7f62f5200640 1 -- 172.21.15.27:0/57959466 shutdown_connections 2024-04-26T22:59:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.351+0000 7f62f5200640 1 -- 172.21.15.27:0/57959466 wait complete. 2024-04-26T22:59:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.352+0000 7f62f5200640 1 Processor -- start 2024-04-26T22:59:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.352+0000 7f62f5200640 1 -- start start 2024-04-26T22:59:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.352+0000 7f62f5200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f01014f0 0x7f62f019b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.352+0000 7f62f5200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62f0101ea0 0x7f62f019b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.353+0000 7f62f5200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62f010f0a0 0x7f62f019fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.353+0000 7f62f5200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0114c50 con 0x7f62f01014f0 2024-04-26T22:59:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.353+0000 7f62f5200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0114ad0 con 0x7f62f0101ea0 2024-04-26T22:59:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.353+0000 7f62f5200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0114dd0 con 0x7f62f010f0a0 2024-04-26T22:59:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.353+0000 7f62ef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f01014f0 0x7f62f019b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.353+0000 7f62ef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f01014f0 0x7f62f019b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46516/0 (socket says 172.21.15.27:46516) 2024-04-26T22:59:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.353+0000 7f62ef400640 1 -- 172.21.15.27:0/1009255969 learned_addr learned my addr 172.21.15.27:0/1009255969 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.353+0000 7f62eea00640 1 --2- 172.21.15.27:0/1009255969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62f0101ea0 0x7f62f019b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.353+0000 7f62efe00640 1 --2- 172.21.15.27:0/1009255969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62f010f0a0 0x7f62f019fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.353+0000 7f62ef400640 1 -- 172.21.15.27:0/1009255969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62f010f0a0 msgr2=0x7f62f019fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.353+0000 7f62ef400640 1 --2- 172.21.15.27:0/1009255969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62f010f0a0 0x7f62f019fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.353+0000 7f62ef400640 1 -- 172.21.15.27:0/1009255969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62f0101ea0 msgr2=0x7f62f019b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.353+0000 7f62ef400640 1 --2- 172.21.15.27:0/1009255969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62f0101ea0 0x7f62f019b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.353+0000 7f62ef400640 1 -- 172.21.15.27:0/1009255969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f01a0180 con 0x7f62f01014f0 2024-04-26T22:59:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.353+0000 7f62efe00640 1 --2- 172.21.15.27:0/1009255969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62f010f0a0 0x7f62f019fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:59:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.354+0000 7f62ef400640 1 --2- 172.21.15.27:0/1009255969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f01014f0 0x7f62f019b190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f62e000ea60 tx=0x7f62e000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.354+0000 7f62e7e00640 1 -- 172.21.15.27:0/1009255969 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e000cde0 con 0x7f62f01014f0 2024-04-26T22:59:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.354+0000 7f62e7e00640 1 -- 172.21.15.27:0/1009255969 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62e00035f0 con 0x7f62f01014f0 2024-04-26T22:59:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.354+0000 7f62f5200640 1 -- 172.21.15.27:0/1009255969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f01a0470 con 0x7f62f01014f0 2024-04-26T22:59:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.354+0000 7f62e7e00640 1 -- 172.21.15.27:0/1009255969 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0010690 con 0x7f62f01014f0 2024-04-26T22:59:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.354+0000 7f62f5200640 1 -- 172.21.15.27:0/1009255969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f01a7d00 con 0x7f62f01014f0 2024-04-26T22:59:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.358+0000 7f62e7e00640 1 -- 172.21.15.27:0/1009255969 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f62e00037a0 con 0x7f62f01014f0 2024-04-26T22:59:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.359+0000 7f62f9784740 1 Processor -- start 2024-04-26T22:59:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.359+0000 7f62f9784740 1 -- start start 2024-04-26T22:59:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.359+0000 7f62f9784740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bbb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.359+0000 7f62f9784740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 0x55f6d3bc5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.359+0000 7f62f9784740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bc61c0 0x55f6d3bc85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.359+0000 7f62f9784740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f6d3b20380 con 0x55f6d3bbb900 2024-04-26T22:59:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.359+0000 7f62f9784740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55f6d3abcc30 con 0x55f6d3bc61c0 2024-04-26T22:59:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.359+0000 7f62f9784740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f6d3b1c340 con 0x55f6d3bbafe0 2024-04-26T22:59:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.360+0000 7f62ef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 0x55f6d3bc5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.360+0000 7f62efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bbb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.360+0000 7f62efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bbb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:56618/0 (socket says 172.21.15.27:56618) 2024-04-26T22:59:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.360+0000 7f62efe00640 1 -- 172.21.15.27:0/21845107 learned_addr learned my addr 172.21.15.27:0/21845107 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.360+0000 7f62eea00640 1 --2- 172.21.15.27:0/21845107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bc61c0 0x55f6d3bc85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.360+0000 7f62e7e00640 1 --2- 172.21.15.27:0/1009255969 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f62d40a3ea0 0x7f62d40a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.360+0000 7f62eea00640 1 -- 172.21.15.27:0/21845107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bbafe0 msgr2=0x55f6d3bbb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.360+0000 7f62efe00640 1 --2- 172.21.15.27:0/1009255969 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f62d40a3ea0 0x7f62d40a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.360+0000 7f62eea00640 1 --2- 172.21.15.27:0/21845107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bbb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.360+0000 7f62eea00640 1 -- 172.21.15.27:0/21845107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 msgr2=0x55f6d3bc5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.360+0000 7f62eea00640 1 --2- 172.21.15.27:0/21845107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 0x55f6d3bc5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.360+0000 7f62eea00640 1 -- 172.21.15.27:0/21845107 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6d3ade940 con 0x55f6d3bc61c0 2024-04-26T22:59:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.360+0000 7f62e7e00640 1 -- 172.21.15.27:0/1009255969 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f62e00d1280 con 0x7f62f01014f0 2024-04-26T22:59:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.360+0000 7f62eea00640 1 --2- 172.21.15.27:0/21845107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bc61c0 0x55f6d3bc85b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f62dc02a9e0 tx=0x7f62dc005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e24121a93b10f011 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.361+0000 7f62e4c00640 1 -- 172.21.15.27:0/21845107 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc035040 con 0x55f6d3bc61c0 2024-04-26T22:59:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.361+0000 7f62efe00640 1 --2- 172.21.15.27:0/1009255969 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f62d40a3ea0 0x7f62d40a6360 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009fd0 tx=0x7f62e8004ab0 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.361+0000 7f62e4c00640 1 -- 172.21.15.27:0/21845107 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62dc0054f0 con 0x55f6d3bc61c0 2024-04-26T22:59:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.361+0000 7f62e4c00640 1 -- 172.21.15.27:0/21845107 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc0035f0 con 0x55f6d3bc61c0 2024-04-26T22:59:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.362+0000 7f62f9784740 1 -- 172.21.15.27:0/21845107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bc61c0 msgr2=0x55f6d3bc85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.362+0000 7f62f9784740 1 --2- 172.21.15.27:0/21845107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bc61c0 0x55f6d3bc85b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f62dc02a9e0 tx=0x7f62dc005c00 comp rx=0 tx=0).stop 2024-04-26T22:59:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.362+0000 7f62f9784740 1 -- 172.21.15.27:0/21845107 shutdown_connections 2024-04-26T22:59:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.362+0000 7f62f9784740 1 --2- 172.21.15.27:0/21845107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bc61c0 0x55f6d3bc85b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.362+0000 7f62f9784740 1 --2- 172.21.15.27:0/21845107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 0x55f6d3bc5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.362+0000 7f62f9784740 1 --2- 172.21.15.27:0/21845107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bbb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.362+0000 7f62f9784740 1 -- 172.21.15.27:0/21845107 >> 172.21.15.27:0/21845107 conn(0x55f6d3bb74f0 msgr2=0x55f6d3bb78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.362+0000 7f62f9784740 1 -- 172.21.15.27:0/21845107 shutdown_connections 2024-04-26T22:59:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.362+0000 7f62f9784740 1 -- 172.21.15.27:0/21845107 wait complete. 2024-04-26T22:59:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.362+0000 7f62f9784740 1 -- start start 2024-04-26T22:59:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.363+0000 7f62f9784740 1 Processor -- start 2024-04-26T22:59:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.363+0000 7f62f9784740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bc3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.363+0000 7f62f9784740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 0x55f6d3bc3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.363+0000 7f62f9784740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bc61c0 0x55f6d3bc3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.363+0000 7f62f9784740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f6d3bc8d90 con 0x55f6d3bbb900 2024-04-26T22:59:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.363+0000 7f62f9784740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55f6d3aa7c60 con 0x55f6d3bbafe0 2024-04-26T22:59:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.363+0000 7f62f9784740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f6d3bc8f10 con 0x55f6d3bc61c0 2024-04-26T22:59:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.363+0000 7f62f9784740 10 client.0 _ll_register_callbacks cb 0x55f6d3ad3e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:59:31.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:31 smithi027 ceph-mon[28034]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T22:59:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.364+0000 7f62efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bc61c0 0x55f6d3bc3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.364+0000 7f62ef400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bc3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.364+0000 7f62efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bc61c0 0x55f6d3bc3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:56632/0 (socket says 172.21.15.27:56632) 2024-04-26T22:59:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.364+0000 7f62ef400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bc3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:55410/0 (socket says 172.21.15.27:55410) 2024-04-26T22:59:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.364+0000 7f62efe00640 1 -- 172.21.15.27:0/2103870124 learned_addr learned my addr 172.21.15.27:0/2103870124 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.364+0000 7f62eea00640 1 --2- 172.21.15.27:0/2103870124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 0x55f6d3bc3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.364+0000 7f62ef400640 1 -- 172.21.15.27:0/2103870124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bc61c0 msgr2=0x55f6d3bc3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.364+0000 7f62ef400640 1 --2- 172.21.15.27:0/2103870124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bc61c0 0x55f6d3bc3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.364+0000 7f62ef400640 1 -- 172.21.15.27:0/2103870124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 msgr2=0x55f6d3bc3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.364+0000 7f62ef400640 1 --2- 172.21.15.27:0/2103870124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 0x55f6d3bc3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.364+0000 7f62ef400640 1 -- 172.21.15.27:0/2103870124 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6d3ab6510 con 0x55f6d3bbafe0 2024-04-26T22:59:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.364+0000 7f62ef400640 1 --2- 172.21.15.27:0/2103870124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bc3130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f62e00ce3b0 tx=0x7f62e00d48e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e89f25edae6d3fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.364+0000 7f62d9600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T22:59:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.365+0000 7f62d9600640 1 -- 172.21.15.27:0/2103870124 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e009de60 con 0x55f6d3bbafe0 2024-04-26T22:59:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.365+0000 7f62f9784740 1 -- 172.21.15.27:0/2103870124 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62dc0054f0 con 0x55f6d3bbafe0 2024-04-26T22:59:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.365+0000 7f62d9600640 1 -- 172.21.15.27:0/2103870124 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f62e0004000 con 0x55f6d3bbafe0 2024-04-26T22:59:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.365+0000 7f62d9600640 1 -- 172.21.15.27:0/2103870124 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f62e009c490 con 0x55f6d3bbafe0 2024-04-26T22:59:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.365+0000 7f62f9784740 10 client.25550 fetch_fsmap learned FSMap version 25 2024-04-26T22:59:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.365+0000 7f62f9784740 1 -- 172.21.15.27:0/2103870124 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f6d3832210 con 0x55f6d3bbafe0 2024-04-26T22:59:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.365+0000 7f62d9600640 1 -- 172.21.15.27:0/2103870124 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62dc0054f0 con 0x55f6d3bbafe0 2024-04-26T22:59:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.365+0000 7f62d9600640 1 -- 172.21.15.27:0/2103870124 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f62e00991a0 con 0x55f6d3bbafe0 2024-04-26T22:59:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.366+0000 7f62f9784740 10 client.25550 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:59:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.366+0000 7f62f9784740 10 client.25550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:59:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.366+0000 7f62f9784740 20 client.25550 populate_metadata read hostname 'smithi027' 2024-04-26T22:59:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.366+0000 7f62f9784740 1 --2- 172.21.15.27:0/2103870124 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55f6d3bd5e80 0x55f6d3bd82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.366+0000 7f62f9784740 4 client.25550 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:59:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.366+0000 7f62f9784740 1 -- 172.21.15.27:0/2103870124 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6d3b2df90 con 0x55f6d3bd5e80 2024-04-26T22:59:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.366+0000 7f62eea00640 1 --2- 172.21.15.27:0/2103870124 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55f6d3bd5e80 0x55f6d3bd82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.367+0000 7f62eea00640 1 --2- 172.21.15.27:0/2103870124 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55f6d3bd5e80 0x55f6d3bd82d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=607faa723921462e server_cookie=d3aa7e2c6ddca7f8 in_seq=0 out_seq=0 2024-04-26T22:59:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.367+0000 7f62d9600640 10 client.25550 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.367+0000 7f62d9600640 1 -- 172.21.15.27:0/2103870124 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6d3b2df90 con 0x55f6d3bd5e80 2024-04-26T22:59:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.367+0000 7f62d9600640 10 client.25550 handle_command_reply: tid=0 2024-04-26T22:59:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.368+0000 7f62f9784740 1 client.25550 shutdown 2024-04-26T22:59:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.368+0000 7f62f9784740 1 -- 172.21.15.27:0/2103870124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bbafe0 msgr2=0x55f6d3bc3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.368+0000 7f62f9784740 1 --2- 172.21.15.27:0/2103870124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bc3130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f62e00ce3b0 tx=0x7f62e00d48e0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.368+0000 7f62f9784740 1 -- 172.21.15.27:0/2103870124 shutdown_connections 2024-04-26T22:59:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.368+0000 7f62f9784740 1 --2- 172.21.15.27:0/2103870124 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55f6d3bd5e80 0x55f6d3bd82d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.368+0000 7f62f9784740 1 --2- 172.21.15.27:0/2103870124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bc61c0 0x55f6d3bc3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.368+0000 7f62d9600640 0 client.25550 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.368+0000 7f62f9784740 1 --2- 172.21.15.27:0/2103870124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 0x55f6d3bc3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.368+0000 7f62f9784740 1 --2- 172.21.15.27:0/2103870124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bc3130 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.368+0000 7f62f9784740 1 -- 172.21.15.27:0/2103870124 >> 172.21.15.27:0/2103870124 conn(0x55f6d3bb74f0 msgr2=0x55f6d3bbd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.368+0000 7f62f9784740 1 -- 172.21.15.27:0/2103870124 shutdown_connections 2024-04-26T22:59:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.368+0000 7f62f9784740 1 -- 172.21.15.27:0/2103870124 wait complete. 2024-04-26T22:59:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.368+0000 7f62f9784740 20 client.25550 trim_cache size 0 max 16384 2024-04-26T22:59:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.384+0000 7f62f9784740 1 Processor -- start 2024-04-26T22:59:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.385+0000 7f62f9784740 1 -- start start 2024-04-26T22:59:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.385+0000 7f62f9784740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bd1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.385+0000 7f62f9784740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bbb900 0x55f6d3bb64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.385+0000 7f62f9784740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bc61c0 0x55f6d3bddc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.385+0000 7f62f9784740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01b4800 con 0x55f6d3bbafe0 2024-04-26T22:59:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.385+0000 7f62f9784740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01b20d0 con 0x55f6d3bc61c0 2024-04-26T22:59:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.385+0000 7f62f9784740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01aff20 con 0x55f6d3bbb900 2024-04-26T22:59:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.385+0000 7f62ef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bd1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.385+0000 7f62ef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bd1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46538/0 (socket says 172.21.15.27:46538) 2024-04-26T22:59:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.385+0000 7f62ef400640 1 -- 172.21.15.27:0/2098173752 learned_addr learned my addr 172.21.15.27:0/2098173752 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.385+0000 7f62eea00640 1 --2- 172.21.15.27:0/2098173752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bbb900 0x55f6d3bb64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.385+0000 7f62efe00640 1 --2- 172.21.15.27:0/2098173752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bc61c0 0x55f6d3bddc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.386+0000 7f62ef400640 1 -- 172.21.15.27:0/2098173752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bbb900 msgr2=0x55f6d3bb64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.386+0000 7f62ef400640 1 --2- 172.21.15.27:0/2098173752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bbb900 0x55f6d3bb64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.386+0000 7f62ef400640 1 -- 172.21.15.27:0/2098173752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bc61c0 msgr2=0x55f6d3bddc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.386+0000 7f62ef400640 1 --2- 172.21.15.27:0/2098173752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bc61c0 0x55f6d3bddc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.386+0000 7f62ef400640 1 -- 172.21.15.27:0/2098173752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6d3bbb730 con 0x55f6d3bbafe0 2024-04-26T22:59:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.386+0000 7f62eea00640 1 --2- 172.21.15.27:0/2098173752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bbb900 0x55f6d3bb64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:59:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.386+0000 7f62ef400640 1 --2- 172.21.15.27:0/2098173752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bd1c50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f62e0004610 tx=0x7f62e00bd050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed47d92e5aeb5878 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.386+0000 7f62dbe00640 1 -- 172.21.15.27:0/2098173752 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e009de60 con 0x55f6d3bbafe0 2024-04-26T22:59:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.386+0000 7f62dbe00640 1 -- 172.21.15.27:0/2098173752 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62e00ce2f0 con 0x55f6d3bbafe0 2024-04-26T22:59:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.386+0000 7f62dbe00640 1 -- 172.21.15.27:0/2098173752 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0004000 con 0x55f6d3bbafe0 2024-04-26T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.387+0000 7f62f9784740 1 -- 172.21.15.27:0/2098173752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbafe0 msgr2=0x55f6d3bd1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.387+0000 7f62f9784740 1 --2- 172.21.15.27:0/2098173752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bd1c50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f62e0004610 tx=0x7f62e00bd050 comp rx=0 tx=0).stop 2024-04-26T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.387+0000 7f62f9784740 1 -- 172.21.15.27:0/2098173752 shutdown_connections 2024-04-26T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.387+0000 7f62f9784740 1 --2- 172.21.15.27:0/2098173752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bc61c0 0x55f6d3bddc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.387+0000 7f62f9784740 1 --2- 172.21.15.27:0/2098173752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bbb900 0x55f6d3bb64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.387+0000 7f62f9784740 1 --2- 172.21.15.27:0/2098173752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bd1c50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.387+0000 7f62f9784740 1 -- 172.21.15.27:0/2098173752 >> 172.21.15.27:0/2098173752 conn(0x55f6d3bb74f0 msgr2=0x55f6d3bb5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.387+0000 7f62f9784740 1 -- 172.21.15.27:0/2098173752 shutdown_connections 2024-04-26T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.387+0000 7f62f9784740 1 -- 172.21.15.27:0/2098173752 wait complete. 2024-04-26T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.388+0000 7f62f9784740 1 -- start start 2024-04-26T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.388+0000 7f62f9784740 1 Processor -- start 2024-04-26T22:59:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.388+0000 7f62f9784740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bc04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.388+0000 7f62f9784740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 0x55f6d3bc0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.388+0000 7f62f9784740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bc61c0 0x55f6d3c3b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.388+0000 7f62f9784740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01b0590 con 0x55f6d3bbb900 2024-04-26T22:59:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.388+0000 7f62f9784740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01b1a00 con 0x55f6d3bbafe0 2024-04-26T22:59:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.388+0000 7f62f9784740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01b0c60 con 0x55f6d3bc61c0 2024-04-26T22:59:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.388+0000 7f62f9784740 10 client.0 _ll_register_callbacks cb 0x7f62e0098790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:59:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.389+0000 7f62eea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 0x55f6d3bc0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.389+0000 7f62eea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 0x55f6d3bc0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46548/0 (socket says 172.21.15.27:46548) 2024-04-26T22:59:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.389+0000 7f62eea00640 1 -- 172.21.15.27:0/851815752 learned_addr learned my addr 172.21.15.27:0/851815752 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.389+0000 7f62ef400640 1 --2- 172.21.15.27:0/851815752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bc04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.389+0000 7f62efe00640 1 --2- 172.21.15.27:0/851815752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bc61c0 0x55f6d3c3b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.389+0000 7f62eea00640 1 -- 172.21.15.27:0/851815752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bc61c0 msgr2=0x55f6d3c3b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.389+0000 7f62eea00640 1 --2- 172.21.15.27:0/851815752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bc61c0 0x55f6d3c3b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.389+0000 7f62eea00640 1 -- 172.21.15.27:0/851815752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bbafe0 msgr2=0x55f6d3bc04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.389+0000 7f62eea00640 1 --2- 172.21.15.27:0/851815752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bc04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.389+0000 7f62eea00640 1 -- 172.21.15.27:0/851815752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6d38fe620 con 0x55f6d3bbb900 2024-04-26T22:59:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.389+0000 7f62ef400640 1 --2- 172.21.15.27:0/851815752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bc04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:59:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.389+0000 7f62eea00640 1 --2- 172.21.15.27:0/851815752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 0x55f6d3bc0a10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0057c0 tx=0x7f62dc004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d79c7e676a6aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.389+0000 7f62d9600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T22:59:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.390+0000 7f62d9600640 1 -- 172.21.15.27:0/851815752 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc059060 con 0x55f6d3bbb900 2024-04-26T22:59:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.390+0000 7f62d9600640 1 -- 172.21.15.27:0/851815752 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f62dc0534f0 con 0x55f6d3bbb900 2024-04-26T22:59:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.390+0000 7f62f9784740 1 -- 172.21.15.27:0/851815752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e00ce2f0 con 0x55f6d3bbb900 2024-04-26T22:59:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.390+0000 7f62d9600640 1 -- 172.21.15.27:0/851815752 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f62dc035040 con 0x55f6d3bbb900 2024-04-26T22:59:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.390+0000 7f62f9784740 10 client.15441 fetch_fsmap learned FSMap version 25 2024-04-26T22:59:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.390+0000 7f62f9784740 1 -- 172.21.15.27:0/851815752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f6d3adacb0 con 0x55f6d3bbb900 2024-04-26T22:59:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.390+0000 7f62d9600640 1 -- 172.21.15.27:0/851815752 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62e00ce2f0 con 0x55f6d3bbb900 2024-04-26T22:59:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.391+0000 7f62d9600640 1 -- 172.21.15.27:0/851815752 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f62dc0314a0 con 0x55f6d3bbb900 2024-04-26T22:59:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.391+0000 7f62f9784740 10 client.15441 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:59:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.391+0000 7f62f9784740 10 client.15441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:59:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.391+0000 7f62f9784740 20 client.15441 populate_metadata read hostname 'smithi027' 2024-04-26T22:59:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.391+0000 7f62f9784740 1 --2- 172.21.15.27:0/851815752 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55f6d3c3cc70 0x55f6d3c3f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.391+0000 7f62f9784740 4 client.15441 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:59:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.391+0000 7f62f9784740 1 -- 172.21.15.27:0/851815752 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6d3b27e00 con 0x55f6d3c3cc70 2024-04-26T22:59:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.391+0000 7f62ef400640 1 --2- 172.21.15.27:0/851815752 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55f6d3c3cc70 0x55f6d3c3f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.392+0000 7f62ef400640 1 --2- 172.21.15.27:0/851815752 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55f6d3c3cc70 0x55f6d3c3f060 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58853095be5e2ec2 server_cookie=5da0542d1b6d044a in_seq=0 out_seq=0 2024-04-26T22:59:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.392+0000 7f62d9600640 10 client.15441 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.395+0000 7f62d9600640 1 -- 172.21.15.27:0/851815752 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f6d3b27e00 con 0x55f6d3c3cc70 2024-04-26T22:59:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.395+0000 7f62d9600640 10 client.15441 handle_command_reply: tid=0 2024-04-26T22:59:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.395+0000 7f62f9784740 1 client.15441 shutdown 2024-04-26T22:59:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.395+0000 7f62f9784740 1 -- 172.21.15.27:0/851815752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 msgr2=0x55f6d3bc0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.395+0000 7f62f9784740 1 --2- 172.21.15.27:0/851815752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 0x55f6d3bc0a10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0057c0 tx=0x7f62dc004410 comp rx=0 tx=0).stop 2024-04-26T22:59:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.396+0000 7f62f9784740 1 -- 172.21.15.27:0/851815752 shutdown_connections 2024-04-26T22:59:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.396+0000 7f62f9784740 1 --2- 172.21.15.27:0/851815752 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55f6d3c3cc70 0x55f6d3c3f060 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.396+0000 7f62f9784740 1 --2- 172.21.15.27:0/851815752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f6d3bc61c0 0x55f6d3c3b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.396+0000 7f62f9784740 1 --2- 172.21.15.27:0/851815752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f6d3bbb900 0x55f6d3bc0a10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.396+0000 7f62d9600640 0 client.15441 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.396+0000 7f62f9784740 1 --2- 172.21.15.27:0/851815752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6d3bbafe0 0x55f6d3bc04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.396+0000 7f62f9784740 1 -- 172.21.15.27:0/851815752 >> 172.21.15.27:0/851815752 conn(0x55f6d3bb74f0 msgr2=0x55f6d3bc3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.396+0000 7f62f9784740 1 -- 172.21.15.27:0/851815752 shutdown_connections 2024-04-26T22:59:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.396+0000 7f62f9784740 1 -- 172.21.15.27:0/851815752 wait complete. 2024-04-26T22:59:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.396+0000 7f62f9784740 20 client.15441 trim_cache size 0 max 16384 2024-04-26T22:59:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T22:59:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T22:59:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T22:59:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T22:59:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.397+0000 7f62f5200640 1 -- 172.21.15.27:0/1009255969 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f62d40a3ea0 msgr2=0x7f62d40a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.397+0000 7f62f5200640 1 --2- 172.21.15.27:0/1009255969 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f62d40a3ea0 0x7f62d40a6360 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009fd0 tx=0x7f62e8004ab0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.397+0000 7f62f5200640 1 -- 172.21.15.27:0/1009255969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f01014f0 msgr2=0x7f62f019b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.397+0000 7f62f5200640 1 --2- 172.21.15.27:0/1009255969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f01014f0 0x7f62f019b190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f62e000ea60 tx=0x7f62e000ef20 comp rx=0 tx=0).stop 2024-04-26T22:59:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.397+0000 7f62f5200640 1 -- 172.21.15.27:0/1009255969 shutdown_connections 2024-04-26T22:59:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.397+0000 7f62f5200640 1 --2- 172.21.15.27:0/1009255969 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f62d40a3ea0 0x7f62d40a6360 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.397+0000 7f62f5200640 1 --2- 172.21.15.27:0/1009255969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62f0101ea0 0x7f62f019b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.397+0000 7f62f5200640 1 --2- 172.21.15.27:0/1009255969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62f010f0a0 0x7f62f019fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.397+0000 7f62f5200640 1 --2- 172.21.15.27:0/1009255969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f01014f0 0x7f62f019b190 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.397+0000 7f62f5200640 1 -- 172.21.15.27:0/1009255969 >> 172.21.15.27:0/1009255969 conn(0x7f62f00fd110 msgr2=0x7f62f00ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.397+0000 7f62f5200640 1 -- 172.21.15.27:0/1009255969 shutdown_connections 2024-04-26T22:59:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:31.397+0000 7f62f5200640 1 -- 172.21.15.27:0/1009255969 wait complete. 2024-04-26T22:59:33.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:33 smithi080 ceph-mon[32324]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-26T22:59:33.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:33 smithi094 ceph-mon[32889]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-26T22:59:33.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:33 smithi027 ceph-mon[28034]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-26T22:59:35.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:35 smithi080 ceph-mon[32324]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-26T22:59:35.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:35 smithi094 ceph-mon[32889]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-26T22:59:35.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:35 smithi027 ceph-mon[28034]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-26T22:59:37.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:37 smithi080 ceph-mon[32324]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:59:37.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:37 smithi094 ceph-mon[32889]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:59:37.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:37 smithi027 ceph-mon[28034]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:59:39.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:39 smithi080 ceph-mon[32324]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T22:59:39.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:39 smithi094 ceph-mon[32889]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T22:59:39.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:39 smithi027 ceph-mon[28034]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T22:59:41.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:41 smithi080 ceph-mon[32324]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:59:41.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:41 smithi094 ceph-mon[32889]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:59:41.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:41 smithi027 ceph-mon[28034]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:59:43.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:43 smithi094 ceph-mon[32889]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T22:59:43.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:43 smithi027 ceph-mon[28034]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T22:59:43.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:43 smithi080 ceph-mon[32324]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T22:59:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.588+0000 7fe634a00640 1 -- 172.21.15.27:0/2627594007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe630104230 msgr2=0x7fe63010c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.588+0000 7fe634a00640 1 --2- 172.21.15.27:0/2627594007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe630104230 0x7fe63010c710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009900 tx=0x7fe61c02a490 comp rx=0 tx=0).stop 2024-04-26T22:59:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.588+0000 7fe634a00640 1 -- 172.21.15.27:0/2627594007 shutdown_connections 2024-04-26T22:59:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.588+0000 7fe634a00640 1 --2- 172.21.15.27:0/2627594007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe63010cd30 0x7fe63010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.588+0000 7fe634a00640 1 --2- 172.21.15.27:0/2627594007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe630104230 0x7fe63010c710 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.588+0000 7fe634a00640 1 --2- 172.21.15.27:0/2627594007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe630103910 0x7fe630103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.588+0000 7fe634a00640 1 -- 172.21.15.27:0/2627594007 >> 172.21.15.27:0/2627594007 conn(0x7fe6300fd110 msgr2=0x7fe6300ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.588+0000 7fe634a00640 1 -- 172.21.15.27:0/2627594007 shutdown_connections 2024-04-26T22:59:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.588+0000 7fe634a00640 1 -- 172.21.15.27:0/2627594007 wait complete. 2024-04-26T22:59:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.590+0000 7fe634a00640 1 Processor -- start 2024-04-26T22:59:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.590+0000 7fe634a00640 1 -- start start 2024-04-26T22:59:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.590+0000 7fe634a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe630103910 0x7fe630198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.590+0000 7fe634a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe630104230 0x7fe630199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.590+0000 7fe634a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe63010cd30 0x7fe63019d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.590+0000 7fe634a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe630112a30 con 0x7fe630103910 2024-04-26T22:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.590+0000 7fe634a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe6301128b0 con 0x7fe63010cd30 2024-04-26T22:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.590+0000 7fe634a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe630112bb0 con 0x7fe630104230 2024-04-26T22:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.591+0000 7fe62de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe630104230 0x7fe630199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.591+0000 7fe62de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe630104230 0x7fe630199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:59528/0 (socket says 172.21.15.27:59528) 2024-04-26T22:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.591+0000 7fe62de00640 1 -- 172.21.15.27:0/3051126246 learned_addr learned my addr 172.21.15.27:0/3051126246 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.591+0000 7fe62e800640 1 --2- 172.21.15.27:0/3051126246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe630103910 0x7fe630198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.591+0000 7fe62f200640 1 --2- 172.21.15.27:0/3051126246 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe63010cd30 0x7fe63019d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.591+0000 7fe62de00640 1 -- 172.21.15.27:0/3051126246 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe63010cd30 msgr2=0x7fe63019d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.591+0000 7fe62de00640 1 --2- 172.21.15.27:0/3051126246 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe63010cd30 0x7fe63019d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.591+0000 7fe62de00640 1 -- 172.21.15.27:0/3051126246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe630103910 msgr2=0x7fe630198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.591+0000 7fe62de00640 1 --2- 172.21.15.27:0/3051126246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe630103910 0x7fe630198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.591+0000 7fe62de00640 1 -- 172.21.15.27:0/3051126246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63019df40 con 0x7fe630104230 2024-04-26T22:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.591+0000 7fe62f200640 1 --2- 172.21.15.27:0/3051126246 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe63010cd30 0x7fe63019d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.592+0000 7fe62de00640 1 --2- 172.21.15.27:0/3051126246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe630104230 0x7fe630199490 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe61c005ab0 tx=0x7fe61c003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.592+0000 7fe627400640 1 -- 172.21.15.27:0/3051126246 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c03f070 con 0x7fe630104230 2024-04-26T22:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.592+0000 7fe627400640 1 -- 172.21.15.27:0/3051126246 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe61c031d60 con 0x7fe630104230 2024-04-26T22:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.592+0000 7fe634a00640 1 -- 172.21.15.27:0/3051126246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe63019e1d0 con 0x7fe630104230 2024-04-26T22:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.592+0000 7fe627400640 1 -- 172.21.15.27:0/3051126246 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c035030 con 0x7fe630104230 2024-04-26T22:59:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.592+0000 7fe634a00640 1 -- 172.21.15.27:0/3051126246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe630101490 con 0x7fe630104230 2024-04-26T22:59:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.597+0000 7fe63822a740 1 Processor -- start 2024-04-26T22:59:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.597+0000 7fe63822a740 1 -- start start 2024-04-26T22:59:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.597+0000 7fe63822a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa60bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.598+0000 7fe63822a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa61110 0x563ebfa6b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.598+0000 7fe63822a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa6b9d0 0x563ebfa6ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.598+0000 7fe63822a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563ebf9bd380 con 0x563ebfa6b9d0 2024-04-26T22:59:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.598+0000 7fe63822a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x563ebf959c30 con 0x563ebfa61110 2024-04-26T22:59:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.598+0000 7fe63822a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563ebf9b9340 con 0x563ebfa607f0 2024-04-26T22:59:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.598+0000 7fe62f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa60bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.598+0000 7fe62f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa60bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:59536/0 (socket says 172.21.15.27:59536) 2024-04-26T22:59:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.598+0000 7fe62f200640 1 -- 172.21.15.27:0/1955976170 learned_addr learned my addr 172.21.15.27:0/1955976170 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.598+0000 7fe62e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa61110 0x563ebfa6b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.598+0000 7fe62f200640 1 -- 172.21.15.27:0/1955976170 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa61110 msgr2=0x563ebfa6b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.598+0000 7fe62f200640 1 --2- 172.21.15.27:0/1955976170 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa61110 0x563ebfa6b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.598+0000 7fe62f200640 1 -- 172.21.15.27:0/1955976170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa6b9d0 msgr2=0x563ebfa6ddc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:59:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.598+0000 7fe62f200640 1 --2- 172.21.15.27:0/1955976170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa6b9d0 0x563ebfa6ddc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.598+0000 7fe62f200640 1 -- 172.21.15.27:0/1955976170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ebf97b940 con 0x563ebfa607f0 2024-04-26T22:59:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.599+0000 7fe627400640 1 -- 172.21.15.27:0/3051126246 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fe61c042070 con 0x7fe630104230 2024-04-26T22:59:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.599+0000 7fe62f200640 1 --2- 172.21.15.27:0/1955976170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa60bd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe628009f30 tx=0x7fe628010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=390eada20118d4a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.599+0000 7fe61b400640 1 -- 172.21.15.27:0/1955976170 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628016070 con 0x563ebfa607f0 2024-04-26T22:59:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.599+0000 7fe61b400640 1 -- 172.21.15.27:0/1955976170 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe62801bb10 con 0x563ebfa607f0 2024-04-26T22:59:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.599+0000 7fe61b400640 1 -- 172.21.15.27:0/1955976170 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628012bc0 con 0x563ebfa607f0 2024-04-26T22:59:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.600+0000 7fe63822a740 1 -- 172.21.15.27:0/1955976170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 msgr2=0x563ebfa60bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.600+0000 7fe63822a740 1 --2- 172.21.15.27:0/1955976170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa60bd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe628009f30 tx=0x7fe628010800 comp rx=0 tx=0).stop 2024-04-26T22:59:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.601+0000 7fe63822a740 1 -- 172.21.15.27:0/1955976170 shutdown_connections 2024-04-26T22:59:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.601+0000 7fe63822a740 1 --2- 172.21.15.27:0/1955976170 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa61110 0x563ebfa6b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.601+0000 7fe63822a740 1 --2- 172.21.15.27:0/1955976170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa6b9d0 0x563ebfa6ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.601+0000 7fe63822a740 1 --2- 172.21.15.27:0/1955976170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa60bd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.601+0000 7fe63822a740 1 -- 172.21.15.27:0/1955976170 >> 172.21.15.27:0/1955976170 conn(0x563ebfa5cd00 msgr2=0x563ebfa5d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.601+0000 7fe63822a740 1 -- 172.21.15.27:0/1955976170 shutdown_connections 2024-04-26T22:59:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.601+0000 7fe63822a740 1 -- 172.21.15.27:0/1955976170 wait complete. 2024-04-26T22:59:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.601+0000 7fe627400640 1 --2- 172.21.15.27:0/3051126246 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe6140a3db0 0x7fe6140a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.601+0000 7fe627400640 1 -- 172.21.15.27:0/3051126246 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fe61c0f1530 con 0x7fe630104230 2024-04-26T22:59:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.602+0000 7fe63822a740 1 -- start start 2024-04-26T22:59:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.602+0000 7fe63822a740 1 Processor -- start 2024-04-26T22:59:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.602+0000 7fe62e800640 1 --2- 172.21.15.27:0/3051126246 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe6140a3db0 0x7fe6140a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.603+0000 7fe63822a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa689d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.603+0000 7fe63822a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa61110 0x563ebfa68f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.603+0000 7fe63822a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa6b9d0 0x563ebfa69450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.603+0000 7fe63822a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563ebfa6e600 con 0x563ebfa61110 2024-04-26T22:59:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.603+0000 7fe63822a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x563ebf944c60 con 0x563ebfa6b9d0 2024-04-26T22:59:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.603+0000 7fe63822a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563ebfa6e780 con 0x563ebfa607f0 2024-04-26T22:59:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.603+0000 7fe63822a740 10 client.0 _ll_register_callbacks cb 0x563ebf970e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:59:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.603+0000 7fe62de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa61110 0x563ebfa68f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.603+0000 7fe62de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa61110 0x563ebfa68f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59796/0 (socket says 172.21.15.27:59796) 2024-04-26T22:59:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.603+0000 7fe62de00640 1 -- 172.21.15.27:0/1299281621 learned_addr learned my addr 172.21.15.27:0/1299281621 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.603+0000 7fe62e800640 1 --2- 172.21.15.27:0/1299281621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.603+0000 7fe62e800640 1 --2- 172.21.15.27:0/3051126246 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe6140a3db0 0x7fe6140a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x563ebfa61d90 tx=0x7fe62000d500 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.604+0000 7fe62f200640 1 --2- 172.21.15.27:0/1299281621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa6b9d0 0x563ebfa69450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.604+0000 7fe62e800640 1 -- 172.21.15.27:0/1299281621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa6b9d0 msgr2=0x563ebfa69450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.604+0000 7fe62e800640 1 --2- 172.21.15.27:0/1299281621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa6b9d0 0x563ebfa69450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.604+0000 7fe62e800640 1 -- 172.21.15.27:0/1299281621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa61110 msgr2=0x563ebfa68f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.604+0000 7fe62e800640 1 --2- 172.21.15.27:0/1299281621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa61110 0x563ebfa68f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.604+0000 7fe62e800640 1 -- 172.21.15.27:0/1299281621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ebf953510 con 0x563ebfa607f0 2024-04-26T22:59:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.604+0000 7fe62de00640 1 --2- 172.21.15.27:0/1299281621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa61110 0x563ebfa68f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:59:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.604+0000 7fe62e800640 1 --2- 172.21.15.27:0/1299281621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa689d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe620004390 tx=0x7fe6200046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d4c25540f1d780b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.604+0000 7fe60fe00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T22:59:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.604+0000 7fe60fe00640 1 -- 172.21.15.27:0/1299281621 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62001b680 con 0x563ebfa607f0 2024-04-26T22:59:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.605+0000 7fe63822a740 1 -- 172.21.15.27:0/1299281621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe62801bb10 con 0x563ebfa607f0 2024-04-26T22:59:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.605+0000 7fe60fe00640 1 -- 172.21.15.27:0/1299281621 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fe62001b820 con 0x563ebfa607f0 2024-04-26T22:59:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.605+0000 7fe60fe00640 1 -- 172.21.15.27:0/1299281621 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe62001ba20 con 0x563ebfa607f0 2024-04-26T22:59:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.605+0000 7fe63822a740 10 client.25594 fetch_fsmap learned FSMap version 25 2024-04-26T22:59:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.605+0000 7fe63822a740 1 -- 172.21.15.27:0/1299281621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563ebf6cf210 con 0x563ebfa607f0 2024-04-26T22:59:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.605+0000 7fe60fe00640 1 -- 172.21.15.27:0/1299281621 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe62801bb10 con 0x563ebfa607f0 2024-04-26T22:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.606+0000 7fe60fe00640 1 -- 172.21.15.27:0/1299281621 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe620024020 con 0x563ebfa607f0 2024-04-26T22:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.606+0000 7fe63822a740 10 client.25594 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.606+0000 7fe63822a740 10 client.25594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.606+0000 7fe63822a740 20 client.25594 populate_metadata read hostname 'smithi027' 2024-04-26T22:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.606+0000 7fe63822a740 1 --2- 172.21.15.27:0/1299281621 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x563ebfa7b7c0 0x563ebfa7dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.606+0000 7fe63822a740 4 client.25594 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.606+0000 7fe63822a740 1 -- 172.21.15.27:0/1299281621 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ebf9caf90 con 0x563ebfa7b7c0 2024-04-26T22:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.606+0000 7fe62de00640 1 --2- 172.21.15.27:0/1299281621 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x563ebfa7b7c0 0x563ebfa7dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.607+0000 7fe62de00640 1 --2- 172.21.15.27:0/1299281621 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x563ebfa7b7c0 0x563ebfa7dbb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74bffa3dfcca135 server_cookie=9a2a5f093ee3beed in_seq=0 out_seq=0 2024-04-26T22:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.607+0000 7fe60fe00640 10 client.25594 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.608+0000 7fe60fe00640 1 -- 172.21.15.27:0/1299281621 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ebf9caf90 con 0x563ebfa7b7c0 2024-04-26T22:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.608+0000 7fe60fe00640 10 client.25594 handle_command_reply: tid=0 2024-04-26T22:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.608+0000 7fe63822a740 1 client.25594 shutdown 2024-04-26T22:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.608+0000 7fe63822a740 1 -- 172.21.15.27:0/1299281621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 msgr2=0x563ebfa689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.608+0000 7fe63822a740 1 --2- 172.21.15.27:0/1299281621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa689d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe620004390 tx=0x7fe6200046c0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.608+0000 7fe63822a740 1 -- 172.21.15.27:0/1299281621 shutdown_connections 2024-04-26T22:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.608+0000 7fe63822a740 1 --2- 172.21.15.27:0/1299281621 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x563ebfa7b7c0 0x563ebfa7dbb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.608+0000 7fe63822a740 1 --2- 172.21.15.27:0/1299281621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa6b9d0 0x563ebfa69450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.608+0000 7fe63822a740 1 --2- 172.21.15.27:0/1299281621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa61110 0x563ebfa68f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.608+0000 7fe63822a740 1 --2- 172.21.15.27:0/1299281621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa689d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.608+0000 7fe63822a740 1 -- 172.21.15.27:0/1299281621 >> 172.21.15.27:0/1299281621 conn(0x563ebfa5cd00 msgr2=0x563ebfa53650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.608+0000 7fe60fe00640 0 client.25594 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.608+0000 7fe63822a740 1 -- 172.21.15.27:0/1299281621 shutdown_connections 2024-04-26T22:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.609+0000 7fe63822a740 1 -- 172.21.15.27:0/1299281621 wait complete. 2024-04-26T22:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.609+0000 7fe63822a740 20 client.25594 trim_cache size 0 max 16384 2024-04-26T22:59:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.640+0000 7fe63822a740 1 Processor -- start 2024-04-26T22:59:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.640+0000 7fe63822a740 1 -- start start 2024-04-26T22:59:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.641+0000 7fe63822a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa631b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.641+0000 7fe63822a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa61110 0x563ebfa834a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.641+0000 7fe63822a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa6b9d0 0x563ebfa839e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.641+0000 7fe63822a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe6301b0560 con 0x563ebfa61110 2024-04-26T22:59:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.641+0000 7fe63822a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe6301ade30 con 0x563ebfa6b9d0 2024-04-26T22:59:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.641+0000 7fe63822a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe6301abc80 con 0x563ebfa607f0 2024-04-26T22:59:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.641+0000 7fe62de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa61110 0x563ebfa834a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.641+0000 7fe62e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.641+0000 7fe62e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa631b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:59550/0 (socket says 172.21.15.27:59550) 2024-04-26T22:59:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.641+0000 7fe62e800640 1 -- 172.21.15.27:0/2891757255 learned_addr learned my addr 172.21.15.27:0/2891757255 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.642+0000 7fe62f200640 1 --2- 172.21.15.27:0/2891757255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa6b9d0 0x563ebfa839e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.642+0000 7fe62e800640 1 -- 172.21.15.27:0/2891757255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa6b9d0 msgr2=0x563ebfa839e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.642+0000 7fe62e800640 1 --2- 172.21.15.27:0/2891757255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa6b9d0 0x563ebfa839e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.642+0000 7fe62e800640 1 -- 172.21.15.27:0/2891757255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa61110 msgr2=0x563ebfa834a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.642+0000 7fe62e800640 1 --2- 172.21.15.27:0/2891757255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa61110 0x563ebfa834a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.642+0000 7fe62e800640 1 -- 172.21.15.27:0/2891757255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ebf79b620 con 0x563ebfa607f0 2024-04-26T22:59:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.642+0000 7fe62de00640 1 --2- 172.21.15.27:0/2891757255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa61110 0x563ebfa834a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:59:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.642+0000 7fe62e800640 1 --2- 172.21.15.27:0/2891757255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa631b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe620004470 tx=0x7fe6200038f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3130a18f9abbd7bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.643+0000 7fe61aa00640 1 -- 172.21.15.27:0/2891757255 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62001bdc0 con 0x563ebfa607f0 2024-04-26T22:59:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.643+0000 7fe61aa00640 1 -- 172.21.15.27:0/2891757255 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe620005040 con 0x563ebfa607f0 2024-04-26T22:59:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.643+0000 7fe61aa00640 1 -- 172.21.15.27:0/2891757255 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62001c610 con 0x563ebfa607f0 2024-04-26T22:59:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.644+0000 7fe63822a740 1 -- 172.21.15.27:0/2891757255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 msgr2=0x563ebfa631b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.644+0000 7fe63822a740 1 --2- 172.21.15.27:0/2891757255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa631b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe620004470 tx=0x7fe6200038f0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.644+0000 7fe63822a740 1 -- 172.21.15.27:0/2891757255 shutdown_connections 2024-04-26T22:59:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.644+0000 7fe63822a740 1 --2- 172.21.15.27:0/2891757255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa6b9d0 0x563ebfa839e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.644+0000 7fe63822a740 1 --2- 172.21.15.27:0/2891757255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa61110 0x563ebfa834a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.644+0000 7fe63822a740 1 --2- 172.21.15.27:0/2891757255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa607f0 0x563ebfa631b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.644+0000 7fe63822a740 1 -- 172.21.15.27:0/2891757255 >> 172.21.15.27:0/2891757255 conn(0x563ebfa5cd00 msgr2=0x563ebfa6a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.644+0000 7fe63822a740 1 -- 172.21.15.27:0/2891757255 shutdown_connections 2024-04-26T22:59:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.644+0000 7fe63822a740 1 -- 172.21.15.27:0/2891757255 wait complete. 2024-04-26T22:59:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.644+0000 7fe63822a740 1 -- start start 2024-04-26T22:59:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.645+0000 7fe63822a740 1 Processor -- start 2024-04-26T22:59:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.645+0000 7fe63822a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa607f0 0x563ebfa65560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.645+0000 7fe63822a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa61110 0x563ebfa65aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.645+0000 7fe63822a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa6b9d0 0x563ebfa65fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.646+0000 7fe63822a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe6301ac2f0 con 0x563ebfa6b9d0 2024-04-26T22:59:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.646+0000 7fe63822a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe6301ad760 con 0x563ebfa607f0 2024-04-26T22:59:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.646+0000 7fe63822a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe6301ac9c0 con 0x563ebfa61110 2024-04-26T22:59:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.646+0000 7fe63822a740 10 client.0 _ll_register_callbacks cb 0x7fe620004200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:59:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.646+0000 7fe62de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa61110 0x563ebfa65aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.646+0000 7fe62f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa6b9d0 0x563ebfa65fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.646+0000 7fe62f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa6b9d0 0x563ebfa65fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59818/0 (socket says 172.21.15.27:59818) 2024-04-26T22:59:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.646+0000 7fe62f200640 1 -- 172.21.15.27:0/230390411 learned_addr learned my addr 172.21.15.27:0/230390411 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.646+0000 7fe62e800640 1 --2- 172.21.15.27:0/230390411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa607f0 0x563ebfa65560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.646+0000 7fe62de00640 1 -- 172.21.15.27:0/230390411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa607f0 msgr2=0x563ebfa65560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.647+0000 7fe62de00640 1 --2- 172.21.15.27:0/230390411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa607f0 0x563ebfa65560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.647+0000 7fe62de00640 1 -- 172.21.15.27:0/230390411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa6b9d0 msgr2=0x563ebfa65fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.647+0000 7fe62de00640 1 --2- 172.21.15.27:0/230390411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa6b9d0 0x563ebfa65fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.647+0000 7fe62de00640 1 -- 172.21.15.27:0/230390411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ebf977cb0 con 0x563ebfa61110 2024-04-26T22:59:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.647+0000 7fe62f200640 1 --2- 172.21.15.27:0/230390411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa6b9d0 0x563ebfa65fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:59:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.647+0000 7fe62e800640 1 --2- 172.21.15.27:0/230390411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa607f0 0x563ebfa65560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:59:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.647+0000 7fe62de00640 1 --2- 172.21.15.27:0/230390411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa61110 0x563ebfa65aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0b9310 tx=0x7fe61c03b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d5d58dbe1767f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.647+0000 7fe60fe00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T22:59:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.647+0000 7fe60fe00640 1 -- 172.21.15.27:0/230390411 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0c1260 con 0x563ebfa61110 2024-04-26T22:59:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.647+0000 7fe60fe00640 1 -- 172.21.15.27:0/230390411 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fe61c0b9520 con 0x563ebfa61110 2024-04-26T22:59:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.647+0000 7fe63822a740 1 -- 172.21.15.27:0/230390411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe620005040 con 0x563ebfa61110 2024-04-26T22:59:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.648+0000 7fe60fe00640 1 -- 172.21.15.27:0/230390411 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe61c0a5050 con 0x563ebfa61110 2024-04-26T22:59:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.648+0000 7fe63822a740 10 client.25600 fetch_fsmap learned FSMap version 25 2024-04-26T22:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.648+0000 7fe63822a740 1 -- 172.21.15.27:0/230390411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563ebfae1d30 con 0x563ebfa61110 2024-04-26T22:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.648+0000 7fe60fe00640 1 -- 172.21.15.27:0/230390411 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe620005040 con 0x563ebfa61110 2024-04-26T22:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.648+0000 7fe60fe00640 1 -- 172.21.15.27:0/230390411 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe61c0d61d0 con 0x563ebfa61110 2024-04-26T22:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.648+0000 7fe63822a740 10 client.25600 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.649+0000 7fe63822a740 10 client.25600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.649+0000 7fe63822a740 20 client.25600 populate_metadata read hostname 'smithi027' 2024-04-26T22:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.649+0000 7fe63822a740 1 --2- 172.21.15.27:0/230390411 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x563ebfae2410 0x563ebfae4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.649+0000 7fe63822a740 4 client.25600 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.649+0000 7fe63822a740 1 -- 172.21.15.27:0/230390411 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ebf9c4e00 con 0x563ebfae2410 2024-04-26T22:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.649+0000 7fe62e800640 1 --2- 172.21.15.27:0/230390411 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x563ebfae2410 0x563ebfae4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.650+0000 7fe62e800640 1 --2- 172.21.15.27:0/230390411 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x563ebfae2410 0x563ebfae4830 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9bb8b8f28a49848 server_cookie=6c8f4e0eb65fb34a in_seq=0 out_seq=0 2024-04-26T22:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.650+0000 7fe60fe00640 10 client.25600 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.654+0000 7fe60fe00640 1 -- 172.21.15.27:0/230390411 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ebf9c4e00 con 0x563ebfae2410 2024-04-26T22:59:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.654+0000 7fe60fe00640 10 client.25600 handle_command_reply: tid=0 2024-04-26T22:59:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.654+0000 7fe63822a740 1 client.25600 shutdown 2024-04-26T22:59:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.654+0000 7fe63822a740 1 -- 172.21.15.27:0/230390411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa61110 msgr2=0x563ebfa65aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.654+0000 7fe63822a740 1 --2- 172.21.15.27:0/230390411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa61110 0x563ebfa65aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0b9310 tx=0x7fe61c03b460 comp rx=0 tx=0).stop 2024-04-26T22:59:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.654+0000 7fe63822a740 1 -- 172.21.15.27:0/230390411 shutdown_connections 2024-04-26T22:59:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.654+0000 7fe63822a740 1 --2- 172.21.15.27:0/230390411 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x563ebfae2410 0x563ebfae4830 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.654+0000 7fe63822a740 1 --2- 172.21.15.27:0/230390411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ebfa6b9d0 0x563ebfa65fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.654+0000 7fe63822a740 1 --2- 172.21.15.27:0/230390411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ebfa61110 0x563ebfa65aa0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.654+0000 7fe60fe00640 0 client.25600 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.654+0000 7fe63822a740 1 --2- 172.21.15.27:0/230390411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ebfa607f0 0x563ebfa65560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.654+0000 7fe63822a740 1 -- 172.21.15.27:0/230390411 >> 172.21.15.27:0/230390411 conn(0x563ebfa5cd00 msgr2=0x563ebfa69d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.655+0000 7fe63822a740 1 -- 172.21.15.27:0/230390411 shutdown_connections 2024-04-26T22:59:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.655+0000 7fe63822a740 1 -- 172.21.15.27:0/230390411 wait complete. 2024-04-26T22:59:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.656+0000 7fe63822a740 20 client.25600 trim_cache size 0 max 16384 2024-04-26T22:59:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T22:59:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T22:59:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T22:59:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T22:59:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.657+0000 7fe634a00640 1 -- 172.21.15.27:0/3051126246 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe6140a3db0 msgr2=0x7fe6140a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.657+0000 7fe634a00640 1 --2- 172.21.15.27:0/3051126246 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe6140a3db0 0x7fe6140a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x563ebfa61d90 tx=0x7fe62000d500 comp rx=0 tx=0).stop 2024-04-26T22:59:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.657+0000 7fe634a00640 1 -- 172.21.15.27:0/3051126246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe630104230 msgr2=0x7fe630199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.657+0000 7fe634a00640 1 --2- 172.21.15.27:0/3051126246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe630104230 0x7fe630199490 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe61c005ab0 tx=0x7fe61c003c80 comp rx=0 tx=0).stop 2024-04-26T22:59:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.657+0000 7fe634a00640 1 -- 172.21.15.27:0/3051126246 shutdown_connections 2024-04-26T22:59:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.657+0000 7fe634a00640 1 --2- 172.21.15.27:0/3051126246 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe6140a3db0 0x7fe6140a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.657+0000 7fe634a00640 1 --2- 172.21.15.27:0/3051126246 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe63010cd30 0x7fe63019d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.657+0000 7fe634a00640 1 --2- 172.21.15.27:0/3051126246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe630104230 0x7fe630199490 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.657+0000 7fe634a00640 1 --2- 172.21.15.27:0/3051126246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe630103910 0x7fe630198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.657+0000 7fe634a00640 1 -- 172.21.15.27:0/3051126246 >> 172.21.15.27:0/3051126246 conn(0x7fe6300fd110 msgr2=0x7fe6300fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.658+0000 7fe634a00640 1 -- 172.21.15.27:0/3051126246 shutdown_connections 2024-04-26T22:59:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:44.658+0000 7fe634a00640 1 -- 172.21.15.27:0/3051126246 wait complete. 2024-04-26T22:59:45.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:45 smithi094 ceph-mon[32889]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T22:59:45.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:45 smithi027 ceph-mon[28034]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T22:59:45.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:45 smithi080 ceph-mon[32324]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T22:59:46.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:46 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:59:46.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:46 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:59:46.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:46 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:59:47.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:47 smithi027 ceph-mon[28034]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-26T22:59:47.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:47 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:59:47.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:47 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:59:47.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:47 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:59:47.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:47 smithi080 ceph-mon[32324]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-26T22:59:47.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:47 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:59:47.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:47 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:59:47.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:47 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:59:47.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:47 smithi094 ceph-mon[32889]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-26T22:59:47.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:47 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:59:47.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:47 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:59:47.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:47 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T22:59:49.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:49 smithi027 ceph-mon[28034]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T22:59:49.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:49 smithi080 ceph-mon[32324]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T22:59:49.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:49 smithi094 ceph-mon[32889]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T22:59:51.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:51 smithi027 ceph-mon[28034]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:59:51.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:51 smithi080 ceph-mon[32324]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:59:51.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:51 smithi094 ceph-mon[32889]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:59:52.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:52 smithi027 ceph-mon[28034]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T22:59:52.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:52 smithi080 ceph-mon[32324]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T22:59:52.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:52 smithi094 ceph-mon[32889]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T22:59:55.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:54 smithi080 ceph-mon[32324]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:59:55.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:54 smithi094 ceph-mon[32889]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:59:55.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:54 smithi027 ceph-mon[28034]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:59:57.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:56 smithi080 ceph-mon[32324]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:59:57.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:56 smithi094 ceph-mon[32889]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:59:57.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:56 smithi027 ceph-mon[28034]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T22:59:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.850+0000 7f6ec7200640 1 -- 172.21.15.27:0/456762239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ec0074d60 msgr2=0x7f6ec0075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.850+0000 7f6ec7200640 1 --2- 172.21.15.27:0/456762239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ec0074d60 0x7f6ec0075140 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009900 tx=0x7f6eb402a490 comp rx=0 tx=0).stop 2024-04-26T22:59:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.850+0000 7f6ec7200640 1 -- 172.21.15.27:0/456762239 shutdown_connections 2024-04-26T22:59:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.850+0000 7f6ec7200640 1 --2- 172.21.15.27:0/456762239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ec010ef90 0x7f6ec01113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.850+0000 7f6ec7200640 1 --2- 172.21.15.27:0/456762239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ec0075680 0x7f6ec010ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.850+0000 7f6ec7200640 1 --2- 172.21.15.27:0/456762239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ec0074d60 0x7f6ec0075140 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.850+0000 7f6ec7200640 1 -- 172.21.15.27:0/456762239 >> 172.21.15.27:0/456762239 conn(0x7f6ec00fe240 msgr2=0x7f6ec0100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.850+0000 7f6ec7200640 1 -- 172.21.15.27:0/456762239 shutdown_connections 2024-04-26T22:59:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.850+0000 7f6ec7200640 1 -- 172.21.15.27:0/456762239 wait complete. 2024-04-26T22:59:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.852+0000 7f6ec7200640 1 Processor -- start 2024-04-26T22:59:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.852+0000 7f6ec7200640 1 -- start start 2024-04-26T22:59:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.853+0000 7f6ec7200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ec0074d60 0x7f6ec019b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.853+0000 7f6ec7200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ec0075680 0x7f6ec019b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.853+0000 7f6ec7200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ec010ef90 0x7f6ec019fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.853+0000 7f6ec7200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec0114c00 con 0x7f6ec0074d60 2024-04-26T22:59:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.853+0000 7f6ec7200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec0114a80 con 0x7f6ec0075680 2024-04-26T22:59:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.853+0000 7f6ec7200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec0114d80 con 0x7f6ec010ef90 2024-04-26T22:59:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.853+0000 7f6ec5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ec0075680 0x7f6ec019b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.853+0000 7f6ec6800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ec010ef90 0x7f6ec019fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.853+0000 7f6ec5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ec0075680 0x7f6ec019b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:51840/0 (socket says 172.21.15.27:51840) 2024-04-26T22:59:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.853+0000 7f6ec5400640 1 -- 172.21.15.27:0/451114482 learned_addr learned my addr 172.21.15.27:0/451114482 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.853+0000 7f6ec5e00640 1 --2- 172.21.15.27:0/451114482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ec0074d60 0x7f6ec019b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.854+0000 7f6ec5400640 1 -- 172.21.15.27:0/451114482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ec010ef90 msgr2=0x7f6ec019fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.854+0000 7f6ec5400640 1 --2- 172.21.15.27:0/451114482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ec010ef90 0x7f6ec019fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.854+0000 7f6ec5400640 1 -- 172.21.15.27:0/451114482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ec0074d60 msgr2=0x7f6ec019b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.854+0000 7f6ec5400640 1 --2- 172.21.15.27:0/451114482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ec0074d60 0x7f6ec019b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.854+0000 7f6ec5400640 1 -- 172.21.15.27:0/451114482 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec01a0140 con 0x7f6ec0075680 2024-04-26T22:59:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.854+0000 7f6ec5e00640 1 --2- 172.21.15.27:0/451114482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ec0074d60 0x7f6ec019b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:59:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.854+0000 7f6ec5400640 1 --2- 172.21.15.27:0/451114482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ec0075680 0x7f6ec019b6e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000e9f0 tx=0x7f6eb000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.855+0000 7f6ebaa00640 1 -- 172.21.15.27:0/451114482 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb000cd80 con 0x7f6ec0075680 2024-04-26T22:59:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.855+0000 7f6ebaa00640 1 -- 172.21.15.27:0/451114482 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb00108e0 con 0x7f6ec0075680 2024-04-26T22:59:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.855+0000 7f6ebaa00640 1 -- 172.21.15.27:0/451114482 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0018980 con 0x7f6ec0075680 2024-04-26T22:59:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.855+0000 7f6ec7200640 1 -- 172.21.15.27:0/451114482 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec01a0430 con 0x7f6ec0075680 2024-04-26T22:59:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.855+0000 7f6ec7200640 1 -- 172.21.15.27:0/451114482 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec01a7cc0 con 0x7f6ec0075680 2024-04-26T22:59:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.859+0000 7f6ecb67b740 1 Processor -- start 2024-04-26T22:59:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.860+0000 7f6ecb67b740 1 -- start start 2024-04-26T22:59:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.860+0000 7f6ebaa00640 1 -- 172.21.15.27:0/451114482 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f6eb0010430 con 0x7f6ec0075680 2024-04-26T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.860+0000 7f6ecb67b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abdfe0 0x555681abe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.860+0000 7f6ecb67b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abe900 0x555681ac8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.861+0000 7f6ecb67b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681ac91c0 0x555681acb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.861+0000 7f6ecb67b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555681a23380 con 0x555681ac91c0 2024-04-26T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.861+0000 7f6ecb67b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5556819bfc30 con 0x555681abe900 2024-04-26T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.861+0000 7f6ecb67b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555681a1f340 con 0x555681abdfe0 2024-04-26T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.861+0000 7f6ec6800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abdfe0 0x555681abe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.861+0000 7f6ec6800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abdfe0 0x555681abe3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:40268/0 (socket says 172.21.15.27:40268) 2024-04-26T22:59:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.861+0000 7f6ec6800640 1 -- 172.21.15.27:0/3510762480 learned_addr learned my addr 172.21.15.27:0/3510762480 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.861+0000 7f6ec5400640 1 --2- 172.21.15.27:0/3510762480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681ac91c0 0x555681acb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.862+0000 7f6ec6800640 1 -- 172.21.15.27:0/3510762480 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abe900 msgr2=0x555681ac8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.862+0000 7f6ec6800640 1 --2- 172.21.15.27:0/3510762480 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abe900 0x555681ac8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.862+0000 7f6ec6800640 1 -- 172.21.15.27:0/3510762480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681ac91c0 msgr2=0x555681acb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.862+0000 7f6ec6800640 1 --2- 172.21.15.27:0/3510762480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681ac91c0 0x555681acb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.862+0000 7f6ec6800640 1 -- 172.21.15.27:0/3510762480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556819e1940 con 0x555681abdfe0 2024-04-26T22:59:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.862+0000 7f6ec6800640 1 --2- 172.21.15.27:0/3510762480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abdfe0 0x555681abe3c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009f50 tx=0x7f6ebc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db01c4fc365397a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.863+0000 7f6eaf000640 1 -- 172.21.15.27:0/3510762480 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc004d20 con 0x555681abdfe0 2024-04-26T22:59:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.863+0000 7f6eaf000640 1 -- 172.21.15.27:0/3510762480 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ebc004050 con 0x555681abdfe0 2024-04-26T22:59:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.863+0000 7f6eaf000640 1 -- 172.21.15.27:0/3510762480 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc005220 con 0x555681abdfe0 2024-04-26T22:59:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.863+0000 7f6ebaa00640 1 --2- 172.21.15.27:0/451114482 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6ea80a3ea0 0x7f6ea80a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.864+0000 7f6ec5e00640 1 --2- 172.21.15.27:0/451114482 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6ea80a3ea0 0x7f6ea80a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.864+0000 7f6ecb67b740 1 -- 172.21.15.27:0/3510762480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abdfe0 msgr2=0x555681abe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.864+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/3510762480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abdfe0 0x555681abe3c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009f50 tx=0x7f6ebc0118d0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.864+0000 7f6ecb67b740 1 -- 172.21.15.27:0/3510762480 shutdown_connections 2024-04-26T22:59:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.864+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/3510762480 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abe900 0x555681ac8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.864+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/3510762480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681ac91c0 0x555681acb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.864+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/3510762480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abdfe0 0x555681abe3c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.864+0000 7f6ecb67b740 1 -- 172.21.15.27:0/3510762480 >> 172.21.15.27:0/3510762480 conn(0x555681aba4f0 msgr2=0x555681aba8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.864+0000 7f6ebaa00640 1 -- 172.21.15.27:0/451114482 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f6eb00d1210 con 0x7f6ec0075680 2024-04-26T22:59:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.865+0000 7f6ecb67b740 1 -- 172.21.15.27:0/3510762480 shutdown_connections 2024-04-26T22:59:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.865+0000 7f6ec5e00640 1 --2- 172.21.15.27:0/451114482 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6ea80a3ea0 0x7f6ea80a6360 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4000c00 tx=0x7f6eb4005a20 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.865+0000 7f6ecb67b740 1 -- 172.21.15.27:0/3510762480 wait complete. 2024-04-26T22:59:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.865+0000 7f6ecb67b740 1 -- start start 2024-04-26T22:59:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.866+0000 7f6ecb67b740 1 Processor -- start 2024-04-26T22:59:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.866+0000 7f6ecb67b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681abdfe0 0x555681ac6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.866+0000 7f6ecb67b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abe900 0x555681ac65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.866+0000 7f6ecb67b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681ac91c0 0x555681ac6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.866+0000 7f6ecb67b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555681acbdf0 con 0x555681abdfe0 2024-04-26T22:59:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.866+0000 7f6ecb67b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5556819aac60 con 0x555681abe900 2024-04-26T22:59:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.866+0000 7f6ecb67b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555681acbf70 con 0x555681ac91c0 2024-04-26T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.867+0000 7f6ecb67b740 10 client.0 _ll_register_callbacks cb 0x5556819d6e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.867+0000 7f6ec6800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681ac91c0 0x555681ac6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.867+0000 7f6ec6800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681ac91c0 0x555681ac6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:40270/0 (socket says 172.21.15.27:40270) 2024-04-26T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.867+0000 7f6ec6800640 1 -- 172.21.15.27:0/1157440769 learned_addr learned my addr 172.21.15.27:0/1157440769 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.867+0000 7f6ec5400640 1 --2- 172.21.15.27:0/1157440769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abe900 0x555681ac65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.867+0000 7f6ec5e00640 1 --2- 172.21.15.27:0/1157440769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681abdfe0 0x555681ac6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.867+0000 7f6ec5400640 1 -- 172.21.15.27:0/1157440769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681ac91c0 msgr2=0x555681ac6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.868+0000 7f6ec5400640 1 --2- 172.21.15.27:0/1157440769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681ac91c0 0x555681ac6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.868+0000 7f6ec5400640 1 -- 172.21.15.27:0/1157440769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681abdfe0 msgr2=0x555681ac6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.868+0000 7f6ec5400640 1 --2- 172.21.15.27:0/1157440769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681abdfe0 0x555681ac6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.868+0000 7f6ec5400640 1 -- 172.21.15.27:0/1157440769 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556819b9510 con 0x555681abe900 2024-04-26T22:59:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.868+0000 7f6ec6800640 1 --2- 172.21.15.27:0/1157440769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681ac91c0 0x555681ac6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:59:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.868+0000 7f6ec5e00640 1 --2- 172.21.15.27:0/1157440769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681abdfe0 0x555681ac6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.868+0000 7f6ec5400640 1 --2- 172.21.15.27:0/1157440769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abe900 0x555681ac65d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00ce3a0 tx=0x7f6eb009c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=944db615a2a60661 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.868+0000 7f6ea3400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.868+0000 7f6ea3400640 1 -- 172.21.15.27:0/1157440769 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb009d220 con 0x555681abe900 2024-04-26T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.869+0000 7f6ea3400640 1 -- 172.21.15.27:0/1157440769 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f6eb00ce680 con 0x555681abe900 2024-04-26T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.869+0000 7f6ecb67b740 1 -- 172.21.15.27:0/1157440769 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ebc004050 con 0x555681abe900 2024-04-26T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.869+0000 7f6ea3400640 1 -- 172.21.15.27:0/1157440769 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6eb009cb20 con 0x555681abe900 2024-04-26T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.869+0000 7f6ecb67b740 10 client.25583 fetch_fsmap learned FSMap version 25 2024-04-26T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.869+0000 7f6ecb67b740 1 -- 172.21.15.27:0/1157440769 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555681735210 con 0x555681abe900 2024-04-26T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.870+0000 7f6ea3400640 1 -- 172.21.15.27:0/1157440769 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ebc004050 con 0x555681abe900 2024-04-26T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.870+0000 7f6ea3400640 1 -- 172.21.15.27:0/1157440769 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6eb00bbce0 con 0x555681abe900 2024-04-26T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.870+0000 7f6ecb67b740 10 client.25583 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.870+0000 7f6ecb67b740 10 client.25583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.870+0000 7f6ecb67b740 20 client.25583 populate_metadata read hostname 'smithi027' 2024-04-26T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.870+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/1157440769 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x555681ad8ef0 0x555681adb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.870+0000 7f6ecb67b740 4 client.25583 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.870+0000 7f6ecb67b740 1 -- 172.21.15.27:0/1157440769 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555681a30f90 con 0x555681ad8ef0 2024-04-26T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.871+0000 7f6ec5e00640 1 --2- 172.21.15.27:0/1157440769 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x555681ad8ef0 0x555681adb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.871+0000 7f6ec5e00640 1 --2- 172.21.15.27:0/1157440769 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x555681ad8ef0 0x555681adb2e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c95db75429ea5952 server_cookie=e1ada6fd18ac47f in_seq=0 out_seq=0 2024-04-26T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.871+0000 7f6ea3400640 10 client.25583 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.872+0000 7f6ea3400640 1 -- 172.21.15.27:0/1157440769 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555681a30f90 con 0x555681ad8ef0 2024-04-26T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.872+0000 7f6ea3400640 10 client.25583 handle_command_reply: tid=0 2024-04-26T22:59:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.872+0000 7f6ecb67b740 1 client.25583 shutdown 2024-04-26T22:59:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.873+0000 7f6ecb67b740 1 -- 172.21.15.27:0/1157440769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abe900 msgr2=0x555681ac65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.873+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/1157440769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abe900 0x555681ac65d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00ce3a0 tx=0x7f6eb009c660 comp rx=0 tx=0).stop 2024-04-26T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.873+0000 7f6ecb67b740 1 -- 172.21.15.27:0/1157440769 shutdown_connections 2024-04-26T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.873+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/1157440769 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x555681ad8ef0 0x555681adb2e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.873+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/1157440769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abe900 0x555681ac65d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.873+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/1157440769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681ac91c0 0x555681ac6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.873+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/1157440769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681abdfe0 0x555681ac6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.873+0000 7f6ea3400640 0 client.25583 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.873+0000 7f6ecb67b740 1 -- 172.21.15.27:0/1157440769 >> 172.21.15.27:0/1157440769 conn(0x555681aba4f0 msgr2=0x555681ac0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.873+0000 7f6ecb67b740 1 -- 172.21.15.27:0/1157440769 shutdown_connections 2024-04-26T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.873+0000 7f6ecb67b740 1 -- 172.21.15.27:0/1157440769 wait complete. 2024-04-26T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.873+0000 7f6ecb67b740 20 client.25583 trim_cache size 0 max 16384 2024-04-26T22:59:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.905+0000 7f6ecb67b740 1 Processor -- start 2024-04-26T22:59:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.905+0000 7f6ecb67b740 1 -- start start 2024-04-26T22:59:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.905+0000 7f6ecb67b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abdfe0 0x555681ad4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.905+0000 7f6ecb67b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abe900 0x555681ac7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.905+0000 7f6ecb67b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681ac91c0 0x555681ae0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.905+0000 7f6ecb67b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01b4980 con 0x555681ac91c0 2024-04-26T22:59:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.905+0000 7f6ecb67b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01b2250 con 0x555681abdfe0 2024-04-26T22:59:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.905+0000 7f6ecb67b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01b00a0 con 0x555681abe900 2024-04-26T22:59:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.906+0000 7f6ec5e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abdfe0 0x555681ad4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.906+0000 7f6ec5e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abdfe0 0x555681ad4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:51866/0 (socket says 172.21.15.27:51866) 2024-04-26T22:59:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.906+0000 7f6ec5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abe900 0x555681ac7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.906+0000 7f6ec5e00640 1 -- 172.21.15.27:0/1094369253 learned_addr learned my addr 172.21.15.27:0/1094369253 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.906+0000 7f6ec6800640 1 --2- 172.21.15.27:0/1094369253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681ac91c0 0x555681ae0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.906+0000 7f6ec5e00640 1 -- 172.21.15.27:0/1094369253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abe900 msgr2=0x555681ac7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.906+0000 7f6ec5e00640 1 --2- 172.21.15.27:0/1094369253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abe900 0x555681ac7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.906+0000 7f6ec5e00640 1 -- 172.21.15.27:0/1094369253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681ac91c0 msgr2=0x555681ae0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.906+0000 7f6ec5e00640 1 --2- 172.21.15.27:0/1094369253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681ac91c0 0x555681ae0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.906+0000 7f6ec5e00640 1 -- 172.21.15.27:0/1094369253 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555681801620 con 0x555681abdfe0 2024-04-26T22:59:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.907+0000 7f6ec5e00640 1 --2- 172.21.15.27:0/1094369253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abdfe0 0x555681ad4cb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40092a0 tx=0x7f6eb403f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47be53dea1f06a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.907+0000 7f6eae600640 1 -- 172.21.15.27:0/1094369253 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4056070 con 0x555681abdfe0 2024-04-26T22:59:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.907+0000 7f6eae600640 1 -- 172.21.15.27:0/1094369253 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb40055f0 con 0x555681abdfe0 2024-04-26T22:59:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.907+0000 7f6eae600640 1 -- 172.21.15.27:0/1094369253 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4035050 con 0x555681abdfe0 2024-04-26T22:59:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.908+0000 7f6ecb67b740 1 -- 172.21.15.27:0/1094369253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abdfe0 msgr2=0x555681ad4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.908+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/1094369253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abdfe0 0x555681ad4cb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40092a0 tx=0x7f6eb403f000 comp rx=0 tx=0).stop 2024-04-26T22:59:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.908+0000 7f6ecb67b740 1 -- 172.21.15.27:0/1094369253 shutdown_connections 2024-04-26T22:59:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.908+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/1094369253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681ac91c0 0x555681ae0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.908+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/1094369253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abe900 0x555681ac7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.908+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/1094369253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681abdfe0 0x555681ad4cb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.908+0000 7f6ecb67b740 1 -- 172.21.15.27:0/1094369253 >> 172.21.15.27:0/1094369253 conn(0x555681aba4f0 msgr2=0x555681ac77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.908+0000 7f6ecb67b740 1 -- 172.21.15.27:0/1094369253 shutdown_connections 2024-04-26T22:59:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.908+0000 7f6ecb67b740 1 -- 172.21.15.27:0/1094369253 wait complete. 2024-04-26T22:59:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.909+0000 7f6ecb67b740 1 -- start start 2024-04-26T22:59:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.909+0000 7f6ecb67b740 1 Processor -- start 2024-04-26T22:59:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.910+0000 7f6ecb67b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abe900 0x555681acce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.910+0000 7f6ecb67b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681ac91c0 0x555681acd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.910+0000 7f6ecb67b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681acd8c0 0x555681b3e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.910+0000 7f6ecb67b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01b0710 con 0x555681ac91c0 2024-04-26T22:59:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.910+0000 7f6ecb67b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01b1b80 con 0x555681acd8c0 2024-04-26T22:59:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.910+0000 7f6ecb67b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01b0de0 con 0x555681abe900 2024-04-26T22:59:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.910+0000 7f6ecb67b740 10 client.0 _ll_register_callbacks cb 0x7f6eb00d4870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:59:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.910+0000 7f6ec5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abe900 0x555681acce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.910+0000 7f6ec5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abe900 0x555681acce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:40284/0 (socket says 172.21.15.27:40284) 2024-04-26T22:59:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.910+0000 7f6ec5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681ac91c0 0x555681acd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.910+0000 7f6ec5e00640 1 -- 172.21.15.27:0/2514462751 learned_addr learned my addr 172.21.15.27:0/2514462751 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T22:59:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.911+0000 7f6ec6800640 1 --2- 172.21.15.27:0/2514462751 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681acd8c0 0x555681b3e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.911+0000 7f6ec5e00640 1 -- 172.21.15.27:0/2514462751 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681acd8c0 msgr2=0x555681b3e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.911+0000 7f6ec5e00640 1 --2- 172.21.15.27:0/2514462751 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681acd8c0 0x555681b3e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.911+0000 7f6ec5e00640 1 -- 172.21.15.27:0/2514462751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681ac91c0 msgr2=0x555681acd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:59:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.911+0000 7f6ec5e00640 1 --2- 172.21.15.27:0/2514462751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681ac91c0 0x555681acd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.911+0000 7f6ec5e00640 1 -- 172.21.15.27:0/2514462751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556819ddcb0 con 0x555681abe900 2024-04-26T22:59:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.911+0000 7f6ec6800640 1 --2- 172.21.15.27:0/2514462751 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681acd8c0 0x555681b3e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:59:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.911+0000 7f6ec5400640 1 --2- 172.21.15.27:0/2514462751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681ac91c0 0x555681acd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:59:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.911+0000 7f6ec5e00640 1 --2- 172.21.15.27:0/2514462751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abe900 0x555681acce40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6eb402a9b0 tx=0x7f6eb40054e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75172e45b562f296 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:59:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.911+0000 7f6ea3400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T22:59:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.911+0000 7f6ea3400640 1 -- 172.21.15.27:0/2514462751 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4056020 con 0x555681abe900 2024-04-26T22:59:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.912+0000 7f6ecb67b740 1 -- 172.21.15.27:0/2514462751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb40055f0 con 0x555681abe900 2024-04-26T22:59:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.912+0000 7f6ea3400640 1 -- 172.21.15.27:0/2514462751 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f6eb40503c0 con 0x555681abe900 2024-04-26T22:59:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.912+0000 7f6ea3400640 1 -- 172.21.15.27:0/2514462751 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6eb4035040 con 0x555681abe900 2024-04-26T22:59:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.912+0000 7f6ecb67b740 10 client.25618 fetch_fsmap learned FSMap version 25 2024-04-26T22:59:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.912+0000 7f6ecb67b740 1 -- 172.21.15.27:0/2514462751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555681b3fbc0 con 0x555681abe900 2024-04-26T22:59:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.912+0000 7f6ea3400640 1 -- 172.21.15.27:0/2514462751 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb40055f0 con 0x555681abe900 2024-04-26T22:59:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.913+0000 7f6ea3400640 1 -- 172.21.15.27:0/2514462751 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6eb40035f0 con 0x555681abe900 2024-04-26T22:59:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.913+0000 7f6ecb67b740 10 client.25618 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:59:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.913+0000 7f6ecb67b740 10 client.25618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:59:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.913+0000 7f6ecb67b740 20 client.25618 populate_metadata read hostname 'smithi027' 2024-04-26T22:59:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.913+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/2514462751 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x555681b401e0 0x555681b42600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:59:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.913+0000 7f6ecb67b740 4 client.25618 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:59:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.913+0000 7f6ecb67b740 1 -- 172.21.15.27:0/2514462751 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555681a2ae00 con 0x555681b401e0 2024-04-26T22:59:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.913+0000 7f6ec5400640 1 --2- 172.21.15.27:0/2514462751 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x555681b401e0 0x555681b42600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:59:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.914+0000 7f6ec5400640 1 --2- 172.21.15.27:0/2514462751 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x555681b401e0 0x555681b42600 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=793e382ae2bc020a server_cookie=209aa0a00ce4f2fe in_seq=0 out_seq=0 2024-04-26T22:59:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.914+0000 7f6ea3400640 10 client.25618 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.920+0000 7f6ea3400640 1 -- 172.21.15.27:0/2514462751 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555681a2ae00 con 0x555681b401e0 2024-04-26T22:59:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.920+0000 7f6ea3400640 10 client.25618 handle_command_reply: tid=0 2024-04-26T22:59:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.920+0000 7f6ecb67b740 1 client.25618 shutdown 2024-04-26T22:59:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.920+0000 7f6ecb67b740 1 -- 172.21.15.27:0/2514462751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abe900 msgr2=0x555681acce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.920+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/2514462751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abe900 0x555681acce40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6eb402a9b0 tx=0x7f6eb40054e0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.920+0000 7f6ecb67b740 1 -- 172.21.15.27:0/2514462751 shutdown_connections 2024-04-26T22:59:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.920+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/2514462751 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x555681b401e0 0x555681b42600 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.920+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/2514462751 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555681acd8c0 0x555681b3e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.920+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/2514462751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555681ac91c0 0x555681acd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.920+0000 7f6ecb67b740 1 --2- 172.21.15.27:0/2514462751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555681abe900 0x555681acce40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.920+0000 7f6ecb67b740 1 -- 172.21.15.27:0/2514462751 >> 172.21.15.27:0/2514462751 conn(0x555681aba4f0 msgr2=0x555681ac68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.920+0000 7f6ea3400640 0 client.25618 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T22:59:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.920+0000 7f6ecb67b740 1 -- 172.21.15.27:0/2514462751 shutdown_connections 2024-04-26T22:59:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.920+0000 7f6ecb67b740 1 -- 172.21.15.27:0/2514462751 wait complete. 2024-04-26T22:59:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.921+0000 7f6ecb67b740 20 client.25618 trim_cache size 0 max 16384 2024-04-26T22:59:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T22:59:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T22:59:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T22:59:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T22:59:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.921+0000 7f6ec7200640 1 -- 172.21.15.27:0/451114482 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6ea80a3ea0 msgr2=0x7f6ea80a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.921+0000 7f6ec7200640 1 --2- 172.21.15.27:0/451114482 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6ea80a3ea0 0x7f6ea80a6360 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4000c00 tx=0x7f6eb4005a20 comp rx=0 tx=0).stop 2024-04-26T22:59:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.921+0000 7f6ec7200640 1 -- 172.21.15.27:0/451114482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ec0075680 msgr2=0x7f6ec019b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:59:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.921+0000 7f6ec7200640 1 --2- 172.21.15.27:0/451114482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ec0075680 0x7f6ec019b6e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000e9f0 tx=0x7f6eb000eeb0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.922+0000 7f6ec7200640 1 -- 172.21.15.27:0/451114482 shutdown_connections 2024-04-26T22:59:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.922+0000 7f6ec7200640 1 --2- 172.21.15.27:0/451114482 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6ea80a3ea0 0x7f6ea80a6360 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.922+0000 7f6ec7200640 1 --2- 172.21.15.27:0/451114482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ec0075680 0x7f6ec019b6e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.922+0000 7f6ec7200640 1 --2- 172.21.15.27:0/451114482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ec010ef90 0x7f6ec019fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.922+0000 7f6ec7200640 1 --2- 172.21.15.27:0/451114482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ec0074d60 0x7f6ec019b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:59:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.922+0000 7f6ec7200640 1 -- 172.21.15.27:0/451114482 >> 172.21.15.27:0/451114482 conn(0x7f6ec00fe240 msgr2=0x7f6ec00feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:59:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.922+0000 7f6ec7200640 1 -- 172.21.15.27:0/451114482 shutdown_connections 2024-04-26T22:59:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T22:59:57.922+0000 7f6ec7200640 1 -- 172.21.15.27:0/451114482 wait complete. 2024-04-26T22:59:59.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 22:59:58 smithi080 ceph-mon[32324]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T22:59:59.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 22:59:58 smithi094 ceph-mon[32889]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T22:59:59.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 22:59:58 smithi027 ceph-mon[28034]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T23:00:01.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:00 smithi080 ceph-mon[32324]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:00:01.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:00 smithi080 ceph-mon[32324]: overall HEALTH_OK 2024-04-26T23:00:01.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:00 smithi094 ceph-mon[32889]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:00:01.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:00 smithi094 ceph-mon[32889]: overall HEALTH_OK 2024-04-26T23:00:01.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:00 smithi027 ceph-mon[28034]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:00:01.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:00 smithi027 ceph-mon[28034]: overall HEALTH_OK 2024-04-26T23:00:03.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:02 smithi080 ceph-mon[32324]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T23:00:03.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:02 smithi094 ceph-mon[32889]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T23:00:03.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:02 smithi027 ceph-mon[28034]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T23:00:05.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:04 smithi080 ceph-mon[32324]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T23:00:05.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:04 smithi094 ceph-mon[32889]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T23:00:05.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:04 smithi027 ceph-mon[28034]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T23:00:07.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:06 smithi080 ceph-mon[32324]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T23:00:07.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:06 smithi094 ceph-mon[32889]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T23:00:07.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:06 smithi027 ceph-mon[28034]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T23:00:08.574 INFO:tasks.workunit.client.0.smithi027.stderr:Updating files: 58% (7135/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 94% (11612/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-26T23:00:09.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:08 smithi080 ceph-mon[32324]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T23:00:09.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:08 smithi080 ceph-mon[32324]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-26T23:00:09.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:08 smithi094 ceph-mon[32889]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T23:00:09.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:08 smithi094 ceph-mon[32889]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-26T23:00:09.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:08 smithi027 ceph-mon[28034]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T23:00:09.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:08 smithi027 ceph-mon[28034]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-26T23:00:09.440 INFO:tasks.workunit.client.0.smithi027.stderr:Note: switching to 'bc612ffb9c1d5bc27536bc917a705262a36d4387'. 2024-04-26T23:00:09.441 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-26T23:00:09.441 INFO:tasks.workunit.client.0.smithi027.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T23:00:09.441 INFO:tasks.workunit.client.0.smithi027.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T23:00:09.441 INFO:tasks.workunit.client.0.smithi027.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T23:00:09.441 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-26T23:00:09.441 INFO:tasks.workunit.client.0.smithi027.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T23:00:09.441 INFO:tasks.workunit.client.0.smithi027.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T23:00:09.441 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-26T23:00:09.441 INFO:tasks.workunit.client.0.smithi027.stderr: git switch -c 2024-04-26T23:00:09.441 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-26T23:00:09.441 INFO:tasks.workunit.client.0.smithi027.stderr:Or undo this operation with: 2024-04-26T23:00:09.441 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-26T23:00:09.441 INFO:tasks.workunit.client.0.smithi027.stderr: git switch - 2024-04-26T23:00:09.441 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-26T23:00:09.442 INFO:tasks.workunit.client.0.smithi027.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T23:00:09.442 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-26T23:00:09.442 INFO:tasks.workunit.client.0.smithi027.stderr:HEAD is now at bc612ffb9c1 Merge PR #55144 into wip-rishabh-testing-20240426.111959 2024-04-26T23:00:09.455 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T23:00:09.564 INFO:tasks.workunit.client.0.smithi027.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T23:00:09.572 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T23:00:09.572 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T23:00:10.503 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T23:00:10.616 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T23:00:10.709 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T23:00:10.715 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T23:00:10.715 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T23:00:10.804 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T23:00:10.815 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T23:00:10.815 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T23:00:10.870 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-26T23:00:10.871 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-26T23:00:10.872 DEBUG:teuthology.orchestra.run.smithi027:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc612ffb9c1d5bc27536bc917a705262a36d4387 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-26T23:00:10.950 INFO:tasks.workunit.client.0.smithi027.stderr:+ mkdir -p fsstress 2024-04-26T23:00:10.955 INFO:tasks.workunit.client.0.smithi027.stderr:+ pushd fsstress 2024-04-26T23:00:10.955 INFO:tasks.workunit.client.0.smithi027.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-26T23:00:10.955 INFO:tasks.workunit.client.0.smithi027.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-26T23:00:11.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:10 smithi080 ceph-mon[32324]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-26T23:00:11.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:10 smithi094 ceph-mon[32889]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-26T23:00:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.076+0000 7f2b6fe00640 1 -- 172.21.15.27:0/4134275837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b70105300 msgr2=0x7f2b70111810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.076+0000 7f2b6fe00640 1 --2- 172.21.15.27:0/4134275837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b70105300 0x7f2b70111810 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009cd0 tx=0x7f2b6802a8f0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.077+0000 7f2b6fe00640 1 -- 172.21.15.27:0/4134275837 shutdown_connections 2024-04-26T23:00:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.077+0000 7f2b6fe00640 1 --2- 172.21.15.27:0/4134275837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b70105300 0x7f2b70111810 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.077+0000 7f2b6fe00640 1 --2- 172.21.15.27:0/4134275837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b70104940 0x7f2b70104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.077+0000 7f2b6fe00640 1 --2- 172.21.15.27:0/4134275837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b7010a950 0x7f2b7010ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.077+0000 7f2b6fe00640 1 -- 172.21.15.27:0/4134275837 >> 172.21.15.27:0/4134275837 conn(0x7f2b70100410 msgr2=0x7f2b70102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.077+0000 7f2b6fe00640 1 -- 172.21.15.27:0/4134275837 shutdown_connections 2024-04-26T23:00:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.077+0000 7f2b6fe00640 1 -- 172.21.15.27:0/4134275837 wait complete. 2024-04-26T23:00:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.078+0000 7f2b6fe00640 1 Processor -- start 2024-04-26T23:00:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.078+0000 7f2b6fe00640 1 -- start start 2024-04-26T23:00:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.079+0000 7f2b6fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b70104940 0x7f2b7019b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.079+0000 7f2b6fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b70105300 0x7f2b7019b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.079+0000 7f2b6fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b7010a950 0x7f2b7019f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.079+0000 7f2b6fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2b70114c60 con 0x7f2b70104940 2024-04-26T23:00:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.079+0000 7f2b6fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2b70114ae0 con 0x7f2b70105300 2024-04-26T23:00:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.079+0000 7f2b6fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2b70114de0 con 0x7f2b7010a950 2024-04-26T23:00:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.079+0000 7f2b6ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b70104940 0x7f2b7019b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.079+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b7010a950 0x7f2b7019f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.079+0000 7f2b6e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b70105300 0x7f2b7019b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.079+0000 7f2b6ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b70104940 0x7f2b7019b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54878/0 (socket says 172.21.15.27:54878) 2024-04-26T23:00:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.079+0000 7f2b6ea00640 1 -- 172.21.15.27:0/3848852924 learned_addr learned my addr 172.21.15.27:0/3848852924 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.079+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b7010a950 0x7f2b7019f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:40526/0 (socket says 172.21.15.27:40526) 2024-04-26T23:00:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.080+0000 7f2b6ea00640 1 -- 172.21.15.27:0/3848852924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b7010a950 msgr2=0x7f2b7019f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.080+0000 7f2b6ea00640 1 --2- 172.21.15.27:0/3848852924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b7010a950 0x7f2b7019f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.080+0000 7f2b6ea00640 1 -- 172.21.15.27:0/3848852924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b70105300 msgr2=0x7f2b7019b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.080+0000 7f2b6ea00640 1 --2- 172.21.15.27:0/3848852924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b70105300 0x7f2b7019b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.080+0000 7f2b6ea00640 1 -- 172.21.15.27:0/3848852924 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b701a0110 con 0x7f2b70104940 2024-04-26T23:00:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.080+0000 7f2b6e000640 1 --2- 172.21.15.27:0/3848852924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b70105300 0x7f2b7019b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:00:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.080+0000 7f2b6ea00640 1 --2- 172.21.15.27:0/3848852924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b70104940 0x7f2b7019b120 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000ea60 tx=0x7f2b6000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.080+0000 7f2b67400640 1 -- 172.21.15.27:0/3848852924 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6000cde0 con 0x7f2b70104940 2024-04-26T23:00:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.080+0000 7f2b67400640 1 -- 172.21.15.27:0/3848852924 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b600035f0 con 0x7f2b70104940 2024-04-26T23:00:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.080+0000 7f2b67400640 1 -- 172.21.15.27:0/3848852924 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60010690 con 0x7f2b70104940 2024-04-26T23:00:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.080+0000 7f2b6fe00640 1 -- 172.21.15.27:0/3848852924 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b701a0400 con 0x7f2b70104940 2024-04-26T23:00:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.080+0000 7f2b6fe00640 1 -- 172.21.15.27:0/3848852924 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b701a7c90 con 0x7f2b70104940 2024-04-26T23:00:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.084+0000 7f2b67400640 1 -- 172.21.15.27:0/3848852924 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2b600037a0 con 0x7f2b70104940 2024-04-26T23:00:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.084+0000 7f2b78a50740 1 Processor -- start 2024-04-26T23:00:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.084+0000 7f2b78a50740 1 -- start start 2024-04-26T23:00:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.084+0000 7f2b78a50740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6bfe0 0x5583a6a6c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.084+0000 7f2b78a50740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6c900 0x5583a6a76c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.084+0000 7f2b78a50740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a771c0 0x5583a6a795b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.084+0000 7f2b78a50740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5583a69d1380 con 0x5583a6a6bfe0 2024-04-26T23:00:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.084+0000 7f2b78a50740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5583a696dc30 con 0x5583a6a6c900 2024-04-26T23:00:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.084+0000 7f2b78a50740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5583a69cd340 con 0x5583a6a771c0 2024-04-26T23:00:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.085+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6bfe0 0x5583a6a6c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.085+0000 7f2b6ea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6c900 0x5583a6a76c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.085+0000 7f2b6e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a771c0 0x5583a6a795b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.085+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6bfe0 0x5583a6a6c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54890/0 (socket says 172.21.15.27:54890) 2024-04-26T23:00:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.085+0000 7f2b6f400640 1 -- 172.21.15.27:0/2689538004 learned_addr learned my addr 172.21.15.27:0/2689538004 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.085+0000 7f2b6f400640 1 -- 172.21.15.27:0/2689538004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a771c0 msgr2=0x5583a6a795b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.085+0000 7f2b6f400640 1 --2- 172.21.15.27:0/2689538004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a771c0 0x5583a6a795b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.085+0000 7f2b6f400640 1 -- 172.21.15.27:0/2689538004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6c900 msgr2=0x5583a6a76c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.085+0000 7f2b6f400640 1 --2- 172.21.15.27:0/2689538004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6c900 0x5583a6a76c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.085+0000 7f2b6f400640 1 -- 172.21.15.27:0/2689538004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583a698f940 con 0x5583a6a6bfe0 2024-04-26T23:00:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.085+0000 7f2b6ea00640 1 --2- 172.21.15.27:0/2689538004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6c900 0x5583a6a76c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:00:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.086+0000 7f2b67400640 1 --2- 172.21.15.27:0/3848852924 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f2b540a3d80 0x7f2b540a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.086+0000 7f2b6f400640 1 --2- 172.21.15.27:0/2689538004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6bfe0 0x5583a6a6c3c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009cd0 tx=0x7f2b68005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc73ffe1d57b6bd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.086+0000 7f2b67400640 1 -- 172.21.15.27:0/3848852924 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f2b600d0dd0 con 0x7f2b70104940 2024-04-26T23:00:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.086+0000 7f2b5be00640 1 -- 172.21.15.27:0/2689538004 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68030800 con 0x5583a6a6bfe0 2024-04-26T23:00:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.086+0000 7f2b6ea00640 1 --2- 172.21.15.27:0/3848852924 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f2b540a3d80 0x7f2b540a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.086+0000 7f2b5be00640 1 -- 172.21.15.27:0/2689538004 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b68036030 con 0x5583a6a6bfe0 2024-04-26T23:00:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.086+0000 7f2b5be00640 1 -- 172.21.15.27:0/2689538004 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68004330 con 0x5583a6a6bfe0 2024-04-26T23:00:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.087+0000 7f2b78a50740 1 -- 172.21.15.27:0/2689538004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6bfe0 msgr2=0x5583a6a6c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.087+0000 7f2b6ea00640 1 --2- 172.21.15.27:0/3848852924 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f2b540a3d80 0x7f2b540a6240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2b600ce190 tx=0x7f2b600d4920 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.087+0000 7f2b78a50740 1 --2- 172.21.15.27:0/2689538004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6bfe0 0x5583a6a6c3c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009cd0 tx=0x7f2b68005f80 comp rx=0 tx=0).stop 2024-04-26T23:00:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.087+0000 7f2b78a50740 1 -- 172.21.15.27:0/2689538004 shutdown_connections 2024-04-26T23:00:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.087+0000 7f2b78a50740 1 --2- 172.21.15.27:0/2689538004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6c900 0x5583a6a76c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.087+0000 7f2b78a50740 1 --2- 172.21.15.27:0/2689538004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a771c0 0x5583a6a795b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.087+0000 7f2b78a50740 1 --2- 172.21.15.27:0/2689538004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6bfe0 0x5583a6a6c3c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.087+0000 7f2b78a50740 1 -- 172.21.15.27:0/2689538004 >> 172.21.15.27:0/2689538004 conn(0x5583a6a684f0 msgr2=0x5583a6a688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.087+0000 7f2b78a50740 1 -- 172.21.15.27:0/2689538004 shutdown_connections 2024-04-26T23:00:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.087+0000 7f2b78a50740 1 -- 172.21.15.27:0/2689538004 wait complete. 2024-04-26T23:00:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.088+0000 7f2b78a50740 1 -- start start 2024-04-26T23:00:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.088+0000 7f2b78a50740 1 Processor -- start 2024-04-26T23:00:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b78a50740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a6bfe0 0x5583a6a740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b78a50740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6c900 0x5583a6a745e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b78a50740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a771c0 0x5583a6a74b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b78a50740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5583a6a824b0 con 0x5583a6a771c0 2024-04-26T23:00:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b78a50740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5583a6958c60 con 0x5583a6a6c900 2024-04-26T23:00:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b78a50740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5583a6a82630 con 0x5583a6a6bfe0 2024-04-26T23:00:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b78a50740 10 client.0 _ll_register_callbacks cb 0x5583a6984e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:00:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a6bfe0 0x5583a6a740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a6bfe0 0x5583a6a740a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:40536/0 (socket says 172.21.15.27:40536) 2024-04-26T23:00:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b6f400640 1 -- 172.21.15.27:0/1238559204 learned_addr learned my addr 172.21.15.27:0/1238559204 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b6f400640 1 --2- 172.21.15.27:0/1238559204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a771c0 0x5583a6a74b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b6e000640 1 --2- 172.21.15.27:0/1238559204 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6c900 0x5583a6a745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b6f400640 1 -- 172.21.15.27:0/1238559204 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6c900 msgr2=0x5583a6a745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b6f400640 1 --2- 172.21.15.27:0/1238559204 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6c900 0x5583a6a745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b6f400640 1 -- 172.21.15.27:0/1238559204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a771c0 msgr2=0x5583a6a74b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b6f400640 1 --2- 172.21.15.27:0/1238559204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a771c0 0x5583a6a74b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.089+0000 7f2b6f400640 1 -- 172.21.15.27:0/1238559204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583a6967510 con 0x5583a6a6bfe0 2024-04-26T23:00:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.090+0000 7f2b6f400640 1 --2- 172.21.15.27:0/1238559204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a6bfe0 0x5583a6a740a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009a60 tx=0x7f2b68004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10da2d1f84287c8b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.090+0000 7f2b58c00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T23:00:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.090+0000 7f2b58c00640 1 -- 172.21.15.27:0/1238559204 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68059060 con 0x5583a6a6bfe0 2024-04-26T23:00:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.090+0000 7f2b78a50740 1 -- 172.21.15.27:0/1238559204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b68036030 con 0x5583a6a6bfe0 2024-04-26T23:00:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.090+0000 7f2b58c00640 1 -- 172.21.15.27:0/1238559204 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f2b68004760 con 0x5583a6a6bfe0 2024-04-26T23:00:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.090+0000 7f2b58c00640 1 -- 172.21.15.27:0/1238559204 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2b68004b40 con 0x5583a6a6bfe0 2024-04-26T23:00:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.090+0000 7f2b78a50740 10 client.25630 fetch_fsmap learned FSMap version 25 2024-04-26T23:00:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.090+0000 7f2b78a50740 1 -- 172.21.15.27:0/1238559204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5583a66e3210 con 0x5583a6a6bfe0 2024-04-26T23:00:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.091+0000 7f2b58c00640 1 -- 172.21.15.27:0/1238559204 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b68036030 con 0x5583a6a6bfe0 2024-04-26T23:00:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.091+0000 7f2b58c00640 1 -- 172.21.15.27:0/1238559204 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2b68043070 con 0x5583a6a6bfe0 2024-04-26T23:00:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.091+0000 7f2b78a50740 10 client.25630 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T23:00:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.091+0000 7f2b78a50740 10 client.25630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:00:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.091+0000 7f2b78a50740 20 client.25630 populate_metadata read hostname 'smithi027' 2024-04-26T23:00:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.091+0000 7f2b78a50740 1 --2- 172.21.15.27:0/1238559204 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5583a6a8f470 0x5583a6a91860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.091+0000 7f2b78a50740 4 client.25630 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:00:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.091+0000 7f2b78a50740 1 -- 172.21.15.27:0/1238559204 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583a69def90 con 0x5583a6a8f470 2024-04-26T23:00:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.091+0000 7f2b6e000640 1 --2- 172.21.15.27:0/1238559204 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5583a6a8f470 0x5583a6a91860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.092+0000 7f2b6e000640 1 --2- 172.21.15.27:0/1238559204 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5583a6a8f470 0x5583a6a91860 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1ec864d371ba113 server_cookie=52b816081c7dd1c2 in_seq=0 out_seq=0 2024-04-26T23:00:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.092+0000 7f2b58c00640 10 client.25630 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:00:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.093+0000 7f2b58c00640 1 -- 172.21.15.27:0/1238559204 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583a69def90 con 0x5583a6a8f470 2024-04-26T23:00:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.093+0000 7f2b58c00640 10 client.25630 handle_command_reply: tid=0 2024-04-26T23:00:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.093+0000 7f2b78a50740 1 client.25630 shutdown 2024-04-26T23:00:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.093+0000 7f2b78a50740 1 -- 172.21.15.27:0/1238559204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a6bfe0 msgr2=0x5583a6a740a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.093+0000 7f2b78a50740 1 --2- 172.21.15.27:0/1238559204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a6bfe0 0x5583a6a740a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009a60 tx=0x7f2b68004040 comp rx=0 tx=0).stop 2024-04-26T23:00:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.093+0000 7f2b78a50740 1 -- 172.21.15.27:0/1238559204 shutdown_connections 2024-04-26T23:00:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.093+0000 7f2b78a50740 1 --2- 172.21.15.27:0/1238559204 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5583a6a8f470 0x5583a6a91860 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.093+0000 7f2b78a50740 1 --2- 172.21.15.27:0/1238559204 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6c900 0x5583a6a745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.093+0000 7f2b78a50740 1 --2- 172.21.15.27:0/1238559204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a771c0 0x5583a6a74b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.093+0000 7f2b78a50740 1 --2- 172.21.15.27:0/1238559204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a6bfe0 0x5583a6a740a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.093+0000 7f2b78a50740 1 -- 172.21.15.27:0/1238559204 >> 172.21.15.27:0/1238559204 conn(0x5583a6a684f0 msgr2=0x5583a6a6e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.093+0000 7f2b58c00640 0 client.25630 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:00:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.093+0000 7f2b78a50740 1 -- 172.21.15.27:0/1238559204 shutdown_connections 2024-04-26T23:00:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.093+0000 7f2b78a50740 1 -- 172.21.15.27:0/1238559204 wait complete. 2024-04-26T23:00:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.094+0000 7f2b78a50740 20 client.25630 trim_cache size 0 max 16384 2024-04-26T23:00:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.112+0000 7f2b78a50740 1 Processor -- start 2024-04-26T23:00:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.112+0000 7f2b78a50740 1 -- start start 2024-04-26T23:00:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.113+0000 7f2b78a50740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6bfe0 0x5583a6a6e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.113+0000 7f2b78a50740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6c900 0x5583a6a73200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.113+0000 7f2b78a50740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a771c0 0x5583a6a75ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.113+0000 7f2b78a50740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2b701b0460 con 0x5583a6a6c900 2024-04-26T23:00:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.113+0000 7f2b78a50740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2b701add30 con 0x5583a6a6bfe0 2024-04-26T23:00:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.113+0000 7f2b78a50740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2b701abb80 con 0x5583a6a771c0 2024-04-26T23:00:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.113+0000 7f2b6e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6c900 0x5583a6a73200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.113+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6bfe0 0x5583a6a6e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.113+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a771c0 0x5583a6a75ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.113+0000 7f2b6e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6c900 0x5583a6a73200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54908/0 (socket says 172.21.15.27:54908) 2024-04-26T23:00:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.113+0000 7f2b6e000640 1 -- 172.21.15.27:0/1803866227 learned_addr learned my addr 172.21.15.27:0/1803866227 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.114+0000 7f2b6f400640 1 -- 172.21.15.27:0/1803866227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6bfe0 msgr2=0x5583a6a6e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.114+0000 7f2b6f400640 1 --2- 172.21.15.27:0/1803866227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6bfe0 0x5583a6a6e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.114+0000 7f2b6f400640 1 -- 172.21.15.27:0/1803866227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6c900 msgr2=0x5583a6a73200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.114+0000 7f2b6f400640 1 --2- 172.21.15.27:0/1803866227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6c900 0x5583a6a73200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.114+0000 7f2b6f400640 1 -- 172.21.15.27:0/1803866227 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583a67af620 con 0x5583a6a771c0 2024-04-26T23:00:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.114+0000 7f2b6e000640 1 --2- 172.21.15.27:0/1803866227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6c900 0x5583a6a73200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:00:11.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:10 smithi027 ceph-mon[28034]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-26T23:00:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.114+0000 7f2b6f400640 1 --2- 172.21.15.27:0/1803866227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a771c0 0x5583a6a75ed0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2b68005e30 tx=0x7f2b68000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58354a194453d8f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.114+0000 7f2b5b400640 1 -- 172.21.15.27:0/1803866227 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68059040 con 0x5583a6a771c0 2024-04-26T23:00:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.114+0000 7f2b5b400640 1 -- 172.21.15.27:0/1803866227 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b68036030 con 0x5583a6a771c0 2024-04-26T23:00:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.114+0000 7f2b5b400640 1 -- 172.21.15.27:0/1803866227 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b680058a0 con 0x5583a6a771c0 2024-04-26T23:00:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.115+0000 7f2b78a50740 1 -- 172.21.15.27:0/1803866227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a771c0 msgr2=0x5583a6a75ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.115+0000 7f2b78a50740 1 --2- 172.21.15.27:0/1803866227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a771c0 0x5583a6a75ed0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2b68005e30 tx=0x7f2b68000f40 comp rx=0 tx=0).stop 2024-04-26T23:00:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.115+0000 7f2b78a50740 1 -- 172.21.15.27:0/1803866227 shutdown_connections 2024-04-26T23:00:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.115+0000 7f2b78a50740 1 --2- 172.21.15.27:0/1803866227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a771c0 0x5583a6a75ed0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.115+0000 7f2b78a50740 1 --2- 172.21.15.27:0/1803866227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6c900 0x5583a6a73200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.115+0000 7f2b78a50740 1 --2- 172.21.15.27:0/1803866227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6bfe0 0x5583a6a6e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.115+0000 7f2b78a50740 1 -- 172.21.15.27:0/1803866227 >> 172.21.15.27:0/1803866227 conn(0x5583a6a684f0 msgr2=0x5583a6a757c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.115+0000 7f2b78a50740 1 -- 172.21.15.27:0/1803866227 shutdown_connections 2024-04-26T23:00:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.115+0000 7f2b78a50740 1 -- 172.21.15.27:0/1803866227 wait complete. 2024-04-26T23:00:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.116+0000 7f2b78a50740 1 -- start start 2024-04-26T23:00:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.116+0000 7f2b78a50740 1 Processor -- start 2024-04-26T23:00:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b78a50740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6bfe0 0x5583a6a94ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b78a50740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6c900 0x5583a6a92070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b78a50740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a771c0 0x5583a6a925f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b78a50740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2b701ac1f0 con 0x5583a6a6bfe0 2024-04-26T23:00:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b78a50740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2b701ad660 con 0x5583a6a6c900 2024-04-26T23:00:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b78a50740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2b701ac8c0 con 0x5583a6a771c0 2024-04-26T23:00:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b78a50740 10 client.0 _ll_register_callbacks cb 0x7f2b600d47f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:00:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6bfe0 0x5583a6a94ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6bfe0 0x5583a6a94ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54918/0 (socket says 172.21.15.27:54918) 2024-04-26T23:00:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b6f400640 1 -- 172.21.15.27:0/2872077719 learned_addr learned my addr 172.21.15.27:0/2872077719 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b6e000640 1 --2- 172.21.15.27:0/2872077719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6c900 0x5583a6a92070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b6f400640 1 --2- 172.21.15.27:0/2872077719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a771c0 0x5583a6a925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b6f400640 1 -- 172.21.15.27:0/2872077719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a771c0 msgr2=0x5583a6a925f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b6f400640 1 --2- 172.21.15.27:0/2872077719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a771c0 0x5583a6a925f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b6f400640 1 -- 172.21.15.27:0/2872077719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6c900 msgr2=0x5583a6a92070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b6f400640 1 --2- 172.21.15.27:0/2872077719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6c900 0x5583a6a92070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.117+0000 7f2b6f400640 1 -- 172.21.15.27:0/2872077719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583a698bcb0 con 0x5583a6a6bfe0 2024-04-26T23:00:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.118+0000 7f2b6f400640 1 --2- 172.21.15.27:0/2872077719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6bfe0 0x5583a6a94ce0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2b68005e00 tx=0x7f2b6802a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb2b1f33cb87092 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.118+0000 7f2b58c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T23:00:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.118+0000 7f2b58c00640 1 -- 172.21.15.27:0/2872077719 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6806e020 con 0x5583a6a6bfe0 2024-04-26T23:00:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.118+0000 7f2b58c00640 1 -- 172.21.15.27:0/2872077719 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f2b680055a0 con 0x5583a6a6bfe0 2024-04-26T23:00:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.118+0000 7f2b78a50740 1 -- 172.21.15.27:0/2872077719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b68036030 con 0x5583a6a6bfe0 2024-04-26T23:00:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.118+0000 7f2b58c00640 1 -- 172.21.15.27:0/2872077719 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2b68076050 con 0x5583a6a6bfe0 2024-04-26T23:00:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.118+0000 7f2b78a50740 10 client.15492 fetch_fsmap learned FSMap version 25 2024-04-26T23:00:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.118+0000 7f2b78a50740 1 -- 172.21.15.27:0/2872077719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5583a6af5530 con 0x5583a6a6bfe0 2024-04-26T23:00:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.119+0000 7f2b58c00640 1 -- 172.21.15.27:0/2872077719 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b68036030 con 0x5583a6a6bfe0 2024-04-26T23:00:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.119+0000 7f2b58c00640 1 -- 172.21.15.27:0/2872077719 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2b68064070 con 0x5583a6a6bfe0 2024-04-26T23:00:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.119+0000 7f2b78a50740 10 client.15492 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T23:00:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.119+0000 7f2b78a50740 10 client.15492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:00:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.119+0000 7f2b78a50740 20 client.15492 populate_metadata read hostname 'smithi027' 2024-04-26T23:00:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.119+0000 7f2b78a50740 1 --2- 172.21.15.27:0/2872077719 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5583a6af5ae0 0x5583a6af7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.119+0000 7f2b78a50740 4 client.15492 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:00:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.119+0000 7f2b78a50740 1 -- 172.21.15.27:0/2872077719 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5583a69d8e00 con 0x5583a6af5ae0 2024-04-26T23:00:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.119+0000 7f2b6e000640 1 --2- 172.21.15.27:0/2872077719 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5583a6af5ae0 0x5583a6af7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.120+0000 7f2b6e000640 1 --2- 172.21.15.27:0/2872077719 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5583a6af5ae0 0x5583a6af7ed0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adb4e1bbc5cef211 server_cookie=cc8029deb27c5812 in_seq=0 out_seq=0 2024-04-26T23:00:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.120+0000 7f2b58c00640 10 client.15492 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:00:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.133+0000 7f2b58c00640 1 -- 172.21.15.27:0/2872077719 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5583a69d8e00 con 0x5583a6af5ae0 2024-04-26T23:00:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.133+0000 7f2b58c00640 10 client.15492 handle_command_reply: tid=0 2024-04-26T23:00:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.133+0000 7f2b78a50740 1 client.15492 shutdown 2024-04-26T23:00:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.134+0000 7f2b78a50740 1 -- 172.21.15.27:0/2872077719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6bfe0 msgr2=0x5583a6a94ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.134+0000 7f2b78a50740 1 --2- 172.21.15.27:0/2872077719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6bfe0 0x5583a6a94ce0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2b68005e00 tx=0x7f2b6802a8f0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.134+0000 7f2b78a50740 1 -- 172.21.15.27:0/2872077719 shutdown_connections 2024-04-26T23:00:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.134+0000 7f2b78a50740 1 --2- 172.21.15.27:0/2872077719 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5583a6af5ae0 0x5583a6af7ed0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.134+0000 7f2b78a50740 1 --2- 172.21.15.27:0/2872077719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a6a6c900 0x5583a6a92070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.134+0000 7f2b78a50740 1 --2- 172.21.15.27:0/2872077719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5583a6a771c0 0x5583a6a925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.134+0000 7f2b78a50740 1 --2- 172.21.15.27:0/2872077719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583a6a6bfe0 0x5583a6a94ce0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.134+0000 7f2b58c00640 0 client.15492 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:00:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.134+0000 7f2b78a50740 1 -- 172.21.15.27:0/2872077719 >> 172.21.15.27:0/2872077719 conn(0x5583a6a684f0 msgr2=0x5583a6a66dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.134+0000 7f2b78a50740 1 -- 172.21.15.27:0/2872077719 shutdown_connections 2024-04-26T23:00:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.134+0000 7f2b78a50740 1 -- 172.21.15.27:0/2872077719 wait complete. 2024-04-26T23:00:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.134+0000 7f2b78a50740 20 client.15492 trim_cache size 0 max 16384 2024-04-26T23:00:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T23:00:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T23:00:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T23:00:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T23:00:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.135+0000 7f2b6fe00640 1 -- 172.21.15.27:0/3848852924 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f2b540a3d80 msgr2=0x7f2b540a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.135+0000 7f2b6fe00640 1 --2- 172.21.15.27:0/3848852924 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f2b540a3d80 0x7f2b540a6240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2b600ce190 tx=0x7f2b600d4920 comp rx=0 tx=0).stop 2024-04-26T23:00:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.135+0000 7f2b6fe00640 1 -- 172.21.15.27:0/3848852924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b70104940 msgr2=0x7f2b7019b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.135+0000 7f2b6fe00640 1 --2- 172.21.15.27:0/3848852924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b70104940 0x7f2b7019b120 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000ea60 tx=0x7f2b6000ef20 comp rx=0 tx=0).stop 2024-04-26T23:00:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.135+0000 7f2b6fe00640 1 -- 172.21.15.27:0/3848852924 shutdown_connections 2024-04-26T23:00:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.135+0000 7f2b6fe00640 1 --2- 172.21.15.27:0/3848852924 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f2b540a3d80 0x7f2b540a6240 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.135+0000 7f2b6fe00640 1 --2- 172.21.15.27:0/3848852924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b70105300 0x7f2b7019b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.135+0000 7f2b6fe00640 1 --2- 172.21.15.27:0/3848852924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b7010a950 0x7f2b7019f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.135+0000 7f2b6fe00640 1 --2- 172.21.15.27:0/3848852924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b70104940 0x7f2b7019b120 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.135+0000 7f2b6fe00640 1 -- 172.21.15.27:0/3848852924 >> 172.21.15.27:0/3848852924 conn(0x7f2b70100410 msgr2=0x7f2b70100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.135+0000 7f2b6fe00640 1 -- 172.21.15.27:0/3848852924 shutdown_connections 2024-04-26T23:00:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:11.135+0000 7f2b6fe00640 1 -- 172.21.15.27:0/3848852924 wait complete. 2024-04-26T23:00:13.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:12 smithi080 ceph-mon[32324]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 KiB/s wr, 27 op/s 2024-04-26T23:00:13.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:12 smithi094 ceph-mon[32889]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 KiB/s wr, 27 op/s 2024-04-26T23:00:13.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:12 smithi027 ceph-mon[28034]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 KiB/s wr, 27 op/s 2024-04-26T23:00:13.518 INFO:tasks.workunit.client.0.smithi027.stderr:+ tar xzf ltp-full.tgz 2024-04-26T23:00:15.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:14 smithi080 ceph-mon[32324]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-04-26T23:00:15.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:14 smithi094 ceph-mon[32889]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-04-26T23:00:15.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:14 smithi027 ceph-mon[28034]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-04-26T23:00:17.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:16 smithi080 ceph-mon[32324]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-04-26T23:00:17.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:16 smithi094 ceph-mon[32889]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-04-26T23:00:17.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:16 smithi027 ceph-mon[28034]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-04-26T23:00:19.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:18 smithi080 ceph-mon[32324]: pgmap v54: 129 pgs: 129 active+clean; 15 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.1 MiB/s wr, 15 op/s 2024-04-26T23:00:19.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:18 smithi094 ceph-mon[32889]: pgmap v54: 129 pgs: 129 active+clean; 15 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.1 MiB/s wr, 15 op/s 2024-04-26T23:00:19.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:18 smithi027 ceph-mon[28034]: pgmap v54: 129 pgs: 129 active+clean; 15 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.1 MiB/s wr, 15 op/s 2024-04-26T23:00:20.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:19 smithi080 ceph-mon[32324]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:00:20.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:19 smithi094 ceph-mon[32889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:00:20.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:19 smithi027 ceph-mon[28034]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:00:21.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:20 smithi094 ceph-mon[32889]: pgmap v55: 129 pgs: 129 active+clean; 15 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 1.1 MiB/s wr, 11 op/s 2024-04-26T23:00:21.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:20 smithi027 ceph-mon[28034]: pgmap v55: 129 pgs: 129 active+clean; 15 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 1.1 MiB/s wr, 11 op/s 2024-04-26T23:00:21.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:20 smithi080 ceph-mon[32324]: pgmap v55: 129 pgs: 129 active+clean; 15 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 1.1 MiB/s wr, 11 op/s 2024-04-26T23:00:23.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:22 smithi094 ceph-mon[32889]: pgmap v56: 129 pgs: 129 active+clean; 18 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.5 MiB/s wr, 91 op/s 2024-04-26T23:00:23.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:22 smithi027 ceph-mon[28034]: pgmap v56: 129 pgs: 129 active+clean; 18 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.5 MiB/s wr, 91 op/s 2024-04-26T23:00:23.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:22 smithi080 ceph-mon[32324]: pgmap v56: 129 pgs: 129 active+clean; 18 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.5 MiB/s wr, 91 op/s 2024-04-26T23:00:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.278+0000 7f6ed2a00640 1 -- 172.21.15.27:0/3724580090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc102750 msgr2=0x7f6ecc102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.278+0000 7f6ed2a00640 1 --2- 172.21.15.27:0/3724580090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc102750 0x7f6ecc102bd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009860 tx=0x7f6eb802a340 comp rx=0 tx=0).stop 2024-04-26T23:00:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.279+0000 7f6ed2a00640 1 -- 172.21.15.27:0/3724580090 shutdown_connections 2024-04-26T23:00:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.279+0000 7f6ed2a00640 1 --2- 172.21.15.27:0/3724580090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc102750 0x7f6ecc102bd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.279+0000 7f6ed2a00640 1 --2- 172.21.15.27:0/3724580090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ecc103110 0x7f6ecc10f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.279+0000 7f6ed2a00640 1 --2- 172.21.15.27:0/3724580090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ecc108760 0x7f6ecc108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.279+0000 7f6ed2a00640 1 -- 172.21.15.27:0/3724580090 >> 172.21.15.27:0/3724580090 conn(0x7f6ecc0fe240 msgr2=0x7f6ecc100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.279+0000 7f6ed2a00640 1 -- 172.21.15.27:0/3724580090 shutdown_connections 2024-04-26T23:00:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.279+0000 7f6ed2a00640 1 -- 172.21.15.27:0/3724580090 wait complete. 2024-04-26T23:00:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.280+0000 7f6ed2a00640 1 Processor -- start 2024-04-26T23:00:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.281+0000 7f6ed2a00640 1 -- start start 2024-04-26T23:00:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.281+0000 7f6ed2a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc102750 0x7f6ecc074c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.281+0000 7f6ed2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ecc103110 0x7f6ecc075170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.281+0000 7f6ed2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ecc108760 0x7f6ecc0756b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.281+0000 7f6ed2a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc112a20 con 0x7f6ecc108760 2024-04-26T23:00:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.281+0000 7f6ed2a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1128a0 con 0x7f6ecc102750 2024-04-26T23:00:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.281+0000 7f6ed2a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc112ba0 con 0x7f6ecc103110 2024-04-26T23:00:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.282+0000 7f6ed0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ecc103110 0x7f6ecc075170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.282+0000 7f6ed1600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc102750 0x7f6ecc074c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.282+0000 7f6ed1600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc102750 0x7f6ecc074c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:53512/0 (socket says 172.21.15.27:53512) 2024-04-26T23:00:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.282+0000 7f6ed0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ecc103110 0x7f6ecc075170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:38812/0 (socket says 172.21.15.27:38812) 2024-04-26T23:00:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.282+0000 7f6ed1600640 1 -- 172.21.15.27:0/1982631135 learned_addr learned my addr 172.21.15.27:0/1982631135 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.282+0000 7f6ed2000640 1 --2- 172.21.15.27:0/1982631135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ecc108760 0x7f6ecc0756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.282+0000 7f6ed1600640 1 -- 172.21.15.27:0/1982631135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ecc103110 msgr2=0x7f6ecc075170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.282+0000 7f6ed1600640 1 --2- 172.21.15.27:0/1982631135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ecc103110 0x7f6ecc075170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.282+0000 7f6ed1600640 1 -- 172.21.15.27:0/1982631135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ecc108760 msgr2=0x7f6ecc0756b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.282+0000 7f6ed1600640 1 --2- 172.21.15.27:0/1982631135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ecc108760 0x7f6ecc0756b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.282+0000 7f6ed1600640 1 -- 172.21.15.27:0/1982631135 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc1a7460 con 0x7f6ecc102750 2024-04-26T23:00:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.282+0000 7f6ed1600640 1 --2- 172.21.15.27:0/1982631135 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc102750 0x7f6ecc074c30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000e9f0 tx=0x7f6ec000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.283+0000 7f6eca000640 1 -- 172.21.15.27:0/1982631135 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec000cd80 con 0x7f6ecc102750 2024-04-26T23:00:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.283+0000 7f6eca000640 1 -- 172.21.15.27:0/1982631135 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec00054d0 con 0x7f6ecc102750 2024-04-26T23:00:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.283+0000 7f6eca000640 1 -- 172.21.15.27:0/1982631135 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0010690 con 0x7f6ecc102750 2024-04-26T23:00:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.283+0000 7f6ed2a00640 1 -- 172.21.15.27:0/1982631135 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ecc1a7750 con 0x7f6ecc102750 2024-04-26T23:00:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.283+0000 7f6ed2a00640 1 -- 172.21.15.27:0/1982631135 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ecc1a7c70 con 0x7f6ecc102750 2024-04-26T23:00:24.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.288+0000 7f6ed6d67740 1 Processor -- start 2024-04-26T23:00:24.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.288+0000 7f6ed6d67740 1 -- start start 2024-04-26T23:00:24.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.288+0000 7f6eca000640 1 -- 172.21.15.27:0/1982631135 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6ec0004e90 con 0x7f6ecc102750 2024-04-26T23:00:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.289+0000 7f6ed6d67740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 0x55e9a013d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.289+0000 7f6ed6d67740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a013d900 0x55e9a0147c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.289+0000 7f6ed6d67740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a01481c0 0x55e9a014a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.289+0000 7f6ed6d67740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e9a00a2380 con 0x55e9a01481c0 2024-04-26T23:00:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.289+0000 7f6ed6d67740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55e9a003ec30 con 0x55e9a013cfe0 2024-04-26T23:00:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.289+0000 7f6ed6d67740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e9a009e340 con 0x55e9a013d900 2024-04-26T23:00:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.289+0000 7f6ed1600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a013d900 0x55e9a0147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.289+0000 7f6ed0c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a01481c0 0x55e9a014a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.289+0000 7f6ed1600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a013d900 0x55e9a0147c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:38826/0 (socket says 172.21.15.27:38826) 2024-04-26T23:00:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.289+0000 7f6ed1600640 1 -- 172.21.15.27:0/3144525901 learned_addr learned my addr 172.21.15.27:0/3144525901 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.289+0000 7f6ed2000640 1 --2- 172.21.15.27:0/3144525901 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 0x55e9a013d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.290+0000 7f6ed1600640 1 -- 172.21.15.27:0/3144525901 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 msgr2=0x55e9a013d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.290+0000 7f6ed1600640 1 --2- 172.21.15.27:0/3144525901 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 0x55e9a013d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.290+0000 7f6ed1600640 1 -- 172.21.15.27:0/3144525901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a01481c0 msgr2=0x55e9a014a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.290+0000 7f6ed1600640 1 --2- 172.21.15.27:0/3144525901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a01481c0 0x55e9a014a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.290+0000 7f6ed1600640 1 -- 172.21.15.27:0/3144525901 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9a0060940 con 0x55e9a013d900 2024-04-26T23:00:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.290+0000 7f6ed0c00640 1 --2- 172.21.15.27:0/3144525901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a01481c0 0x55e9a014a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:00:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.290+0000 7f6ed1600640 1 --2- 172.21.15.27:0/3144525901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a013d900 0x55e9a0147c80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6ec00ce260 tx=0x7f6ec00d91a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acce0c3a63d436db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.291+0000 7f6ebe600640 1 -- 172.21.15.27:0/3144525901 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec00e4070 con 0x55e9a013d900 2024-04-26T23:00:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.291+0000 7f6ebe600640 1 -- 172.21.15.27:0/3144525901 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec00d9bd0 con 0x55e9a013d900 2024-04-26T23:00:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.291+0000 7f6ebe600640 1 -- 172.21.15.27:0/3144525901 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec00df400 con 0x55e9a013d900 2024-04-26T23:00:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.291+0000 7f6eca000640 1 --2- 172.21.15.27:0/1982631135 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6eb40a3d80 0x7f6eb40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.291+0000 7f6eca000640 1 -- 172.21.15.27:0/1982631135 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f6ec00d1070 con 0x7f6ecc102750 2024-04-26T23:00:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.292+0000 7f6ed6d67740 1 -- 172.21.15.27:0/3144525901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a013d900 msgr2=0x55e9a0147c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.292+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/3144525901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a013d900 0x55e9a0147c80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6ec00ce260 tx=0x7f6ec00d91a0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.293+0000 7f6ed2000640 1 --2- 172.21.15.27:0/1982631135 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6eb40a3d80 0x7f6eb40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.293+0000 7f6ed6d67740 1 -- 172.21.15.27:0/3144525901 shutdown_connections 2024-04-26T23:00:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.293+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/3144525901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a01481c0 0x55e9a014a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.293+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/3144525901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a013d900 0x55e9a0147c80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.293+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/3144525901 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 0x55e9a013d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.293+0000 7f6ed6d67740 1 -- 172.21.15.27:0/3144525901 >> 172.21.15.27:0/3144525901 conn(0x55e9a01394f0 msgr2=0x55e9a01398d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.293+0000 7f6ed6d67740 1 -- 172.21.15.27:0/3144525901 shutdown_connections 2024-04-26T23:00:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.293+0000 7f6ed6d67740 1 -- 172.21.15.27:0/3144525901 wait complete. 2024-04-26T23:00:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.294+0000 7f6ed2000640 1 --2- 172.21.15.27:0/1982631135 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6eb40a3d80 0x7f6eb40a6240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4005fd0 tx=0x7f6ec4011040 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.295+0000 7f6ed6d67740 1 -- start start 2024-04-26T23:00:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.295+0000 7f6ed6d67740 1 Processor -- start 2024-04-26T23:00:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.296+0000 7f6ed6d67740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 0x55e9a0145160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.296+0000 7f6ed6d67740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a013d900 0x55e9a01456a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.296+0000 7f6ed6d67740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a01481c0 0x55e9a0145be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.296+0000 7f6ed6d67740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e9a014adf0 con 0x55e9a013d900 2024-04-26T23:00:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.296+0000 7f6ed6d67740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55e9a0029c60 con 0x55e9a013cfe0 2024-04-26T23:00:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.296+0000 7f6ed6d67740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e9a014af70 con 0x55e9a01481c0 2024-04-26T23:00:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.296+0000 7f6ed6d67740 10 client.0 _ll_register_callbacks cb 0x55e9a0055e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:00:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.296+0000 7f6ed1600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 0x55e9a0145160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.296+0000 7f6ed1600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 0x55e9a0145160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:53536/0 (socket says 172.21.15.27:53536) 2024-04-26T23:00:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.296+0000 7f6ed1600640 1 -- 172.21.15.27:0/518944426 learned_addr learned my addr 172.21.15.27:0/518944426 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.297+0000 7f6ed2000640 1 --2- 172.21.15.27:0/518944426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a01481c0 0x55e9a0145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.297+0000 7f6ed1600640 1 -- 172.21.15.27:0/518944426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a01481c0 msgr2=0x55e9a0145be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.297+0000 7f6ed1600640 1 --2- 172.21.15.27:0/518944426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a01481c0 0x55e9a0145be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.297+0000 7f6ed1600640 1 -- 172.21.15.27:0/518944426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a013d900 msgr2=0x55e9a01456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.297+0000 7f6ed1600640 1 --2- 172.21.15.27:0/518944426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a013d900 0x55e9a01456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.297+0000 7f6ed1600640 1 -- 172.21.15.27:0/518944426 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9a0038510 con 0x55e9a013cfe0 2024-04-26T23:00:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.297+0000 7f6ed1600640 1 --2- 172.21.15.27:0/518944426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 0x55e9a0145160 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0099540 tx=0x7f6ec00d3390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cb41d83b72cdc51 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.297+0000 7f6eb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T23:00:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.298+0000 7f6eb2a00640 1 -- 172.21.15.27:0/518944426 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0098e10 con 0x55e9a013cfe0 2024-04-26T23:00:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.298+0000 7f6eb2a00640 1 -- 172.21.15.27:0/518944426 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f6ec00046c0 con 0x55e9a013cfe0 2024-04-26T23:00:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.299+0000 7f6ed6d67740 1 -- 172.21.15.27:0/518944426 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ec00d9bd0 con 0x55e9a013cfe0 2024-04-26T23:00:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.299+0000 7f6eb2a00640 1 -- 172.21.15.27:0/518944426 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ec0005180 con 0x55e9a013cfe0 2024-04-26T23:00:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.299+0000 7f6ed6d67740 10 client.25616 fetch_fsmap learned FSMap version 26 2024-04-26T23:00:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.299+0000 7f6ed6d67740 1 -- 172.21.15.27:0/518944426 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e99fdb4210 con 0x55e9a013cfe0 2024-04-26T23:00:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.299+0000 7f6eb2a00640 1 -- 172.21.15.27:0/518944426 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec00d9bd0 con 0x55e9a013cfe0 2024-04-26T23:00:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.300+0000 7f6eb2a00640 1 -- 172.21.15.27:0/518944426 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f6ec00b7030 con 0x55e9a013cfe0 2024-04-26T23:00:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.300+0000 7f6ed6d67740 10 client.25616 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T23:00:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.300+0000 7f6ed6d67740 10 client.25616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:00:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.300+0000 7f6ed6d67740 20 client.25616 populate_metadata read hostname 'smithi027' 2024-04-26T23:00:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.300+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/518944426 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e9a01605b0 0x55e9a01629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.300+0000 7f6ed6d67740 4 client.25616 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:00:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.300+0000 7f6ed6d67740 1 -- 172.21.15.27:0/518944426 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9a00aff90 con 0x55e9a01605b0 2024-04-26T23:00:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.300+0000 7f6ed0c00640 1 --2- 172.21.15.27:0/518944426 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e9a01605b0 0x55e9a01629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.301+0000 7f6ed0c00640 1 --2- 172.21.15.27:0/518944426 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e9a01605b0 0x55e9a01629a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b87a9a79107f5c65 server_cookie=90ee6ea1678aff64 in_seq=0 out_seq=0 2024-04-26T23:00:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.301+0000 7f6eb2a00640 10 client.25616 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:00:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.302+0000 7f6eb2a00640 1 -- 172.21.15.27:0/518944426 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9a00aff90 con 0x55e9a01605b0 2024-04-26T23:00:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.302+0000 7f6eb2a00640 10 client.25616 handle_command_reply: tid=0 2024-04-26T23:00:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.302+0000 7f6ed6d67740 1 client.25616 shutdown 2024-04-26T23:00:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.302+0000 7f6ed6d67740 1 -- 172.21.15.27:0/518944426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 msgr2=0x55e9a0145160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.302+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/518944426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 0x55e9a0145160 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0099540 tx=0x7f6ec00d3390 comp rx=0 tx=0).stop 2024-04-26T23:00:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.302+0000 7f6ed6d67740 1 -- 172.21.15.27:0/518944426 shutdown_connections 2024-04-26T23:00:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.302+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/518944426 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e9a01605b0 0x55e9a01629a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.302+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/518944426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a01481c0 0x55e9a0145be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.302+0000 7f6eb2a00640 0 client.25616 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:00:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.302+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/518944426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a013d900 0x55e9a01456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.302+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/518944426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 0x55e9a0145160 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.302+0000 7f6ed6d67740 1 -- 172.21.15.27:0/518944426 >> 172.21.15.27:0/518944426 conn(0x55e9a01394f0 msgr2=0x55e9a013f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.302+0000 7f6ed6d67740 1 -- 172.21.15.27:0/518944426 shutdown_connections 2024-04-26T23:00:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.302+0000 7f6ed6d67740 1 -- 172.21.15.27:0/518944426 wait complete. 2024-04-26T23:00:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.303+0000 7f6ed6d67740 20 client.25616 trim_cache size 0 max 16384 2024-04-26T23:00:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.323+0000 7f6ed6d67740 1 Processor -- start 2024-04-26T23:00:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.323+0000 7f6ed6d67740 1 -- start start 2024-04-26T23:00:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.324+0000 7f6ed6d67740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 0x55e9a0147890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.324+0000 7f6ed6d67740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a013d900 0x55e9a0144010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.324+0000 7f6ed6d67740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a01481c0 0x55e9a01659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.324+0000 7f6ed6d67740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc0748f0 con 0x55e9a013d900 2024-04-26T23:00:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.324+0000 7f6ed6d67740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc0721c0 con 0x55e9a013cfe0 2024-04-26T23:00:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.324+0000 7f6ed6d67740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc070010 con 0x55e9a01481c0 2024-04-26T23:00:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.324+0000 7f6ed0c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a013d900 0x55e9a0144010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.324+0000 7f6ed1600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 0x55e9a0147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.324+0000 7f6ed0c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a013d900 0x55e9a0144010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35550/0 (socket says 172.21.15.27:35550) 2024-04-26T23:00:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.324+0000 7f6ed0c00640 1 -- 172.21.15.27:0/2976793283 learned_addr learned my addr 172.21.15.27:0/2976793283 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.324+0000 7f6ed2000640 1 --2- 172.21.15.27:0/2976793283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a01481c0 0x55e9a01659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.325+0000 7f6ed1600640 1 -- 172.21.15.27:0/2976793283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a01481c0 msgr2=0x55e9a01659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.325+0000 7f6ed1600640 1 --2- 172.21.15.27:0/2976793283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a01481c0 0x55e9a01659c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.325+0000 7f6ed1600640 1 -- 172.21.15.27:0/2976793283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a013d900 msgr2=0x55e9a0144010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.325+0000 7f6ed1600640 1 --2- 172.21.15.27:0/2976793283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a013d900 0x55e9a0144010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.325+0000 7f6ed1600640 1 -- 172.21.15.27:0/2976793283 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e99fe80620 con 0x55e9a013cfe0 2024-04-26T23:00:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.325+0000 7f6ed1600640 1 --2- 172.21.15.27:0/2976793283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 0x55e9a0147890 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6ec00994f0 tx=0x7f6ec009b080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcb7adbd8b03000b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.325+0000 7f6ebdc00640 1 -- 172.21.15.27:0/2976793283 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec009c4d0 con 0x55e9a013cfe0 2024-04-26T23:00:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.325+0000 7f6ebdc00640 1 -- 172.21.15.27:0/2976793283 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec00d96a0 con 0x55e9a013cfe0 2024-04-26T23:00:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.325+0000 7f6ebdc00640 1 -- 172.21.15.27:0/2976793283 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec009c6b0 con 0x55e9a013cfe0 2024-04-26T23:00:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.326+0000 7f6ed6d67740 1 -- 172.21.15.27:0/2976793283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 msgr2=0x55e9a0147890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.326+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/2976793283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 0x55e9a0147890 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6ec00994f0 tx=0x7f6ec009b080 comp rx=0 tx=0).stop 2024-04-26T23:00:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.326+0000 7f6ed6d67740 1 -- 172.21.15.27:0/2976793283 shutdown_connections 2024-04-26T23:00:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.326+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/2976793283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a01481c0 0x55e9a01659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.326+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/2976793283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a013d900 0x55e9a0144010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.326+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/2976793283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013cfe0 0x55e9a0147890 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.326+0000 7f6ed6d67740 1 -- 172.21.15.27:0/2976793283 >> 172.21.15.27:0/2976793283 conn(0x55e9a01394f0 msgr2=0x55e9a0143900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.326+0000 7f6ed6d67740 1 -- 172.21.15.27:0/2976793283 shutdown_connections 2024-04-26T23:00:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.326+0000 7f6ed6d67740 1 -- 172.21.15.27:0/2976793283 wait complete. 2024-04-26T23:00:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.327+0000 7f6ed6d67740 1 -- start start 2024-04-26T23:00:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.327+0000 7f6ed6d67740 1 Processor -- start 2024-04-26T23:00:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.328+0000 7f6ed6d67740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a013cfe0 0x55e9a0165df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.328+0000 7f6ed6d67740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013d900 0x55e9a01631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.328+0000 7f6ed6d67740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a01481c0 0x55e9a0163730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.328+0000 7f6ed6d67740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc070680 con 0x55e9a01481c0 2024-04-26T23:00:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.328+0000 7f6ed6d67740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc071af0 con 0x55e9a013d900 2024-04-26T23:00:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.328+0000 7f6ed6d67740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc070d50 con 0x55e9a013cfe0 2024-04-26T23:00:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.328+0000 7f6ed6d67740 10 client.0 _ll_register_callbacks cb 0x7f6ec00e2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:00:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.328+0000 7f6ed1600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a013cfe0 0x55e9a0165df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.328+0000 7f6ed2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a01481c0 0x55e9a0163730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.328+0000 7f6ed0c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013d900 0x55e9a01631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.328+0000 7f6ed1600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a013cfe0 0x55e9a0165df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:38840/0 (socket says 172.21.15.27:38840) 2024-04-26T23:00:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.328+0000 7f6ed2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a01481c0 0x55e9a0163730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35558/0 (socket says 172.21.15.27:35558) 2024-04-26T23:00:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.328+0000 7f6ed2000640 1 -- 172.21.15.27:0/3049402024 learned_addr learned my addr 172.21.15.27:0/3049402024 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.328+0000 7f6ed0c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013d900 0x55e9a01631d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:53562/0 (socket says 172.21.15.27:53562) 2024-04-26T23:00:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.329+0000 7f6ed0c00640 1 -- 172.21.15.27:0/3049402024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a013cfe0 msgr2=0x55e9a0165df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.329+0000 7f6ed0c00640 1 --2- 172.21.15.27:0/3049402024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a013cfe0 0x55e9a0165df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.329+0000 7f6ed0c00640 1 -- 172.21.15.27:0/3049402024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a01481c0 msgr2=0x55e9a0163730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.329+0000 7f6ed0c00640 1 --2- 172.21.15.27:0/3049402024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a01481c0 0x55e9a0163730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.329+0000 7f6ed0c00640 1 -- 172.21.15.27:0/3049402024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9a005ccb0 con 0x55e9a013d900 2024-04-26T23:00:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.329+0000 7f6ed0c00640 1 --2- 172.21.15.27:0/3049402024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013d900 0x55e9a01631d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009780 tx=0x7f6eb8030f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf0372551a83f067 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.329+0000 7f6eb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T23:00:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.329+0000 7f6eb2a00640 1 -- 172.21.15.27:0/3049402024 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8037040 con 0x55e9a013d900 2024-04-26T23:00:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.329+0000 7f6eb2a00640 1 -- 172.21.15.27:0/3049402024 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f6eb8030740 con 0x55e9a013d900 2024-04-26T23:00:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.329+0000 7f6ed6d67740 1 -- 172.21.15.27:0/3049402024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ec00d96a0 con 0x55e9a013d900 2024-04-26T23:00:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.330+0000 7f6eb2a00640 1 -- 172.21.15.27:0/3049402024 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6eb8053050 con 0x55e9a013d900 2024-04-26T23:00:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.330+0000 7f6ed6d67740 10 client.25622 fetch_fsmap learned FSMap version 26 2024-04-26T23:00:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.330+0000 7f6ed6d67740 1 -- 172.21.15.27:0/3049402024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e9a01c66b0 con 0x55e9a013d900 2024-04-26T23:00:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.330+0000 7f6eb2a00640 1 -- 172.21.15.27:0/3049402024 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec00d96a0 con 0x55e9a013d900 2024-04-26T23:00:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.331+0000 7f6eb2a00640 1 -- 172.21.15.27:0/3049402024 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f6eb80426b0 con 0x55e9a013d900 2024-04-26T23:00:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.331+0000 7f6ed6d67740 10 client.25622 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T23:00:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.331+0000 7f6ed6d67740 10 client.25622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:00:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.331+0000 7f6ed6d67740 20 client.25622 populate_metadata read hostname 'smithi027' 2024-04-26T23:00:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.331+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/3049402024 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e9a01c6ee0 0x55e9a01c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.331+0000 7f6ed6d67740 4 client.25622 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:00:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.331+0000 7f6ed6d67740 1 -- 172.21.15.27:0/3049402024 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e9a00a9e00 con 0x55e9a01c6ee0 2024-04-26T23:00:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.331+0000 7f6ed1600640 1 --2- 172.21.15.27:0/3049402024 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e9a01c6ee0 0x55e9a01c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.331+0000 7f6ed1600640 1 --2- 172.21.15.27:0/3049402024 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e9a01c6ee0 0x55e9a01c9300 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=628ecb910f73b841 server_cookie=71e8716e5f3e1287 in_seq=0 out_seq=0 2024-04-26T23:00:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.331+0000 7f6eb2a00640 10 client.25622 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:00:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.342+0000 7f6eb2a00640 1 -- 172.21.15.27:0/3049402024 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e9a00a9e00 con 0x55e9a01c6ee0 2024-04-26T23:00:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.342+0000 7f6eb2a00640 10 client.25622 handle_command_reply: tid=0 2024-04-26T23:00:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.343+0000 7f6ed6d67740 1 client.25622 shutdown 2024-04-26T23:00:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.343+0000 7f6ed6d67740 1 -- 172.21.15.27:0/3049402024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013d900 msgr2=0x55e9a01631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.343+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/3049402024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013d900 0x55e9a01631d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009780 tx=0x7f6eb8030f10 comp rx=0 tx=0).stop 2024-04-26T23:00:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.343+0000 7f6ed6d67740 1 -- 172.21.15.27:0/3049402024 shutdown_connections 2024-04-26T23:00:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.343+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/3049402024 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e9a01c6ee0 0x55e9a01c9300 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.343+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/3049402024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e9a013d900 0x55e9a01631d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.343+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/3049402024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9a01481c0 0x55e9a0163730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.343+0000 7f6ed6d67740 1 --2- 172.21.15.27:0/3049402024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9a013cfe0 0x55e9a0165df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.343+0000 7f6eb2a00640 0 client.25622 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:00:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.343+0000 7f6ed6d67740 1 -- 172.21.15.27:0/3049402024 >> 172.21.15.27:0/3049402024 conn(0x55e9a01394f0 msgr2=0x55e9a015c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.343+0000 7f6ed6d67740 1 -- 172.21.15.27:0/3049402024 shutdown_connections 2024-04-26T23:00:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.343+0000 7f6ed6d67740 1 -- 172.21.15.27:0/3049402024 wait complete. 2024-04-26T23:00:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.343+0000 7f6ed6d67740 20 client.25622 trim_cache size 0 max 16384 2024-04-26T23:00:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T23:00:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T23:00:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T23:00:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T23:00:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.344+0000 7f6ed2a00640 1 -- 172.21.15.27:0/1982631135 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6eb40a3d80 msgr2=0x7f6eb40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.344+0000 7f6ed2a00640 1 --2- 172.21.15.27:0/1982631135 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6eb40a3d80 0x7f6eb40a6240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4005fd0 tx=0x7f6ec4011040 comp rx=0 tx=0).stop 2024-04-26T23:00:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.344+0000 7f6ed2a00640 1 -- 172.21.15.27:0/1982631135 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc102750 msgr2=0x7f6ecc074c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.344+0000 7f6ed2a00640 1 --2- 172.21.15.27:0/1982631135 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc102750 0x7f6ecc074c30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000e9f0 tx=0x7f6ec000edf0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.345+0000 7f6ed2a00640 1 -- 172.21.15.27:0/1982631135 shutdown_connections 2024-04-26T23:00:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.345+0000 7f6ed2a00640 1 --2- 172.21.15.27:0/1982631135 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f6eb40a3d80 0x7f6eb40a6240 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.345+0000 7f6ed2a00640 1 --2- 172.21.15.27:0/1982631135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ecc108760 0x7f6ecc0756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.345+0000 7f6ed2a00640 1 --2- 172.21.15.27:0/1982631135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ecc103110 0x7f6ecc075170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.345+0000 7f6ed2a00640 1 --2- 172.21.15.27:0/1982631135 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ecc102750 0x7f6ecc074c30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.345+0000 7f6ed2a00640 1 -- 172.21.15.27:0/1982631135 >> 172.21.15.27:0/1982631135 conn(0x7f6ecc0fe240 msgr2=0x7f6ecc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.345+0000 7f6ed2a00640 1 -- 172.21.15.27:0/1982631135 shutdown_connections 2024-04-26T23:00:24.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:24.345+0000 7f6ed2a00640 1 -- 172.21.15.27:0/1982631135 wait complete. 2024-04-26T23:00:25.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:24 smithi027 ceph-mon[28034]: pgmap v57: 129 pgs: 129 active+clean; 18 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-04-26T23:00:25.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:24 smithi080 ceph-mon[32324]: pgmap v57: 129 pgs: 129 active+clean; 18 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-04-26T23:00:25.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:24 smithi094 ceph-mon[32889]: pgmap v57: 129 pgs: 129 active+clean; 18 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-04-26T23:00:27.087 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:26 smithi094 ceph-mon[32889]: pgmap v58: 129 pgs: 129 active+clean; 18 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-04-26T23:00:27.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:26 smithi027 ceph-mon[28034]: pgmap v58: 129 pgs: 129 active+clean; 18 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-04-26T23:00:27.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:26 smithi080 ceph-mon[32324]: pgmap v58: 129 pgs: 129 active+clean; 18 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-04-26T23:00:29.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:28 smithi027 ceph-mon[28034]: pgmap v59: 129 pgs: 129 active+clean; 24 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 2.0 MiB/s wr, 104 op/s 2024-04-26T23:00:29.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:28 smithi080 ceph-mon[32324]: pgmap v59: 129 pgs: 129 active+clean; 24 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 2.0 MiB/s wr, 104 op/s 2024-04-26T23:00:29.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:28 smithi094 ceph-mon[32889]: pgmap v59: 129 pgs: 129 active+clean; 24 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 2.0 MiB/s wr, 104 op/s 2024-04-26T23:00:31.073 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:30 smithi080 ceph-mon[32324]: pgmap v60: 129 pgs: 129 active+clean; 24 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 865 KiB/s wr, 98 op/s 2024-04-26T23:00:31.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:30 smithi027 ceph-mon[28034]: pgmap v60: 129 pgs: 129 active+clean; 24 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 865 KiB/s wr, 98 op/s 2024-04-26T23:00:31.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:30 smithi094 ceph-mon[32889]: pgmap v60: 129 pgs: 129 active+clean; 24 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 865 KiB/s wr, 98 op/s 2024-04-26T23:00:33.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:32 smithi027 ceph-mon[28034]: pgmap v61: 129 pgs: 129 active+clean; 27 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-04-26T23:00:33.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:32 smithi080 ceph-mon[32324]: pgmap v61: 129 pgs: 129 active+clean; 27 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-04-26T23:00:33.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:32 smithi094 ceph-mon[32889]: pgmap v61: 129 pgs: 129 active+clean; 27 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-04-26T23:00:35.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:34 smithi027 ceph-mon[28034]: pgmap v62: 129 pgs: 129 active+clean; 27 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 808 KiB/s wr, 28 op/s 2024-04-26T23:00:35.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:34 smithi080 ceph-mon[32324]: pgmap v62: 129 pgs: 129 active+clean; 27 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 808 KiB/s wr, 28 op/s 2024-04-26T23:00:35.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:34 smithi094 ceph-mon[32889]: pgmap v62: 129 pgs: 129 active+clean; 27 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 808 KiB/s wr, 28 op/s 2024-04-26T23:00:37.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:36 smithi027 ceph-mon[28034]: pgmap v63: 129 pgs: 129 active+clean; 27 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 768 KiB/s wr, 21 op/s 2024-04-26T23:00:37.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:36 smithi080 ceph-mon[32324]: pgmap v63: 129 pgs: 129 active+clean; 27 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 768 KiB/s wr, 21 op/s 2024-04-26T23:00:37.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:36 smithi094 ceph-mon[32889]: pgmap v63: 129 pgs: 129 active+clean; 27 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 768 KiB/s wr, 21 op/s 2024-04-26T23:00:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.563+0000 7ff549c00640 1 -- 172.21.15.27:0/3668616710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff544076e60 msgr2=0x7ff544077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.563+0000 7ff549c00640 1 --2- 172.21.15.27:0/3668616710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff544076e60 0x7ff544077240 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff534009930 tx=0x7ff53402a420 comp rx=0 tx=0).stop 2024-04-26T23:00:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.563+0000 7ff549c00640 1 -- 172.21.15.27:0/3668616710 shutdown_connections 2024-04-26T23:00:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.563+0000 7ff549c00640 1 --2- 172.21.15.27:0/3668616710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5441137f0 0x7ff544115be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.563+0000 7ff549c00640 1 --2- 172.21.15.27:0/3668616710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff544077780 0x7ff5441132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.563+0000 7ff549c00640 1 --2- 172.21.15.27:0/3668616710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff544076e60 0x7ff544077240 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.563+0000 7ff549c00640 1 -- 172.21.15.27:0/3668616710 >> 172.21.15.27:0/3668616710 conn(0x7ff5441005e0 msgr2=0x7ff544102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.563+0000 7ff549c00640 1 -- 172.21.15.27:0/3668616710 shutdown_connections 2024-04-26T23:00:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.563+0000 7ff549c00640 1 -- 172.21.15.27:0/3668616710 wait complete. 2024-04-26T23:00:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.565+0000 7ff549c00640 1 Processor -- start 2024-04-26T23:00:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.565+0000 7ff549c00640 1 -- start start 2024-04-26T23:00:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.566+0000 7ff549c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff544076e60 0x7ff5441a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.566+0000 7ff549c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff544077780 0x7ff5441a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.566+0000 7ff549c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5441137f0 0x7ff54419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.566+0000 7ff549c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff5441192d0 con 0x7ff544076e60 2024-04-26T23:00:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.566+0000 7ff549c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff544119150 con 0x7ff544077780 2024-04-26T23:00:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.566+0000 7ff549c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff544119450 con 0x7ff5441137f0 2024-04-26T23:00:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.566+0000 7ff542000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff544077780 0x7ff5441a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.566+0000 7ff542000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff544077780 0x7ff5441a5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:54194/0 (socket says 172.21.15.27:54194) 2024-04-26T23:00:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.566+0000 7ff542000640 1 -- 172.21.15.27:0/1842485647 learned_addr learned my addr 172.21.15.27:0/1842485647 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.566+0000 7ff542a00640 1 --2- 172.21.15.27:0/1842485647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff544076e60 0x7ff5441a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.566+0000 7ff543400640 1 --2- 172.21.15.27:0/1842485647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5441137f0 0x7ff54419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.567+0000 7ff543400640 1 -- 172.21.15.27:0/1842485647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff544077780 msgr2=0x7ff5441a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.567+0000 7ff543400640 1 --2- 172.21.15.27:0/1842485647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff544077780 0x7ff5441a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.567+0000 7ff543400640 1 -- 172.21.15.27:0/1842485647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff544076e60 msgr2=0x7ff5441a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.568+0000 7ff543400640 1 --2- 172.21.15.27:0/1842485647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff544076e60 0x7ff5441a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.568+0000 7ff543400640 1 -- 172.21.15.27:0/1842485647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54419fef0 con 0x7ff5441137f0 2024-04-26T23:00:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.568+0000 7ff542a00640 1 --2- 172.21.15.27:0/1842485647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff544076e60 0x7ff5441a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:00:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.569+0000 7ff543400640 1 --2- 172.21.15.27:0/1842485647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5441137f0 0x7ff54419f630 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff53c005fd0 tx=0x7ff53c00edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.569+0000 7ff53b400640 1 -- 172.21.15.27:0/1842485647 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c004d20 con 0x7ff5441137f0 2024-04-26T23:00:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.570+0000 7ff53b400640 1 -- 172.21.15.27:0/1842485647 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff53c005020 con 0x7ff5441137f0 2024-04-26T23:00:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.570+0000 7ff53b400640 1 -- 172.21.15.27:0/1842485647 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0116b0 con 0x7ff5441137f0 2024-04-26T23:00:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.570+0000 7ff549c00640 1 -- 172.21.15.27:0/1842485647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5441a01e0 con 0x7ff5441137f0 2024-04-26T23:00:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.570+0000 7ff549c00640 1 -- 172.21.15.27:0/1842485647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5441ac4f0 con 0x7ff5441137f0 2024-04-26T23:00:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.576+0000 7ff53b400640 1 -- 172.21.15.27:0/1842485647 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff53c011850 con 0x7ff5441137f0 2024-04-26T23:00:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.578+0000 7ff53b400640 1 --2- 172.21.15.27:0/1842485647 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7ff5280a3f20 0x7ff5280a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.578+0000 7ff53b400640 1 -- 172.21.15.27:0/1842485647 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7ff53c0d1bb0 con 0x7ff5441137f0 2024-04-26T23:00:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.578+0000 7ff54ca34740 1 Processor -- start 2024-04-26T23:00:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.579+0000 7ff542a00640 1 --2- 172.21.15.27:0/1842485647 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7ff5280a3f20 0x7ff5280a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.579+0000 7ff54ca34740 1 -- start start 2024-04-26T23:00:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.579+0000 7ff54ca34740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c35ddd0 0x559b0c35e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.579+0000 7ff54ca34740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 0x559b0c368a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.579+0000 7ff54ca34740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c368fb0 0x559b0c36b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.580+0000 7ff54ca34740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559b0c2c3380 con 0x559b0c35e6f0 2024-04-26T23:00:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.580+0000 7ff54ca34740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x559b0c25fc30 con 0x559b0c368fb0 2024-04-26T23:00:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.580+0000 7ff54ca34740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559b0c2bf340 con 0x559b0c35ddd0 2024-04-26T23:00:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.580+0000 7ff542000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 0x559b0c368a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.580+0000 7ff542000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 0x559b0c368a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59886/0 (socket says 172.21.15.27:59886) 2024-04-26T23:00:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.580+0000 7ff542000640 1 -- 172.21.15.27:0/3725729716 learned_addr learned my addr 172.21.15.27:0/3725729716 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.580+0000 7ff542a00640 1 --2- 172.21.15.27:0/1842485647 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7ff5280a3f20 0x7ff5280a63e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff534009850 tx=0x7ff534000bc0 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.581+0000 7ff542000640 1 -- 172.21.15.27:0/3725729716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c35ddd0 msgr2=0x559b0c35e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.581+0000 7ff542000640 1 --2- 172.21.15.27:0/3725729716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c35ddd0 0x559b0c35e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.581+0000 7ff542000640 1 -- 172.21.15.27:0/3725729716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c368fb0 msgr2=0x559b0c36b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:00:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.581+0000 7ff542000640 1 --2- 172.21.15.27:0/3725729716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c368fb0 0x559b0c36b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.581+0000 7ff542000640 1 -- 172.21.15.27:0/3725729716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b0c281940 con 0x559b0c35e6f0 2024-04-26T23:00:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.581+0000 7ff542000640 1 --2- 172.21.15.27:0/3725729716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 0x559b0c368a70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff530009ae0 tx=0x7ff53000e1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c918fe2d6b4e206b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.582+0000 7ff52fe00640 1 -- 172.21.15.27:0/3725729716 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff530017070 con 0x559b0c35e6f0 2024-04-26T23:00:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.582+0000 7ff52fe00640 1 -- 172.21.15.27:0/3725729716 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff53000ebb0 con 0x559b0c35e6f0 2024-04-26T23:00:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.582+0000 7ff52fe00640 1 -- 172.21.15.27:0/3725729716 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff530006800 con 0x559b0c35e6f0 2024-04-26T23:00:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.584+0000 7ff54ca34740 1 -- 172.21.15.27:0/3725729716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 msgr2=0x559b0c368a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.584+0000 7ff54ca34740 1 --2- 172.21.15.27:0/3725729716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 0x559b0c368a70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff530009ae0 tx=0x7ff53000e1d0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.584+0000 7ff54ca34740 1 -- 172.21.15.27:0/3725729716 shutdown_connections 2024-04-26T23:00:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.584+0000 7ff54ca34740 1 --2- 172.21.15.27:0/3725729716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c368fb0 0x559b0c36b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.584+0000 7ff54ca34740 1 --2- 172.21.15.27:0/3725729716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 0x559b0c368a70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.584+0000 7ff54ca34740 1 --2- 172.21.15.27:0/3725729716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c35ddd0 0x559b0c35e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.584+0000 7ff54ca34740 1 -- 172.21.15.27:0/3725729716 >> 172.21.15.27:0/3725729716 conn(0x559b0c35a2e0 msgr2=0x559b0c35a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.585+0000 7ff54ca34740 1 -- 172.21.15.27:0/3725729716 shutdown_connections 2024-04-26T23:00:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.585+0000 7ff54ca34740 1 -- 172.21.15.27:0/3725729716 wait complete. 2024-04-26T23:00:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.586+0000 7ff54ca34740 1 -- start start 2024-04-26T23:00:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.587+0000 7ff54ca34740 1 Processor -- start 2024-04-26T23:00:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.588+0000 7ff54ca34740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35ddd0 0x559b0c365fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.588+0000 7ff54ca34740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 0x559b0c366500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.588+0000 7ff54ca34740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c366a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.589+0000 7ff542a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35ddd0 0x559b0c365fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.589+0000 7ff542a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35ddd0 0x559b0c365fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:54214/0 (socket says 172.21.15.27:54214) 2024-04-26T23:00:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.589+0000 7ff54ca34740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559b0c36bbe0 con 0x559b0c35e6f0 2024-04-26T23:00:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.589+0000 7ff54ca34740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x559b0c24ac60 con 0x559b0c35ddd0 2024-04-26T23:00:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.589+0000 7ff543400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c366a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.589+0000 7ff54ca34740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559b0c36bd60 con 0x559b0c368fb0 2024-04-26T23:00:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.589+0000 7ff543400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c366a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:38382/0 (socket says 172.21.15.27:38382) 2024-04-26T23:00:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.589+0000 7ff543400640 1 -- 172.21.15.27:0/816268176 learned_addr learned my addr 172.21.15.27:0/816268176 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.589+0000 7ff542000640 1 --2- 172.21.15.27:0/816268176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 0x559b0c366500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.589+0000 7ff54ca34740 10 client.0 _ll_register_callbacks cb 0x559b0c276e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:00:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.590+0000 7ff543400640 1 -- 172.21.15.27:0/816268176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35ddd0 msgr2=0x559b0c365fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.590+0000 7ff543400640 1 --2- 172.21.15.27:0/816268176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35ddd0 0x559b0c365fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.590+0000 7ff543400640 1 -- 172.21.15.27:0/816268176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 msgr2=0x559b0c366500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.590+0000 7ff543400640 1 --2- 172.21.15.27:0/816268176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 0x559b0c366500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.590+0000 7ff543400640 1 -- 172.21.15.27:0/816268176 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b0c259510 con 0x559b0c368fb0 2024-04-26T23:00:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.590+0000 7ff542000640 1 --2- 172.21.15.27:0/816268176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 0x559b0c366500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:00:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.590+0000 7ff542a00640 1 --2- 172.21.15.27:0/816268176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35ddd0 0x559b0c365fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-26T23:00:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.590+0000 7ff543400640 1 --2- 172.21.15.27:0/816268176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c366a40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff53c09d3b0 tx=0x7ff53c09d2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8be9c40290e24b48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.590+0000 7ff52cc00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T23:00:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.591+0000 7ff52cc00640 1 -- 172.21.15.27:0/816268176 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c01f040 con 0x559b0c368fb0 2024-04-26T23:00:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.592+0000 7ff52cc00640 1 -- 172.21.15.27:0/816268176 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7ff53c011d40 con 0x559b0c368fb0 2024-04-26T23:00:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.592+0000 7ff54ca34740 1 -- 172.21.15.27:0/816268176 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff53000ebb0 con 0x559b0c368fb0 2024-04-26T23:00:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.593+0000 7ff52cc00640 1 -- 172.21.15.27:0/816268176 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff53c0cf2f0 con 0x559b0c368fb0 2024-04-26T23:00:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.593+0000 7ff54ca34740 10 client.25663 fetch_fsmap learned FSMap version 26 2024-04-26T23:00:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.593+0000 7ff54ca34740 1 -- 172.21.15.27:0/816268176 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b0bfd5210 con 0x559b0c368fb0 2024-04-26T23:00:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.593+0000 7ff52cc00640 1 -- 172.21.15.27:0/816268176 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff53c0a1a60 con 0x559b0c368fb0 2024-04-26T23:00:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.593+0000 7ff52cc00640 1 -- 172.21.15.27:0/816268176 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7ff53c00f040 con 0x559b0c368fb0 2024-04-26T23:00:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.594+0000 7ff54ca34740 10 client.25663 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T23:00:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.594+0000 7ff54ca34740 10 client.25663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:00:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.594+0000 7ff54ca34740 20 client.25663 populate_metadata read hostname 'smithi027' 2024-04-26T23:00:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.594+0000 7ff54ca34740 1 --2- 172.21.15.27:0/816268176 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x559b0c378ea0 0x559b0c37b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.594+0000 7ff54ca34740 4 client.25663 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:00:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.594+0000 7ff54ca34740 1 -- 172.21.15.27:0/816268176 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff53000ebb0 con 0x559b0c378ea0 2024-04-26T23:00:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.594+0000 7ff542a00640 1 --2- 172.21.15.27:0/816268176 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x559b0c378ea0 0x559b0c37b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.595+0000 7ff542a00640 1 --2- 172.21.15.27:0/816268176 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x559b0c378ea0 0x559b0c37b290 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42417c2129740f19 server_cookie=4818ddff90cb3ae8 in_seq=0 out_seq=0 2024-04-26T23:00:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.595+0000 7ff52cc00640 10 client.25663 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:00:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.596+0000 7ff52cc00640 1 -- 172.21.15.27:0/816268176 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7ff53000ebb0 con 0x559b0c378ea0 2024-04-26T23:00:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.596+0000 7ff52cc00640 10 client.25663 handle_command_reply: tid=0 2024-04-26T23:00:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.596+0000 7ff54ca34740 1 client.25663 shutdown 2024-04-26T23:00:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.597+0000 7ff54ca34740 1 -- 172.21.15.27:0/816268176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 msgr2=0x559b0c366a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.597+0000 7ff54ca34740 1 --2- 172.21.15.27:0/816268176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c366a40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff53c09d3b0 tx=0x7ff53c09d2e0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.598+0000 7ff54ca34740 1 -- 172.21.15.27:0/816268176 shutdown_connections 2024-04-26T23:00:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.598+0000 7ff54ca34740 1 --2- 172.21.15.27:0/816268176 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x559b0c378ea0 0x559b0c37b290 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.598+0000 7ff52cc00640 0 client.25663 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:00:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.598+0000 7ff54ca34740 1 --2- 172.21.15.27:0/816268176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c366a40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.598+0000 7ff54ca34740 1 --2- 172.21.15.27:0/816268176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 0x559b0c366500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.598+0000 7ff54ca34740 1 --2- 172.21.15.27:0/816268176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35ddd0 0x559b0c365fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.598+0000 7ff54ca34740 1 -- 172.21.15.27:0/816268176 >> 172.21.15.27:0/816268176 conn(0x559b0c35a2e0 msgr2=0x559b0c360010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.598+0000 7ff54ca34740 1 -- 172.21.15.27:0/816268176 shutdown_connections 2024-04-26T23:00:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.598+0000 7ff54ca34740 1 -- 172.21.15.27:0/816268176 wait complete. 2024-04-26T23:00:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.598+0000 7ff54ca34740 20 client.25663 trim_cache size 0 max 16384 2024-04-26T23:00:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.632+0000 7ff54ca34740 1 Processor -- start 2024-04-26T23:00:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.632+0000 7ff54ca34740 1 -- start start 2024-04-26T23:00:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.632+0000 7ff54ca34740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35ddd0 0x559b0c366860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.632+0000 7ff54ca34740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35e6f0 0x559b0c360790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.632+0000 7ff54ca34740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c367df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.632+0000 7ff54ca34740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff524012a80 con 0x559b0c35ddd0 2024-04-26T23:00:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.632+0000 7ff54ca34740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff524010350 con 0x559b0c35e6f0 2024-04-26T23:00:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.632+0000 7ff54ca34740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff52400e1a0 con 0x559b0c368fb0 2024-04-26T23:00:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.633+0000 7ff542000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35e6f0 0x559b0c360790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.633+0000 7ff542000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35e6f0 0x559b0c360790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:54230/0 (socket says 172.21.15.27:54230) 2024-04-26T23:00:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.633+0000 7ff542000640 1 -- 172.21.15.27:0/1267874682 learned_addr learned my addr 172.21.15.27:0/1267874682 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.633+0000 7ff543400640 1 --2- 172.21.15.27:0/1267874682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c367df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.633+0000 7ff542a00640 1 --2- 172.21.15.27:0/1267874682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35ddd0 0x559b0c366860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.633+0000 7ff543400640 1 -- 172.21.15.27:0/1267874682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35e6f0 msgr2=0x559b0c360790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.634+0000 7ff543400640 1 --2- 172.21.15.27:0/1267874682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35e6f0 0x559b0c360790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.634+0000 7ff543400640 1 -- 172.21.15.27:0/1267874682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35ddd0 msgr2=0x559b0c366860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.634+0000 7ff543400640 1 --2- 172.21.15.27:0/1267874682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35ddd0 0x559b0c366860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.634+0000 7ff543400640 1 -- 172.21.15.27:0/1267874682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b0c37b600 con 0x559b0c368fb0 2024-04-26T23:00:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.634+0000 7ff543400640 1 --2- 172.21.15.27:0/1267874682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c367df0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0a5df0 tx=0x7ff53c09d830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4623d3779b929596 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.634+0000 7ff52f400640 1 -- 172.21.15.27:0/1267874682 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c01f040 con 0x559b0c368fb0 2024-04-26T23:00:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.635+0000 7ff52f400640 1 -- 172.21.15.27:0/1267874682 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff53c0a6320 con 0x559b0c368fb0 2024-04-26T23:00:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.635+0000 7ff52f400640 1 -- 172.21.15.27:0/1267874682 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c09ce30 con 0x559b0c368fb0 2024-04-26T23:00:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.636+0000 7ff54ca34740 1 -- 172.21.15.27:0/1267874682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 msgr2=0x559b0c367df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.636+0000 7ff54ca34740 1 --2- 172.21.15.27:0/1267874682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c367df0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0a5df0 tx=0x7ff53c09d830 comp rx=0 tx=0).stop 2024-04-26T23:00:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.636+0000 7ff54ca34740 1 -- 172.21.15.27:0/1267874682 shutdown_connections 2024-04-26T23:00:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.636+0000 7ff54ca34740 1 --2- 172.21.15.27:0/1267874682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35e6f0 0x559b0c360790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.636+0000 7ff54ca34740 1 --2- 172.21.15.27:0/1267874682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c367df0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.636+0000 7ff54ca34740 1 --2- 172.21.15.27:0/1267874682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35ddd0 0x559b0c366860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.636+0000 7ff54ca34740 1 -- 172.21.15.27:0/1267874682 >> 172.21.15.27:0/1267874682 conn(0x559b0c35a2e0 msgr2=0x559b0c3676e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.636+0000 7ff54ca34740 1 -- 172.21.15.27:0/1267874682 shutdown_connections 2024-04-26T23:00:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.636+0000 7ff54ca34740 1 -- 172.21.15.27:0/1267874682 wait complete. 2024-04-26T23:00:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.637+0000 7ff54ca34740 1 -- start start 2024-04-26T23:00:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.637+0000 7ff54ca34740 1 Processor -- start 2024-04-26T23:00:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.638+0000 7ff54ca34740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35ddd0 0x559b0c371340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.638+0000 7ff54ca34740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 0x559b0c3628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.638+0000 7ff54ca34740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c362e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.638+0000 7ff54ca34740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff52400e810 con 0x559b0c35e6f0 2024-04-26T23:00:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.638+0000 7ff54ca34740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff52400fc80 con 0x559b0c35ddd0 2024-04-26T23:00:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.638+0000 7ff54ca34740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff52400eee0 con 0x559b0c368fb0 2024-04-26T23:00:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.638+0000 7ff54ca34740 10 client.0 _ll_register_callbacks cb 0x7ff53c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:00:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.639+0000 7ff543400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c362e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.639+0000 7ff543400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c362e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:38398/0 (socket says 172.21.15.27:38398) 2024-04-26T23:00:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.639+0000 7ff543400640 1 -- 172.21.15.27:0/2853403044 learned_addr learned my addr 172.21.15.27:0/2853403044 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.639+0000 7ff542000640 1 --2- 172.21.15.27:0/2853403044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 0x559b0c3628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.639+0000 7ff542a00640 1 --2- 172.21.15.27:0/2853403044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35ddd0 0x559b0c371340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.640+0000 7ff543400640 1 -- 172.21.15.27:0/2853403044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35ddd0 msgr2=0x559b0c371340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.640+0000 7ff543400640 1 --2- 172.21.15.27:0/2853403044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35ddd0 0x559b0c371340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.640+0000 7ff543400640 1 -- 172.21.15.27:0/2853403044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 msgr2=0x559b0c3628f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.640+0000 7ff543400640 1 --2- 172.21.15.27:0/2853403044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 0x559b0c3628f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.640+0000 7ff543400640 1 -- 172.21.15.27:0/2853403044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b0c0a1620 con 0x559b0c368fb0 2024-04-26T23:00:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.640+0000 7ff542a00640 1 --2- 172.21.15.27:0/2853403044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35ddd0 0x559b0c371340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:00:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.640+0000 7ff543400640 1 --2- 172.21.15.27:0/2853403044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c362e30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00c640 tx=0x7ff53c09d1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c4745ba254b0342 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.640+0000 7ff52cc00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T23:00:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.641+0000 7ff52cc00640 1 -- 172.21.15.27:0/2853403044 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0aa050 con 0x559b0c368fb0 2024-04-26T23:00:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.641+0000 7ff52cc00640 1 -- 172.21.15.27:0/2853403044 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7ff53c00cd20 con 0x559b0c368fb0 2024-04-26T23:00:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.641+0000 7ff54ca34740 1 -- 172.21.15.27:0/2853403044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff53c0a6320 con 0x559b0c368fb0 2024-04-26T23:00:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.641+0000 7ff52cc00640 1 -- 172.21.15.27:0/2853403044 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff53c082050 con 0x559b0c368fb0 2024-04-26T23:00:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.641+0000 7ff54ca34740 10 client.25669 fetch_fsmap learned FSMap version 26 2024-04-26T23:00:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.641+0000 7ff54ca34740 1 -- 172.21.15.27:0/2853403044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b0c27dcb0 con 0x559b0c368fb0 2024-04-26T23:00:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.641+0000 7ff52cc00640 1 -- 172.21.15.27:0/2853403044 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff53c0a6320 con 0x559b0c368fb0 2024-04-26T23:00:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.642+0000 7ff52cc00640 1 -- 172.21.15.27:0/2853403044 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7ff53c0a2c60 con 0x559b0c368fb0 2024-04-26T23:00:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.642+0000 7ff54ca34740 10 client.25669 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T23:00:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.642+0000 7ff54ca34740 10 client.25669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:00:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.642+0000 7ff54ca34740 20 client.25669 populate_metadata read hostname 'smithi027' 2024-04-26T23:00:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.642+0000 7ff54ca34740 1 --2- 172.21.15.27:0/2853403044 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x559b0c378ea0 0x559b0c3e1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.642+0000 7ff54ca34740 4 client.25669 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:00:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.642+0000 7ff54ca34740 1 -- 172.21.15.27:0/2853403044 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b0c2d0f90 con 0x559b0c378ea0 2024-04-26T23:00:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.642+0000 7ff542a00640 1 --2- 172.21.15.27:0/2853403044 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x559b0c378ea0 0x559b0c3e1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.644+0000 7ff542a00640 1 --2- 172.21.15.27:0/2853403044 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x559b0c378ea0 0x559b0c3e1370 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcfcaa0b6c0fd8de server_cookie=ee06860ea47ff1fb in_seq=0 out_seq=0 2024-04-26T23:00:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.644+0000 7ff52cc00640 10 client.25669 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:00:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.663+0000 7ff52cc00640 1 -- 172.21.15.27:0/2853403044 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b0c2d0f90 con 0x559b0c378ea0 2024-04-26T23:00:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.663+0000 7ff52cc00640 10 client.25669 handle_command_reply: tid=0 2024-04-26T23:00:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.664+0000 7ff54ca34740 1 client.25669 shutdown 2024-04-26T23:00:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.664+0000 7ff54ca34740 1 -- 172.21.15.27:0/2853403044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 msgr2=0x559b0c362e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.664+0000 7ff54ca34740 1 --2- 172.21.15.27:0/2853403044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c362e30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00c640 tx=0x7ff53c09d1f0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.664+0000 7ff54ca34740 1 -- 172.21.15.27:0/2853403044 shutdown_connections 2024-04-26T23:00:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.664+0000 7ff54ca34740 1 --2- 172.21.15.27:0/2853403044 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x559b0c378ea0 0x559b0c3e1370 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.664+0000 7ff54ca34740 1 --2- 172.21.15.27:0/2853403044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0c368fb0 0x559b0c362e30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.664+0000 7ff54ca34740 1 --2- 172.21.15.27:0/2853403044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b0c35e6f0 0x559b0c3628f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.664+0000 7ff54ca34740 1 --2- 172.21.15.27:0/2853403044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559b0c35ddd0 0x559b0c371340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.664+0000 7ff54ca34740 1 -- 172.21.15.27:0/2853403044 >> 172.21.15.27:0/2853403044 conn(0x559b0c35a2e0 msgr2=0x559b0c367e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.664+0000 7ff52cc00640 0 client.25669 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:00:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.665+0000 7ff54ca34740 1 -- 172.21.15.27:0/2853403044 shutdown_connections 2024-04-26T23:00:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.665+0000 7ff54ca34740 1 -- 172.21.15.27:0/2853403044 wait complete. 2024-04-26T23:00:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.665+0000 7ff54ca34740 20 client.25669 trim_cache size 0 max 16384 2024-04-26T23:00:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T23:00:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T23:00:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T23:00:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T23:00:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.666+0000 7ff549c00640 1 -- 172.21.15.27:0/1842485647 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7ff5280a3f20 msgr2=0x7ff5280a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.666+0000 7ff549c00640 1 --2- 172.21.15.27:0/1842485647 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7ff5280a3f20 0x7ff5280a63e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff534009850 tx=0x7ff534000bc0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.666+0000 7ff549c00640 1 -- 172.21.15.27:0/1842485647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5441137f0 msgr2=0x7ff54419f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.666+0000 7ff549c00640 1 --2- 172.21.15.27:0/1842485647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5441137f0 0x7ff54419f630 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff53c005fd0 tx=0x7ff53c00edd0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.667+0000 7ff549c00640 1 -- 172.21.15.27:0/1842485647 shutdown_connections 2024-04-26T23:00:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.667+0000 7ff549c00640 1 --2- 172.21.15.27:0/1842485647 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7ff5280a3f20 0x7ff5280a63e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.667+0000 7ff549c00640 1 --2- 172.21.15.27:0/1842485647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff544077780 0x7ff5441a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.667+0000 7ff549c00640 1 --2- 172.21.15.27:0/1842485647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5441137f0 0x7ff54419f630 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.667+0000 7ff549c00640 1 --2- 172.21.15.27:0/1842485647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff544076e60 0x7ff5441a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.667+0000 7ff549c00640 1 -- 172.21.15.27:0/1842485647 >> 172.21.15.27:0/1842485647 conn(0x7ff5441005e0 msgr2=0x7ff544114460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.669+0000 7ff549c00640 1 -- 172.21.15.27:0/1842485647 shutdown_connections 2024-04-26T23:00:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:37.669+0000 7ff549c00640 1 -- 172.21.15.27:0/1842485647 wait complete. 2024-04-26T23:00:39.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:38 smithi027 ceph-mon[28034]: pgmap v64: 129 pgs: 129 active+clean; 28 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 917 KiB/s wr, 28 op/s 2024-04-26T23:00:39.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:38 smithi080 ceph-mon[32324]: pgmap v64: 129 pgs: 129 active+clean; 28 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 917 KiB/s wr, 28 op/s 2024-04-26T23:00:39.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:38 smithi094 ceph-mon[32889]: pgmap v64: 129 pgs: 129 active+clean; 28 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 917 KiB/s wr, 28 op/s 2024-04-26T23:00:41.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:40 smithi027 ceph-mon[28034]: pgmap v65: 129 pgs: 129 active+clean; 28 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 444 KiB/s wr, 16 op/s 2024-04-26T23:00:41.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:40 smithi080 ceph-mon[32324]: pgmap v65: 129 pgs: 129 active+clean; 28 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 444 KiB/s wr, 16 op/s 2024-04-26T23:00:41.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:40 smithi094 ceph-mon[32889]: pgmap v65: 129 pgs: 129 active+clean; 28 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 444 KiB/s wr, 16 op/s 2024-04-26T23:00:43.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:42 smithi027 ceph-mon[28034]: pgmap v66: 129 pgs: 129 active+clean; 48 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 204 op/s 2024-04-26T23:00:43.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:42 smithi080 ceph-mon[32324]: pgmap v66: 129 pgs: 129 active+clean; 48 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 204 op/s 2024-04-26T23:00:43.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:42 smithi094 ceph-mon[32889]: pgmap v66: 129 pgs: 129 active+clean; 48 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 204 op/s 2024-04-26T23:00:45.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:44 smithi027 ceph-mon[28034]: pgmap v67: 129 pgs: 129 active+clean; 50 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-04-26T23:00:45.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:44 smithi080 ceph-mon[32324]: pgmap v67: 129 pgs: 129 active+clean; 50 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-04-26T23:00:45.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:44 smithi094 ceph-mon[32889]: pgmap v67: 129 pgs: 129 active+clean; 50 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-04-26T23:00:47.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:46 smithi027 ceph-mon[28034]: pgmap v68: 129 pgs: 129 active+clean; 50 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-04-26T23:00:47.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:46 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:00:47.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:46 smithi080 ceph-mon[32324]: pgmap v68: 129 pgs: 129 active+clean; 50 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-04-26T23:00:47.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:46 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:00:47.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:46 smithi094 ceph-mon[32889]: pgmap v68: 129 pgs: 129 active+clean; 50 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-04-26T23:00:47.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:46 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:00:49.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:48 smithi027 ceph-mon[28034]: pgmap v69: 129 pgs: 129 active+clean; 54 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.4 MiB/s wr, 245 op/s 2024-04-26T23:00:49.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:48 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:00:49.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:48 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:00:49.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:48 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:00:49.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:48 smithi080 ceph-mon[32324]: pgmap v69: 129 pgs: 129 active+clean; 54 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.4 MiB/s wr, 245 op/s 2024-04-26T23:00:49.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:48 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:00:49.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:48 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:00:49.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:48 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:00:49.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:48 smithi094 ceph-mon[32889]: pgmap v69: 129 pgs: 129 active+clean; 54 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.4 MiB/s wr, 245 op/s 2024-04-26T23:00:49.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:48 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:00:49.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:48 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:00:49.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:48 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:00:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.820+0000 7f8585c00640 1 -- 172.21.15.27:0/279476310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85800755f0 msgr2=0x7f8580113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.820+0000 7f8585c00640 1 --2- 172.21.15.27:0/279476310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85800755f0 0x7f8580113200 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f856c009900 tx=0x7f856c02a480 comp rx=0 tx=0).stop 2024-04-26T23:00:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.820+0000 7f8585c00640 1 -- 172.21.15.27:0/279476310 shutdown_connections 2024-04-26T23:00:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.820+0000 7f8585c00640 1 --2- 172.21.15.27:0/279476310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8580113740 0x7f8580115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.821+0000 7f8585c00640 1 --2- 172.21.15.27:0/279476310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85800755f0 0x7f8580113200 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.821+0000 7f8585c00640 1 --2- 172.21.15.27:0/279476310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8580074cd0 0x7f85800750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.821+0000 7f8585c00640 1 -- 172.21.15.27:0/279476310 >> 172.21.15.27:0/279476310 conn(0x7f85800fe260 msgr2=0x7f8580100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.821+0000 7f8585c00640 1 -- 172.21.15.27:0/279476310 shutdown_connections 2024-04-26T23:00:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.821+0000 7f8585c00640 1 -- 172.21.15.27:0/279476310 wait complete. 2024-04-26T23:00:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.822+0000 7f8585c00640 1 Processor -- start 2024-04-26T23:00:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.822+0000 7f8585c00640 1 -- start start 2024-04-26T23:00:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.822+0000 7f8585c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8580074cd0 0x7f858019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.822+0000 7f8585c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85800755f0 0x7f858019fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.822+0000 7f8585c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8580113740 0x7f85801a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.822+0000 7f8585c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8580119020 con 0x7f8580074cd0 2024-04-26T23:00:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.822+0000 7f8585c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8580118ea0 con 0x7f85800755f0 2024-04-26T23:00:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.822+0000 7f8585c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f85801191a0 con 0x7f8580113740 2024-04-26T23:00:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.823+0000 7f857fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8580074cd0 0x7f858019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.823+0000 7f857f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85800755f0 0x7f858019fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.823+0000 7f857f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85800755f0 0x7f858019fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:49646/0 (socket says 172.21.15.27:49646) 2024-04-26T23:00:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.823+0000 7f857f400640 1 -- 172.21.15.27:0/586638061 learned_addr learned my addr 172.21.15.27:0/586638061 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.823+0000 7f8585200640 1 --2- 172.21.15.27:0/586638061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8580113740 0x7f85801a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.823+0000 7f857fe00640 1 -- 172.21.15.27:0/586638061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8580113740 msgr2=0x7f85801a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.823+0000 7f857fe00640 1 --2- 172.21.15.27:0/586638061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8580113740 0x7f85801a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.823+0000 7f857fe00640 1 -- 172.21.15.27:0/586638061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85800755f0 msgr2=0x7f858019fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.823+0000 7f857fe00640 1 --2- 172.21.15.27:0/586638061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85800755f0 0x7f858019fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.823+0000 7f857fe00640 1 -- 172.21.15.27:0/586638061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85801a4580 con 0x7f8580074cd0 2024-04-26T23:00:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.824+0000 7f857fe00640 1 --2- 172.21.15.27:0/586638061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8580074cd0 0x7f858019f5e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8570005fd0 tx=0x7f857000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.824+0000 7f857cc00640 1 -- 172.21.15.27:0/586638061 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570017070 con 0x7f8580074cd0 2024-04-26T23:00:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.824+0000 7f8585c00640 1 -- 172.21.15.27:0/586638061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85801a4870 con 0x7f8580074cd0 2024-04-26T23:00:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.824+0000 7f857cc00640 1 -- 172.21.15.27:0/586638061 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8570009c30 con 0x7f8580074cd0 2024-04-26T23:00:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.824+0000 7f857cc00640 1 -- 172.21.15.27:0/586638061 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857000f620 con 0x7f8580074cd0 2024-04-26T23:00:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.824+0000 7f8585c00640 1 -- 172.21.15.27:0/586638061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85801ac100 con 0x7f8580074cd0 2024-04-26T23:00:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.828+0000 7f858a1ea740 1 Processor -- start 2024-04-26T23:00:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.828+0000 7f857cc00640 1 -- 172.21.15.27:0/586638061 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f857001a070 con 0x7f8580074cd0 2024-04-26T23:00:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.828+0000 7f858a1ea740 1 -- start start 2024-04-26T23:00:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.829+0000 7f858a1ea740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b77fe0 0x556eb2b783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.829+0000 7f858a1ea740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b78900 0x556eb2b82c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.829+0000 7f858a1ea740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b831c0 0x556eb2b855b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.829+0000 7f858a1ea740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556eb2add380 con 0x556eb2b831c0 2024-04-26T23:00:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.829+0000 7f858a1ea740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x556eb2a79c30 con 0x556eb2b77fe0 2024-04-26T23:00:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.829+0000 7f858a1ea740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556eb2ad9340 con 0x556eb2b78900 2024-04-26T23:00:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.829+0000 7f857f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b831c0 0x556eb2b855b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.829+0000 7f8585200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b77fe0 0x556eb2b783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.829+0000 7f857fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b78900 0x556eb2b82c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.829+0000 7f8585200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b77fe0 0x556eb2b783c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:49662/0 (socket says 172.21.15.27:49662) 2024-04-26T23:00:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.829+0000 7f8585200640 1 -- 172.21.15.27:0/3547978572 learned_addr learned my addr 172.21.15.27:0/3547978572 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.830+0000 7f8585200640 1 -- 172.21.15.27:0/3547978572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b78900 msgr2=0x556eb2b82c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.830+0000 7f8585200640 1 --2- 172.21.15.27:0/3547978572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b78900 0x556eb2b82c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.830+0000 7f8585200640 1 -- 172.21.15.27:0/3547978572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b831c0 msgr2=0x556eb2b855b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.830+0000 7f8585200640 1 --2- 172.21.15.27:0/3547978572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b831c0 0x556eb2b855b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.830+0000 7f8585200640 1 -- 172.21.15.27:0/3547978572 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eb2a9b940 con 0x556eb2b77fe0 2024-04-26T23:00:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.830+0000 7f8585200640 1 --2- 172.21.15.27:0/3547978572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b77fe0 0x556eb2b783c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8578009f40 tx=0x7f8578010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6905c97051ec0b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.830+0000 7f857cc00640 1 --2- 172.21.15.27:0/586638061 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f85640a3d80 0x7f85640a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.831+0000 7f8575600640 1 -- 172.21.15.27:0/3547978572 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578016070 con 0x556eb2b77fe0 2024-04-26T23:00:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.831+0000 7f8575600640 1 -- 172.21.15.27:0/3547978572 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8578004d20 con 0x556eb2b77fe0 2024-04-26T23:00:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.831+0000 7f8575600640 1 -- 172.21.15.27:0/3547978572 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85780127f0 con 0x556eb2b77fe0 2024-04-26T23:00:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.831+0000 7f857cc00640 1 -- 172.21.15.27:0/586638061 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f85700d0200 con 0x7f8580074cd0 2024-04-26T23:00:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.832+0000 7f857fe00640 1 --2- 172.21.15.27:0/586638061 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f85640a3d80 0x7f85640a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.832+0000 7f858a1ea740 1 -- 172.21.15.27:0/3547978572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b77fe0 msgr2=0x556eb2b783c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.832+0000 7f858a1ea740 1 --2- 172.21.15.27:0/3547978572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b77fe0 0x556eb2b783c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8578009f40 tx=0x7f8578010800 comp rx=0 tx=0).stop 2024-04-26T23:00:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.832+0000 7f858a1ea740 1 -- 172.21.15.27:0/3547978572 shutdown_connections 2024-04-26T23:00:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.832+0000 7f858a1ea740 1 --2- 172.21.15.27:0/3547978572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b831c0 0x556eb2b855b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.832+0000 7f858a1ea740 1 --2- 172.21.15.27:0/3547978572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b78900 0x556eb2b82c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.832+0000 7f858a1ea740 1 --2- 172.21.15.27:0/3547978572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b77fe0 0x556eb2b783c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.832+0000 7f858a1ea740 1 -- 172.21.15.27:0/3547978572 >> 172.21.15.27:0/3547978572 conn(0x556eb2b744f0 msgr2=0x556eb2b748d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.832+0000 7f858a1ea740 1 -- 172.21.15.27:0/3547978572 shutdown_connections 2024-04-26T23:00:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.832+0000 7f858a1ea740 1 -- 172.21.15.27:0/3547978572 wait complete. 2024-04-26T23:00:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.832+0000 7f857fe00640 1 --2- 172.21.15.27:0/586638061 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f85640a3d80 0x7f85640a6240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x556eb2b79580 tx=0x7f85700d3cf0 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.833+0000 7f858a1ea740 1 -- start start 2024-04-26T23:00:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.833+0000 7f858a1ea740 1 Processor -- start 2024-04-26T23:00:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.833+0000 7f858a1ea740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b77fe0 0x556eb2b800f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.833+0000 7f858a1ea740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b78900 0x556eb2b80630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.833+0000 7f858a1ea740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b831c0 0x556eb2b80b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.833+0000 7f858a1ea740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556eb2b85df0 con 0x556eb2b77fe0 2024-04-26T23:00:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.833+0000 7f858a1ea740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x556eb2a64c60 con 0x556eb2b78900 2024-04-26T23:00:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.833+0000 7f858a1ea740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556eb2b85f70 con 0x556eb2b831c0 2024-04-26T23:00:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.834+0000 7f858a1ea740 10 client.0 _ll_register_callbacks cb 0x556eb2a90e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.834+0000 7f8585200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b77fe0 0x556eb2b800f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.834+0000 7f857f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b78900 0x556eb2b80630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.834+0000 7f8585200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b77fe0 0x556eb2b800f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53262/0 (socket says 172.21.15.27:53262) 2024-04-26T23:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.834+0000 7f8585200640 1 -- 172.21.15.27:0/3243869580 learned_addr learned my addr 172.21.15.27:0/3243869580 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.834+0000 7f8585200640 1 --2- 172.21.15.27:0/3243869580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b831c0 0x556eb2b80b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.834+0000 7f8585200640 1 -- 172.21.15.27:0/3243869580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b831c0 msgr2=0x556eb2b80b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.834+0000 7f8585200640 1 --2- 172.21.15.27:0/3243869580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b831c0 0x556eb2b80b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.834+0000 7f8585200640 1 -- 172.21.15.27:0/3243869580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b78900 msgr2=0x556eb2b80630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.834+0000 7f8585200640 1 --2- 172.21.15.27:0/3243869580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b78900 0x556eb2b80630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.834+0000 7f8585200640 1 -- 172.21.15.27:0/3243869580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eb2a73510 con 0x556eb2b77fe0 2024-04-26T23:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.835+0000 7f8585200640 1 --2- 172.21.15.27:0/3243869580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b77fe0 0x556eb2b800f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f85780124b0 tx=0x7f8578009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13bed6cfdd16de08 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.835+0000 7f856a000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T23:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.835+0000 7f856a000640 1 -- 172.21.15.27:0/3243869580 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578016070 con 0x556eb2b77fe0 2024-04-26T23:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.835+0000 7f856a000640 1 -- 172.21.15.27:0/3243869580 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f8578005510 con 0x556eb2b77fe0 2024-04-26T23:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.835+0000 7f858a1ea740 1 -- 172.21.15.27:0/3243869580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8578004d20 con 0x556eb2b77fe0 2024-04-26T23:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.836+0000 7f856a000640 1 -- 172.21.15.27:0/3243869580 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8578012550 con 0x556eb2b77fe0 2024-04-26T23:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.836+0000 7f858a1ea740 10 client.15528 fetch_fsmap learned FSMap version 26 2024-04-26T23:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.836+0000 7f858a1ea740 1 -- 172.21.15.27:0/3243869580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556eb27ef210 con 0x556eb2b77fe0 2024-04-26T23:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.836+0000 7f856a000640 1 -- 172.21.15.27:0/3243869580 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8578032070 con 0x556eb2b77fe0 2024-04-26T23:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.836+0000 7f856a000640 1 -- 172.21.15.27:0/3243869580 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f857800f040 con 0x556eb2b77fe0 2024-04-26T23:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.836+0000 7f858a1ea740 10 client.15528 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T23:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.836+0000 7f858a1ea740 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.836+0000 7f858a1ea740 20 client.15528 populate_metadata read hostname 'smithi027' 2024-04-26T23:00:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.836+0000 7f858a1ea740 1 --2- 172.21.15.27:0/3243869580 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x556eb2b92df0 0x556eb2b951e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.836+0000 7f858a1ea740 4 client.15528 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:00:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.836+0000 7f858a1ea740 1 -- 172.21.15.27:0/3243869580 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eb2aeaf90 con 0x556eb2b92df0 2024-04-26T23:00:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.837+0000 7f857f400640 1 --2- 172.21.15.27:0/3243869580 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x556eb2b92df0 0x556eb2b951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.837+0000 7f857f400640 1 --2- 172.21.15.27:0/3243869580 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x556eb2b92df0 0x556eb2b951e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37d6bad305aec3d server_cookie=31c38a42f246f6b6 in_seq=0 out_seq=0 2024-04-26T23:00:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.837+0000 7f856a000640 10 client.15528 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:00:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.838+0000 7f856a000640 1 -- 172.21.15.27:0/3243869580 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556eb2aeaf90 con 0x556eb2b92df0 2024-04-26T23:00:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.838+0000 7f856a000640 10 client.15528 handle_command_reply: tid=0 2024-04-26T23:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.838+0000 7f858a1ea740 1 client.15528 shutdown 2024-04-26T23:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.838+0000 7f858a1ea740 1 -- 172.21.15.27:0/3243869580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b77fe0 msgr2=0x556eb2b800f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.838+0000 7f858a1ea740 1 --2- 172.21.15.27:0/3243869580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b77fe0 0x556eb2b800f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f85780124b0 tx=0x7f8578009920 comp rx=0 tx=0).stop 2024-04-26T23:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.839+0000 7f858a1ea740 1 -- 172.21.15.27:0/3243869580 shutdown_connections 2024-04-26T23:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.839+0000 7f858a1ea740 1 --2- 172.21.15.27:0/3243869580 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x556eb2b92df0 0x556eb2b951e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.839+0000 7f858a1ea740 1 --2- 172.21.15.27:0/3243869580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b78900 0x556eb2b80630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.839+0000 7f858a1ea740 1 --2- 172.21.15.27:0/3243869580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b831c0 0x556eb2b80b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.839+0000 7f858a1ea740 1 --2- 172.21.15.27:0/3243869580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b77fe0 0x556eb2b800f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.839+0000 7f856a000640 0 client.15528 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.839+0000 7f858a1ea740 1 -- 172.21.15.27:0/3243869580 >> 172.21.15.27:0/3243869580 conn(0x556eb2b744f0 msgr2=0x556eb2b7a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.839+0000 7f858a1ea740 1 -- 172.21.15.27:0/3243869580 shutdown_connections 2024-04-26T23:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.839+0000 7f858a1ea740 1 -- 172.21.15.27:0/3243869580 wait complete. 2024-04-26T23:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.839+0000 7f858a1ea740 20 client.15528 trim_cache size 0 max 16384 2024-04-26T23:00:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.859+0000 7f858a1ea740 1 Processor -- start 2024-04-26T23:00:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.859+0000 7f858a1ea740 1 -- start start 2024-04-26T23:00:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.860+0000 7f858a1ea740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b77fe0 0x556eb2b8ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.860+0000 7f858a1ea740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b78900 0x556eb2b822e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.860+0000 7f858a1ea740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b831c0 0x556eb2b7f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.860+0000 7f858a1ea740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b8d20 con 0x556eb2b77fe0 2024-04-26T23:00:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.860+0000 7f858a1ea740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b65f0 con 0x556eb2b831c0 2024-04-26T23:00:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.860+0000 7f858a1ea740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b4440 con 0x556eb2b78900 2024-04-26T23:00:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.860+0000 7f8585200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b77fe0 0x556eb2b8ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.860+0000 7f857f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b78900 0x556eb2b822e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.860+0000 7f8585200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b77fe0 0x556eb2b8ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53264/0 (socket says 172.21.15.27:53264) 2024-04-26T23:00:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.860+0000 7f857f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b78900 0x556eb2b822e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:39528/0 (socket says 172.21.15.27:39528) 2024-04-26T23:00:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.860+0000 7f8585200640 1 -- 172.21.15.27:0/136734473 learned_addr learned my addr 172.21.15.27:0/136734473 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.861+0000 7f8585200640 1 --2- 172.21.15.27:0/136734473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b831c0 0x556eb2b7f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.861+0000 7f857f400640 1 -- 172.21.15.27:0/136734473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b831c0 msgr2=0x556eb2b7f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.861+0000 7f857f400640 1 --2- 172.21.15.27:0/136734473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b831c0 0x556eb2b7f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.861+0000 7f857f400640 1 -- 172.21.15.27:0/136734473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b77fe0 msgr2=0x556eb2b8ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.861+0000 7f857f400640 1 --2- 172.21.15.27:0/136734473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b77fe0 0x556eb2b8ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.861+0000 7f857f400640 1 -- 172.21.15.27:0/136734473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eb28bb620 con 0x556eb2b78900 2024-04-26T23:00:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.861+0000 7f8585200640 1 --2- 172.21.15.27:0/136734473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b831c0 0x556eb2b7f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:00:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.861+0000 7f857f400640 1 --2- 172.21.15.27:0/136734473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b78900 0x556eb2b822e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f856c000c00 tx=0x7f856c02ac00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88827dedbc2b46b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.861+0000 7f8574c00640 1 -- 172.21.15.27:0/136734473 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c035040 con 0x556eb2b78900 2024-04-26T23:00:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.861+0000 7f8574c00640 1 -- 172.21.15.27:0/136734473 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f856c0054d0 con 0x556eb2b78900 2024-04-26T23:00:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.861+0000 7f8574c00640 1 -- 172.21.15.27:0/136734473 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c004c70 con 0x556eb2b78900 2024-04-26T23:00:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.863+0000 7f858a1ea740 1 -- 172.21.15.27:0/136734473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b78900 msgr2=0x556eb2b822e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.863+0000 7f858a1ea740 1 --2- 172.21.15.27:0/136734473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b78900 0x556eb2b822e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f856c000c00 tx=0x7f856c02ac00 comp rx=0 tx=0).stop 2024-04-26T23:00:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.863+0000 7f858a1ea740 1 -- 172.21.15.27:0/136734473 shutdown_connections 2024-04-26T23:00:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.863+0000 7f858a1ea740 1 --2- 172.21.15.27:0/136734473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b831c0 0x556eb2b7f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.863+0000 7f858a1ea740 1 --2- 172.21.15.27:0/136734473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b78900 0x556eb2b822e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.863+0000 7f858a1ea740 1 --2- 172.21.15.27:0/136734473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b77fe0 0x556eb2b8ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.863+0000 7f858a1ea740 1 -- 172.21.15.27:0/136734473 >> 172.21.15.27:0/136734473 conn(0x556eb2b744f0 msgr2=0x556eb2b7a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.863+0000 7f858a1ea740 1 -- 172.21.15.27:0/136734473 shutdown_connections 2024-04-26T23:00:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.863+0000 7f858a1ea740 1 -- 172.21.15.27:0/136734473 wait complete. 2024-04-26T23:00:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.863+0000 7f858a1ea740 1 -- start start 2024-04-26T23:00:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.864+0000 7f858a1ea740 1 Processor -- start 2024-04-26T23:00:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.864+0000 7f858a1ea740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b77fe0 0x556eb2b8b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.864+0000 7f858a1ea740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b78900 0x556eb2b867c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.864+0000 7f858a1ea740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b831c0 0x556eb2b86d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.865+0000 7f858a1ea740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b4ab0 con 0x556eb2b831c0 2024-04-26T23:00:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.865+0000 7f858a1ea740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b5f20 con 0x556eb2b78900 2024-04-26T23:00:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.865+0000 7f858a1ea740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b5180 con 0x556eb2b77fe0 2024-04-26T23:00:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.865+0000 7f858a1ea740 10 client.0 _ll_register_callbacks cb 0x7f856c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:00:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.865+0000 7f857f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b78900 0x556eb2b867c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.865+0000 7f8585200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b77fe0 0x556eb2b8b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.865+0000 7f857f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b78900 0x556eb2b867c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:49680/0 (socket says 172.21.15.27:49680) 2024-04-26T23:00:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.865+0000 7f857f400640 1 -- 172.21.15.27:0/2463057206 learned_addr learned my addr 172.21.15.27:0/2463057206 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:00:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.865+0000 7f8585200640 1 --2- 172.21.15.27:0/2463057206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b831c0 0x556eb2b86d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.865+0000 7f8585200640 1 -- 172.21.15.27:0/2463057206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b78900 msgr2=0x556eb2b867c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.865+0000 7f8585200640 1 --2- 172.21.15.27:0/2463057206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b78900 0x556eb2b867c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.865+0000 7f8585200640 1 -- 172.21.15.27:0/2463057206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b831c0 msgr2=0x556eb2b86d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:00:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.865+0000 7f8585200640 1 --2- 172.21.15.27:0/2463057206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b831c0 0x556eb2b86d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.865+0000 7f8585200640 1 -- 172.21.15.27:0/2463057206 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eb2a97cb0 con 0x556eb2b77fe0 2024-04-26T23:00:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.866+0000 7f8585200640 1 --2- 172.21.15.27:0/2463057206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b77fe0 0x556eb2b8b450 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f857801a8e0 tx=0x7f857801c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ed8e25d8687fae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:00:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.866+0000 7f856a000640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T23:00:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.866+0000 7f856a000640 1 -- 172.21.15.27:0/2463057206 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578016070 con 0x556eb2b77fe0 2024-04-26T23:00:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.866+0000 7f858a1ea740 1 -- 172.21.15.27:0/2463057206 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f856c0054d0 con 0x556eb2b77fe0 2024-04-26T23:00:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.866+0000 7f856a000640 1 -- 172.21.15.27:0/2463057206 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f85780128f0 con 0x556eb2b77fe0 2024-04-26T23:00:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.866+0000 7f856a000640 1 -- 172.21.15.27:0/2463057206 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8578012af0 con 0x556eb2b77fe0 2024-04-26T23:00:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.867+0000 7f858a1ea740 10 client.25687 fetch_fsmap learned FSMap version 26 2024-04-26T23:00:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.867+0000 7f858a1ea740 1 -- 172.21.15.27:0/2463057206 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556eb2bf8cf0 con 0x556eb2b77fe0 2024-04-26T23:00:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.867+0000 7f856a000640 1 -- 172.21.15.27:0/2463057206 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f856c0054d0 con 0x556eb2b77fe0 2024-04-26T23:00:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.867+0000 7f856a000640 1 -- 172.21.15.27:0/2463057206 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f8578005960 con 0x556eb2b77fe0 2024-04-26T23:00:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.867+0000 7f858a1ea740 10 client.25687 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T23:00:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.867+0000 7f858a1ea740 10 client.25687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:00:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.867+0000 7f858a1ea740 20 client.25687 populate_metadata read hostname 'smithi027' 2024-04-26T23:00:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.867+0000 7f858a1ea740 1 --2- 172.21.15.27:0/2463057206 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x556eb2bf9410 0x556eb2bfb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:00:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.867+0000 7f858a1ea740 4 client.25687 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:00:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.867+0000 7f858a1ea740 1 -- 172.21.15.27:0/2463057206 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556eb2ae4e00 con 0x556eb2bf9410 2024-04-26T23:00:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.868+0000 7f857f400640 1 --2- 172.21.15.27:0/2463057206 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x556eb2bf9410 0x556eb2bfb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:00:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.868+0000 7f857f400640 1 --2- 172.21.15.27:0/2463057206 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x556eb2bf9410 0x556eb2bfb830 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3275ed38eeafc4e server_cookie=bdbf4197b7de9b0 in_seq=0 out_seq=0 2024-04-26T23:00:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.868+0000 7f856a000640 10 client.25687 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:00:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.877+0000 7f856a000640 1 -- 172.21.15.27:0/2463057206 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556eb2ae4e00 con 0x556eb2bf9410 2024-04-26T23:00:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.877+0000 7f856a000640 10 client.25687 handle_command_reply: tid=0 2024-04-26T23:00:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.877+0000 7f858a1ea740 1 client.25687 shutdown 2024-04-26T23:00:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.877+0000 7f858a1ea740 1 -- 172.21.15.27:0/2463057206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b77fe0 msgr2=0x556eb2b8b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.877+0000 7f858a1ea740 1 --2- 172.21.15.27:0/2463057206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b77fe0 0x556eb2b8b450 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f857801a8e0 tx=0x7f857801c040 comp rx=0 tx=0).stop 2024-04-26T23:00:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.877+0000 7f858a1ea740 1 -- 172.21.15.27:0/2463057206 shutdown_connections 2024-04-26T23:00:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.877+0000 7f858a1ea740 1 --2- 172.21.15.27:0/2463057206 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x556eb2bf9410 0x556eb2bfb830 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.877+0000 7f858a1ea740 1 --2- 172.21.15.27:0/2463057206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556eb2b78900 0x556eb2b867c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.877+0000 7f858a1ea740 1 --2- 172.21.15.27:0/2463057206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eb2b831c0 0x556eb2b86d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.877+0000 7f858a1ea740 1 --2- 172.21.15.27:0/2463057206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556eb2b77fe0 0x556eb2b8b450 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.877+0000 7f856a000640 0 client.25687 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:00:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.877+0000 7f858a1ea740 1 -- 172.21.15.27:0/2463057206 >> 172.21.15.27:0/2463057206 conn(0x556eb2b744f0 msgr2=0x556eb2b800b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.877+0000 7f858a1ea740 1 -- 172.21.15.27:0/2463057206 shutdown_connections 2024-04-26T23:00:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.877+0000 7f858a1ea740 1 -- 172.21.15.27:0/2463057206 wait complete. 2024-04-26T23:00:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.877+0000 7f858a1ea740 20 client.25687 trim_cache size 0 max 16384 2024-04-26T23:00:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T23:00:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T23:00:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T23:00:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T23:00:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.878+0000 7f8585c00640 1 -- 172.21.15.27:0/586638061 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f85640a3d80 msgr2=0x7f85640a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.878+0000 7f8585c00640 1 --2- 172.21.15.27:0/586638061 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f85640a3d80 0x7f85640a6240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x556eb2b79580 tx=0x7f85700d3cf0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.878+0000 7f8585c00640 1 -- 172.21.15.27:0/586638061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8580074cd0 msgr2=0x7f858019f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:00:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.878+0000 7f8585c00640 1 --2- 172.21.15.27:0/586638061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8580074cd0 0x7f858019f5e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8570005fd0 tx=0x7f857000dde0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.878+0000 7f8585c00640 1 -- 172.21.15.27:0/586638061 shutdown_connections 2024-04-26T23:00:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.878+0000 7f8585c00640 1 --2- 172.21.15.27:0/586638061 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f85640a3d80 0x7f85640a6240 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.878+0000 7f8585c00640 1 --2- 172.21.15.27:0/586638061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85800755f0 0x7f858019fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.878+0000 7f8585c00640 1 --2- 172.21.15.27:0/586638061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8580113740 0x7f85801a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.878+0000 7f8585c00640 1 --2- 172.21.15.27:0/586638061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8580074cd0 0x7f858019f5e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:00:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.878+0000 7f8585c00640 1 -- 172.21.15.27:0/586638061 >> 172.21.15.27:0/586638061 conn(0x7f85800fe260 msgr2=0x7f85800ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:00:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.879+0000 7f8585c00640 1 -- 172.21.15.27:0/586638061 shutdown_connections 2024-04-26T23:00:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:00:50.879+0000 7f8585c00640 1 -- 172.21.15.27:0/586638061 wait complete. 2024-04-26T23:00:51.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:50 smithi027 ceph-mon[28034]: pgmap v70: 129 pgs: 129 active+clean; 54 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 2.2 MiB/s wr, 237 op/s 2024-04-26T23:00:51.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:50 smithi080 ceph-mon[32324]: pgmap v70: 129 pgs: 129 active+clean; 54 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 2.2 MiB/s wr, 237 op/s 2024-04-26T23:00:51.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:50 smithi094 ceph-mon[32889]: pgmap v70: 129 pgs: 129 active+clean; 54 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 2.2 MiB/s wr, 237 op/s 2024-04-26T23:00:53.274 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:52 smithi080 ceph-mon[32324]: pgmap v71: 129 pgs: 129 active+clean; 57 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 251 op/s 2024-04-26T23:00:53.274 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:52 smithi080 ceph-mon[32324]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:00:53.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:52 smithi094 ceph-mon[32889]: pgmap v71: 129 pgs: 129 active+clean; 57 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 251 op/s 2024-04-26T23:00:53.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:52 smithi094 ceph-mon[32889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:00:53.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:52 smithi027 ceph-mon[28034]: pgmap v71: 129 pgs: 129 active+clean; 57 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 251 op/s 2024-04-26T23:00:53.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:52 smithi027 ceph-mon[28034]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:00:55.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:54 smithi080 ceph-mon[32324]: pgmap v72: 129 pgs: 129 active+clean; 57 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 854 KiB/s wr, 64 op/s 2024-04-26T23:00:55.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:54 smithi094 ceph-mon[32889]: pgmap v72: 129 pgs: 129 active+clean; 57 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 854 KiB/s wr, 64 op/s 2024-04-26T23:00:55.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:54 smithi027 ceph-mon[28034]: pgmap v72: 129 pgs: 129 active+clean; 57 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 854 KiB/s wr, 64 op/s 2024-04-26T23:00:57.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:56 smithi080 ceph-mon[32324]: pgmap v73: 129 pgs: 129 active+clean; 57 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 686 KiB/s wr, 39 op/s 2024-04-26T23:00:57.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:56 smithi094 ceph-mon[32889]: pgmap v73: 129 pgs: 129 active+clean; 57 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 686 KiB/s wr, 39 op/s 2024-04-26T23:00:57.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:56 smithi027 ceph-mon[28034]: pgmap v73: 129 pgs: 129 active+clean; 57 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 686 KiB/s wr, 39 op/s 2024-04-26T23:00:59.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:00:58 smithi080 ceph-mon[32324]: pgmap v74: 129 pgs: 129 active+clean; 63 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.2 MiB/s wr, 51 op/s 2024-04-26T23:00:59.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:00:58 smithi094 ceph-mon[32889]: pgmap v74: 129 pgs: 129 active+clean; 63 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.2 MiB/s wr, 51 op/s 2024-04-26T23:00:59.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:00:58 smithi027 ceph-mon[28034]: pgmap v74: 129 pgs: 129 active+clean; 63 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.2 MiB/s wr, 51 op/s 2024-04-26T23:01:01.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:00 smithi080 ceph-mon[32324]: pgmap v75: 129 pgs: 129 active+clean; 63 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 856 KiB/s wr, 26 op/s 2024-04-26T23:01:01.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:00 smithi094 ceph-mon[32889]: pgmap v75: 129 pgs: 129 active+clean; 63 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 856 KiB/s wr, 26 op/s 2024-04-26T23:01:01.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:00 smithi027 ceph-mon[28034]: pgmap v75: 129 pgs: 129 active+clean; 63 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 856 KiB/s wr, 26 op/s 2024-04-26T23:01:03.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:02 smithi080 ceph-mon[32324]: pgmap v76: 129 pgs: 129 active+clean; 67 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 35 op/s 2024-04-26T23:01:03.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:02 smithi094 ceph-mon[32889]: pgmap v76: 129 pgs: 129 active+clean; 67 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 35 op/s 2024-04-26T23:01:03.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:02 smithi027 ceph-mon[28034]: pgmap v76: 129 pgs: 129 active+clean; 67 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 35 op/s 2024-04-26T23:01:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.087+0000 7fd691800640 1 -- 172.21.15.27:0/2083604360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd68c074d00 msgr2=0x7fd68c0750e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.087+0000 7fd691800640 1 --2- 172.21.15.27:0/2083604360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd68c074d00 0x7fd68c0750e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd678009900 tx=0x7fd67802a480 comp rx=0 tx=0).stop 2024-04-26T23:01:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.089+0000 7fd691800640 1 -- 172.21.15.27:0/2083604360 shutdown_connections 2024-04-26T23:01:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.089+0000 7fd691800640 1 --2- 172.21.15.27:0/2083604360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd68c075620 0x7fd68c10ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.089+0000 7fd691800640 1 --2- 172.21.15.27:0/2083604360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd68c10f1c0 0x7fd68c1115d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.089+0000 7fd691800640 1 --2- 172.21.15.27:0/2083604360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd68c074d00 0x7fd68c0750e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.089+0000 7fd691800640 1 -- 172.21.15.27:0/2083604360 >> 172.21.15.27:0/2083604360 conn(0x7fd68c0fe3f0 msgr2=0x7fd68c100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.093+0000 7fd691800640 1 -- 172.21.15.27:0/2083604360 shutdown_connections 2024-04-26T23:01:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.093+0000 7fd691800640 1 -- 172.21.15.27:0/2083604360 wait complete. 2024-04-26T23:01:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.094+0000 7fd691800640 1 Processor -- start 2024-04-26T23:01:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.094+0000 7fd691800640 1 -- start start 2024-04-26T23:01:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.095+0000 7fd691800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd68c074d00 0x7fd68c10d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.095+0000 7fd691800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd68c075620 0x7fd68c10dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.095+0000 7fd691800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd68c10f1c0 0x7fd68c1a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.095+0000 7fd691800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c103a10 con 0x7fd68c075620 2024-04-26T23:01:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.095+0000 7fd691800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c103890 con 0x7fd68c10f1c0 2024-04-26T23:01:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.095+0000 7fd691800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c103b90 con 0x7fd68c074d00 2024-04-26T23:01:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.095+0000 7fd68aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd68c074d00 0x7fd68c10d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.095+0000 7fd68aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd68c074d00 0x7fd68c10d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:42000/0 (socket says 172.21.15.27:42000) 2024-04-26T23:01:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.095+0000 7fd68aa00640 1 -- 172.21.15.27:0/3120545987 learned_addr learned my addr 172.21.15.27:0/3120545987 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.096+0000 7fd68aa00640 1 -- 172.21.15.27:0/3120545987 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd68c10f1c0 msgr2=0x7fd68c1a5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.096+0000 7fd68aa00640 1 --2- 172.21.15.27:0/3120545987 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd68c10f1c0 0x7fd68c1a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.096+0000 7fd68aa00640 1 -- 172.21.15.27:0/3120545987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd68c075620 msgr2=0x7fd68c10dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.096+0000 7fd68aa00640 1 --2- 172.21.15.27:0/3120545987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd68c075620 0x7fd68c10dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.096+0000 7fd68aa00640 1 -- 172.21.15.27:0/3120545987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c1a59f0 con 0x7fd68c074d00 2024-04-26T23:01:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.096+0000 7fd68aa00640 1 --2- 172.21.15.27:0/3120545987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd68c074d00 0x7fd68c10d750 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd678031440 tx=0x7fd678003d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.096+0000 7fd683400640 1 -- 172.21.15.27:0/3120545987 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67803f070 con 0x7fd68c074d00 2024-04-26T23:01:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.096+0000 7fd683400640 1 -- 172.21.15.27:0/3120545987 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd678031d60 con 0x7fd68c074d00 2024-04-26T23:01:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.096+0000 7fd683400640 1 -- 172.21.15.27:0/3120545987 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678035030 con 0x7fd68c074d00 2024-04-26T23:01:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.096+0000 7fd691800640 1 -- 172.21.15.27:0/3120545987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68c1a5c80 con 0x7fd68c074d00 2024-04-26T23:01:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.097+0000 7fd691800640 1 -- 172.21.15.27:0/3120545987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68c1ac320 con 0x7fd68c074d00 2024-04-26T23:01:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.101+0000 7fd6941fd740 1 Processor -- start 2024-04-26T23:01:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.101+0000 7fd6941fd740 1 -- start start 2024-04-26T23:01:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.102+0000 7fd6941fd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 0x561735d131b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.102+0000 7fd6941fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 0x561735d1da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.102+0000 7fd6941fd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d1dfb0 0x561735d203a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.102+0000 7fd6941fd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561735c78380 con 0x561735d12dd0 2024-04-26T23:01:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.102+0000 7fd6941fd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x561735c14c30 con 0x561735d1dfb0 2024-04-26T23:01:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.102+0000 7fd6941fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561735c74340 con 0x561735d136f0 2024-04-26T23:01:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.104+0000 7fd68b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 0x561735d131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.104+0000 7fd68b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 0x561735d131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54724/0 (socket says 172.21.15.27:54724) 2024-04-26T23:01:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.104+0000 7fd68b400640 1 -- 172.21.15.27:0/792429815 learned_addr learned my addr 172.21.15.27:0/792429815 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.104+0000 7fd68a000640 1 --2- 172.21.15.27:0/792429815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d1dfb0 0x561735d203a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.105+0000 7fd68b400640 1 -- 172.21.15.27:0/792429815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 msgr2=0x561735d1da70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:01:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.105+0000 7fd68b400640 1 --2- 172.21.15.27:0/792429815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 0x561735d1da70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.105+0000 7fd68b400640 1 -- 172.21.15.27:0/792429815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d1dfb0 msgr2=0x561735d203a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.105+0000 7fd68b400640 1 --2- 172.21.15.27:0/792429815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d1dfb0 0x561735d203a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.105+0000 7fd68b400640 1 -- 172.21.15.27:0/792429815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561735c36940 con 0x561735d12dd0 2024-04-26T23:01:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.105+0000 7fd683400640 1 -- 172.21.15.27:0/3120545987 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd678042070 con 0x7fd68c074d00 2024-04-26T23:01:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.108+0000 7fd683400640 1 --2- 172.21.15.27:0/3120545987 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd6700a3ea0 0x7fd6700a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.108+0000 7fd683400640 1 -- 172.21.15.27:0/3120545987 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fd6780f1580 con 0x7fd68c074d00 2024-04-26T23:01:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.109+0000 7fd68aa00640 1 --2- 172.21.15.27:0/3120545987 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd6700a3ea0 0x7fd6700a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.109+0000 7fd68b400640 1 --2- 172.21.15.27:0/792429815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 0x561735d131b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd684007c90 tx=0x7fd68400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6a01eebfa54fb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.110+0000 7fd677e00640 1 -- 172.21.15.27:0/792429815 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684014070 con 0x561735d12dd0 2024-04-26T23:01:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.111+0000 7fd677e00640 1 -- 172.21.15.27:0/792429815 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd68400fb00 con 0x561735d12dd0 2024-04-26T23:01:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.111+0000 7fd677e00640 1 -- 172.21.15.27:0/792429815 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68400ea30 con 0x561735d12dd0 2024-04-26T23:01:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.112+0000 7fd6941fd740 1 -- 172.21.15.27:0/792429815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 msgr2=0x561735d131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.112+0000 7fd6941fd740 1 --2- 172.21.15.27:0/792429815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 0x561735d131b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd684007c90 tx=0x7fd68400c6a0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.112+0000 7fd68aa00640 1 --2- 172.21.15.27:0/3120545987 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd6700a3ea0 0x7fd6700a6360 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x561735d14370 tx=0x7fd67803b890 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.114+0000 7fd6941fd740 1 -- 172.21.15.27:0/792429815 shutdown_connections 2024-04-26T23:01:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.114+0000 7fd6941fd740 1 --2- 172.21.15.27:0/792429815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d1dfb0 0x561735d203a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.114+0000 7fd6941fd740 1 --2- 172.21.15.27:0/792429815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 0x561735d1da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.114+0000 7fd6941fd740 1 --2- 172.21.15.27:0/792429815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 0x561735d131b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.114+0000 7fd6941fd740 1 -- 172.21.15.27:0/792429815 >> 172.21.15.27:0/792429815 conn(0x561735d0f2e0 msgr2=0x561735d0f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.114+0000 7fd6941fd740 1 -- 172.21.15.27:0/792429815 shutdown_connections 2024-04-26T23:01:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.114+0000 7fd6941fd740 1 -- 172.21.15.27:0/792429815 wait complete. 2024-04-26T23:01:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.115+0000 7fd6941fd740 1 -- start start 2024-04-26T23:01:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.115+0000 7fd6941fd740 1 Processor -- start 2024-04-26T23:01:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.116+0000 7fd6941fd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 0x561735d1af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.116+0000 7fd6941fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 0x561735d1b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.116+0000 7fd6941fd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d1dfb0 0x561735d1ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.116+0000 7fd6941fd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561735d20be0 con 0x561735d12dd0 2024-04-26T23:01:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.116+0000 7fd6941fd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x561735bffc60 con 0x561735d1dfb0 2024-04-26T23:01:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.116+0000 7fd6941fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561735d20d60 con 0x561735d136f0 2024-04-26T23:01:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.116+0000 7fd6941fd740 10 client.0 _ll_register_callbacks cb 0x561735c2be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:01:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.120+0000 7fd68a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 0x561735d1b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.120+0000 7fd68a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 0x561735d1b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:42004/0 (socket says 172.21.15.27:42004) 2024-04-26T23:01:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.120+0000 7fd68a000640 1 -- 172.21.15.27:0/839211526 learned_addr learned my addr 172.21.15.27:0/839211526 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.120+0000 7fd68b400640 1 --2- 172.21.15.27:0/839211526 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d1dfb0 0x561735d1ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.120+0000 7fd68a000640 1 -- 172.21.15.27:0/839211526 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d1dfb0 msgr2=0x561735d1ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.120+0000 7fd68a000640 1 --2- 172.21.15.27:0/839211526 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d1dfb0 0x561735d1ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.121+0000 7fd68a000640 1 -- 172.21.15.27:0/839211526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 msgr2=0x561735d1af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.121+0000 7fd68a000640 1 --2- 172.21.15.27:0/839211526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 0x561735d1af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.121+0000 7fd68a000640 1 -- 172.21.15.27:0/839211526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561735c0e510 con 0x561735d136f0 2024-04-26T23:01:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.122+0000 7fd68a000640 1 --2- 172.21.15.27:0/839211526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 0x561735d1b4c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009fd0 tx=0x7fd67c00e6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f57014d74d3677a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.122+0000 7fd674c00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T23:01:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.122+0000 7fd674c00640 1 -- 172.21.15.27:0/839211526 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c017070 con 0x561735d136f0 2024-04-26T23:01:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.122+0000 7fd6941fd740 1 -- 172.21.15.27:0/839211526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd68400fb00 con 0x561735d136f0 2024-04-26T23:01:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.123+0000 7fd674c00640 1 -- 172.21.15.27:0/839211526 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fd67c00f040 con 0x561735d136f0 2024-04-26T23:01:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.124+0000 7fd674c00640 1 -- 172.21.15.27:0/839211526 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd67c023050 con 0x561735d136f0 2024-04-26T23:01:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.124+0000 7fd6941fd740 10 client.25696 fetch_fsmap learned FSMap version 27 2024-04-26T23:01:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.124+0000 7fd6941fd740 1 -- 172.21.15.27:0/839211526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56173598a210 con 0x561735d136f0 2024-04-26T23:01:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.125+0000 7fd674c00640 1 -- 172.21.15.27:0/839211526 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd68400fb00 con 0x561735d136f0 2024-04-26T23:01:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.125+0000 7fd674c00640 1 -- 172.21.15.27:0/839211526 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fd67c0126e0 con 0x561735d136f0 2024-04-26T23:01:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.125+0000 7fd6941fd740 10 client.25696 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T23:01:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.125+0000 7fd6941fd740 10 client.25696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:01:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.125+0000 7fd6941fd740 20 client.25696 populate_metadata read hostname 'smithi027' 2024-04-26T23:01:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.125+0000 7fd6941fd740 1 --2- 172.21.15.27:0/839211526 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x561735d2e240 0x561735d306c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.125+0000 7fd6941fd740 4 client.25696 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:01:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.125+0000 7fd6941fd740 1 -- 172.21.15.27:0/839211526 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561735c85f90 con 0x561735d2e240 2024-04-26T23:01:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.126+0000 7fd68b400640 1 --2- 172.21.15.27:0/839211526 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x561735d2e240 0x561735d306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.128+0000 7fd68b400640 1 --2- 172.21.15.27:0/839211526 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x561735d2e240 0x561735d306c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6d2ee51c8f0a905 server_cookie=dcff5d910eb5f3bd in_seq=0 out_seq=0 2024-04-26T23:01:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.128+0000 7fd674c00640 10 client.25696 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:01:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.129+0000 7fd674c00640 1 -- 172.21.15.27:0/839211526 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561735c85f90 con 0x561735d2e240 2024-04-26T23:01:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.129+0000 7fd674c00640 10 client.25696 handle_command_reply: tid=0 2024-04-26T23:01:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.129+0000 7fd6941fd740 1 client.25696 shutdown 2024-04-26T23:01:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.129+0000 7fd6941fd740 1 -- 172.21.15.27:0/839211526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 msgr2=0x561735d1b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.129+0000 7fd6941fd740 1 --2- 172.21.15.27:0/839211526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 0x561735d1b4c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009fd0 tx=0x7fd67c00e6b0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.129+0000 7fd6941fd740 1 -- 172.21.15.27:0/839211526 shutdown_connections 2024-04-26T23:01:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.129+0000 7fd6941fd740 1 --2- 172.21.15.27:0/839211526 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x561735d2e240 0x561735d306c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.129+0000 7fd6941fd740 1 --2- 172.21.15.27:0/839211526 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d1dfb0 0x561735d1ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.129+0000 7fd6941fd740 1 --2- 172.21.15.27:0/839211526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 0x561735d1b4c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.129+0000 7fd6941fd740 1 --2- 172.21.15.27:0/839211526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 0x561735d1af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.129+0000 7fd674c00640 0 client.25696 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:01:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.129+0000 7fd6941fd740 1 -- 172.21.15.27:0/839211526 >> 172.21.15.27:0/839211526 conn(0x561735d0f2e0 msgr2=0x561735d15010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.130+0000 7fd6941fd740 1 -- 172.21.15.27:0/839211526 shutdown_connections 2024-04-26T23:01:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.130+0000 7fd6941fd740 1 -- 172.21.15.27:0/839211526 wait complete. 2024-04-26T23:01:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.131+0000 7fd6941fd740 20 client.25696 trim_cache size 0 max 16384 2024-04-26T23:01:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.169+0000 7fd6941fd740 1 Processor -- start 2024-04-26T23:01:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.169+0000 7fd6941fd740 1 -- start start 2024-04-26T23:01:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.170+0000 7fd6941fd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d12dd0 0x561735d1d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.170+0000 7fd6941fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 0x561735d29aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.170+0000 7fd6941fd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d1dfb0 0x561735d1b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.170+0000 7fd6941fd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1b8e90 con 0x561735d1dfb0 2024-04-26T23:01:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.170+0000 7fd6941fd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1b6700 con 0x561735d12dd0 2024-04-26T23:01:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.170+0000 7fd6941fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1b4550 con 0x561735d136f0 2024-04-26T23:01:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.170+0000 7fd68a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 0x561735d29aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.170+0000 7fd68a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 0x561735d29aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:42018/0 (socket says 172.21.15.27:42018) 2024-04-26T23:01:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.170+0000 7fd68a000640 1 -- 172.21.15.27:0/2108409884 learned_addr learned my addr 172.21.15.27:0/2108409884 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.170+0000 7fd68a000640 1 -- 172.21.15.27:0/2108409884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d12dd0 msgr2=0x561735d1d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:01:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.171+0000 7fd68b400640 1 --2- 172.21.15.27:0/2108409884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d12dd0 0x561735d1d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.171+0000 7fd68a000640 1 --2- 172.21.15.27:0/2108409884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d12dd0 0x561735d1d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.171+0000 7fd68a000640 1 -- 172.21.15.27:0/2108409884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d1dfb0 msgr2=0x561735d1b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.171+0000 7fd68a000640 1 --2- 172.21.15.27:0/2108409884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d1dfb0 0x561735d1b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.171+0000 7fd68a000640 1 -- 172.21.15.27:0/2108409884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561735a56620 con 0x561735d136f0 2024-04-26T23:01:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.171+0000 7fd68a000640 1 --2- 172.21.15.27:0/2108409884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 0x561735d29aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd67c006bc0 tx=0x7fd67c012f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5660b6a3756bd4f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.171+0000 7fd68b400640 1 --2- 172.21.15.27:0/2108409884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d12dd0 0x561735d1d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:01:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.171+0000 7fd677400640 1 -- 172.21.15.27:0/2108409884 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c017070 con 0x561735d136f0 2024-04-26T23:01:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.171+0000 7fd677400640 1 -- 172.21.15.27:0/2108409884 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd67c020070 con 0x561735d136f0 2024-04-26T23:01:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.171+0000 7fd677400640 1 -- 172.21.15.27:0/2108409884 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c013d90 con 0x561735d136f0 2024-04-26T23:01:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.172+0000 7fd6941fd740 1 -- 172.21.15.27:0/2108409884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 msgr2=0x561735d29aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.172+0000 7fd6941fd740 1 --2- 172.21.15.27:0/2108409884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 0x561735d29aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd67c006bc0 tx=0x7fd67c012f30 comp rx=0 tx=0).stop 2024-04-26T23:01:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.172+0000 7fd6941fd740 1 -- 172.21.15.27:0/2108409884 shutdown_connections 2024-04-26T23:01:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.172+0000 7fd6941fd740 1 --2- 172.21.15.27:0/2108409884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d1dfb0 0x561735d1b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.172+0000 7fd6941fd740 1 --2- 172.21.15.27:0/2108409884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d136f0 0x561735d29aa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.172+0000 7fd6941fd740 1 --2- 172.21.15.27:0/2108409884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d12dd0 0x561735d1d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.172+0000 7fd6941fd740 1 -- 172.21.15.27:0/2108409884 >> 172.21.15.27:0/2108409884 conn(0x561735d0f2e0 msgr2=0x561735d16230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.173+0000 7fd6941fd740 1 -- 172.21.15.27:0/2108409884 shutdown_connections 2024-04-26T23:01:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.173+0000 7fd6941fd740 1 -- 172.21.15.27:0/2108409884 wait complete. 2024-04-26T23:01:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.174+0000 7fd6941fd740 1 -- start start 2024-04-26T23:01:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.174+0000 7fd6941fd740 1 Processor -- start 2024-04-26T23:01:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.175+0000 7fd6941fd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 0x561735d211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.175+0000 7fd68b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 0x561735d211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.176+0000 7fd6941fd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d136f0 0x561735d21720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.176+0000 7fd6941fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d1dfb0 0x561735d8c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.176+0000 7fd6941fd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1b4bc0 con 0x561735d12dd0 2024-04-26T23:01:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.176+0000 7fd6941fd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1b6030 con 0x561735d136f0 2024-04-26T23:01:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.176+0000 7fd6941fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1b5290 con 0x561735d1dfb0 2024-04-26T23:01:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.176+0000 7fd6941fd740 10 client.0 _ll_register_callbacks cb 0x7fd67c009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:01:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.176+0000 7fd68b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 0x561735d211e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54754/0 (socket says 172.21.15.27:54754) 2024-04-26T23:01:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.176+0000 7fd68b400640 1 -- 172.21.15.27:0/3118568594 learned_addr learned my addr 172.21.15.27:0/3118568594 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.176+0000 7fd68a000640 1 --2- 172.21.15.27:0/3118568594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d136f0 0x561735d21720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.176+0000 7fd68b400640 1 --2- 172.21.15.27:0/3118568594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d1dfb0 0x561735d8c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.176+0000 7fd68b400640 1 -- 172.21.15.27:0/3118568594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d1dfb0 msgr2=0x561735d8c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.176+0000 7fd68b400640 1 --2- 172.21.15.27:0/3118568594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d1dfb0 0x561735d8c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.176+0000 7fd68b400640 1 -- 172.21.15.27:0/3118568594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d136f0 msgr2=0x561735d21720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.176+0000 7fd68b400640 1 --2- 172.21.15.27:0/3118568594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d136f0 0x561735d21720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.176+0000 7fd68b400640 1 -- 172.21.15.27:0/3118568594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561735c32cb0 con 0x561735d12dd0 2024-04-26T23:01:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.177+0000 7fd68b400640 1 --2- 172.21.15.27:0/3118568594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 0x561735d211e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd68400d010 tx=0x7fd684007bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107e6f17776575dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.177+0000 7fd674c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T23:01:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.177+0000 7fd674c00640 1 -- 172.21.15.27:0/3118568594 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684014050 con 0x561735d12dd0 2024-04-26T23:01:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.177+0000 7fd6941fd740 1 -- 172.21.15.27:0/3118568594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd67c020070 con 0x561735d12dd0 2024-04-26T23:01:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.178+0000 7fd674c00640 1 -- 172.21.15.27:0/3118568594 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fd684003e00 con 0x561735d12dd0 2024-04-26T23:01:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.179+0000 7fd674c00640 1 -- 172.21.15.27:0/3118568594 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd684026320 con 0x561735d12dd0 2024-04-26T23:01:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.179+0000 7fd6941fd740 10 client.15540 fetch_fsmap learned FSMap version 27 2024-04-26T23:01:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.179+0000 7fd6941fd740 1 -- 172.21.15.27:0/3118568594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561735d93fc0 con 0x561735d12dd0 2024-04-26T23:01:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.179+0000 7fd674c00640 1 -- 172.21.15.27:0/3118568594 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd67c020070 con 0x561735d12dd0 2024-04-26T23:01:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.180+0000 7fd674c00640 1 -- 172.21.15.27:0/3118568594 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fd68402f070 con 0x561735d12dd0 2024-04-26T23:01:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.180+0000 7fd6941fd740 10 client.15540 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T23:01:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.180+0000 7fd6941fd740 10 client.15540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:01:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.180+0000 7fd6941fd740 20 client.15540 populate_metadata read hostname 'smithi027' 2024-04-26T23:01:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.180+0000 7fd6941fd740 1 --2- 172.21.15.27:0/3118568594 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x561735d94490 0x561735d968b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.180+0000 7fd6941fd740 4 client.15540 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:01:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.180+0000 7fd6941fd740 1 -- 172.21.15.27:0/3118568594 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561735c7fe00 con 0x561735d94490 2024-04-26T23:01:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.181+0000 7fd68a000640 1 --2- 172.21.15.27:0/3118568594 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x561735d94490 0x561735d968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.182+0000 7fd68a000640 1 --2- 172.21.15.27:0/3118568594 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x561735d94490 0x561735d968b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1445002f2b0a7107 server_cookie=e4e998433474094f in_seq=0 out_seq=0 2024-04-26T23:01:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:04.182+0000 7fd674c00640 10 client.15540 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:01:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.049+0000 7fd674c00640 1 -- 172.21.15.27:0/3118568594 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561735c7fe00 con 0x561735d94490 2024-04-26T23:01:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.049+0000 7fd674c00640 10 client.15540 handle_command_reply: tid=0 2024-04-26T23:01:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.049+0000 7fd6941fd740 1 client.15540 shutdown 2024-04-26T23:01:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.050+0000 7fd6941fd740 1 -- 172.21.15.27:0/3118568594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 msgr2=0x561735d211e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.050+0000 7fd6941fd740 1 --2- 172.21.15.27:0/3118568594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 0x561735d211e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd68400d010 tx=0x7fd684007bf0 comp rx=0 tx=0).stop 2024-04-26T23:01:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.050+0000 7fd6941fd740 1 -- 172.21.15.27:0/3118568594 shutdown_connections 2024-04-26T23:01:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.050+0000 7fd6941fd740 1 --2- 172.21.15.27:0/3118568594 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x561735d94490 0x561735d968b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.050+0000 7fd6941fd740 1 --2- 172.21.15.27:0/3118568594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561735d136f0 0x561735d21720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.050+0000 7fd6941fd740 1 --2- 172.21.15.27:0/3118568594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561735d1dfb0 0x561735d8c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.050+0000 7fd6941fd740 1 --2- 172.21.15.27:0/3118568594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561735d12dd0 0x561735d211e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.050+0000 7fd674c00640 0 client.15540 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:01:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.050+0000 7fd6941fd740 1 -- 172.21.15.27:0/3118568594 >> 172.21.15.27:0/3118568594 conn(0x561735d0f2e0 msgr2=0x561735d14780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.050+0000 7fd6941fd740 1 -- 172.21.15.27:0/3118568594 shutdown_connections 2024-04-26T23:01:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.050+0000 7fd6941fd740 1 -- 172.21.15.27:0/3118568594 wait complete. 2024-04-26T23:01:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.050+0000 7fd6941fd740 20 client.15540 trim_cache size 0 max 16384 2024-04-26T23:01:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T23:01:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T23:01:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T23:01:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T23:01:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.051+0000 7fd691800640 1 -- 172.21.15.27:0/3120545987 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd6700a3ea0 msgr2=0x7fd6700a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.051+0000 7fd691800640 1 --2- 172.21.15.27:0/3120545987 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd6700a3ea0 0x7fd6700a6360 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x561735d14370 tx=0x7fd67803b890 comp rx=0 tx=0).stop 2024-04-26T23:01:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.051+0000 7fd691800640 1 -- 172.21.15.27:0/3120545987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd68c074d00 msgr2=0x7fd68c10d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.051+0000 7fd691800640 1 --2- 172.21.15.27:0/3120545987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd68c074d00 0x7fd68c10d750 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd678031440 tx=0x7fd678003d80 comp rx=0 tx=0).stop 2024-04-26T23:01:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.051+0000 7fd691800640 1 -- 172.21.15.27:0/3120545987 shutdown_connections 2024-04-26T23:01:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.051+0000 7fd691800640 1 --2- 172.21.15.27:0/3120545987 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd6700a3ea0 0x7fd6700a6360 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.051+0000 7fd691800640 1 --2- 172.21.15.27:0/3120545987 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd68c10f1c0 0x7fd68c1a5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.051+0000 7fd691800640 1 --2- 172.21.15.27:0/3120545987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd68c075620 0x7fd68c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.051+0000 7fd691800640 1 --2- 172.21.15.27:0/3120545987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd68c074d00 0x7fd68c10d750 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.051+0000 7fd691800640 1 -- 172.21.15.27:0/3120545987 >> 172.21.15.27:0/3120545987 conn(0x7fd68c0fe3f0 msgr2=0x7fd68c0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.051+0000 7fd691800640 1 -- 172.21.15.27:0/3120545987 shutdown_connections 2024-04-26T23:01:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:05.051+0000 7fd691800640 1 -- 172.21.15.27:0/3120545987 wait complete. 2024-04-26T23:01:05.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:04 smithi080 ceph-mon[32324]: pgmap v77: 129 pgs: 129 active+clean; 67 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 894 KiB/s wr, 34 op/s 2024-04-26T23:01:05.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:04 smithi094 ceph-mon[32889]: pgmap v77: 129 pgs: 129 active+clean; 67 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 894 KiB/s wr, 34 op/s 2024-04-26T23:01:05.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:04 smithi027 ceph-mon[28034]: pgmap v77: 129 pgs: 129 active+clean; 67 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 894 KiB/s wr, 34 op/s 2024-04-26T23:01:07.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:06 smithi080 ceph-mon[32324]: pgmap v78: 129 pgs: 129 active+clean; 67 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 893 KiB/s wr, 33 op/s 2024-04-26T23:01:07.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:06 smithi094 ceph-mon[32889]: pgmap v78: 129 pgs: 129 active+clean; 67 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 893 KiB/s wr, 33 op/s 2024-04-26T23:01:07.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:06 smithi027 ceph-mon[28034]: pgmap v78: 129 pgs: 129 active+clean; 67 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 893 KiB/s wr, 33 op/s 2024-04-26T23:01:09.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:08 smithi080 ceph-mon[32324]: pgmap v79: 129 pgs: 129 active+clean; 88 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 777 op/s 2024-04-26T23:01:09.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:08 smithi094 ceph-mon[32889]: pgmap v79: 129 pgs: 129 active+clean; 88 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 777 op/s 2024-04-26T23:01:09.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:08 smithi027 ceph-mon[28034]: pgmap v79: 129 pgs: 129 active+clean; 88 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 777 op/s 2024-04-26T23:01:11.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:10 smithi080 ceph-mon[32324]: pgmap v80: 129 pgs: 129 active+clean; 88 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 764 op/s 2024-04-26T23:01:11.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:10 smithi094 ceph-mon[32889]: pgmap v80: 129 pgs: 129 active+clean; 88 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 764 op/s 2024-04-26T23:01:11.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:10 smithi027 ceph-mon[28034]: pgmap v80: 129 pgs: 129 active+clean; 88 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 764 op/s 2024-04-26T23:01:13.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:12 smithi080 ceph-mon[32324]: pgmap v81: 129 pgs: 129 active+clean; 115 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.1 MiB/s wr, 899 op/s 2024-04-26T23:01:13.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:12 smithi094 ceph-mon[32889]: pgmap v81: 129 pgs: 129 active+clean; 115 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.1 MiB/s wr, 899 op/s 2024-04-26T23:01:13.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:12 smithi027 ceph-mon[28034]: pgmap v81: 129 pgs: 129 active+clean; 115 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.1 MiB/s wr, 899 op/s 2024-04-26T23:01:15.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:14 smithi080 ceph-mon[32324]: pgmap v82: 129 pgs: 129 active+clean; 120 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 906 op/s 2024-04-26T23:01:15.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:14 smithi094 ceph-mon[32889]: pgmap v82: 129 pgs: 129 active+clean; 120 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 906 op/s 2024-04-26T23:01:15.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:14 smithi027 ceph-mon[28034]: pgmap v82: 129 pgs: 129 active+clean; 120 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 906 op/s 2024-04-26T23:01:15.461 INFO:tasks.workunit.client.0.smithi027.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-26T23:01:15.464 INFO:tasks.workunit.client.0.smithi027.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-26T23:01:15.465 INFO:tasks.workunit.client.0.smithi027.stderr:+ make 2024-04-26T23:01:15.792 INFO:tasks.workunit.client.0.smithi027.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-26T23:01:17.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:16 smithi080 ceph-mon[32324]: pgmap v83: 129 pgs: 129 active+clean; 120 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 893 op/s 2024-04-26T23:01:17.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:16 smithi094 ceph-mon[32889]: pgmap v83: 129 pgs: 129 active+clean; 120 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 893 op/s 2024-04-26T23:01:17.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:16 smithi027 ceph-mon[28034]: pgmap v83: 129 pgs: 129 active+clean; 120 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 893 op/s 2024-04-26T23:01:18.006 INFO:tasks.workunit.client.0.smithi027.stderr:++ readlink -f fsstress 2024-04-26T23:01:18.010 INFO:tasks.workunit.client.0.smithi027.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-26T23:01:18.010 INFO:tasks.workunit.client.0.smithi027.stderr:+ popd 2024-04-26T23:01:18.012 INFO:tasks.workunit.client.0.smithi027.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-26T23:01:18.013 INFO:tasks.workunit.client.0.smithi027.stderr:+ popd 2024-04-26T23:01:18.013 INFO:tasks.workunit.client.0.smithi027.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-26T23:01:18.014 INFO:tasks.workunit.client.0.smithi027.stderr:++ mktemp -d -p . 2024-04-26T23:01:18.020 INFO:tasks.workunit.client.0.smithi027.stderr:+ T=./tmp.l3JEpWWxh6 2024-04-26T23:01:18.020 INFO:tasks.workunit.client.0.smithi027.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.l3JEpWWxh6 -l 1 -n 1000 -p 10 -v 2024-04-26T23:01:18.024 INFO:tasks.workunit.client.0.smithi027.stdout:seed = 1714180598 2024-04-26T23:01:18.029 INFO:tasks.workunit.client.0.smithi027.stdout:0/0: readlink - no filename 2024-04-26T23:01:18.029 INFO:tasks.workunit.client.0.smithi027.stdout:0/1: chown . 24262 1 2024-04-26T23:01:18.029 INFO:tasks.workunit.client.0.smithi027.stdout:0/2: stat - no entries 2024-04-26T23:01:18.031 INFO:tasks.workunit.client.0.smithi027.stdout:9/0: chown . 2771 1 2024-04-26T23:01:18.032 INFO:tasks.workunit.client.0.smithi027.stdout:5/0: mknod c0 0 2024-04-26T23:01:18.033 INFO:tasks.workunit.client.0.smithi027.stdout:1/0: dwrite - no filename 2024-04-26T23:01:18.033 INFO:tasks.workunit.client.0.smithi027.stdout:1/1: rename - no filename 2024-04-26T23:01:18.033 INFO:tasks.workunit.client.0.smithi027.stdout:1/2: dwrite - no filename 2024-04-26T23:01:18.033 INFO:tasks.workunit.client.0.smithi027.stdout:9/1: mknod c0 0 2024-04-26T23:01:18.034 INFO:tasks.workunit.client.0.smithi027.stdout:9/2: fsync - no filename 2024-04-26T23:01:18.034 INFO:tasks.workunit.client.0.smithi027.stdout:9/3: write - no filename 2024-04-26T23:01:18.034 INFO:tasks.workunit.client.0.smithi027.stdout:9/4: truncate - no filename 2024-04-26T23:01:18.034 INFO:tasks.workunit.client.0.smithi027.stdout:9/5: rmdir - no directory 2024-04-26T23:01:18.035 INFO:tasks.workunit.client.0.smithi027.stdout:5/1: creat f1 x:0 0 0 2024-04-26T23:01:18.038 INFO:tasks.workunit.client.0.smithi027.stdout:1/3: mknod c0 0 2024-04-26T23:01:18.038 INFO:tasks.workunit.client.0.smithi027.stdout:1/4: chown c0 887 1 2024-04-26T23:01:18.038 INFO:tasks.workunit.client.0.smithi027.stdout:1/5: dread - no filename 2024-04-26T23:01:18.039 INFO:tasks.workunit.client.0.smithi027.stdout:1/6: dread - no filename 2024-04-26T23:01:18.039 INFO:tasks.workunit.client.0.smithi027.stdout:2/0: stat - no entries 2024-04-26T23:01:18.039 INFO:tasks.workunit.client.0.smithi027.stdout:2/1: dread - no filename 2024-04-26T23:01:18.039 INFO:tasks.workunit.client.0.smithi027.stdout:9/6: mknod c1 0 2024-04-26T23:01:18.040 INFO:tasks.workunit.client.0.smithi027.stdout:5/2: mknod c2 0 2024-04-26T23:01:18.042 INFO:tasks.workunit.client.0.smithi027.stdout:3/0: dwrite - no filename 2024-04-26T23:01:18.042 INFO:tasks.workunit.client.0.smithi027.stdout:3/1: read - no filename 2024-04-26T23:01:18.042 INFO:tasks.workunit.client.0.smithi027.stdout:3/2: rmdir - no directory 2024-04-26T23:01:18.043 INFO:tasks.workunit.client.0.smithi027.stdout:1/7: unlink c0 0 2024-04-26T23:01:18.043 INFO:tasks.workunit.client.0.smithi027.stdout:2/2: symlink l0 0 2024-04-26T23:01:18.044 INFO:tasks.workunit.client.0.smithi027.stdout:2/3: dread - no filename 2024-04-26T23:01:18.044 INFO:tasks.workunit.client.0.smithi027.stdout:2/4: dwrite - no filename 2024-04-26T23:01:18.050 INFO:tasks.workunit.client.0.smithi027.stdout:5/3: link c0 c3 0 2024-04-26T23:01:18.050 INFO:tasks.workunit.client.0.smithi027.stdout:5/4: fdatasync f1 0 2024-04-26T23:01:18.050 INFO:tasks.workunit.client.0.smithi027.stdout:5/5: dread - f1 zero size 2024-04-26T23:01:18.050 INFO:tasks.workunit.client.0.smithi027.stdout:9/7: rename c1 to c2 0 2024-04-26T23:01:18.058 INFO:tasks.workunit.client.0.smithi027.stdout:3/3: mknod c0 0 2024-04-26T23:01:18.058 INFO:tasks.workunit.client.0.smithi027.stdout:2/5: rename l0 to l1 0 2024-04-26T23:01:18.058 INFO:tasks.workunit.client.0.smithi027.stdout:3/4: truncate - no filename 2024-04-26T23:01:18.058 INFO:tasks.workunit.client.0.smithi027.stdout:5/6: mkdir d4 0 2024-04-26T23:01:18.058 INFO:tasks.workunit.client.0.smithi027.stdout:5/7: chown c3 7199 1 2024-04-26T23:01:18.058 INFO:tasks.workunit.client.0.smithi027.stdout:4/0: write - no filename 2024-04-26T23:01:18.059 INFO:tasks.workunit.client.0.smithi027.stdout:4/1: dwrite - no filename 2024-04-26T23:01:18.059 INFO:tasks.workunit.client.0.smithi027.stdout:4/2: fdatasync - no filename 2024-04-26T23:01:18.059 INFO:tasks.workunit.client.0.smithi027.stdout:4/3: dread - no filename 2024-04-26T23:01:18.059 INFO:tasks.workunit.client.0.smithi027.stdout:4/4: rename - no filename 2024-04-26T23:01:18.059 INFO:tasks.workunit.client.0.smithi027.stdout:4/5: write - no filename 2024-04-26T23:01:18.059 INFO:tasks.workunit.client.0.smithi027.stdout:5/8: truncate f1 640144 0 2024-04-26T23:01:18.059 INFO:tasks.workunit.client.0.smithi027.stdout:4/6: write - no filename 2024-04-26T23:01:18.059 INFO:tasks.workunit.client.0.smithi027.stdout:4/7: dread - no filename 2024-04-26T23:01:18.059 INFO:tasks.workunit.client.0.smithi027.stdout:4/8: dwrite - no filename 2024-04-26T23:01:18.059 INFO:tasks.workunit.client.0.smithi027.stdout:5/9: getdents d4 0 2024-04-26T23:01:18.064 INFO:tasks.workunit.client.0.smithi027.stdout:1/8: mkdir d1 0 2024-04-26T23:01:18.064 INFO:tasks.workunit.client.0.smithi027.stdout:1/9: truncate - no filename 2024-04-26T23:01:18.064 INFO:tasks.workunit.client.0.smithi027.stdout:1/10: getdents d1 0 2024-04-26T23:01:18.064 INFO:tasks.workunit.client.0.smithi027.stdout:1/11: getdents d1 0 2024-04-26T23:01:18.064 INFO:tasks.workunit.client.0.smithi027.stdout:1/12: chown d1 349101 1 2024-04-26T23:01:18.072 INFO:tasks.workunit.client.0.smithi027.stdout:2/6: mkdir d2 0 2024-04-26T23:01:18.072 INFO:tasks.workunit.client.0.smithi027.stdout:2/7: readlink l1 0 2024-04-26T23:01:18.072 INFO:tasks.workunit.client.0.smithi027.stdout:2/8: write - no filename 2024-04-26T23:01:18.072 INFO:tasks.workunit.client.0.smithi027.stdout:2/9: chown d2 3757728 1 2024-04-26T23:01:18.072 INFO:tasks.workunit.client.0.smithi027.stdout:6/0: dread - no filename 2024-04-26T23:01:18.072 INFO:tasks.workunit.client.0.smithi027.stdout:6/1: fdatasync - no filename 2024-04-26T23:01:18.073 INFO:tasks.workunit.client.0.smithi027.stdout:6/2: dread - no filename 2024-04-26T23:01:18.073 INFO:tasks.workunit.client.0.smithi027.stdout:4/9: creat f0 x:0 0 0 2024-04-26T23:01:18.073 INFO:tasks.workunit.client.0.smithi027.stdout:4/10: dread - f0 zero size 2024-04-26T23:01:18.073 INFO:tasks.workunit.client.0.smithi027.stdout:9/8: mknod c3 0 2024-04-26T23:01:18.073 INFO:tasks.workunit.client.0.smithi027.stdout:3/5: symlink l1 0 2024-04-26T23:01:18.074 INFO:tasks.workunit.client.0.smithi027.stdout:1/13: symlink d1/l2 0 2024-04-26T23:01:18.076 INFO:tasks.workunit.client.0.smithi027.stdout:7/0: truncate - no filename 2024-04-26T23:01:18.076 INFO:tasks.workunit.client.0.smithi027.stdout:2/10: mkdir d2/d3 0 2024-04-26T23:01:18.077 INFO:tasks.workunit.client.0.smithi027.stdout:6/3: mknod c0 0 2024-04-26T23:01:18.077 INFO:tasks.workunit.client.0.smithi027.stdout:6/4: stat c0 0 2024-04-26T23:01:18.080 INFO:tasks.workunit.client.0.smithi027.stdout:3/6: creat f2 x:0 0 0 2024-04-26T23:01:18.084 INFO:tasks.workunit.client.0.smithi027.stdout:7/1: mkdir d0 0 2024-04-26T23:01:18.084 INFO:tasks.workunit.client.0.smithi027.stdout:9/9: mkdir d4 0 2024-04-26T23:01:18.085 INFO:tasks.workunit.client.0.smithi027.stdout:6/5: mkdir d1 0 2024-04-26T23:01:18.085 INFO:tasks.workunit.client.0.smithi027.stdout:6/6: truncate - no filename 2024-04-26T23:01:18.086 INFO:tasks.workunit.client.0.smithi027.stdout:6/7: readlink - no filename 2024-04-26T23:01:18.091 INFO:tasks.workunit.client.0.smithi027.stdout:1/14: creat d1/f3 x:0 0 0 2024-04-26T23:01:18.092 INFO:tasks.workunit.client.0.smithi027.stdout:1/15: dread - d1/f3 zero size 2024-04-26T23:01:18.092 INFO:tasks.workunit.client.0.smithi027.stdout:9/10: symlink d4/l5 0 2024-04-26T23:01:18.092 INFO:tasks.workunit.client.0.smithi027.stdout:8/0: chown . 117506633 1 2024-04-26T23:01:18.092 INFO:tasks.workunit.client.0.smithi027.stdout:7/2: symlink d0/l1 0 2024-04-26T23:01:18.092 INFO:tasks.workunit.client.0.smithi027.stdout:1/16: creat d1/f4 x:0 0 0 2024-04-26T23:01:18.092 INFO:tasks.workunit.client.0.smithi027.stdout:1/17: dread - d1/f4 zero size 2024-04-26T23:01:18.092 INFO:tasks.workunit.client.0.smithi027.stdout:1/18: read - d1/f4 zero size 2024-04-26T23:01:18.092 INFO:tasks.workunit.client.0.smithi027.stdout:9/11: symlink d4/l6 0 2024-04-26T23:01:18.092 INFO:tasks.workunit.client.0.smithi027.stdout:9/12: write - no filename 2024-04-26T23:01:18.092 INFO:tasks.workunit.client.0.smithi027.stdout:9/13: write - no filename 2024-04-26T23:01:18.092 INFO:tasks.workunit.client.0.smithi027.stdout:9/14: dread - no filename 2024-04-26T23:01:18.093 INFO:tasks.workunit.client.0.smithi027.stdout:6/8: creat d1/f2 x:0 0 0 2024-04-26T23:01:18.093 INFO:tasks.workunit.client.0.smithi027.stdout:8/1: creat f0 x:0 0 0 2024-04-26T23:01:18.093 INFO:tasks.workunit.client.0.smithi027.stdout:8/2: dread - f0 zero size 2024-04-26T23:01:18.093 INFO:tasks.workunit.client.0.smithi027.stdout:8/3: write f0 [640078,1573] 0 2024-04-26T23:01:18.094 INFO:tasks.workunit.client.0.smithi027.stdout:1/19: mkdir d1/d5 0 2024-04-26T23:01:18.096 INFO:tasks.workunit.client.0.smithi027.stdout:9/15: mknod d4/c7 0 2024-04-26T23:01:18.097 INFO:tasks.workunit.client.0.smithi027.stdout:9/16: chown c0 1351707315 1 2024-04-26T23:01:18.103 INFO:tasks.workunit.client.0.smithi027.stdout:8/4: creat f1 x:0 0 0 2024-04-26T23:01:18.103 INFO:tasks.workunit.client.0.smithi027.stdout:6/9: unlink c0 0 2024-04-26T23:01:18.106 INFO:tasks.workunit.client.0.smithi027.stdout:1/20: creat d1/f6 x:0 0 0 2024-04-26T23:01:18.107 INFO:tasks.workunit.client.0.smithi027.stdout:9/17: rename c3 to d4/c8 0 2024-04-26T23:01:18.107 INFO:tasks.workunit.client.0.smithi027.stdout:6/10: creat d1/f3 x:0 0 0 2024-04-26T23:01:18.107 INFO:tasks.workunit.client.0.smithi027.stdout:9/18: symlink d4/l9 0 2024-04-26T23:01:18.107 INFO:tasks.workunit.client.0.smithi027.stdout:9/19: fdatasync - no filename 2024-04-26T23:01:18.107 INFO:tasks.workunit.client.0.smithi027.stdout:9/20: dread - no filename 2024-04-26T23:01:18.107 INFO:tasks.workunit.client.0.smithi027.stdout:9/21: dwrite - no filename 2024-04-26T23:01:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.294+0000 7fe4d4a00640 1 -- 172.21.15.27:0/4261670040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4d0105ce0 msgr2=0x7fe4d0106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.294+0000 7fe4d4a00640 1 --2- 172.21.15.27:0/4261670040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4d0105ce0 0x7fe4d0106170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009900 tx=0x7fe4bc02a3f0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.294+0000 7fe4d4a00640 1 -- 172.21.15.27:0/4261670040 shutdown_connections 2024-04-26T23:01:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.294+0000 7fe4d4a00640 1 --2- 172.21.15.27:0/4261670040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d0101950 0x7fe4d01057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.294+0000 7fe4d4a00640 1 --2- 172.21.15.27:0/4261670040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4d0105ce0 0x7fe4d0106170 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.294+0000 7fe4d4a00640 1 --2- 172.21.15.27:0/4261670040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d0101030 0x7fe4d0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.295+0000 7fe4d4a00640 1 -- 172.21.15.27:0/4261670040 >> 172.21.15.27:0/4261670040 conn(0x7fe4d0076740 msgr2=0x7fe4d0076b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.295+0000 7fe4d4a00640 1 -- 172.21.15.27:0/4261670040 shutdown_connections 2024-04-26T23:01:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.295+0000 7fe4d4a00640 1 -- 172.21.15.27:0/4261670040 wait complete. 2024-04-26T23:01:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.297+0000 7fe4d4a00640 1 Processor -- start 2024-04-26T23:01:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.298+0000 7fe4d4a00640 1 -- start start 2024-04-26T23:01:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.298+0000 7fe4d4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4d0101030 0x7fe4d019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.298+0000 7fe4d4a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d0101950 0x7fe4d019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.298+0000 7fe4d4a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d0105ce0 0x7fe4d019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.298+0000 7fe4d4a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d010ea60 con 0x7fe4d0101950 2024-04-26T23:01:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.298+0000 7fe4d4a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d010e8e0 con 0x7fe4d0105ce0 2024-04-26T23:01:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.298+0000 7fe4d4a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d010ebe0 con 0x7fe4d0101030 2024-04-26T23:01:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.299+0000 7fe4cde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4d0101030 0x7fe4d019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.299+0000 7fe4cde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4d0101030 0x7fe4d019b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:51132/0 (socket says 172.21.15.27:51132) 2024-04-26T23:01:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.299+0000 7fe4cde00640 1 -- 172.21.15.27:0/2950391044 learned_addr learned my addr 172.21.15.27:0/2950391044 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.299+0000 7fe4ce800640 1 --2- 172.21.15.27:0/2950391044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d0105ce0 0x7fe4d019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.299+0000 7fe4cde00640 1 -- 172.21.15.27:0/2950391044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d0105ce0 msgr2=0x7fe4d019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.299+0000 7fe4cde00640 1 --2- 172.21.15.27:0/2950391044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d0105ce0 0x7fe4d019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.299+0000 7fe4cd400640 1 --2- 172.21.15.27:0/2950391044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d0101950 0x7fe4d019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.299+0000 7fe4cde00640 1 -- 172.21.15.27:0/2950391044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d0101950 msgr2=0x7fe4d019b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.299+0000 7fe4cde00640 1 --2- 172.21.15.27:0/2950391044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d0101950 0x7fe4d019b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.299+0000 7fe4cde00640 1 -- 172.21.15.27:0/2950391044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d01a02d0 con 0x7fe4d0101030 2024-04-26T23:01:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.299+0000 7fe4ce800640 1 --2- 172.21.15.27:0/2950391044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d0105ce0 0x7fe4d019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:01:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.299+0000 7fe4cd400640 1 --2- 172.21.15.27:0/2950391044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d0101950 0x7fe4d019b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:01:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.300+0000 7fe4cde00640 1 --2- 172.21.15.27:0/2950391044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4d0101030 0x7fe4d019b2e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800cce0 tx=0x7fe4c800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.301+0000 7fe4c6a00640 1 -- 172.21.15.27:0/2950391044 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8004b20 con 0x7fe4d0101030 2024-04-26T23:01:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.301+0000 7fe4c6a00640 1 -- 172.21.15.27:0/2950391044 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c8004cc0 con 0x7fe4d0101030 2024-04-26T23:01:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.301+0000 7fe4c6a00640 1 -- 172.21.15.27:0/2950391044 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c800f6b0 con 0x7fe4d0101030 2024-04-26T23:01:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.301+0000 7fe4d4a00640 1 -- 172.21.15.27:0/2950391044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d01a05c0 con 0x7fe4d0101030 2024-04-26T23:01:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.302+0000 7fe4d4a00640 1 -- 172.21.15.27:0/2950391044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d01a7e50 con 0x7fe4d0101030 2024-04-26T23:01:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.309+0000 7fe4c6a00640 1 -- 172.21.15.27:0/2950391044 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe4c800f850 con 0x7fe4d0101030 2024-04-26T23:01:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.311+0000 7fe4d7813740 1 Processor -- start 2024-04-26T23:01:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.312+0000 7fe4d7813740 1 -- start start 2024-04-26T23:01:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.313+0000 7fe4d7813740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588917935e0 0x5588917939c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.313+0000 7fe4c6a00640 1 --2- 172.21.15.27:0/2950391044 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe4b40a3d90 0x7fe4b40a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.314+0000 7fe4c6a00640 1 -- 172.21.15.27:0/2950391044 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fe4c80d0970 con 0x7fe4d0101030 2024-04-26T23:01:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.314+0000 7fe4d7813740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558891793f00 0x55889179e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.314+0000 7fe4d7813740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55889179e7c0 0x5588917a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.314+0000 7fe4d7813740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5588916f0380 con 0x558891793f00 2024-04-26T23:01:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.314+0000 7fe4d7813740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55889168cc30 con 0x5588917935e0 2024-04-26T23:01:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.314+0000 7fe4d7813740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5588916ec340 con 0x55889179e7c0 2024-04-26T23:01:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.314+0000 7fe4ce800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588917935e0 0x5588917939c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.315+0000 7fe4cde00640 1 --2- 172.21.15.27:0/2950391044 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe4b40a3d90 0x7fe4b40a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.315+0000 7fe4cd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558891793f00 0x55889179e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.315+0000 7fe4ce800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588917935e0 0x5588917939c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:59536/0 (socket says 172.21.15.27:59536) 2024-04-26T23:01:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.315+0000 7fe4ce800640 1 -- 172.21.15.27:0/3306950256 learned_addr learned my addr 172.21.15.27:0/3306950256 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.315+0000 7fe4ce800640 1 --2- 172.21.15.27:0/3306950256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55889179e7c0 0x5588917a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.316+0000 7fe4cd400640 1 -- 172.21.15.27:0/3306950256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55889179e7c0 msgr2=0x5588917a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.317+0000 7fe4cde00640 1 --2- 172.21.15.27:0/2950391044 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe4b40a3d90 0x7fe4b40a6250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe4c809c1d0 tx=0x7fe4c8098040 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.317+0000 7fe4cd400640 1 --2- 172.21.15.27:0/3306950256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55889179e7c0 0x5588917a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.317+0000 7fe4cd400640 1 -- 172.21.15.27:0/3306950256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588917935e0 msgr2=0x5588917939c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.317+0000 7fe4cd400640 1 --2- 172.21.15.27:0/3306950256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588917935e0 0x5588917939c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.317+0000 7fe4cd400640 1 -- 172.21.15.27:0/3306950256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588916ae940 con 0x558891793f00 2024-04-26T23:01:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.317+0000 7fe4ce800640 1 --2- 172.21.15.27:0/3306950256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588917935e0 0x5588917939c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:01:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.319+0000 7fe4cd400640 1 --2- 172.21.15.27:0/3306950256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558891793f00 0x55889179e280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009ad0 tx=0x7fe4c0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d9d3c19653efd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.320+0000 7fe4bb000640 1 -- 172.21.15.27:0/3306950256 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0016070 con 0x558891793f00 2024-04-26T23:01:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.320+0000 7fe4bb000640 1 -- 172.21.15.27:0/3306950256 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c00054d0 con 0x558891793f00 2024-04-26T23:01:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.320+0000 7fe4bb000640 1 -- 172.21.15.27:0/3306950256 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0004db0 con 0x558891793f00 2024-04-26T23:01:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.321+0000 7fe4d7813740 1 -- 172.21.15.27:0/3306950256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558891793f00 msgr2=0x55889179e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.321+0000 7fe4d7813740 1 --2- 172.21.15.27:0/3306950256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558891793f00 0x55889179e280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009ad0 tx=0x7fe4c0009f90 comp rx=0 tx=0).stop 2024-04-26T23:01:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.322+0000 7fe4d7813740 1 -- 172.21.15.27:0/3306950256 shutdown_connections 2024-04-26T23:01:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.322+0000 7fe4d7813740 1 --2- 172.21.15.27:0/3306950256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55889179e7c0 0x5588917a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.322+0000 7fe4d7813740 1 --2- 172.21.15.27:0/3306950256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558891793f00 0x55889179e280 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.322+0000 7fe4d7813740 1 --2- 172.21.15.27:0/3306950256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588917935e0 0x5588917939c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.322+0000 7fe4d7813740 1 -- 172.21.15.27:0/3306950256 >> 172.21.15.27:0/3306950256 conn(0x55889178faf0 msgr2=0x55889178fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.322+0000 7fe4d7813740 1 -- 172.21.15.27:0/3306950256 shutdown_connections 2024-04-26T23:01:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.322+0000 7fe4d7813740 1 -- 172.21.15.27:0/3306950256 wait complete. 2024-04-26T23:01:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.323+0000 7fe4d7813740 1 -- start start 2024-04-26T23:01:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.323+0000 7fe4d7813740 1 Processor -- start 2024-04-26T23:01:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.325+0000 7fe4d7813740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588917935e0 0x55889179b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.325+0000 7fe4d7813740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558891793f00 0x55889179bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.325+0000 7fe4d7813740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 0x55889179c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.325+0000 7fe4d7813740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5588917a13f0 con 0x5588917935e0 2024-04-26T23:01:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.325+0000 7fe4d7813740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x558891677c60 con 0x55889179e7c0 2024-04-26T23:01:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.325+0000 7fe4d7813740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5588917a1570 con 0x558891793f00 2024-04-26T23:01:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.325+0000 7fe4d7813740 10 client.0 _ll_register_callbacks cb 0x5588916a3e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:01:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.326+0000 7fe4ce800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 0x55889179c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.326+0000 7fe4ce800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 0x55889179c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:59546/0 (socket says 172.21.15.27:59546) 2024-04-26T23:01:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.326+0000 7fe4ce800640 1 -- 172.21.15.27:0/1337850186 learned_addr learned my addr 172.21.15.27:0/1337850186 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.326+0000 7fe4ce800640 1 --2- 172.21.15.27:0/1337850186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588917935e0 0x55889179b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.326+0000 7fe4cd400640 1 --2- 172.21.15.27:0/1337850186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558891793f00 0x55889179bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.327+0000 7fe4ce800640 1 -- 172.21.15.27:0/1337850186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558891793f00 msgr2=0x55889179bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.327+0000 7fe4ce800640 1 --2- 172.21.15.27:0/1337850186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558891793f00 0x55889179bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.327+0000 7fe4ce800640 1 -- 172.21.15.27:0/1337850186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588917935e0 msgr2=0x55889179b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.327+0000 7fe4ce800640 1 --2- 172.21.15.27:0/1337850186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588917935e0 0x55889179b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.327+0000 7fe4ce800640 1 -- 172.21.15.27:0/1337850186 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558891686510 con 0x55889179e7c0 2024-04-26T23:01:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.329+0000 7fe4ce800640 1 --2- 172.21.15.27:0/1337850186 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 0x55889179c0e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc006010 tx=0x7fe4bc02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d3164e4d6ea1172 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.329+0000 7fe4af400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T23:01:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.330+0000 7fe4af400640 1 -- 172.21.15.27:0/1337850186 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc035040 con 0x55889179e7c0 2024-04-26T23:01:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.330+0000 7fe4af400640 1 -- 172.21.15.27:0/1337850186 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fe4bc005220 con 0x55889179e7c0 2024-04-26T23:01:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.330+0000 7fe4d7813740 1 -- 172.21.15.27:0/1337850186 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c00054d0 con 0x55889179e7c0 2024-04-26T23:01:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.330+0000 7fe4af400640 1 -- 172.21.15.27:0/1337850186 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4bc005420 con 0x55889179e7c0 2024-04-26T23:01:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.330+0000 7fe4d7813740 10 client.25670 fetch_fsmap learned FSMap version 27 2024-04-26T23:01:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.330+0000 7fe4d7813740 1 -- 172.21.15.27:0/1337850186 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558891402210 con 0x55889179e7c0 2024-04-26T23:01:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.331+0000 7fe4af400640 1 -- 172.21.15.27:0/1337850186 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c00054d0 con 0x55889179e7c0 2024-04-26T23:01:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.331+0000 7fe4af400640 1 -- 172.21.15.27:0/1337850186 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fe4bc03a040 con 0x55889179e7c0 2024-04-26T23:01:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.331+0000 7fe4d7813740 10 client.25670 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T23:01:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.331+0000 7fe4d7813740 10 client.25670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:01:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.331+0000 7fe4d7813740 20 client.25670 populate_metadata read hostname 'smithi027' 2024-04-26T23:01:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.331+0000 7fe4d7813740 1 --2- 172.21.15.27:0/1337850186 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5588917ae4b0 0x5588917b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.331+0000 7fe4d7813740 4 client.25670 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:01:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.331+0000 7fe4d7813740 1 -- 172.21.15.27:0/1337850186 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588916fdf90 con 0x5588917ae4b0 2024-04-26T23:01:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.333+0000 7fe4cd400640 1 --2- 172.21.15.27:0/1337850186 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5588917ae4b0 0x5588917b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.333+0000 7fe4cd400640 1 --2- 172.21.15.27:0/1337850186 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5588917ae4b0 0x5588917b08a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fc254f8bfa97436 server_cookie=b51df8bc30f522f6 in_seq=0 out_seq=0 2024-04-26T23:01:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.333+0000 7fe4af400640 10 client.25670 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:01:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.334+0000 7fe4af400640 1 -- 172.21.15.27:0/1337850186 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588916fdf90 con 0x5588917ae4b0 2024-04-26T23:01:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.334+0000 7fe4af400640 10 client.25670 handle_command_reply: tid=0 2024-04-26T23:01:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.334+0000 7fe4d7813740 1 client.25670 shutdown 2024-04-26T23:01:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.335+0000 7fe4d7813740 1 -- 172.21.15.27:0/1337850186 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 msgr2=0x55889179c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.335+0000 7fe4d7813740 1 --2- 172.21.15.27:0/1337850186 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 0x55889179c0e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc006010 tx=0x7fe4bc02a3f0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.335+0000 7fe4d7813740 1 -- 172.21.15.27:0/1337850186 shutdown_connections 2024-04-26T23:01:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.335+0000 7fe4d7813740 1 --2- 172.21.15.27:0/1337850186 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5588917ae4b0 0x5588917b08a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.335+0000 7fe4d7813740 1 --2- 172.21.15.27:0/1337850186 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 0x55889179c0e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.335+0000 7fe4d7813740 1 --2- 172.21.15.27:0/1337850186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558891793f00 0x55889179bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.335+0000 7fe4d7813740 1 --2- 172.21.15.27:0/1337850186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588917935e0 0x55889179b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.335+0000 7fe4d7813740 1 -- 172.21.15.27:0/1337850186 >> 172.21.15.27:0/1337850186 conn(0x55889178faf0 msgr2=0x558891786440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.336+0000 7fe4af400640 0 client.25670 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:01:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.336+0000 7fe4d7813740 1 -- 172.21.15.27:0/1337850186 shutdown_connections 2024-04-26T23:01:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.336+0000 7fe4d7813740 1 -- 172.21.15.27:0/1337850186 wait complete. 2024-04-26T23:01:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.336+0000 7fe4d7813740 20 client.25670 trim_cache size 0 max 16384 2024-04-26T23:01:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.371+0000 7fe4d7813740 1 Processor -- start 2024-04-26T23:01:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.372+0000 7fe4d7813740 1 -- start start 2024-04-26T23:01:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.372+0000 7fe4d7813740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588917935e0 0x5588917aa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.372+0000 7fe4d7813740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558891793f00 0x5588917b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.372+0000 7fe4d7813740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 0x55889179a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.372+0000 7fe4d7813740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01b0800 con 0x558891793f00 2024-04-26T23:01:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.372+0000 7fe4d7813740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01ae070 con 0x55889179e7c0 2024-04-26T23:01:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.372+0000 7fe4d7813740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01abee0 con 0x5588917935e0 2024-04-26T23:01:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.373+0000 7fe4ce800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588917935e0 0x5588917aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.373+0000 7fe4ce800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588917935e0 0x5588917aa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:51148/0 (socket says 172.21.15.27:51148) 2024-04-26T23:01:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.373+0000 7fe4ce800640 1 -- 172.21.15.27:0/1137214124 learned_addr learned my addr 172.21.15.27:0/1137214124 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.373+0000 7fe4ce800640 1 --2- 172.21.15.27:0/1137214124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 0x55889179a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.374+0000 7fe4ce800640 1 -- 172.21.15.27:0/1137214124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 msgr2=0x55889179a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.374+0000 7fe4ce800640 1 --2- 172.21.15.27:0/1137214124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 0x55889179a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.374+0000 7fe4ce800640 1 -- 172.21.15.27:0/1137214124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558891793f00 msgr2=0x5588917b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.374+0000 7fe4ce800640 1 --2- 172.21.15.27:0/1137214124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558891793f00 0x5588917b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.374+0000 7fe4ce800640 1 -- 172.21.15.27:0/1137214124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588914ce620 con 0x5588917935e0 2024-04-26T23:01:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.375+0000 7fe4ce800640 1 --2- 172.21.15.27:0/1137214124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588917935e0 0x5588917aa300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc003ea0 tx=0x7fe4bc009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc5f0cf3971d68f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.375+0000 7fe4ba600640 1 -- 172.21.15.27:0/1137214124 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc035070 con 0x5588917935e0 2024-04-26T23:01:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.375+0000 7fe4ba600640 1 -- 172.21.15.27:0/1137214124 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4bc032da0 con 0x5588917935e0 2024-04-26T23:01:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.375+0000 7fe4ba600640 1 -- 172.21.15.27:0/1137214124 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc0051e0 con 0x5588917935e0 2024-04-26T23:01:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.376+0000 7fe4d7813740 1 -- 172.21.15.27:0/1137214124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588917935e0 msgr2=0x5588917aa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.376+0000 7fe4d7813740 1 --2- 172.21.15.27:0/1137214124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588917935e0 0x5588917aa300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc003ea0 tx=0x7fe4bc009560 comp rx=0 tx=0).stop 2024-04-26T23:01:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.377+0000 7fe4d7813740 1 -- 172.21.15.27:0/1137214124 shutdown_connections 2024-04-26T23:01:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.377+0000 7fe4d7813740 1 --2- 172.21.15.27:0/1137214124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 0x55889179a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.377+0000 7fe4d7813740 1 --2- 172.21.15.27:0/1137214124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558891793f00 0x5588917b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.377+0000 7fe4d7813740 1 --2- 172.21.15.27:0/1137214124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588917935e0 0x5588917aa300 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.377+0000 7fe4d7813740 1 -- 172.21.15.27:0/1137214124 >> 172.21.15.27:0/1137214124 conn(0x55889178faf0 msgr2=0x558891794fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.377+0000 7fe4d7813740 1 -- 172.21.15.27:0/1137214124 shutdown_connections 2024-04-26T23:01:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.378+0000 7fe4d7813740 1 -- 172.21.15.27:0/1137214124 wait complete. 2024-04-26T23:01:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.379+0000 7fe4d7813740 1 -- start start 2024-04-26T23:01:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.380+0000 7fe4d7813740 1 Processor -- start 2024-04-26T23:01:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.381+0000 7fe4d7813740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588917935e0 0x5588917a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.381+0000 7fe4d7813740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558891793f00 0x5588917a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.381+0000 7fe4d7813740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 0x55889180c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.381+0000 7fe4d7813740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01ac530 con 0x558891793f00 2024-04-26T23:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.381+0000 7fe4d7813740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01ad9a0 con 0x55889179e7c0 2024-04-26T23:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.381+0000 7fe4d7813740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01acc00 con 0x5588917935e0 2024-04-26T23:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.381+0000 7fe4d7813740 10 client.0 _ll_register_callbacks cb 0x7fe4bc003d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.381+0000 7fe4ce800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588917935e0 0x5588917a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.381+0000 7fe4ce800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588917935e0 0x5588917a1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:51164/0 (socket says 172.21.15.27:51164) 2024-04-26T23:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.381+0000 7fe4ce800640 1 -- 172.21.15.27:0/3751424754 learned_addr learned my addr 172.21.15.27:0/3751424754 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.381+0000 7fe4ce800640 1 --2- 172.21.15.27:0/3751424754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 0x55889180c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.382+0000 7fe4ce800640 1 -- 172.21.15.27:0/3751424754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 msgr2=0x55889180c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.382+0000 7fe4ce800640 1 --2- 172.21.15.27:0/3751424754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 0x55889180c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.382+0000 7fe4ce800640 1 -- 172.21.15.27:0/3751424754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558891793f00 msgr2=0x5588917a1fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.382+0000 7fe4ce800640 1 --2- 172.21.15.27:0/3751424754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558891793f00 0x5588917a1fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.382+0000 7fe4ce800640 1 -- 172.21.15.27:0/3751424754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588916aacb0 con 0x5588917935e0 2024-04-26T23:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.382+0000 7fe4ce800640 1 --2- 172.21.15.27:0/3751424754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588917935e0 0x5588917a1aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc004a00 tx=0x7fe4bc004010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df9fab98d8fa5b19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.382+0000 7fe4af400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T23:01:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.384+0000 7fe4af400640 1 -- 172.21.15.27:0/3751424754 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc035070 con 0x5588917935e0 2024-04-26T23:01:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.384+0000 7fe4d7813740 1 -- 172.21.15.27:0/3751424754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4bc032da0 con 0x5588917935e0 2024-04-26T23:01:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.384+0000 7fe4af400640 1 -- 172.21.15.27:0/3751424754 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fe4bc0046c0 con 0x5588917935e0 2024-04-26T23:01:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.385+0000 7fe4af400640 1 -- 172.21.15.27:0/3751424754 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4bc0044d0 con 0x5588917935e0 2024-04-26T23:01:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.385+0000 7fe4d7813740 10 client.25720 fetch_fsmap learned FSMap version 27 2024-04-26T23:01:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.385+0000 7fe4d7813740 1 -- 172.21.15.27:0/3751424754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558891787340 con 0x5588917935e0 2024-04-26T23:01:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.386+0000 7fe4af400640 1 -- 172.21.15.27:0/3751424754 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4bc032da0 con 0x5588917935e0 2024-04-26T23:01:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.387+0000 7fe4af400640 1 -- 172.21.15.27:0/3751424754 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fe4bc03dcb0 con 0x5588917935e0 2024-04-26T23:01:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.387+0000 7fe4d7813740 10 client.25720 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T23:01:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.387+0000 7fe4d7813740 10 client.25720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:01:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.387+0000 7fe4d7813740 20 client.25720 populate_metadata read hostname 'smithi027' 2024-04-26T23:01:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.387+0000 7fe4d7813740 1 --2- 172.21.15.27:0/3751424754 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x558891787ac0 0x558891789ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.387+0000 7fe4d7813740 4 client.25720 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:01:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.387+0000 7fe4d7813740 1 -- 172.21.15.27:0/3751424754 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588916f7e00 con 0x558891787ac0 2024-04-26T23:01:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.388+0000 7fe4cd400640 1 --2- 172.21.15.27:0/3751424754 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x558891787ac0 0x558891789ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.389+0000 7fe4cd400640 1 --2- 172.21.15.27:0/3751424754 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x558891787ac0 0x558891789ee0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1cb3fae668138 server_cookie=b3ff11258d83eb29 in_seq=0 out_seq=0 2024-04-26T23:01:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:18.389+0000 7fe4af400640 10 client.25720 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:01:18.446 INFO:tasks.workunit.client.0.smithi027.stdout:4/11: fsync f0 0 2024-04-26T23:01:18.447 INFO:tasks.workunit.client.0.smithi027.stdout:4/12: write f0 [884387,127625] 0 2024-04-26T23:01:18.447 INFO:tasks.workunit.client.0.smithi027.stdout:2/11: getdents d2 0 2024-04-26T23:01:18.447 INFO:tasks.workunit.client.0.smithi027.stdout:2/12: getdents d2 0 2024-04-26T23:01:18.449 INFO:tasks.workunit.client.0.smithi027.stdout:4/13: mknod c1 0 2024-04-26T23:01:18.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/13: link l1 d2/l4 0 2024-04-26T23:01:18.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/14: dwrite - no filename 2024-04-26T23:01:18.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/15: write - no filename 2024-04-26T23:01:18.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/16: dwrite - no filename 2024-04-26T23:01:18.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/17: rename d2/d3 to d2/d5 0 2024-04-26T23:01:18.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/18: dwrite - no filename 2024-04-26T23:01:18.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/19: chown d2/d5 28988 1 2024-04-26T23:01:18.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/20: fsync - no filename 2024-04-26T23:01:18.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/21: creat d2/d5/f6 x:0 0 0 2024-04-26T23:01:18.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/22: chown d2/l4 12282 1 2024-04-26T23:01:18.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/23: dread - d2/d5/f6 zero size 2024-04-26T23:01:18.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/24: dread - d2/d5/f6 zero size 2024-04-26T23:01:18.467 INFO:tasks.workunit.client.0.smithi027.stdout:2/25: chown l1 23600 1 2024-04-26T23:01:18.467 INFO:tasks.workunit.client.0.smithi027.stdout:2/26: mkdir d2/d5/d7 0 2024-04-26T23:01:18.471 INFO:tasks.workunit.client.0.smithi027.stdout:2/27: link l1 d2/d5/d7/l8 0 2024-04-26T23:01:18.473 INFO:tasks.workunit.client.0.smithi027.stdout:2/28: creat d2/d5/f9 x:0 0 0 2024-04-26T23:01:18.508 INFO:tasks.workunit.client.0.smithi027.stdout:6/11: getdents d1 0 2024-04-26T23:01:18.512 INFO:tasks.workunit.client.0.smithi027.stdout:9/22: getdents d4 0 2024-04-26T23:01:18.516 INFO:tasks.workunit.client.0.smithi027.stdout:9/23: link d4/l6 d4/la 0 2024-04-26T23:01:18.516 INFO:tasks.workunit.client.0.smithi027.stdout:9/24: truncate - no filename 2024-04-26T23:01:18.516 INFO:tasks.workunit.client.0.smithi027.stdout:9/25: dwrite - no filename 2024-04-26T23:01:18.516 INFO:tasks.workunit.client.0.smithi027.stdout:9/26: chown d4/c8 231 1 2024-04-26T23:01:18.517 INFO:tasks.workunit.client.0.smithi027.stdout:8/5: dwrite f0 [0,4194304] 0 2024-04-26T23:01:18.519 INFO:tasks.workunit.client.0.smithi027.stdout:3/7: dwrite f2 [0,4194304] 0 2024-04-26T23:01:18.519 INFO:tasks.workunit.client.0.smithi027.stdout:3/8: fdatasync f2 0 2024-04-26T23:01:18.519 INFO:tasks.workunit.client.0.smithi027.stdout:3/9: chown f2 6 1 2024-04-26T23:01:18.520 INFO:tasks.workunit.client.0.smithi027.stdout:9/27: rename d4/l9 to d4/lb 0 2024-04-26T23:01:18.522 INFO:tasks.workunit.client.0.smithi027.stdout:8/6: creat f2 x:0 0 0 2024-04-26T23:01:18.522 INFO:tasks.workunit.client.0.smithi027.stdout:8/7: fdatasync f2 0 2024-04-26T23:01:18.523 INFO:tasks.workunit.client.0.smithi027.stdout:3/10: mknod c3 0 2024-04-26T23:01:18.525 INFO:tasks.workunit.client.0.smithi027.stdout:9/28: link c2 d4/cc 0 2024-04-26T23:01:18.525 INFO:tasks.workunit.client.0.smithi027.stdout:9/29: truncate - no filename 2024-04-26T23:01:18.525 INFO:tasks.workunit.client.0.smithi027.stdout:9/30: rename d4 to d4/dd 22 2024-04-26T23:01:18.526 INFO:tasks.workunit.client.0.smithi027.stdout:8/8: creat f3 x:0 0 0 2024-04-26T23:01:18.527 INFO:tasks.workunit.client.0.smithi027.stdout:3/11: symlink l4 0 2024-04-26T23:01:18.527 INFO:tasks.workunit.client.0.smithi027.stdout:3/12: chown f2 756790 1 2024-04-26T23:01:18.527 INFO:tasks.workunit.client.0.smithi027.stdout:3/13: truncate f2 4449444 0 2024-04-26T23:01:18.532 INFO:tasks.workunit.client.0.smithi027.stdout:9/31: creat d4/fe x:0 0 0 2024-04-26T23:01:18.532 INFO:tasks.workunit.client.0.smithi027.stdout:9/32: fdatasync d4/fe 0 2024-04-26T23:01:18.532 INFO:tasks.workunit.client.0.smithi027.stdout:8/9: rename f0 to f4 0 2024-04-26T23:01:18.532 INFO:tasks.workunit.client.0.smithi027.stdout:8/10: truncate f1 1046747 0 2024-04-26T23:01:18.532 INFO:tasks.workunit.client.0.smithi027.stdout:3/14: mkdir d5 0 2024-04-26T23:01:18.532 INFO:tasks.workunit.client.0.smithi027.stdout:8/11: mknod c5 0 2024-04-26T23:01:18.533 INFO:tasks.workunit.client.0.smithi027.stdout:3/15: rmdir d5 0 2024-04-26T23:01:18.537 INFO:tasks.workunit.client.0.smithi027.stdout:8/12: creat f6 x:0 0 0 2024-04-26T23:01:18.537 INFO:tasks.workunit.client.0.smithi027.stdout:3/16: mkdir d6 0 2024-04-26T23:01:18.538 INFO:tasks.workunit.client.0.smithi027.stdout:3/17: creat d6/f7 x:0 0 0 2024-04-26T23:01:18.595 INFO:tasks.workunit.client.0.smithi027.stdout:5/10: dwrite f1 [0,4194304] 0 2024-04-26T23:01:18.643 INFO:tasks.workunit.client.0.smithi027.stdout:3/18: dread f2 [0,4194304] 0 2024-04-26T23:01:18.668 INFO:tasks.workunit.client.0.smithi027.stdout:3/19: write f2 [2510451,41373] 0 2024-04-26T23:01:18.668 INFO:tasks.workunit.client.0.smithi027.stdout:3/20: read - d6/f7 zero size 2024-04-26T23:01:18.669 INFO:tasks.workunit.client.0.smithi027.stdout:3/21: creat d6/f8 x:0 0 0 2024-04-26T23:01:18.669 INFO:tasks.workunit.client.0.smithi027.stdout:3/22: write d6/f8 [498828,110832] 0 2024-04-26T23:01:18.670 INFO:tasks.workunit.client.0.smithi027.stdout:3/23: dread - d6/f7 zero size 2024-04-26T23:01:18.680 INFO:tasks.workunit.client.0.smithi027.stdout:9/33: dwrite d4/fe [0,4194304] 0 2024-04-26T23:01:18.681 INFO:tasks.workunit.client.0.smithi027.stdout:9/34: chown d4 278608 1 2024-04-26T23:01:18.721 INFO:tasks.workunit.client.0.smithi027.stdout:1/21: dwrite d1/f4 [0,4194304] 0 2024-04-26T23:01:18.721 INFO:tasks.workunit.client.0.smithi027.stdout:1/22: chown d1/f6 429 1 2024-04-26T23:01:18.721 INFO:tasks.workunit.client.0.smithi027.stdout:1/23: dread - d1/f6 zero size 2024-04-26T23:01:18.780 INFO:tasks.workunit.client.0.smithi027.stdout:9/35: dwrite d4/fe [0,4194304] 0 2024-04-26T23:01:18.820 INFO:tasks.workunit.client.0.smithi027.stdout:9/36: dread d4/fe [0,4194304] 0 2024-04-26T23:01:18.875 INFO:tasks.workunit.client.0.smithi027.stdout:2/29: dwrite d2/d5/f6 [0,4194304] 0 2024-04-26T23:01:18.884 INFO:tasks.workunit.client.0.smithi027.stdout:1/24: creat d1/f7 x:0 0 0 2024-04-26T23:01:18.884 INFO:tasks.workunit.client.0.smithi027.stdout:1/25: chown d1/f3 394945 1 2024-04-26T23:01:18.892 INFO:tasks.workunit.client.0.smithi027.stdout:9/37: mknod d4/cf 0 2024-04-26T23:01:18.909 INFO:tasks.workunit.client.0.smithi027.stdout:9/38: mkdir d4/d10 0 2024-04-26T23:01:18.909 INFO:tasks.workunit.client.0.smithi027.stdout:1/26: symlink d1/d5/l8 0 2024-04-26T23:01:18.909 INFO:tasks.workunit.client.0.smithi027.stdout:2/30: rename d2/d5/f6 to d2/d5/fa 0 2024-04-26T23:01:18.911 INFO:tasks.workunit.client.0.smithi027.stdout:1/27: rmdir d1/d5 39 2024-04-26T23:01:18.917 INFO:tasks.workunit.client.0.smithi027.stdout:1/28: mknod d1/c9 0 2024-04-26T23:01:18.923 INFO:tasks.workunit.client.0.smithi027.stdout:1/29: unlink d1/d5/l8 0 2024-04-26T23:01:18.923 INFO:tasks.workunit.client.0.smithi027.stdout:1/30: truncate d1/f7 837450 0 2024-04-26T23:01:18.923 INFO:tasks.workunit.client.0.smithi027.stdout:1/31: chown d1/d5 13184 1 2024-04-26T23:01:18.924 INFO:tasks.workunit.client.0.smithi027.stdout:1/32: chown d1/d5 49563 1 2024-04-26T23:01:18.924 INFO:tasks.workunit.client.0.smithi027.stdout:1/33: dread - d1/f6 zero size 2024-04-26T23:01:18.941 INFO:tasks.workunit.client.0.smithi027.stdout:1/34: symlink d1/d5/la 0 2024-04-26T23:01:18.941 INFO:tasks.workunit.client.0.smithi027.stdout:1/35: chown d1/f7 98485 1 2024-04-26T23:01:19.161 INFO:tasks.workunit.client.0.smithi027.stdout:8/13: getdents . 0 2024-04-26T23:01:19.165 INFO:tasks.workunit.client.0.smithi027.stdout:8/14: creat f7 x:0 0 0 2024-04-26T23:01:19.193 INFO:tasks.workunit.client.0.smithi027.stdout:5/11: truncate f1 1524321 0 2024-04-26T23:01:19.209 INFO:tasks.workunit.client.0.smithi027.stdout:3/24: getdents d6 0 2024-04-26T23:01:19.210 INFO:tasks.workunit.client.0.smithi027.stdout:3/25: write d6/f8 [709331,89190] 0 2024-04-26T23:01:19.212 INFO:tasks.workunit.client.0.smithi027.stdout:3/26: rename l1 to d6/l9 0 2024-04-26T23:01:19.215 INFO:tasks.workunit.client.0.smithi027.stdout:3/27: mknod d6/ca 0 2024-04-26T23:01:19.224 INFO:tasks.workunit.client.0.smithi027.stdout:3/28: rename c3 to d6/cb 0 2024-04-26T23:01:19.233 INFO:tasks.workunit.client.0.smithi027.stdout:9/39: write d4/fe [1971613,52147] 0 2024-04-26T23:01:19.236 INFO:tasks.workunit.client.0.smithi027.stdout:9/40: creat d4/f11 x:0 0 0 2024-04-26T23:01:19.238 INFO:tasks.workunit.client.0.smithi027.stdout:9/41: creat d4/d10/f12 x:0 0 0 2024-04-26T23:01:19.238 INFO:tasks.workunit.client.0.smithi027.stdout:9/42: chown d4/c8 2143229086 1 2024-04-26T23:01:19.241 INFO:tasks.workunit.client.0.smithi027.stdout:9/43: link d4/cf d4/d10/c13 0 2024-04-26T23:01:19.246 INFO:tasks.workunit.client.0.smithi027.stdout:9/44: creat d4/f14 x:0 0 0 2024-04-26T23:01:19.246 INFO:tasks.workunit.client.0.smithi027.stdout:9/45: symlink d4/l15 0 2024-04-26T23:01:19.246 INFO:tasks.workunit.client.0.smithi027.stdout:9/46: dread - d4/d10/f12 zero size 2024-04-26T23:01:19.248 INFO:tasks.workunit.client.0.smithi027.stdout:9/47: rename d4/c8 to d4/d10/c16 0 2024-04-26T23:01:19.262 INFO:tasks.workunit.client.0.smithi027.stdout:8/15: dread f4 [0,4194304] 0 2024-04-26T23:01:19.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:18 smithi080 ceph-mon[32324]: pgmap v84: 129 pgs: 129 active+clean; 124 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.5 MiB/s wr, 921 op/s 2024-04-26T23:01:19.274 INFO:tasks.workunit.client.0.smithi027.stdout:3/29: read d6/f8 [59325,58887] 0 2024-04-26T23:01:19.281 INFO:tasks.workunit.client.0.smithi027.stdout:3/30: mkdir d6/dc 0 2024-04-26T23:01:19.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:18 smithi094 ceph-mon[32889]: pgmap v84: 129 pgs: 129 active+clean; 124 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.5 MiB/s wr, 921 op/s 2024-04-26T23:01:19.287 INFO:tasks.workunit.client.0.smithi027.stdout:3/31: creat d6/dc/fd x:0 0 0 2024-04-26T23:01:19.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:18 smithi027 ceph-mon[28034]: pgmap v84: 129 pgs: 129 active+clean; 124 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.5 MiB/s wr, 921 op/s 2024-04-26T23:01:19.427 INFO:tasks.workunit.client.0.smithi027.stdout:3/32: dread d6/f8 [0,4194304] 0 2024-04-26T23:01:19.429 INFO:tasks.workunit.client.0.smithi027.stdout:3/33: truncate f2 2365434 0 2024-04-26T23:01:19.432 INFO:tasks.workunit.client.0.smithi027.stdout:3/34: rename l4 to d6/le 0 2024-04-26T23:01:19.707 INFO:tasks.workunit.client.0.smithi027.stdout:1/36: dread d1/f4 [0,4194304] 0 2024-04-26T23:01:19.707 INFO:tasks.workunit.client.0.smithi027.stdout:1/37: truncate d1/f3 846053 0 2024-04-26T23:01:19.712 INFO:tasks.workunit.client.0.smithi027.stdout:9/48: dread d4/fe [0,4194304] 0 2024-04-26T23:01:19.712 INFO:tasks.workunit.client.0.smithi027.stdout:9/49: chown d4/l15 412374 1 2024-04-26T23:01:19.712 INFO:tasks.workunit.client.0.smithi027.stdout:9/50: chown d4/d10/c16 10 1 2024-04-26T23:01:19.712 INFO:tasks.workunit.client.0.smithi027.stdout:9/51: dread - d4/f11 zero size 2024-04-26T23:01:19.714 INFO:tasks.workunit.client.0.smithi027.stdout:9/52: mkdir d4/d17 0 2024-04-26T23:01:19.715 INFO:tasks.workunit.client.0.smithi027.stdout:9/53: symlink d4/l18 0 2024-04-26T23:01:19.718 INFO:tasks.workunit.client.0.smithi027.stdout:9/54: creat d4/d10/f19 x:0 0 0 2024-04-26T23:01:19.718 INFO:tasks.workunit.client.0.smithi027.stdout:9/55: write d4/f14 [393839,115920] 0 2024-04-26T23:01:19.718 INFO:tasks.workunit.client.0.smithi027.stdout:9/56: chown c2 23413 1 2024-04-26T23:01:19.718 INFO:tasks.workunit.client.0.smithi027.stdout:9/57: chown d4/d17 13 1 2024-04-26T23:01:19.837 INFO:tasks.workunit.client.0.smithi027.stdout:1/38: dread d1/f3 [0,4194304] 0 2024-04-26T23:01:19.841 INFO:tasks.workunit.client.0.smithi027.stdout:1/39: truncate d1/f4 2028433 0 2024-04-26T23:01:19.846 INFO:tasks.workunit.client.0.smithi027.stdout:1/40: creat d1/fb x:0 0 0 2024-04-26T23:01:19.862 INFO:tasks.workunit.client.0.smithi027.stdout:5/12: truncate f1 2133870 0 2024-04-26T23:01:19.862 INFO:tasks.workunit.client.0.smithi027.stdout:5/13: symlink d4/l5 0 2024-04-26T23:01:19.862 INFO:tasks.workunit.client.0.smithi027.stdout:5/14: chown d4/l5 409168158 1 2024-04-26T23:01:19.864 INFO:tasks.workunit.client.0.smithi027.stdout:5/15: rename c2 to d4/c6 0 2024-04-26T23:01:19.864 INFO:tasks.workunit.client.0.smithi027.stdout:5/16: fdatasync f1 0 2024-04-26T23:01:19.865 INFO:tasks.workunit.client.0.smithi027.stdout:5/17: mkdir d4/d7 0 2024-04-26T23:01:19.873 INFO:tasks.workunit.client.0.smithi027.stdout:3/35: truncate f2 2977310 0 2024-04-26T23:01:19.951 INFO:tasks.workunit.client.0.smithi027.stdout:3/36: dwrite d6/dc/fd [0,4194304] 0 2024-04-26T23:01:19.951 INFO:tasks.workunit.client.0.smithi027.stdout:3/37: dread - d6/f7 zero size 2024-04-26T23:01:19.952 INFO:tasks.workunit.client.0.smithi027.stdout:3/38: read d6/f8 [19652,62662] 0 2024-04-26T23:01:19.952 INFO:tasks.workunit.client.0.smithi027.stdout:3/39: stat f2 0 2024-04-26T23:01:19.954 INFO:tasks.workunit.client.0.smithi027.stdout:3/40: rename d6/ca to d6/cf 0 2024-04-26T23:01:19.985 INFO:tasks.workunit.client.0.smithi027.stdout:5/18: dread f1 [0,4194304] 0 2024-04-26T23:01:19.987 INFO:tasks.workunit.client.0.smithi027.stdout:5/19: creat d4/f8 x:0 0 0 2024-04-26T23:01:19.991 INFO:tasks.workunit.client.0.smithi027.stdout:5/20: creat d4/f9 x:0 0 0 2024-04-26T23:01:20.014 INFO:tasks.workunit.client.0.smithi027.stdout:5/21: write f1 [673555,105004] 0 2024-04-26T23:01:20.014 INFO:tasks.workunit.client.0.smithi027.stdout:5/22: truncate d4/f8 295831 0 2024-04-26T23:01:20.026 INFO:tasks.workunit.client.0.smithi027.stdout:3/41: dwrite d6/f7 [0,4194304] 0 2024-04-26T23:01:20.028 INFO:tasks.workunit.client.0.smithi027.stdout:3/42: link d6/f8 d6/dc/f10 0 2024-04-26T23:01:20.030 INFO:tasks.workunit.client.0.smithi027.stdout:3/43: truncate f2 869894 0 2024-04-26T23:01:20.030 INFO:tasks.workunit.client.0.smithi027.stdout:3/44: chown f2 340197881 1 2024-04-26T23:01:20.031 INFO:tasks.workunit.client.0.smithi027.stdout:3/45: creat d6/dc/f11 x:0 0 0 2024-04-26T23:01:20.063 INFO:tasks.workunit.client.0.smithi027.stdout:3/46: symlink d6/dc/l12 0 2024-04-26T23:01:20.065 INFO:tasks.workunit.client.0.smithi027.stdout:3/47: chown d6/dc/f11 446079 1 2024-04-26T23:01:20.065 INFO:tasks.workunit.client.0.smithi027.stdout:3/48: chown d6 35129452 1 2024-04-26T23:01:20.065 INFO:tasks.workunit.client.0.smithi027.stdout:3/49: creat d6/f13 x:0 0 0 2024-04-26T23:01:20.066 INFO:tasks.workunit.client.0.smithi027.stdout:3/50: symlink d6/dc/l14 0 2024-04-26T23:01:20.066 INFO:tasks.workunit.client.0.smithi027.stdout:3/51: readlink d6/le 0 2024-04-26T23:01:20.066 INFO:tasks.workunit.client.0.smithi027.stdout:3/52: mknod d6/dc/c15 0 2024-04-26T23:01:20.066 INFO:tasks.workunit.client.0.smithi027.stdout:3/53: mknod d6/c16 0 2024-04-26T23:01:20.066 INFO:tasks.workunit.client.0.smithi027.stdout:9/58: fsync d4/f14 0 2024-04-26T23:01:20.066 INFO:tasks.workunit.client.0.smithi027.stdout:9/59: dread - d4/d10/f12 zero size 2024-04-26T23:01:20.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/60: creat d4/d10/f1a x:0 0 0 2024-04-26T23:01:20.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/61: write d4/d10/f1a [1018971,73513] 0 2024-04-26T23:01:20.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/62: chown d4/d10 285 1 2024-04-26T23:01:20.068 INFO:tasks.workunit.client.0.smithi027.stdout:9/63: dread d4/f14 [0,4194304] 0 2024-04-26T23:01:20.068 INFO:tasks.workunit.client.0.smithi027.stdout:9/64: mknod d4/c1b 0 2024-04-26T23:01:20.112 INFO:tasks.workunit.client.0.smithi027.stdout:6/12: dwrite d1/f2 [0,4194304] 0 2024-04-26T23:01:20.129 INFO:tasks.workunit.client.0.smithi027.stdout:5/23: dwrite f1 [0,4194304] 0 2024-04-26T23:01:20.154 INFO:tasks.workunit.client.0.smithi027.stdout:3/54: dwrite d6/f13 [0,4194304] 0 2024-04-26T23:01:20.154 INFO:tasks.workunit.client.0.smithi027.stdout:1/41: dwrite d1/fb [0,4194304] 0 2024-04-26T23:01:20.166 INFO:tasks.workunit.client.0.smithi027.stdout:9/65: dwrite d4/f14 [0,4194304] 0 2024-04-26T23:01:20.229 INFO:tasks.workunit.client.0.smithi027.stdout:1/42: dwrite d1/f4 [0,4194304] 0 2024-04-26T23:01:20.229 INFO:tasks.workunit.client.0.smithi027.stdout:1/43: readlink d1/d5/la 0 2024-04-26T23:01:20.251 INFO:tasks.workunit.client.0.smithi027.stdout:5/24: dwrite d4/f9 [0,4194304] 0 2024-04-26T23:01:20.253 INFO:tasks.workunit.client.0.smithi027.stdout:5/25: write d4/f8 [3091,87720] 0 2024-04-26T23:01:20.264 INFO:tasks.workunit.client.0.smithi027.stdout:6/13: mkdir d1/d4 0 2024-04-26T23:01:20.270 INFO:tasks.workunit.client.0.smithi027.stdout:9/66: link d4/l6 d4/l1c 0 2024-04-26T23:01:20.276 INFO:tasks.workunit.client.0.smithi027.stdout:1/44: mkdir d1/dc 0 2024-04-26T23:01:20.276 INFO:tasks.workunit.client.0.smithi027.stdout:5/26: mknod d4/ca 0 2024-04-26T23:01:20.276 INFO:tasks.workunit.client.0.smithi027.stdout:5/27: mkdir d4/db 0 2024-04-26T23:01:20.277 INFO:tasks.workunit.client.0.smithi027.stdout:9/67: rmdir d4/d10 39 2024-04-26T23:01:20.277 INFO:tasks.workunit.client.0.smithi027.stdout:6/14: rmdir d1/d4 0 2024-04-26T23:01:20.278 INFO:tasks.workunit.client.0.smithi027.stdout:9/68: unlink c2 0 2024-04-26T23:01:20.278 INFO:tasks.workunit.client.0.smithi027.stdout:9/69: read d4/fe [804060,39316] 0 2024-04-26T23:01:20.279 INFO:tasks.workunit.client.0.smithi027.stdout:9/70: fsync d4/f11 0 2024-04-26T23:01:20.279 INFO:tasks.workunit.client.0.smithi027.stdout:9/71: truncate d4/f11 941017 0 2024-04-26T23:01:20.279 INFO:tasks.workunit.client.0.smithi027.stdout:9/72: truncate d4/d10/f1a 1758984 0 2024-04-26T23:01:20.279 INFO:tasks.workunit.client.0.smithi027.stdout:3/55: dwrite d6/dc/fd [0,4194304] 0 2024-04-26T23:01:20.281 INFO:tasks.workunit.client.0.smithi027.stdout:6/15: unlink d1/f3 0 2024-04-26T23:01:20.282 INFO:tasks.workunit.client.0.smithi027.stdout:9/73: getdents d4/d10 0 2024-04-26T23:01:20.282 INFO:tasks.workunit.client.0.smithi027.stdout:9/74: chown d4/d17 28353 1 2024-04-26T23:01:20.282 INFO:tasks.workunit.client.0.smithi027.stdout:9/75: dread - d4/d10/f12 zero size 2024-04-26T23:01:20.284 INFO:tasks.workunit.client.0.smithi027.stdout:3/56: mkdir d6/dc/d17 0 2024-04-26T23:01:20.293 INFO:tasks.workunit.client.0.smithi027.stdout:5/28: dread d4/f8 [0,4194304] 0 2024-04-26T23:01:20.293 INFO:tasks.workunit.client.0.smithi027.stdout:5/29: readlink d4/l5 0 2024-04-26T23:01:20.326 INFO:tasks.workunit.client.0.smithi027.stdout:1/45: read d1/fb [2393154,68356] 0 2024-04-26T23:01:20.327 INFO:tasks.workunit.client.0.smithi027.stdout:1/46: symlink d1/dc/ld 0 2024-04-26T23:01:20.327 INFO:tasks.workunit.client.0.smithi027.stdout:1/47: read - d1/f6 zero size 2024-04-26T23:01:20.328 INFO:tasks.workunit.client.0.smithi027.stdout:1/48: creat d1/fe x:0 0 0 2024-04-26T23:01:20.330 INFO:tasks.workunit.client.0.smithi027.stdout:1/49: truncate d1/fb 1368735 0 2024-04-26T23:01:20.331 INFO:tasks.workunit.client.0.smithi027.stdout:1/50: creat d1/dc/ff x:0 0 0 2024-04-26T23:01:20.335 INFO:tasks.workunit.client.0.smithi027.stdout:1/51: creat d1/f10 x:0 0 0 2024-04-26T23:01:20.336 INFO:tasks.workunit.client.0.smithi027.stdout:1/52: chown d1/dc 13742592 1 2024-04-26T23:01:20.336 INFO:tasks.workunit.client.0.smithi027.stdout:1/53: readlink d1/l2 0 2024-04-26T23:01:20.359 INFO:tasks.workunit.client.0.smithi027.stdout:5/30: truncate d4/f8 243145 0 2024-04-26T23:01:20.364 INFO:tasks.workunit.client.0.smithi027.stdout:5/31: mkdir d4/db/dc 0 2024-04-26T23:01:20.368 INFO:tasks.workunit.client.0.smithi027.stdout:3/57: dwrite d6/f13 [0,4194304] 0 2024-04-26T23:01:20.368 INFO:tasks.workunit.client.0.smithi027.stdout:3/58: fsync d6/dc/fd 0 2024-04-26T23:01:20.372 INFO:tasks.workunit.client.0.smithi027.stdout:3/59: creat d6/f18 x:0 0 0 2024-04-26T23:01:20.372 INFO:tasks.workunit.client.0.smithi027.stdout:3/60: write d6/f13 [5228462,78652] 0 2024-04-26T23:01:20.373 INFO:tasks.workunit.client.0.smithi027.stdout:3/61: truncate d6/f8 451473 0 2024-04-26T23:01:20.374 INFO:tasks.workunit.client.0.smithi027.stdout:3/62: getdents d6/dc 0 2024-04-26T23:01:20.375 INFO:tasks.workunit.client.0.smithi027.stdout:3/63: mknod d6/c19 0 2024-04-26T23:01:20.375 INFO:tasks.workunit.client.0.smithi027.stdout:3/64: truncate d6/dc/f11 745339 0 2024-04-26T23:01:20.382 INFO:tasks.workunit.client.0.smithi027.stdout:5/32: write d4/f8 [12254,91247] 0 2024-04-26T23:01:20.388 INFO:tasks.workunit.client.0.smithi027.stdout:5/33: mknod d4/db/cd 0 2024-04-26T23:01:20.388 INFO:tasks.workunit.client.0.smithi027.stdout:3/65: write d6/f8 [1484169,28548] 0 2024-04-26T23:01:20.388 INFO:tasks.workunit.client.0.smithi027.stdout:3/66: symlink d6/dc/l1a 0 2024-04-26T23:01:20.388 INFO:tasks.workunit.client.0.smithi027.stdout:3/67: write d6/dc/f11 [1159872,127846] 0 2024-04-26T23:01:20.388 INFO:tasks.workunit.client.0.smithi027.stdout:3/68: fdatasync d6/dc/fd 0 2024-04-26T23:01:20.389 INFO:tasks.workunit.client.0.smithi027.stdout:3/69: rename d6/dc/l1a to d6/l1b 0 2024-04-26T23:01:20.401 INFO:tasks.workunit.client.0.smithi027.stdout:3/70: write d6/f8 [664321,118488] 0 2024-04-26T23:01:20.401 INFO:tasks.workunit.client.0.smithi027.stdout:3/71: write d6/f8 [1714622,66175] 0 2024-04-26T23:01:20.421 INFO:tasks.workunit.client.0.smithi027.stdout:5/34: write d4/f9 [1718605,52845] 0 2024-04-26T23:01:20.421 INFO:tasks.workunit.client.0.smithi027.stdout:5/35: chown d4/f8 53830 1 2024-04-26T23:01:20.422 INFO:tasks.workunit.client.0.smithi027.stdout:5/36: chown f1 264 1 2024-04-26T23:01:20.426 INFO:tasks.workunit.client.0.smithi027.stdout:5/37: link f1 d4/db/fe 0 2024-04-26T23:01:20.432 INFO:tasks.workunit.client.0.smithi027.stdout:9/76: dwrite d4/d10/f1a [0,4194304] 0 2024-04-26T23:01:20.435 INFO:tasks.workunit.client.0.smithi027.stdout:1/54: dwrite d1/dc/ff [0,4194304] 0 2024-04-26T23:01:20.466 INFO:tasks.workunit.client.0.smithi027.stdout:1/55: write d1/f4 [2208388,81701] 0 2024-04-26T23:01:20.499 INFO:tasks.workunit.client.0.smithi027.stdout:5/38: creat d4/d7/ff x:0 0 0 2024-04-26T23:01:20.499 INFO:tasks.workunit.client.0.smithi027.stdout:5/39: readlink d4/l5 0 2024-04-26T23:01:20.499 INFO:tasks.workunit.client.0.smithi027.stdout:5/40: dread - d4/d7/ff zero size 2024-04-26T23:01:20.537 INFO:tasks.workunit.client.0.smithi027.stdout:1/56: dwrite d1/fb [0,4194304] 0 2024-04-26T23:01:20.537 INFO:tasks.workunit.client.0.smithi027.stdout:1/57: truncate d1/f10 508725 0 2024-04-26T23:01:20.538 INFO:tasks.workunit.client.0.smithi027.stdout:1/58: getdents d1/dc 0 2024-04-26T23:01:20.540 INFO:tasks.workunit.client.0.smithi027.stdout:3/72: dwrite d6/dc/fd [0,4194304] 0 2024-04-26T23:01:20.550 INFO:tasks.workunit.client.0.smithi027.stdout:3/73: dread f2 [0,4194304] 0 2024-04-26T23:01:20.550 INFO:tasks.workunit.client.0.smithi027.stdout:3/74: dread - d6/f18 zero size 2024-04-26T23:01:20.550 INFO:tasks.workunit.client.0.smithi027.stdout:3/75: chown d6/l9 9743651 1 2024-04-26T23:01:20.551 INFO:tasks.workunit.client.0.smithi027.stdout:3/76: link d6/dc/fd d6/dc/d17/f1c 0 2024-04-26T23:01:20.557 INFO:tasks.workunit.client.0.smithi027.stdout:3/77: readlink d6/l1b 0 2024-04-26T23:01:20.557 INFO:tasks.workunit.client.0.smithi027.stdout:3/78: chown d6/dc/d17 12 1 2024-04-26T23:01:20.567 INFO:tasks.workunit.client.0.smithi027.stdout:9/77: dwrite d4/d10/f12 [0,4194304] 0 2024-04-26T23:01:20.567 INFO:tasks.workunit.client.0.smithi027.stdout:1/59: truncate d1/fb 3077012 0 2024-04-26T23:01:20.582 INFO:tasks.workunit.client.0.smithi027.stdout:3/79: write d6/dc/fd [1447316,43825] 0 2024-04-26T23:01:20.604 INFO:tasks.workunit.client.0.smithi027.stdout:1/60: creat d1/dc/f11 x:0 0 0 2024-04-26T23:01:20.606 INFO:tasks.workunit.client.0.smithi027.stdout:3/80: mkdir d6/dc/d1d 0 2024-04-26T23:01:20.606 INFO:tasks.workunit.client.0.smithi027.stdout:3/81: truncate f2 1706313 0 2024-04-26T23:01:20.608 INFO:tasks.workunit.client.0.smithi027.stdout:1/61: symlink d1/l12 0 2024-04-26T23:01:20.608 INFO:tasks.workunit.client.0.smithi027.stdout:1/62: fdatasync d1/f6 0 2024-04-26T23:01:20.608 INFO:tasks.workunit.client.0.smithi027.stdout:9/78: creat d4/d17/f1d x:0 0 0 2024-04-26T23:01:20.609 INFO:tasks.workunit.client.0.smithi027.stdout:1/63: symlink d1/dc/l13 0 2024-04-26T23:01:20.611 INFO:tasks.workunit.client.0.smithi027.stdout:9/79: creat d4/f1e x:0 0 0 2024-04-26T23:01:20.611 INFO:tasks.workunit.client.0.smithi027.stdout:9/80: truncate d4/d10/f19 591423 0 2024-04-26T23:01:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.612+0000 7fe4af400640 1 -- 172.21.15.27:0/3751424754 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588916f7e00 con 0x558891787ac0 2024-04-26T23:01:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.612+0000 7fe4af400640 10 client.25720 handle_command_reply: tid=0 2024-04-26T23:01:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.612+0000 7fe4d7813740 1 client.25720 shutdown 2024-04-26T23:01:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.612+0000 7fe4d7813740 1 -- 172.21.15.27:0/3751424754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588917935e0 msgr2=0x5588917a1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.612+0000 7fe4d7813740 1 --2- 172.21.15.27:0/3751424754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588917935e0 0x5588917a1aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc004a00 tx=0x7fe4bc004010 comp rx=0 tx=0).stop 2024-04-26T23:01:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.614+0000 7fe4d7813740 1 -- 172.21.15.27:0/3751424754 shutdown_connections 2024-04-26T23:01:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.614+0000 7fe4d7813740 1 --2- 172.21.15.27:0/3751424754 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x558891787ac0 0x558891789ee0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.614+0000 7fe4d7813740 1 --2- 172.21.15.27:0/3751424754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55889179e7c0 0x55889180c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.614+0000 7fe4af400640 0 client.25720 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:01:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.614+0000 7fe4d7813740 1 --2- 172.21.15.27:0/3751424754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558891793f00 0x5588917a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.614+0000 7fe4d7813740 1 --2- 172.21.15.27:0/3751424754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588917935e0 0x5588917a1aa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.614+0000 7fe4d7813740 1 -- 172.21.15.27:0/3751424754 >> 172.21.15.27:0/3751424754 conn(0x55889178faf0 msgr2=0x55889179bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:20.615 INFO:tasks.workunit.client.0.smithi027.stdout:9/81: rename d4/cc to d4/d10/c1f 0 2024-04-26T23:01:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.615+0000 7fe4d7813740 1 -- 172.21.15.27:0/3751424754 shutdown_connections 2024-04-26T23:01:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.615+0000 7fe4d7813740 1 -- 172.21.15.27:0/3751424754 wait complete. 2024-04-26T23:01:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.615+0000 7fe4d7813740 20 client.25720 trim_cache size 0 max 16384 2024-04-26T23:01:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T23:01:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T23:01:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T23:01:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T23:01:20.617 INFO:tasks.workunit.client.0.smithi027.stdout:9/82: mknod d4/d10/c20 0 2024-04-26T23:01:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.616+0000 7fe4d4a00640 1 -- 172.21.15.27:0/2950391044 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe4b40a3d90 msgr2=0x7fe4b40a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.616+0000 7fe4d4a00640 1 --2- 172.21.15.27:0/2950391044 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe4b40a3d90 0x7fe4b40a6250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe4c809c1d0 tx=0x7fe4c8098040 comp rx=0 tx=0).stop 2024-04-26T23:01:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.617+0000 7fe4d4a00640 1 -- 172.21.15.27:0/2950391044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4d0101030 msgr2=0x7fe4d019b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.617+0000 7fe4d4a00640 1 --2- 172.21.15.27:0/2950391044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4d0101030 0x7fe4d019b2e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800cce0 tx=0x7fe4c800a620 comp rx=0 tx=0).stop 2024-04-26T23:01:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.619+0000 7fe4d4a00640 1 -- 172.21.15.27:0/2950391044 shutdown_connections 2024-04-26T23:01:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.619+0000 7fe4d4a00640 1 --2- 172.21.15.27:0/2950391044 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe4b40a3d90 0x7fe4b40a6250 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.619+0000 7fe4d4a00640 1 --2- 172.21.15.27:0/2950391044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d0105ce0 0x7fe4d019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.619+0000 7fe4d4a00640 1 --2- 172.21.15.27:0/2950391044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d0101950 0x7fe4d019b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.619+0000 7fe4d4a00640 1 --2- 172.21.15.27:0/2950391044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4d0101030 0x7fe4d019b2e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.619+0000 7fe4d4a00640 1 -- 172.21.15.27:0/2950391044 >> 172.21.15.27:0/2950391044 conn(0x7fe4d0076740 msgr2=0x7fe4d00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.619+0000 7fe4d4a00640 1 -- 172.21.15.27:0/2950391044 shutdown_connections 2024-04-26T23:01:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:20.619+0000 7fe4d4a00640 1 -- 172.21.15.27:0/2950391044 wait complete. 2024-04-26T23:01:20.699 INFO:tasks.workunit.client.0.smithi027.stdout:3/82: dwrite d6/f7 [0,4194304] 0 2024-04-26T23:01:20.709 INFO:tasks.workunit.client.0.smithi027.stdout:3/83: write d6/dc/f11 [1136959,117976] 0 2024-04-26T23:01:20.709 INFO:tasks.workunit.client.0.smithi027.stdout:3/84: write d6/f18 [1042458,11256] 0 2024-04-26T23:01:20.717 INFO:tasks.workunit.client.0.smithi027.stdout:1/64: dwrite d1/fe [0,4194304] 0 2024-04-26T23:01:20.732 INFO:tasks.workunit.client.0.smithi027.stdout:1/65: dread d1/fb [0,4194304] 0 2024-04-26T23:01:20.733 INFO:tasks.workunit.client.0.smithi027.stdout:1/66: unlink d1/l2 0 2024-04-26T23:01:20.757 INFO:tasks.workunit.client.0.smithi027.stdout:1/67: dread d1/f10 [0,4194304] 0 2024-04-26T23:01:20.757 INFO:tasks.workunit.client.0.smithi027.stdout:1/68: chown d1/f3 610 1 2024-04-26T23:01:20.769 INFO:tasks.workunit.client.0.smithi027.stdout:3/85: dwrite d6/dc/d17/f1c [0,4194304] 0 2024-04-26T23:01:20.771 INFO:tasks.workunit.client.0.smithi027.stdout:3/86: creat d6/dc/d1d/f1e x:0 0 0 2024-04-26T23:01:20.772 INFO:tasks.workunit.client.0.smithi027.stdout:3/87: rmdir d6/dc/d17 39 2024-04-26T23:01:20.772 INFO:tasks.workunit.client.0.smithi027.stdout:3/88: write d6/f13 [5552778,97640] 0 2024-04-26T23:01:20.772 INFO:tasks.workunit.client.0.smithi027.stdout:1/69: read d1/fb [39494,35326] 0 2024-04-26T23:01:20.773 INFO:tasks.workunit.client.0.smithi027.stdout:3/89: write d6/f7 [4711140,44134] 0 2024-04-26T23:01:20.774 INFO:tasks.workunit.client.0.smithi027.stdout:1/70: getdents d1/dc 0 2024-04-26T23:01:20.774 INFO:tasks.workunit.client.0.smithi027.stdout:1/71: getdents d1/dc 0 2024-04-26T23:01:20.775 INFO:tasks.workunit.client.0.smithi027.stdout:3/90: truncate d6/f8 1628899 0 2024-04-26T23:01:20.776 INFO:tasks.workunit.client.0.smithi027.stdout:1/72: link d1/fe d1/dc/f14 0 2024-04-26T23:01:20.777 INFO:tasks.workunit.client.0.smithi027.stdout:1/73: symlink d1/d5/l15 0 2024-04-26T23:01:20.782 INFO:tasks.workunit.client.0.smithi027.stdout:1/74: rename d1/f4 to d1/dc/f16 0 2024-04-26T23:01:20.819 INFO:tasks.workunit.client.0.smithi027.stdout:1/75: dwrite d1/f7 [0,4194304] 0 2024-04-26T23:01:20.822 INFO:tasks.workunit.client.0.smithi027.stdout:1/76: truncate d1/dc/f14 3191453 0 2024-04-26T23:01:20.824 INFO:tasks.workunit.client.0.smithi027.stdout:1/77: symlink d1/dc/l17 0 2024-04-26T23:01:20.824 INFO:tasks.workunit.client.0.smithi027.stdout:1/78: chown d1/fb 126068 1 2024-04-26T23:01:20.826 INFO:tasks.workunit.client.0.smithi027.stdout:1/79: symlink d1/dc/l18 0 2024-04-26T23:01:20.828 INFO:tasks.workunit.client.0.smithi027.stdout:1/80: link d1/fe d1/d5/f19 0 2024-04-26T23:01:20.828 INFO:tasks.workunit.client.0.smithi027.stdout:1/81: write d1/fb [2451266,246] 0 2024-04-26T23:01:20.830 INFO:tasks.workunit.client.0.smithi027.stdout:1/82: rename d1/dc/ld to d1/d5/l1a 0 2024-04-26T23:01:20.914 INFO:tasks.workunit.client.0.smithi027.stdout:1/83: dwrite d1/dc/ff [0,4194304] 0 2024-04-26T23:01:20.955 INFO:tasks.workunit.client.0.smithi027.stdout:1/84: dwrite d1/dc/f14 [0,4194304] 0 2024-04-26T23:01:20.955 INFO:tasks.workunit.client.0.smithi027.stdout:1/85: chown d1/fe 4249833 1 2024-04-26T23:01:20.956 INFO:tasks.workunit.client.0.smithi027.stdout:1/86: truncate d1/dc/f11 373614 0 2024-04-26T23:01:20.956 INFO:tasks.workunit.client.0.smithi027.stdout:1/87: write d1/fb [1786415,42792] 0 2024-04-26T23:01:20.957 INFO:tasks.workunit.client.0.smithi027.stdout:1/88: getdents d1 0 2024-04-26T23:01:20.958 INFO:tasks.workunit.client.0.smithi027.stdout:1/89: creat d1/f1b x:0 0 0 2024-04-26T23:01:20.962 INFO:tasks.workunit.client.0.smithi027.stdout:1/90: getdents d1/dc 0 2024-04-26T23:01:20.963 INFO:tasks.workunit.client.0.smithi027.stdout:1/91: write d1/fb [92947,14613] 0 2024-04-26T23:01:20.988 INFO:tasks.workunit.client.0.smithi027.stdout:1/92: write d1/dc/ff [1171374,104110] 0 2024-04-26T23:01:20.989 INFO:tasks.workunit.client.0.smithi027.stdout:1/93: symlink d1/dc/l1c 0 2024-04-26T23:01:20.990 INFO:tasks.workunit.client.0.smithi027.stdout:1/94: creat d1/dc/f1d x:0 0 0 2024-04-26T23:01:20.991 INFO:tasks.workunit.client.0.smithi027.stdout:1/95: fdatasync d1/f3 0 2024-04-26T23:01:20.991 INFO:tasks.workunit.client.0.smithi027.stdout:1/96: chown d1/dc/f16 4321 1 2024-04-26T23:01:20.992 INFO:tasks.workunit.client.0.smithi027.stdout:1/97: mknod d1/d5/c1e 0 2024-04-26T23:01:20.996 INFO:tasks.workunit.client.0.smithi027.stdout:1/98: rename d1/fe to d1/f1f 0 2024-04-26T23:01:21.019 INFO:tasks.workunit.client.0.smithi027.stdout:0/3: sync 2024-04-26T23:01:21.019 INFO:tasks.workunit.client.0.smithi027.stdout:4/14: sync 2024-04-26T23:01:21.019 INFO:tasks.workunit.client.0.smithi027.stdout:8/16: sync 2024-04-26T23:01:21.019 INFO:tasks.workunit.client.0.smithi027.stdout:2/31: sync 2024-04-26T23:01:21.019 INFO:tasks.workunit.client.0.smithi027.stdout:0/4: read - no filename 2024-04-26T23:01:21.019 INFO:tasks.workunit.client.0.smithi027.stdout:0/5: unlink - no file 2024-04-26T23:01:21.019 INFO:tasks.workunit.client.0.smithi027.stdout:7/3: sync 2024-04-26T23:01:21.019 INFO:tasks.workunit.client.0.smithi027.stdout:0/6: chown . 10647 1 2024-04-26T23:01:21.020 INFO:tasks.workunit.client.0.smithi027.stdout:8/17: dread - f2 zero size 2024-04-26T23:01:21.020 INFO:tasks.workunit.client.0.smithi027.stdout:7/4: chown d0 7018 1 2024-04-26T23:01:21.020 INFO:tasks.workunit.client.0.smithi027.stdout:8/18: dread - f3 zero size 2024-04-26T23:01:21.020 INFO:tasks.workunit.client.0.smithi027.stdout:8/19: read - f6 zero size 2024-04-26T23:01:21.020 INFO:tasks.workunit.client.0.smithi027.stdout:2/32: rmdir d2/d5/d7 39 2024-04-26T23:01:21.020 INFO:tasks.workunit.client.0.smithi027.stdout:2/33: read - d2/d5/f9 zero size 2024-04-26T23:01:21.021 INFO:tasks.workunit.client.0.smithi027.stdout:4/15: creat f2 x:0 0 0 2024-04-26T23:01:21.021 INFO:tasks.workunit.client.0.smithi027.stdout:4/16: chown c1 863049 1 2024-04-26T23:01:21.021 INFO:tasks.workunit.client.0.smithi027.stdout:7/5: rmdir d0 39 2024-04-26T23:01:21.022 INFO:tasks.workunit.client.0.smithi027.stdout:7/6: truncate - no filename 2024-04-26T23:01:21.022 INFO:tasks.workunit.client.0.smithi027.stdout:7/7: write - no filename 2024-04-26T23:01:21.022 INFO:tasks.workunit.client.0.smithi027.stdout:7/8: readlink d0/l1 0 2024-04-26T23:01:21.022 INFO:tasks.workunit.client.0.smithi027.stdout:7/9: dread - no filename 2024-04-26T23:01:21.022 INFO:tasks.workunit.client.0.smithi027.stdout:7/10: dread - no filename 2024-04-26T23:01:21.022 INFO:tasks.workunit.client.0.smithi027.stdout:0/7: mkdir d0 0 2024-04-26T23:01:21.023 INFO:tasks.workunit.client.0.smithi027.stdout:8/20: creat f8 x:0 0 0 2024-04-26T23:01:21.024 INFO:tasks.workunit.client.0.smithi027.stdout:2/34: mkdir d2/d5/db 0 2024-04-26T23:01:21.024 INFO:tasks.workunit.client.0.smithi027.stdout:2/35: chown l1 28314 1 2024-04-26T23:01:21.025 INFO:tasks.workunit.client.0.smithi027.stdout:7/11: mknod d0/c2 0 2024-04-26T23:01:21.025 INFO:tasks.workunit.client.0.smithi027.stdout:0/8: creat d0/f1 x:0 0 0 2024-04-26T23:01:21.026 INFO:tasks.workunit.client.0.smithi027.stdout:8/21: mkdir d9 0 2024-04-26T23:01:21.026 INFO:tasks.workunit.client.0.smithi027.stdout:8/22: read - f8 zero size 2024-04-26T23:01:21.027 INFO:tasks.workunit.client.0.smithi027.stdout:2/36: mknod d2/d5/d7/cc 0 2024-04-26T23:01:21.027 INFO:tasks.workunit.client.0.smithi027.stdout:7/12: mkdir d0/d3 0 2024-04-26T23:01:21.027 INFO:tasks.workunit.client.0.smithi027.stdout:7/13: dread - no filename 2024-04-26T23:01:21.029 INFO:tasks.workunit.client.0.smithi027.stdout:0/9: symlink d0/l2 0 2024-04-26T23:01:21.029 INFO:tasks.workunit.client.0.smithi027.stdout:4/17: fsync f2 0 2024-04-26T23:01:21.031 INFO:tasks.workunit.client.0.smithi027.stdout:7/14: creat d0/d3/f4 x:0 0 0 2024-04-26T23:01:21.031 INFO:tasks.workunit.client.0.smithi027.stdout:7/15: read - d0/d3/f4 zero size 2024-04-26T23:01:21.032 INFO:tasks.workunit.client.0.smithi027.stdout:0/10: creat d0/f3 x:0 0 0 2024-04-26T23:01:21.032 INFO:tasks.workunit.client.0.smithi027.stdout:0/11: dread - d0/f1 zero size 2024-04-26T23:01:21.037 INFO:tasks.workunit.client.0.smithi027.stdout:2/37: rmdir d2 39 2024-04-26T23:01:21.037 INFO:tasks.workunit.client.0.smithi027.stdout:2/38: fsync d2/d5/fa 0 2024-04-26T23:01:21.037 INFO:tasks.workunit.client.0.smithi027.stdout:7/16: unlink d0/c2 0 2024-04-26T23:01:21.041 INFO:tasks.workunit.client.0.smithi027.stdout:7/17: symlink d0/l5 0 2024-04-26T23:01:21.042 INFO:tasks.workunit.client.0.smithi027.stdout:7/18: link d0/d3/f4 d0/f6 0 2024-04-26T23:01:21.044 INFO:tasks.workunit.client.0.smithi027.stdout:7/19: link d0/l5 d0/l7 0 2024-04-26T23:01:21.062 INFO:tasks.workunit.client.0.smithi027.stdout:8/23: dwrite f3 [0,4194304] 0 2024-04-26T23:01:21.065 INFO:tasks.workunit.client.0.smithi027.stdout:1/99: dwrite d1/dc/f14 [0,4194304] 0 2024-04-26T23:01:21.065 INFO:tasks.workunit.client.0.smithi027.stdout:1/100: stat d1/d5 0 2024-04-26T23:01:21.076 INFO:tasks.workunit.client.0.smithi027.stdout:4/18: dwrite f0 [0,4194304] 0 2024-04-26T23:01:21.076 INFO:tasks.workunit.client.0.smithi027.stdout:2/39: write d2/d5/fa [3069578,113362] 0 2024-04-26T23:01:21.080 INFO:tasks.workunit.client.0.smithi027.stdout:1/101: truncate d1/dc/ff 480262 0 2024-04-26T23:01:21.083 INFO:tasks.workunit.client.0.smithi027.stdout:2/40: unlink d2/d5/d7/l8 0 2024-04-26T23:01:21.089 INFO:tasks.workunit.client.0.smithi027.stdout:1/102: getdents d1/d5 0 2024-04-26T23:01:21.090 INFO:tasks.workunit.client.0.smithi027.stdout:1/103: write d1/f3 [984639,93086] 0 2024-04-26T23:01:21.091 INFO:tasks.workunit.client.0.smithi027.stdout:1/104: write d1/f10 [1000799,26030] 0 2024-04-26T23:01:21.112 INFO:tasks.workunit.client.0.smithi027.stdout:4/19: dwrite f0 [0,4194304] 0 2024-04-26T23:01:21.120 INFO:tasks.workunit.client.0.smithi027.stdout:7/20: dwrite d0/f6 [0,4194304] 0 2024-04-26T23:01:21.120 INFO:tasks.workunit.client.0.smithi027.stdout:7/21: chown d0/l1 7 1 2024-04-26T23:01:21.133 INFO:tasks.workunit.client.0.smithi027.stdout:4/20: write f0 [369385,105498] 0 2024-04-26T23:01:21.135 INFO:tasks.workunit.client.0.smithi027.stdout:4/21: rename f0 to f3 0 2024-04-26T23:01:21.135 INFO:tasks.workunit.client.0.smithi027.stdout:4/22: dread - f2 zero size 2024-04-26T23:01:21.136 INFO:tasks.workunit.client.0.smithi027.stdout:4/23: creat f4 x:0 0 0 2024-04-26T23:01:21.138 INFO:tasks.workunit.client.0.smithi027.stdout:4/24: mknod c5 0 2024-04-26T23:01:21.145 INFO:tasks.workunit.client.0.smithi027.stdout:4/25: fsync f3 0 2024-04-26T23:01:21.145 INFO:tasks.workunit.client.0.smithi027.stdout:4/26: write f3 [4460249,31396] 0 2024-04-26T23:01:21.145 INFO:tasks.workunit.client.0.smithi027.stdout:4/27: write f4 [25747,18040] 0 2024-04-26T23:01:21.146 INFO:tasks.workunit.client.0.smithi027.stdout:4/28: mknod c6 0 2024-04-26T23:01:21.148 INFO:tasks.workunit.client.0.smithi027.stdout:4/29: symlink l7 0 2024-04-26T23:01:21.148 INFO:tasks.workunit.client.0.smithi027.stdout:4/30: rmdir - no directory 2024-04-26T23:01:21.149 INFO:tasks.workunit.client.0.smithi027.stdout:4/31: creat f8 x:0 0 0 2024-04-26T23:01:21.150 INFO:tasks.workunit.client.0.smithi027.stdout:8/24: dwrite f6 [0,4194304] 0 2024-04-26T23:01:21.165 INFO:tasks.workunit.client.0.smithi027.stdout:7/22: write d0/d3/f4 [2978109,97659] 0 2024-04-26T23:01:21.166 INFO:tasks.workunit.client.0.smithi027.stdout:1/105: dwrite d1/f1f [0,4194304] 0 2024-04-26T23:01:21.166 INFO:tasks.workunit.client.0.smithi027.stdout:7/23: creat d0/d3/f8 x:0 0 0 2024-04-26T23:01:21.177 INFO:tasks.workunit.client.0.smithi027.stdout:7/24: unlink d0/d3/f8 0 2024-04-26T23:01:21.177 INFO:tasks.workunit.client.0.smithi027.stdout:1/106: rename d1/dc/l18 to d1/l20 0 2024-04-26T23:01:21.178 INFO:tasks.workunit.client.0.smithi027.stdout:1/107: write d1/dc/f16 [2591373,101358] 0 2024-04-26T23:01:21.178 INFO:tasks.workunit.client.0.smithi027.stdout:1/108: chown d1/dc/l1c 1 1 2024-04-26T23:01:21.178 INFO:tasks.workunit.client.0.smithi027.stdout:7/25: mknod d0/c9 0 2024-04-26T23:01:21.180 INFO:tasks.workunit.client.0.smithi027.stdout:1/109: mkdir d1/d5/d21 0 2024-04-26T23:01:21.181 INFO:tasks.workunit.client.0.smithi027.stdout:7/26: mkdir d0/da 0 2024-04-26T23:01:21.185 INFO:tasks.workunit.client.0.smithi027.stdout:1/110: rename d1/f7 to d1/d5/d21/f22 0 2024-04-26T23:01:21.200 INFO:tasks.workunit.client.0.smithi027.stdout:8/25: write f6 [3446204,81032] 0 2024-04-26T23:01:21.200 INFO:tasks.workunit.client.0.smithi027.stdout:2/41: dwrite d2/d5/fa [4194304,4194304] 0 2024-04-26T23:01:21.201 INFO:tasks.workunit.client.0.smithi027.stdout:4/32: dwrite f4 [0,4194304] 0 2024-04-26T23:01:21.201 INFO:tasks.workunit.client.0.smithi027.stdout:4/33: rmdir - no directory 2024-04-26T23:01:21.205 INFO:tasks.workunit.client.0.smithi027.stdout:2/42: creat d2/d5/d7/fd x:0 0 0 2024-04-26T23:01:21.205 INFO:tasks.workunit.client.0.smithi027.stdout:2/43: readlink d2/l4 0 2024-04-26T23:01:21.208 INFO:tasks.workunit.client.0.smithi027.stdout:2/44: unlink l1 0 2024-04-26T23:01:21.208 INFO:tasks.workunit.client.0.smithi027.stdout:2/45: fdatasync d2/d5/d7/fd 0 2024-04-26T23:01:21.211 INFO:tasks.workunit.client.0.smithi027.stdout:2/46: mkdir d2/d5/db/de 0 2024-04-26T23:01:21.221 INFO:tasks.workunit.client.0.smithi027.stdout:2/47: rename d2/d5/d7 to d2/d5/db/df 0 2024-04-26T23:01:21.221 INFO:tasks.workunit.client.0.smithi027.stdout:2/48: write d2/d5/fa [395635,94412] 0 2024-04-26T23:01:21.221 INFO:tasks.workunit.client.0.smithi027.stdout:2/49: chown d2 6974 1 2024-04-26T23:01:21.221 INFO:tasks.workunit.client.0.smithi027.stdout:4/34: dread f4 [0,4194304] 0 2024-04-26T23:01:21.222 INFO:tasks.workunit.client.0.smithi027.stdout:4/35: dread - f2 zero size 2024-04-26T23:01:21.223 INFO:tasks.workunit.client.0.smithi027.stdout:2/50: read - d2/d5/f9 zero size 2024-04-26T23:01:21.223 INFO:tasks.workunit.client.0.smithi027.stdout:2/51: fdatasync d2/d5/db/df/fd 0 2024-04-26T23:01:21.228 INFO:tasks.workunit.client.0.smithi027.stdout:4/36: creat f9 x:0 0 0 2024-04-26T23:01:21.228 INFO:tasks.workunit.client.0.smithi027.stdout:1/111: dwrite d1/d5/d21/f22 [4194304,4194304] 0 2024-04-26T23:01:21.228 INFO:tasks.workunit.client.0.smithi027.stdout:2/52: symlink d2/d5/db/l10 0 2024-04-26T23:01:21.228 INFO:tasks.workunit.client.0.smithi027.stdout:2/53: dread - d2/d5/f9 zero size 2024-04-26T23:01:21.229 INFO:tasks.workunit.client.0.smithi027.stdout:2/54: getdents d2/d5 0 2024-04-26T23:01:21.229 INFO:tasks.workunit.client.0.smithi027.stdout:2/55: stat d2/d5/f9 0 2024-04-26T23:01:21.230 INFO:tasks.workunit.client.0.smithi027.stdout:2/56: chown d2/d5/f9 9872 1 2024-04-26T23:01:21.230 INFO:tasks.workunit.client.0.smithi027.stdout:2/57: creat d2/f11 x:0 0 0 2024-04-26T23:01:21.232 INFO:tasks.workunit.client.0.smithi027.stdout:2/58: mkdir d2/d5/db/df/d12 0 2024-04-26T23:01:21.243 INFO:tasks.workunit.client.0.smithi027.stdout:4/37: dread f4 [0,4194304] 0 2024-04-26T23:01:21.246 INFO:tasks.workunit.client.0.smithi027.stdout:4/38: unlink c5 0 2024-04-26T23:01:21.246 INFO:tasks.workunit.client.0.smithi027.stdout:4/39: write f4 [4571051,79145] 0 2024-04-26T23:01:21.246 INFO:tasks.workunit.client.0.smithi027.stdout:4/40: truncate f4 5095863 0 2024-04-26T23:01:21.246 INFO:tasks.workunit.client.0.smithi027.stdout:4/41: dread - f8 zero size 2024-04-26T23:01:21.248 INFO:tasks.workunit.client.0.smithi027.stdout:4/42: mkdir da 0 2024-04-26T23:01:21.248 INFO:tasks.workunit.client.0.smithi027.stdout:4/43: chown l7 51789 1 2024-04-26T23:01:21.252 INFO:tasks.workunit.client.0.smithi027.stdout:8/26: dwrite f7 [0,4194304] 0 2024-04-26T23:01:21.255 INFO:tasks.workunit.client.0.smithi027.stdout:8/27: mkdir d9/da 0 2024-04-26T23:01:21.264 INFO:tasks.workunit.client.0.smithi027.stdout:4/44: dread f3 [0,4194304] 0 2024-04-26T23:01:21.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:20 smithi080 ceph-mon[32324]: pgmap v85: 129 pgs: 129 active+clean; 124 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-04-26T23:01:21.278 INFO:tasks.workunit.client.0.smithi027.stdout:2/59: dread d2/d5/fa [4194304,4194304] 0 2024-04-26T23:01:21.279 INFO:tasks.workunit.client.0.smithi027.stdout:2/60: dread - d2/d5/f9 zero size 2024-04-26T23:01:21.280 INFO:tasks.workunit.client.0.smithi027.stdout:2/61: link d2/f11 d2/d5/f13 0 2024-04-26T23:01:21.281 INFO:tasks.workunit.client.0.smithi027.stdout:2/62: chown d2/d5/db/df/fd 728 1 2024-04-26T23:01:21.281 INFO:tasks.workunit.client.0.smithi027.stdout:1/112: dwrite d1/dc/ff [0,4194304] 0 2024-04-26T23:01:21.282 INFO:tasks.workunit.client.0.smithi027.stdout:2/63: link d2/d5/db/df/fd d2/d5/db/f14 0 2024-04-26T23:01:21.284 INFO:tasks.workunit.client.0.smithi027.stdout:1/113: mknod d1/d5/c23 0 2024-04-26T23:01:21.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:20 smithi094 ceph-mon[32889]: pgmap v85: 129 pgs: 129 active+clean; 124 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-04-26T23:01:21.285 INFO:tasks.workunit.client.0.smithi027.stdout:1/114: creat d1/d5/d21/f24 x:0 0 0 2024-04-26T23:01:21.312 INFO:tasks.workunit.client.0.smithi027.stdout:4/45: dwrite f4 [4194304,4194304] 0 2024-04-26T23:01:21.312 INFO:tasks.workunit.client.0.smithi027.stdout:4/46: fsync f9 0 2024-04-26T23:01:21.314 INFO:tasks.workunit.client.0.smithi027.stdout:4/47: rename l7 to da/lb 0 2024-04-26T23:01:21.314 INFO:tasks.workunit.client.0.smithi027.stdout:4/48: chown da 5814728 1 2024-04-26T23:01:21.314 INFO:tasks.workunit.client.0.smithi027.stdout:4/49: readlink da/lb 0 2024-04-26T23:01:21.326 INFO:tasks.workunit.client.0.smithi027.stdout:2/64: dwrite d2/d5/f13 [0,4194304] 0 2024-04-26T23:01:21.329 INFO:tasks.workunit.client.0.smithi027.stdout:2/65: symlink d2/d5/db/df/d12/l15 0 2024-04-26T23:01:21.329 INFO:tasks.workunit.client.0.smithi027.stdout:2/66: read - d2/d5/db/f14 zero size 2024-04-26T23:01:21.329 INFO:tasks.workunit.client.0.smithi027.stdout:2/67: truncate d2/d5/f13 4812193 0 2024-04-26T23:01:21.329 INFO:tasks.workunit.client.0.smithi027.stdout:2/68: fdatasync d2/d5/fa 0 2024-04-26T23:01:21.337 INFO:tasks.workunit.client.0.smithi027.stdout:1/115: dwrite d1/d5/f19 [4194304,4194304] 0 2024-04-26T23:01:21.342 INFO:tasks.workunit.client.0.smithi027.stdout:1/116: link d1/d5/d21/f22 d1/dc/f25 0 2024-04-26T23:01:21.344 INFO:tasks.workunit.client.0.smithi027.stdout:2/69: dread d2/d5/fa [4194304,4194304] 0 2024-04-26T23:01:21.345 INFO:tasks.workunit.client.0.smithi027.stdout:1/117: mkdir d1/d26 0 2024-04-26T23:01:21.346 INFO:tasks.workunit.client.0.smithi027.stdout:4/50: dwrite f4 [4194304,4194304] 0 2024-04-26T23:01:21.350 INFO:tasks.workunit.client.0.smithi027.stdout:1/118: rmdir d1 39 2024-04-26T23:01:21.352 INFO:tasks.workunit.client.0.smithi027.stdout:1/119: creat d1/d5/d21/f27 x:0 0 0 2024-04-26T23:01:21.354 INFO:tasks.workunit.client.0.smithi027.stdout:1/120: rmdir d1 39 2024-04-26T23:01:21.354 INFO:tasks.workunit.client.0.smithi027.stdout:1/121: chown d1/dc/f16 80119 1 2024-04-26T23:01:21.354 INFO:tasks.workunit.client.0.smithi027.stdout:1/122: truncate d1/d5/d21/f24 990855 0 2024-04-26T23:01:21.356 INFO:tasks.workunit.client.0.smithi027.stdout:1/123: mknod d1/c28 0 2024-04-26T23:01:21.357 INFO:tasks.workunit.client.0.smithi027.stdout:1/124: link d1/d5/d21/f24 d1/d26/f29 0 2024-04-26T23:01:21.361 INFO:tasks.workunit.client.0.smithi027.stdout:4/51: read f4 [4351548,115983] 0 2024-04-26T23:01:21.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:20 smithi027 ceph-mon[28034]: pgmap v85: 129 pgs: 129 active+clean; 124 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-04-26T23:01:21.381 INFO:tasks.workunit.client.0.smithi027.stdout:1/125: dread d1/dc/f14 [4194304,4194304] 0 2024-04-26T23:01:21.396 INFO:tasks.workunit.client.0.smithi027.stdout:4/52: dwrite f2 [0,4194304] 0 2024-04-26T23:01:21.396 INFO:tasks.workunit.client.0.smithi027.stdout:4/53: fdatasync f9 0 2024-04-26T23:01:21.398 INFO:tasks.workunit.client.0.smithi027.stdout:4/54: rename f8 to da/fc 0 2024-04-26T23:01:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:3/91: sync 2024-04-26T23:01:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:9/83: sync 2024-04-26T23:01:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:5/41: sync 2024-04-26T23:01:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:6/16: sync 2024-04-26T23:01:22.066 INFO:tasks.workunit.client.0.smithi027.stdout:3/92: truncate d6/f7 4183268 0 2024-04-26T23:01:22.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/84: rename d4/f1e to d4/d17/f21 0 2024-04-26T23:01:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:3/93: unlink d6/dc/c15 0 2024-04-26T23:01:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:5/42: creat d4/db/f10 x:0 0 0 2024-04-26T23:01:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:3/94: mknod d6/dc/d1d/c1f 0 2024-04-26T23:01:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:5/43: link d4/db/cd d4/d7/c11 0 2024-04-26T23:01:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:5/44: chown d4/d7/c11 92 1 2024-04-26T23:01:22.074 INFO:tasks.workunit.client.0.smithi027.stdout:9/85: write d4/f11 [541368,108472] 0 2024-04-26T23:01:22.075 INFO:tasks.workunit.client.0.smithi027.stdout:5/45: rename d4/l5 to d4/l12 0 2024-04-26T23:01:22.077 INFO:tasks.workunit.client.0.smithi027.stdout:5/46: rmdir d4/d7 39 2024-04-26T23:01:22.086 INFO:tasks.workunit.client.0.smithi027.stdout:5/47: creat d4/d7/f13 x:0 0 0 2024-04-26T23:01:22.154 INFO:tasks.workunit.client.0.smithi027.stdout:6/17: dwrite d1/f2 [0,4194304] 0 2024-04-26T23:01:22.156 INFO:tasks.workunit.client.0.smithi027.stdout:6/18: mknod d1/c5 0 2024-04-26T23:01:22.158 INFO:tasks.workunit.client.0.smithi027.stdout:6/19: unlink d1/c5 0 2024-04-26T23:01:22.159 INFO:tasks.workunit.client.0.smithi027.stdout:3/95: dwrite d6/f8 [0,4194304] 0 2024-04-26T23:01:22.160 INFO:tasks.workunit.client.0.smithi027.stdout:6/20: creat d1/f6 x:0 0 0 2024-04-26T23:01:22.163 INFO:tasks.workunit.client.0.smithi027.stdout:3/96: creat d6/dc/d17/f20 x:0 0 0 2024-04-26T23:01:22.173 INFO:tasks.workunit.client.0.smithi027.stdout:6/21: getdents d1 0 2024-04-26T23:01:22.174 INFO:tasks.workunit.client.0.smithi027.stdout:6/22: mkdir d1/d7 0 2024-04-26T23:01:22.174 INFO:tasks.workunit.client.0.smithi027.stdout:6/23: chown d1/f2 96 1 2024-04-26T23:01:22.190 INFO:tasks.workunit.client.0.smithi027.stdout:3/97: write d6/dc/d17/f1c [4122448,65003] 0 2024-04-26T23:01:22.190 INFO:tasks.workunit.client.0.smithi027.stdout:3/98: chown d6/cf 3 1 2024-04-26T23:01:22.191 INFO:tasks.workunit.client.0.smithi027.stdout:3/99: creat d6/dc/f21 x:0 0 0 2024-04-26T23:01:22.193 INFO:tasks.workunit.client.0.smithi027.stdout:5/48: dwrite d4/f8 [0,4194304] 0 2024-04-26T23:01:22.193 INFO:tasks.workunit.client.0.smithi027.stdout:5/49: write d4/d7/f13 [60624,35372] 0 2024-04-26T23:01:22.195 INFO:tasks.workunit.client.0.smithi027.stdout:5/50: creat d4/f14 x:0 0 0 2024-04-26T23:01:22.195 INFO:tasks.workunit.client.0.smithi027.stdout:5/51: write d4/d7/ff [603699,65328] 0 2024-04-26T23:01:22.196 INFO:tasks.workunit.client.0.smithi027.stdout:9/86: dwrite d4/f14 [0,4194304] 0 2024-04-26T23:01:22.199 INFO:tasks.workunit.client.0.smithi027.stdout:5/52: rename d4/d7/f13 to d4/f15 0 2024-04-26T23:01:22.200 INFO:tasks.workunit.client.0.smithi027.stdout:9/87: mknod d4/d10/c22 0 2024-04-26T23:01:22.201 INFO:tasks.workunit.client.0.smithi027.stdout:9/88: mknod d4/d10/c23 0 2024-04-26T23:01:22.204 INFO:tasks.workunit.client.0.smithi027.stdout:5/53: fdatasync d4/d7/ff 0 2024-04-26T23:01:22.205 INFO:tasks.workunit.client.0.smithi027.stdout:5/54: mkdir d4/db/d16 0 2024-04-26T23:01:22.208 INFO:tasks.workunit.client.0.smithi027.stdout:1/126: sync 2024-04-26T23:01:22.208 INFO:tasks.workunit.client.0.smithi027.stdout:2/70: sync 2024-04-26T23:01:22.208 INFO:tasks.workunit.client.0.smithi027.stdout:7/27: sync 2024-04-26T23:01:22.210 INFO:tasks.workunit.client.0.smithi027.stdout:4/55: sync 2024-04-26T23:01:22.211 INFO:tasks.workunit.client.0.smithi027.stdout:0/12: sync 2024-04-26T23:01:22.211 INFO:tasks.workunit.client.0.smithi027.stdout:8/28: sync 2024-04-26T23:01:22.213 INFO:tasks.workunit.client.0.smithi027.stdout:7/28: rmdir d0/d3 39 2024-04-26T23:01:22.225 INFO:tasks.workunit.client.0.smithi027.stdout:0/13: mkdir d0/d4 0 2024-04-26T23:01:22.226 INFO:tasks.workunit.client.0.smithi027.stdout:2/71: creat d2/d5/db/df/f16 x:0 0 0 2024-04-26T23:01:22.226 INFO:tasks.workunit.client.0.smithi027.stdout:8/29: creat d9/fb x:0 0 0 2024-04-26T23:01:22.226 INFO:tasks.workunit.client.0.smithi027.stdout:2/72: fdatasync d2/d5/db/df/fd 0 2024-04-26T23:01:22.226 INFO:tasks.workunit.client.0.smithi027.stdout:2/73: truncate d2/d5/db/df/fd 877008 0 2024-04-26T23:01:22.226 INFO:tasks.workunit.client.0.smithi027.stdout:2/74: write d2/d5/db/f14 [901328,128374] 0 2024-04-26T23:01:22.227 INFO:tasks.workunit.client.0.smithi027.stdout:0/14: unlink d0/f1 0 2024-04-26T23:01:22.227 INFO:tasks.workunit.client.0.smithi027.stdout:0/15: fsync d0/f3 0 2024-04-26T23:01:22.227 INFO:tasks.workunit.client.0.smithi027.stdout:0/16: chown d0 31 1 2024-04-26T23:01:22.227 INFO:tasks.workunit.client.0.smithi027.stdout:0/17: dread - d0/f3 zero size 2024-04-26T23:01:22.227 INFO:tasks.workunit.client.0.smithi027.stdout:0/18: dread - d0/f3 zero size 2024-04-26T23:01:22.229 INFO:tasks.workunit.client.0.smithi027.stdout:8/30: mknod d9/da/cc 0 2024-04-26T23:01:22.229 INFO:tasks.workunit.client.0.smithi027.stdout:3/100: dread d6/f8 [0,4194304] 0 2024-04-26T23:01:22.229 INFO:tasks.workunit.client.0.smithi027.stdout:0/19: creat d0/d4/f5 x:0 0 0 2024-04-26T23:01:22.230 INFO:tasks.workunit.client.0.smithi027.stdout:0/20: symlink d0/l6 0 2024-04-26T23:01:22.233 INFO:tasks.workunit.client.0.smithi027.stdout:3/101: getdents d6/dc/d17 0 2024-04-26T23:01:22.234 INFO:tasks.workunit.client.0.smithi027.stdout:3/102: write d6/dc/f11 [2024500,21012] 0 2024-04-26T23:01:22.234 INFO:tasks.workunit.client.0.smithi027.stdout:4/56: dread f3 [4194304,4194304] 0 2024-04-26T23:01:22.239 INFO:tasks.workunit.client.0.smithi027.stdout:3/103: link d6/f7 d6/dc/d1d/f22 0 2024-04-26T23:01:22.239 INFO:tasks.workunit.client.0.smithi027.stdout:3/104: truncate d6/f8 1661514 0 2024-04-26T23:01:22.239 INFO:tasks.workunit.client.0.smithi027.stdout:9/89: write d4/f14 [1916838,105952] 0 2024-04-26T23:01:22.244 INFO:tasks.workunit.client.0.smithi027.stdout:3/105: symlink d6/l23 0 2024-04-26T23:01:22.245 INFO:tasks.workunit.client.0.smithi027.stdout:6/24: dwrite d1/f2 [0,4194304] 0 2024-04-26T23:01:22.258 INFO:tasks.workunit.client.0.smithi027.stdout:7/29: dread d0/d3/f4 [0,4194304] 0 2024-04-26T23:01:22.269 INFO:tasks.workunit.client.0.smithi027.stdout:7/30: symlink d0/d3/lb 0 2024-04-26T23:01:22.272 INFO:tasks.workunit.client.0.smithi027.stdout:7/31: unlink d0/f6 0 2024-04-26T23:01:22.275 INFO:tasks.workunit.client.0.smithi027.stdout:7/32: rename d0/da to d0/dc 0 2024-04-26T23:01:22.276 INFO:tasks.workunit.client.0.smithi027.stdout:7/33: stat d0/d3/f4 0 2024-04-26T23:01:22.277 INFO:tasks.workunit.client.0.smithi027.stdout:5/55: dwrite d4/db/f10 [0,4194304] 0 2024-04-26T23:01:22.280 INFO:tasks.workunit.client.0.smithi027.stdout:7/34: mknod d0/cd 0 2024-04-26T23:01:22.281 INFO:tasks.workunit.client.0.smithi027.stdout:9/90: write d4/d10/f1a [3369689,67044] 0 2024-04-26T23:01:22.281 INFO:tasks.workunit.client.0.smithi027.stdout:5/56: rmdir d4/d7 39 2024-04-26T23:01:22.282 INFO:tasks.workunit.client.0.smithi027.stdout:5/57: stat c0 0 2024-04-26T23:01:22.285 INFO:tasks.workunit.client.0.smithi027.stdout:2/75: dwrite d2/d5/f9 [0,4194304] 0 2024-04-26T23:01:22.292 INFO:tasks.workunit.client.0.smithi027.stdout:1/127: dwrite d1/dc/f1d [0,4194304] 0 2024-04-26T23:01:22.296 INFO:tasks.workunit.client.0.smithi027.stdout:2/76: mknod d2/c17 0 2024-04-26T23:01:22.297 INFO:tasks.workunit.client.0.smithi027.stdout:0/21: getdents d0/d4 0 2024-04-26T23:01:22.297 INFO:tasks.workunit.client.0.smithi027.stdout:1/128: getdents d1/d26 0 2024-04-26T23:01:22.297 INFO:tasks.workunit.client.0.smithi027.stdout:0/22: getdents d0 0 2024-04-26T23:01:22.297 INFO:tasks.workunit.client.0.smithi027.stdout:9/91: fdatasync d4/d10/f1a 0 2024-04-26T23:01:22.298 INFO:tasks.workunit.client.0.smithi027.stdout:4/57: dwrite f3 [4194304,4194304] 0 2024-04-26T23:01:22.299 INFO:tasks.workunit.client.0.smithi027.stdout:0/23: creat d0/d4/f7 x:0 0 0 2024-04-26T23:01:22.299 INFO:tasks.workunit.client.0.smithi027.stdout:0/24: chown d0/d4/f7 7019610 1 2024-04-26T23:01:22.299 INFO:tasks.workunit.client.0.smithi027.stdout:8/31: dwrite f3 [0,4194304] 0 2024-04-26T23:01:22.311 INFO:tasks.workunit.client.0.smithi027.stdout:9/92: creat d4/f24 x:0 0 0 2024-04-26T23:01:22.311 INFO:tasks.workunit.client.0.smithi027.stdout:3/106: write d6/dc/d1d/f22 [3329195,111382] 0 2024-04-26T23:01:22.311 INFO:tasks.workunit.client.0.smithi027.stdout:3/107: write d6/dc/d17/f20 [883954,109652] 0 2024-04-26T23:01:22.311 INFO:tasks.workunit.client.0.smithi027.stdout:3/108: chown d6/f13 852988420 1 2024-04-26T23:01:22.314 INFO:tasks.workunit.client.0.smithi027.stdout:7/35: rmdir d0/dc 0 2024-04-26T23:01:22.316 INFO:tasks.workunit.client.0.smithi027.stdout:0/25: getdents d0/d4 0 2024-04-26T23:01:22.316 INFO:tasks.workunit.client.0.smithi027.stdout:0/26: getdents d0/d4 0 2024-04-26T23:01:22.319 INFO:tasks.workunit.client.0.smithi027.stdout:9/93: mkdir d4/d10/d25 0 2024-04-26T23:01:22.322 INFO:tasks.workunit.client.0.smithi027.stdout:6/25: dwrite d1/f6 [0,4194304] 0 2024-04-26T23:01:22.327 INFO:tasks.workunit.client.0.smithi027.stdout:4/58: write f2 [3338374,67962] 0 2024-04-26T23:01:22.327 INFO:tasks.workunit.client.0.smithi027.stdout:9/94: unlink c0 0 2024-04-26T23:01:22.329 INFO:tasks.workunit.client.0.smithi027.stdout:3/109: mknod d6/dc/d1d/c24 0 2024-04-26T23:01:22.334 INFO:tasks.workunit.client.0.smithi027.stdout:1/129: dread d1/dc/ff [0,4194304] 0 2024-04-26T23:01:22.341 INFO:tasks.workunit.client.0.smithi027.stdout:0/27: creat d0/f8 x:0 0 0 2024-04-26T23:01:22.348 INFO:tasks.workunit.client.0.smithi027.stdout:1/130: mkdir d1/d2a 0 2024-04-26T23:01:22.360 INFO:tasks.workunit.client.0.smithi027.stdout:9/95: write d4/d10/f12 [941010,82059] 0 2024-04-26T23:01:22.379 INFO:tasks.workunit.client.0.smithi027.stdout:8/32: dwrite d9/fb [0,4194304] 0 2024-04-26T23:01:22.380 INFO:tasks.workunit.client.0.smithi027.stdout:8/33: stat d9 0 2024-04-26T23:01:22.381 INFO:tasks.workunit.client.0.smithi027.stdout:8/34: creat d9/da/fd x:0 0 0 2024-04-26T23:01:22.383 INFO:tasks.workunit.client.0.smithi027.stdout:8/35: mkdir d9/de 0 2024-04-26T23:01:22.387 INFO:tasks.workunit.client.0.smithi027.stdout:8/36: rename f8 to d9/de/ff 0 2024-04-26T23:01:22.388 INFO:tasks.workunit.client.0.smithi027.stdout:9/96: dread d4/d10/f12 [0,4194304] 0 2024-04-26T23:01:22.389 INFO:tasks.workunit.client.0.smithi027.stdout:4/59: dread f2 [0,4194304] 0 2024-04-26T23:01:22.389 INFO:tasks.workunit.client.0.smithi027.stdout:4/60: write f9 [586310,25309] 0 2024-04-26T23:01:22.389 INFO:tasks.workunit.client.0.smithi027.stdout:4/61: chown c6 3927 1 2024-04-26T23:01:22.389 INFO:tasks.workunit.client.0.smithi027.stdout:9/97: write d4/d10/f1a [4474088,114987] 0 2024-04-26T23:01:22.391 INFO:tasks.workunit.client.0.smithi027.stdout:9/98: symlink d4/d17/l26 0 2024-04-26T23:01:22.393 INFO:tasks.workunit.client.0.smithi027.stdout:9/99: getdents d4 0 2024-04-26T23:01:22.394 INFO:tasks.workunit.client.0.smithi027.stdout:9/100: mknod d4/d17/c27 0 2024-04-26T23:01:22.395 INFO:tasks.workunit.client.0.smithi027.stdout:3/110: dwrite d6/dc/f21 [0,4194304] 0 2024-04-26T23:01:22.396 INFO:tasks.workunit.client.0.smithi027.stdout:9/101: unlink d4/d17/l26 0 2024-04-26T23:01:22.397 INFO:tasks.workunit.client.0.smithi027.stdout:3/111: creat d6/dc/f25 x:0 0 0 2024-04-26T23:01:22.397 INFO:tasks.workunit.client.0.smithi027.stdout:3/112: fsync d6/f13 0 2024-04-26T23:01:22.398 INFO:tasks.workunit.client.0.smithi027.stdout:9/102: creat d4/d10/f28 x:0 0 0 2024-04-26T23:01:22.399 INFO:tasks.workunit.client.0.smithi027.stdout:3/113: creat d6/dc/d17/f26 x:0 0 0 2024-04-26T23:01:22.399 INFO:tasks.workunit.client.0.smithi027.stdout:9/103: creat d4/d17/f29 x:0 0 0 2024-04-26T23:01:22.400 INFO:tasks.workunit.client.0.smithi027.stdout:9/104: fdatasync d4/d10/f19 0 2024-04-26T23:01:22.400 INFO:tasks.workunit.client.0.smithi027.stdout:3/114: mknod d6/c27 0 2024-04-26T23:01:22.401 INFO:tasks.workunit.client.0.smithi027.stdout:0/28: dwrite d0/f8 [0,4194304] 0 2024-04-26T23:01:22.401 INFO:tasks.workunit.client.0.smithi027.stdout:9/105: symlink d4/d10/d25/l2a 0 2024-04-26T23:01:22.401 INFO:tasks.workunit.client.0.smithi027.stdout:0/29: write d0/d4/f5 [356775,126258] 0 2024-04-26T23:01:22.405 INFO:tasks.workunit.client.0.smithi027.stdout:0/30: creat d0/f9 x:0 0 0 2024-04-26T23:01:22.405 INFO:tasks.workunit.client.0.smithi027.stdout:0/31: readlink d0/l2 0 2024-04-26T23:01:22.405 INFO:tasks.workunit.client.0.smithi027.stdout:0/32: truncate d0/f8 4736225 0 2024-04-26T23:01:22.408 INFO:tasks.workunit.client.0.smithi027.stdout:0/33: rmdir d0/d4 39 2024-04-26T23:01:22.409 INFO:tasks.workunit.client.0.smithi027.stdout:4/62: dread f2 [0,4194304] 0 2024-04-26T23:01:22.409 INFO:tasks.workunit.client.0.smithi027.stdout:4/63: fsync f3 0 2024-04-26T23:01:22.409 INFO:tasks.workunit.client.0.smithi027.stdout:4/64: chown f3 122785564 1 2024-04-26T23:01:22.410 INFO:tasks.workunit.client.0.smithi027.stdout:4/65: mkdir da/dd 0 2024-04-26T23:01:22.416 INFO:tasks.workunit.client.0.smithi027.stdout:5/58: dwrite d4/f8 [0,4194304] 0 2024-04-26T23:01:22.416 INFO:tasks.workunit.client.0.smithi027.stdout:2/77: dwrite d2/d5/fa [4194304,4194304] 0 2024-04-26T23:01:22.416 INFO:tasks.workunit.client.0.smithi027.stdout:2/78: symlink d2/d5/db/df/l18 0 2024-04-26T23:01:22.432 INFO:tasks.workunit.client.0.smithi027.stdout:2/79: unlink d2/d5/db/df/d12/l15 0 2024-04-26T23:01:22.435 INFO:tasks.workunit.client.0.smithi027.stdout:3/115: getdents d6 0 2024-04-26T23:01:22.435 INFO:tasks.workunit.client.0.smithi027.stdout:0/34: getdents d0 0 2024-04-26T23:01:22.437 INFO:tasks.workunit.client.0.smithi027.stdout:7/36: dwrite d0/d3/f4 [0,4194304] 0 2024-04-26T23:01:22.438 INFO:tasks.workunit.client.0.smithi027.stdout:7/37: chown d0/d3/f4 66295387 1 2024-04-26T23:01:22.439 INFO:tasks.workunit.client.0.smithi027.stdout:5/59: write d4/f8 [765218,55971] 0 2024-04-26T23:01:22.440 INFO:tasks.workunit.client.0.smithi027.stdout:6/26: dwrite d1/f6 [0,4194304] 0 2024-04-26T23:01:22.440 INFO:tasks.workunit.client.0.smithi027.stdout:6/27: write d1/f6 [4241694,21000] 0 2024-04-26T23:01:22.443 INFO:tasks.workunit.client.0.smithi027.stdout:2/80: mkdir d2/d5/db/df/d12/d19 0 2024-04-26T23:01:22.443 INFO:tasks.workunit.client.0.smithi027.stdout:2/81: truncate d2/d5/db/df/f16 54393 0 2024-04-26T23:01:22.449 INFO:tasks.workunit.client.0.smithi027.stdout:3/116: mkdir d6/d28 0 2024-04-26T23:01:22.449 INFO:tasks.workunit.client.0.smithi027.stdout:2/82: write d2/d5/db/df/f16 [18110,82536] 0 2024-04-26T23:01:22.449 INFO:tasks.workunit.client.0.smithi027.stdout:2/83: chown d2/d5/fa 28 1 2024-04-26T23:01:22.449 INFO:tasks.workunit.client.0.smithi027.stdout:2/84: unlink d2/f11 0 2024-04-26T23:01:22.459 INFO:tasks.workunit.client.0.smithi027.stdout:1/131: write d1/f1f [287749,57476] 0 2024-04-26T23:01:22.459 INFO:tasks.workunit.client.0.smithi027.stdout:1/132: fsync d1/d5/d21/f24 0 2024-04-26T23:01:22.468 INFO:tasks.workunit.client.0.smithi027.stdout:6/28: write d1/f6 [800302,19362] 0 2024-04-26T23:01:22.485 INFO:tasks.workunit.client.0.smithi027.stdout:3/117: dwrite d6/f18 [0,4194304] 0 2024-04-26T23:01:22.486 INFO:tasks.workunit.client.0.smithi027.stdout:8/37: dwrite f4 [0,4194304] 0 2024-04-26T23:01:22.486 INFO:tasks.workunit.client.0.smithi027.stdout:8/38: read - d9/de/ff zero size 2024-04-26T23:01:22.488 INFO:tasks.workunit.client.0.smithi027.stdout:4/66: dwrite f3 [0,4194304] 0 2024-04-26T23:01:22.488 INFO:tasks.workunit.client.0.smithi027.stdout:4/67: fdatasync f9 0 2024-04-26T23:01:22.488 INFO:tasks.workunit.client.0.smithi027.stdout:3/118: rename d6/dc/d17/f26 to d6/f29 0 2024-04-26T23:01:22.488 INFO:tasks.workunit.client.0.smithi027.stdout:3/119: stat d6/dc 0 2024-04-26T23:01:22.491 INFO:tasks.workunit.client.0.smithi027.stdout:8/39: symlink d9/l10 0 2024-04-26T23:01:22.491 INFO:tasks.workunit.client.0.smithi027.stdout:5/60: read d4/f15 [7374,116367] 0 2024-04-26T23:01:22.491 INFO:tasks.workunit.client.0.smithi027.stdout:4/68: mkdir da/dd/de 0 2024-04-26T23:01:22.492 INFO:tasks.workunit.client.0.smithi027.stdout:4/69: mkdir da/df 0 2024-04-26T23:01:22.493 INFO:tasks.workunit.client.0.smithi027.stdout:4/70: stat f4 0 2024-04-26T23:01:22.493 INFO:tasks.workunit.client.0.smithi027.stdout:4/71: write da/fc [652779,64817] 0 2024-04-26T23:01:22.494 INFO:tasks.workunit.client.0.smithi027.stdout:4/72: mknod da/c10 0 2024-04-26T23:01:22.496 INFO:tasks.workunit.client.0.smithi027.stdout:4/73: mkdir da/df/d11 0 2024-04-26T23:01:22.499 INFO:tasks.workunit.client.0.smithi027.stdout:4/74: link da/c10 da/dd/c12 0 2024-04-26T23:01:22.502 INFO:tasks.workunit.client.0.smithi027.stdout:5/61: dread d4/d7/ff [0,4194304] 0 2024-04-26T23:01:22.502 INFO:tasks.workunit.client.0.smithi027.stdout:5/62: truncate d4/f8 4913823 0 2024-04-26T23:01:22.502 INFO:tasks.workunit.client.0.smithi027.stdout:5/63: read d4/f9 [849781,113096] 0 2024-04-26T23:01:22.502 INFO:tasks.workunit.client.0.smithi027.stdout:5/64: write d4/f15 [720523,100464] 0 2024-04-26T23:01:22.502 INFO:tasks.workunit.client.0.smithi027.stdout:5/65: read d4/f9 [2171605,119094] 0 2024-04-26T23:01:22.504 INFO:tasks.workunit.client.0.smithi027.stdout:5/66: creat d4/d7/f17 x:0 0 0 2024-04-26T23:01:22.504 INFO:tasks.workunit.client.0.smithi027.stdout:5/67: write d4/f9 [4450510,10815] 0 2024-04-26T23:01:22.504 INFO:tasks.workunit.client.0.smithi027.stdout:5/68: chown d4/f15 50084188 1 2024-04-26T23:01:22.507 INFO:tasks.workunit.client.0.smithi027.stdout:5/69: rename d4/f14 to d4/f18 0 2024-04-26T23:01:22.507 INFO:tasks.workunit.client.0.smithi027.stdout:5/70: write d4/db/f10 [4806063,21208] 0 2024-04-26T23:01:22.518 INFO:tasks.workunit.client.0.smithi027.stdout:2/85: write d2/d5/f9 [2793533,38112] 0 2024-04-26T23:01:22.518 INFO:tasks.workunit.client.0.smithi027.stdout:0/35: dwrite d0/f8 [0,4194304] 0 2024-04-26T23:01:22.521 INFO:tasks.workunit.client.0.smithi027.stdout:5/71: fsync d4/f9 0 2024-04-26T23:01:22.521 INFO:tasks.workunit.client.0.smithi027.stdout:5/72: readlink d4/l12 0 2024-04-26T23:01:22.525 INFO:tasks.workunit.client.0.smithi027.stdout:0/36: mkdir d0/d4/da 0 2024-04-26T23:01:22.525 INFO:tasks.workunit.client.0.smithi027.stdout:4/75: dread f3 [0,4194304] 0 2024-04-26T23:01:22.525 INFO:tasks.workunit.client.0.smithi027.stdout:4/76: rename da/df/d11 to da/df/d11/d13 22 2024-04-26T23:01:22.526 INFO:tasks.workunit.client.0.smithi027.stdout:4/77: chown da/dd 26 1 2024-04-26T23:01:22.530 INFO:tasks.workunit.client.0.smithi027.stdout:0/37: creat d0/d4/da/fb x:0 0 0 2024-04-26T23:01:22.530 INFO:tasks.workunit.client.0.smithi027.stdout:4/78: rename da/df/d11 to da/dd/de/d14 0 2024-04-26T23:01:22.531 INFO:tasks.workunit.client.0.smithi027.stdout:5/73: fsync d4/d7/f17 0 2024-04-26T23:01:22.532 INFO:tasks.workunit.client.0.smithi027.stdout:5/74: dread - d4/f18 zero size 2024-04-26T23:01:22.532 INFO:tasks.workunit.client.0.smithi027.stdout:4/79: creat da/dd/de/f15 x:0 0 0 2024-04-26T23:01:22.532 INFO:tasks.workunit.client.0.smithi027.stdout:4/80: stat f3 0 2024-04-26T23:01:22.534 INFO:tasks.workunit.client.0.smithi027.stdout:7/38: dwrite d0/d3/f4 [0,4194304] 0 2024-04-26T23:01:22.534 INFO:tasks.workunit.client.0.smithi027.stdout:4/81: truncate f3 7388475 0 2024-04-26T23:01:22.545 INFO:tasks.workunit.client.0.smithi027.stdout:3/120: dwrite d6/dc/f25 [0,4194304] 0 2024-04-26T23:01:22.546 INFO:tasks.workunit.client.0.smithi027.stdout:3/121: write d6/dc/d1d/f22 [4789826,87839] 0 2024-04-26T23:01:22.547 INFO:tasks.workunit.client.0.smithi027.stdout:3/122: getdents d6/dc 0 2024-04-26T23:01:22.549 INFO:tasks.workunit.client.0.smithi027.stdout:3/123: mknod d6/dc/d1d/c2a 0 2024-04-26T23:01:22.551 INFO:tasks.workunit.client.0.smithi027.stdout:3/124: creat d6/dc/d17/f2b x:0 0 0 2024-04-26T23:01:22.553 INFO:tasks.workunit.client.0.smithi027.stdout:3/125: unlink d6/dc/d1d/c2a 0 2024-04-26T23:01:22.561 INFO:tasks.workunit.client.0.smithi027.stdout:5/75: fsync d4/f15 0 2024-04-26T23:01:22.561 INFO:tasks.workunit.client.0.smithi027.stdout:1/133: dread d1/f1f [0,4194304] 0 2024-04-26T23:01:22.561 INFO:tasks.workunit.client.0.smithi027.stdout:1/134: dread - d1/f1b zero size 2024-04-26T23:01:22.562 INFO:tasks.workunit.client.0.smithi027.stdout:5/76: symlink d4/d7/l19 0 2024-04-26T23:01:22.564 INFO:tasks.workunit.client.0.smithi027.stdout:1/135: mkdir d1/d2b 0 2024-04-26T23:01:22.566 INFO:tasks.workunit.client.0.smithi027.stdout:1/136: creat d1/d5/d21/f2c x:0 0 0 2024-04-26T23:01:22.568 INFO:tasks.workunit.client.0.smithi027.stdout:7/39: write d0/d3/f4 [2000274,76075] 0 2024-04-26T23:01:22.571 INFO:tasks.workunit.client.0.smithi027.stdout:7/40: rename d0/d3/f4 to d0/fe 0 2024-04-26T23:01:22.576 INFO:tasks.workunit.client.0.smithi027.stdout:8/40: dwrite f2 [0,4194304] 0 2024-04-26T23:01:22.578 INFO:tasks.workunit.client.0.smithi027.stdout:5/77: dread d4/d7/ff [0,4194304] 0 2024-04-26T23:01:22.578 INFO:tasks.workunit.client.0.smithi027.stdout:5/78: chown d4/db/dc 14084 1 2024-04-26T23:01:22.580 INFO:tasks.workunit.client.0.smithi027.stdout:8/41: unlink d9/fb 0 2024-04-26T23:01:22.581 INFO:tasks.workunit.client.0.smithi027.stdout:0/38: dwrite d0/f9 [0,4194304] 0 2024-04-26T23:01:22.581 INFO:tasks.workunit.client.0.smithi027.stdout:5/79: truncate d4/f15 85499 0 2024-04-26T23:01:22.581 INFO:tasks.workunit.client.0.smithi027.stdout:5/80: write d4/d7/f17 [509060,104434] 0 2024-04-26T23:01:22.585 INFO:tasks.workunit.client.0.smithi027.stdout:4/82: dread f3 [0,4194304] 0 2024-04-26T23:01:22.585 INFO:tasks.workunit.client.0.smithi027.stdout:4/83: write f4 [4268782,70202] 0 2024-04-26T23:01:22.585 INFO:tasks.workunit.client.0.smithi027.stdout:4/84: fdatasync f3 0 2024-04-26T23:01:22.585 INFO:tasks.workunit.client.0.smithi027.stdout:0/39: symlink d0/d4/lc 0 2024-04-26T23:01:22.585 INFO:tasks.workunit.client.0.smithi027.stdout:0/40: readlink d0/d4/lc 0 2024-04-26T23:01:22.586 INFO:tasks.workunit.client.0.smithi027.stdout:4/85: creat da/df/f16 x:0 0 0 2024-04-26T23:01:22.589 INFO:tasks.workunit.client.0.smithi027.stdout:1/137: dread d1/fb [0,4194304] 0 2024-04-26T23:01:22.589 INFO:tasks.workunit.client.0.smithi027.stdout:1/138: chown d1/dc/f14 701257 1 2024-04-26T23:01:22.590 INFO:tasks.workunit.client.0.smithi027.stdout:1/139: mkdir d1/d2b/d2d 0 2024-04-26T23:01:22.602 INFO:tasks.workunit.client.0.smithi027.stdout:8/42: write f3 [2589565,102069] 0 2024-04-26T23:01:22.602 INFO:tasks.workunit.client.0.smithi027.stdout:8/43: fsync f1 0 2024-04-26T23:01:22.617 INFO:tasks.workunit.client.0.smithi027.stdout:0/41: write d0/f9 [2334191,16720] 0 2024-04-26T23:01:22.617 INFO:tasks.workunit.client.0.smithi027.stdout:0/42: chown d0/l6 1584 1 2024-04-26T23:01:22.626 INFO:tasks.workunit.client.0.smithi027.stdout:0/43: rename d0/l2 to d0/ld 0 2024-04-26T23:01:22.626 INFO:tasks.workunit.client.0.smithi027.stdout:0/44: chown d0/d4/da/fb 3 1 2024-04-26T23:01:22.627 INFO:tasks.workunit.client.0.smithi027.stdout:0/45: creat d0/d4/da/fe x:0 0 0 2024-04-26T23:01:22.628 INFO:tasks.workunit.client.0.smithi027.stdout:1/140: write d1/dc/f1d [1199350,41758] 0 2024-04-26T23:01:22.628 INFO:tasks.workunit.client.0.smithi027.stdout:1/141: write d1/f10 [1557375,54191] 0 2024-04-26T23:01:22.628 INFO:tasks.workunit.client.0.smithi027.stdout:1/142: chown d1/dc 0 1 2024-04-26T23:01:22.628 INFO:tasks.workunit.client.0.smithi027.stdout:0/46: mkdir d0/df 0 2024-04-26T23:01:22.629 INFO:tasks.workunit.client.0.smithi027.stdout:0/47: stat d0/f3 0 2024-04-26T23:01:22.629 INFO:tasks.workunit.client.0.smithi027.stdout:0/48: truncate d0/d4/da/fe 694912 0 2024-04-26T23:01:22.629 INFO:tasks.workunit.client.0.smithi027.stdout:0/49: chown d0/df 3967 1 2024-04-26T23:01:22.630 INFO:tasks.workunit.client.0.smithi027.stdout:2/86: dwrite d2/d5/fa [0,4194304] 0 2024-04-26T23:01:22.630 INFO:tasks.workunit.client.0.smithi027.stdout:1/143: creat d1/d5/f2e x:0 0 0 2024-04-26T23:01:22.630 INFO:tasks.workunit.client.0.smithi027.stdout:1/144: write d1/dc/f14 [64622,89005] 0 2024-04-26T23:01:22.631 INFO:tasks.workunit.client.0.smithi027.stdout:2/87: write d2/d5/db/df/f16 [780721,122075] 0 2024-04-26T23:01:22.633 INFO:tasks.workunit.client.0.smithi027.stdout:1/145: creat d1/d5/f2f x:0 0 0 2024-04-26T23:01:22.633 INFO:tasks.workunit.client.0.smithi027.stdout:1/146: truncate d1/d5/f19 9325956 0 2024-04-26T23:01:22.637 INFO:tasks.workunit.client.0.smithi027.stdout:1/147: unlink d1/d5/d21/f24 0 2024-04-26T23:01:22.640 INFO:tasks.workunit.client.0.smithi027.stdout:1/148: rename d1/dc/ff to d1/d2b/d2d/f30 0 2024-04-26T23:01:22.642 INFO:tasks.workunit.client.0.smithi027.stdout:1/149: symlink d1/d2b/d2d/l31 0 2024-04-26T23:01:22.643 INFO:tasks.workunit.client.0.smithi027.stdout:1/150: rmdir d1/d2b 39 2024-04-26T23:01:22.643 INFO:tasks.workunit.client.0.smithi027.stdout:1/151: chown d1/f1f 5 1 2024-04-26T23:01:22.657 INFO:tasks.workunit.client.0.smithi027.stdout:4/86: dwrite f4 [8388608,4194304] 0 2024-04-26T23:01:22.657 INFO:tasks.workunit.client.0.smithi027.stdout:8/44: write f2 [1041894,68651] 0 2024-04-26T23:01:22.658 INFO:tasks.workunit.client.0.smithi027.stdout:7/41: dwrite d0/fe [0,4194304] 0 2024-04-26T23:01:22.658 INFO:tasks.workunit.client.0.smithi027.stdout:7/42: fdatasync d0/fe 0 2024-04-26T23:01:22.662 INFO:tasks.workunit.client.0.smithi027.stdout:1/152: truncate d1/f1f 899022 0 2024-04-26T23:01:22.662 INFO:tasks.workunit.client.0.smithi027.stdout:8/45: link f3 d9/de/f11 0 2024-04-26T23:01:22.663 INFO:tasks.workunit.client.0.smithi027.stdout:7/43: creat d0/d3/ff x:0 0 0 2024-04-26T23:01:22.663 INFO:tasks.workunit.client.0.smithi027.stdout:7/44: fdatasync d0/d3/ff 0 2024-04-26T23:01:22.663 INFO:tasks.workunit.client.0.smithi027.stdout:7/45: fdatasync d0/d3/ff 0 2024-04-26T23:01:22.666 INFO:tasks.workunit.client.0.smithi027.stdout:8/46: rename d9/l10 to d9/da/l12 0 2024-04-26T23:01:22.668 INFO:tasks.workunit.client.0.smithi027.stdout:1/153: rmdir d1/dc 39 2024-04-26T23:01:22.672 INFO:tasks.workunit.client.0.smithi027.stdout:4/87: read f3 [37841,13308] 0 2024-04-26T23:01:22.672 INFO:tasks.workunit.client.0.smithi027.stdout:1/154: link d1/fb d1/f32 0 2024-04-26T23:01:22.672 INFO:tasks.workunit.client.0.smithi027.stdout:1/155: readlink d1/d5/l1a 0 2024-04-26T23:01:22.672 INFO:tasks.workunit.client.0.smithi027.stdout:4/88: write da/dd/de/f15 [578767,6987] 0 2024-04-26T23:01:22.675 INFO:tasks.workunit.client.0.smithi027.stdout:1/156: link d1/dc/l17 d1/d2b/d2d/l33 0 2024-04-26T23:01:22.677 INFO:tasks.workunit.client.0.smithi027.stdout:4/89: mkdir da/dd/de/d14/d17 0 2024-04-26T23:01:22.678 INFO:tasks.workunit.client.0.smithi027.stdout:1/157: write d1/f10 [2224000,110789] 0 2024-04-26T23:01:22.678 INFO:tasks.workunit.client.0.smithi027.stdout:4/90: rmdir da/dd/de/d14 39 2024-04-26T23:01:22.680 INFO:tasks.workunit.client.0.smithi027.stdout:0/50: dwrite d0/f9 [0,4194304] 0 2024-04-26T23:01:22.680 INFO:tasks.workunit.client.0.smithi027.stdout:0/51: readlink d0/d4/lc 0 2024-04-26T23:01:22.681 INFO:tasks.workunit.client.0.smithi027.stdout:1/158: creat d1/d5/d21/f34 x:0 0 0 2024-04-26T23:01:22.682 INFO:tasks.workunit.client.0.smithi027.stdout:4/91: unlink da/dd/c12 0 2024-04-26T23:01:22.682 INFO:tasks.workunit.client.0.smithi027.stdout:4/92: truncate da/dd/de/f15 725632 0 2024-04-26T23:01:22.683 INFO:tasks.workunit.client.0.smithi027.stdout:4/93: readlink da/lb 0 2024-04-26T23:01:22.683 INFO:tasks.workunit.client.0.smithi027.stdout:4/94: write f4 [12869429,17318] 0 2024-04-26T23:01:22.685 INFO:tasks.workunit.client.0.smithi027.stdout:0/52: symlink d0/l10 0 2024-04-26T23:01:22.685 INFO:tasks.workunit.client.0.smithi027.stdout:0/53: fsync d0/d4/f5 0 2024-04-26T23:01:22.698 INFO:tasks.workunit.client.0.smithi027.stdout:1/159: dread d1/dc/f14 [0,4194304] 0 2024-04-26T23:01:22.705 INFO:tasks.workunit.client.0.smithi027.stdout:7/46: dwrite d0/d3/ff [0,4194304] 0 2024-04-26T23:01:22.708 INFO:tasks.workunit.client.0.smithi027.stdout:8/47: dwrite f4 [0,4194304] 0 2024-04-26T23:01:22.713 INFO:tasks.workunit.client.0.smithi027.stdout:1/160: mkdir d1/d2b/d35 0 2024-04-26T23:01:22.713 INFO:tasks.workunit.client.0.smithi027.stdout:1/161: fdatasync d1/f1f 0 2024-04-26T23:01:22.717 INFO:tasks.workunit.client.0.smithi027.stdout:7/47: mknod d0/d3/c10 0 2024-04-26T23:01:22.717 INFO:tasks.workunit.client.0.smithi027.stdout:8/48: rename d9/de/ff to d9/da/f13 0 2024-04-26T23:01:22.717 INFO:tasks.workunit.client.0.smithi027.stdout:8/49: write d9/de/f11 [2611551,121902] 0 2024-04-26T23:01:22.721 INFO:tasks.workunit.client.0.smithi027.stdout:7/48: creat d0/f11 x:0 0 0 2024-04-26T23:01:22.721 INFO:tasks.workunit.client.0.smithi027.stdout:7/49: chown d0 10546 1 2024-04-26T23:01:22.722 INFO:tasks.workunit.client.0.smithi027.stdout:8/50: getdents d9 0 2024-04-26T23:01:22.726 INFO:tasks.workunit.client.0.smithi027.stdout:7/50: unlink d0/fe 0 2024-04-26T23:01:22.726 INFO:tasks.workunit.client.0.smithi027.stdout:0/54: dwrite d0/f9 [0,4194304] 0 2024-04-26T23:01:22.726 INFO:tasks.workunit.client.0.smithi027.stdout:4/95: dwrite f3 [0,4194304] 0 2024-04-26T23:01:22.728 INFO:tasks.workunit.client.0.smithi027.stdout:7/51: symlink d0/d3/l12 0 2024-04-26T23:01:22.730 INFO:tasks.workunit.client.0.smithi027.stdout:1/162: dread d1/f32 [0,4194304] 0 2024-04-26T23:01:22.731 INFO:tasks.workunit.client.0.smithi027.stdout:0/55: creat d0/d4/da/f11 x:0 0 0 2024-04-26T23:01:22.731 INFO:tasks.workunit.client.0.smithi027.stdout:0/56: fsync d0/d4/da/fb 0 2024-04-26T23:01:22.733 INFO:tasks.workunit.client.0.smithi027.stdout:0/57: creat d0/f12 x:0 0 0 2024-04-26T23:01:22.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/58: rename d0/d4/da/fe to d0/f13 0 2024-04-26T23:01:22.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/59: write d0/f3 [91923,45238] 0 2024-04-26T23:01:22.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/60: chown d0/d4 834 1 2024-04-26T23:01:22.737 INFO:tasks.workunit.client.0.smithi027.stdout:0/61: creat d0/d4/da/f14 x:0 0 0 2024-04-26T23:01:22.739 INFO:tasks.workunit.client.0.smithi027.stdout:8/51: truncate f4 2369738 0 2024-04-26T23:01:22.741 INFO:tasks.workunit.client.0.smithi027.stdout:4/96: truncate f4 8807203 0 2024-04-26T23:01:22.741 INFO:tasks.workunit.client.0.smithi027.stdout:4/97: write da/fc [1232814,118256] 0 2024-04-26T23:01:22.744 INFO:tasks.workunit.client.0.smithi027.stdout:8/52: link d9/da/f13 d9/de/f14 0 2024-04-26T23:01:22.744 INFO:tasks.workunit.client.0.smithi027.stdout:9/106: sync 2024-04-26T23:01:22.746 INFO:tasks.workunit.client.0.smithi027.stdout:6/29: sync 2024-04-26T23:01:22.746 INFO:tasks.workunit.client.0.smithi027.stdout:3/126: sync 2024-04-26T23:01:22.746 INFO:tasks.workunit.client.0.smithi027.stdout:6/30: write d1/f6 [4848658,24302] 0 2024-04-26T23:01:22.750 INFO:tasks.workunit.client.0.smithi027.stdout:8/53: mkdir d9/da/d15 0 2024-04-26T23:01:22.750 INFO:tasks.workunit.client.0.smithi027.stdout:9/107: link d4/f14 d4/f2b 0 2024-04-26T23:01:22.758 INFO:tasks.workunit.client.0.smithi027.stdout:6/31: creat d1/d7/f8 x:0 0 0 2024-04-26T23:01:22.763 INFO:tasks.workunit.client.0.smithi027.stdout:6/32: mknod d1/d7/c9 0 2024-04-26T23:01:22.766 INFO:tasks.workunit.client.0.smithi027.stdout:6/33: symlink d1/la 0 2024-04-26T23:01:22.767 INFO:tasks.workunit.client.0.smithi027.stdout:4/98: read f3 [3330857,31519] 0 2024-04-26T23:01:22.767 INFO:tasks.workunit.client.0.smithi027.stdout:6/34: symlink d1/d7/lb 0 2024-04-26T23:01:22.769 INFO:tasks.workunit.client.0.smithi027.stdout:4/99: write da/fc [1040225,118754] 0 2024-04-26T23:01:22.769 INFO:tasks.workunit.client.0.smithi027.stdout:4/100: write da/df/f16 [820699,105] 0 2024-04-26T23:01:22.769 INFO:tasks.workunit.client.0.smithi027.stdout:4/101: chown da/dd 43206 1 2024-04-26T23:01:22.770 INFO:tasks.workunit.client.0.smithi027.stdout:6/35: creat d1/d7/fc x:0 0 0 2024-04-26T23:01:22.771 INFO:tasks.workunit.client.0.smithi027.stdout:4/102: creat da/dd/de/d14/f18 x:0 0 0 2024-04-26T23:01:22.773 INFO:tasks.workunit.client.0.smithi027.stdout:4/103: rename da/dd/de/d14/f18 to da/df/f19 0 2024-04-26T23:01:22.773 INFO:tasks.workunit.client.0.smithi027.stdout:4/104: stat da/fc 0 2024-04-26T23:01:22.795 INFO:tasks.workunit.client.0.smithi027.stdout:6/36: fdatasync d1/f6 0 2024-04-26T23:01:22.795 INFO:tasks.workunit.client.0.smithi027.stdout:4/105: write da/df/f16 [177650,64366] 0 2024-04-26T23:01:22.795 INFO:tasks.workunit.client.0.smithi027.stdout:4/106: stat da/df/f16 0 2024-04-26T23:01:22.795 INFO:tasks.workunit.client.0.smithi027.stdout:4/107: rename da/dd/de to da/dd/de/d1a 22 2024-04-26T23:01:22.795 INFO:tasks.workunit.client.0.smithi027.stdout:9/108: dwrite d4/fe [4194304,4194304] 0 2024-04-26T23:01:22.797 INFO:tasks.workunit.client.0.smithi027.stdout:6/37: creat d1/d7/fd x:0 0 0 2024-04-26T23:01:22.798 INFO:tasks.workunit.client.0.smithi027.stdout:9/109: unlink d4/c1b 0 2024-04-26T23:01:22.809 INFO:tasks.workunit.client.0.smithi027.stdout:3/127: dwrite d6/dc/f10 [0,4194304] 0 2024-04-26T23:01:22.809 INFO:tasks.workunit.client.0.smithi027.stdout:8/54: dread f2 [0,4194304] 0 2024-04-26T23:01:22.811 INFO:tasks.workunit.client.0.smithi027.stdout:0/62: dwrite d0/d4/da/f14 [0,4194304] 0 2024-04-26T23:01:22.813 INFO:tasks.workunit.client.0.smithi027.stdout:3/128: getdents d6/dc/d1d 0 2024-04-26T23:01:22.814 INFO:tasks.workunit.client.0.smithi027.stdout:0/63: unlink d0/f3 0 2024-04-26T23:01:22.815 INFO:tasks.workunit.client.0.smithi027.stdout:8/55: rename d9/de/f11 to d9/f16 0 2024-04-26T23:01:22.815 INFO:tasks.workunit.client.0.smithi027.stdout:0/64: truncate d0/f9 3430002 0 2024-04-26T23:01:22.820 INFO:tasks.workunit.client.0.smithi027.stdout:8/56: rename f4 to d9/f17 0 2024-04-26T23:01:22.822 INFO:tasks.workunit.client.0.smithi027.stdout:8/57: creat d9/de/f18 x:0 0 0 2024-04-26T23:01:22.823 INFO:tasks.workunit.client.0.smithi027.stdout:8/58: symlink d9/da/d15/l19 0 2024-04-26T23:01:22.823 INFO:tasks.workunit.client.0.smithi027.stdout:8/59: truncate d9/da/f13 105314 0 2024-04-26T23:01:22.823 INFO:tasks.workunit.client.0.smithi027.stdout:8/60: write d9/de/f18 [338990,67165] 0 2024-04-26T23:01:22.824 INFO:tasks.workunit.client.0.smithi027.stdout:8/61: mknod d9/da/d15/c1a 0 2024-04-26T23:01:22.830 INFO:tasks.workunit.client.0.smithi027.stdout:8/62: creat d9/f1b x:0 0 0 2024-04-26T23:01:22.830 INFO:tasks.workunit.client.0.smithi027.stdout:8/63: write d9/f1b [156331,26871] 0 2024-04-26T23:01:22.831 INFO:tasks.workunit.client.0.smithi027.stdout:6/38: write d1/f2 [1528577,3039] 0 2024-04-26T23:01:22.831 INFO:tasks.workunit.client.0.smithi027.stdout:6/39: write d1/d7/f8 [656050,11691] 0 2024-04-26T23:01:22.833 INFO:tasks.workunit.client.0.smithi027.stdout:6/40: mkdir d1/de 0 2024-04-26T23:01:22.833 INFO:tasks.workunit.client.0.smithi027.stdout:6/41: readlink d1/la 0 2024-04-26T23:01:22.839 INFO:tasks.workunit.client.0.smithi027.stdout:6/42: rmdir d1 39 2024-04-26T23:01:22.842 INFO:tasks.workunit.client.0.smithi027.stdout:8/64: dread d9/da/f13 [0,4194304] 0 2024-04-26T23:01:22.843 INFO:tasks.workunit.client.0.smithi027.stdout:6/43: rename d1/d7 to d1/df 0 2024-04-26T23:01:22.845 INFO:tasks.workunit.client.0.smithi027.stdout:9/110: dwrite d4/d10/f19 [0,4194304] 0 2024-04-26T23:01:22.845 INFO:tasks.workunit.client.0.smithi027.stdout:9/111: chown d4/l1c 187687 1 2024-04-26T23:01:22.856 INFO:tasks.workunit.client.0.smithi027.stdout:6/44: getdents d1 0 2024-04-26T23:01:22.858 INFO:tasks.workunit.client.0.smithi027.stdout:6/45: getdents d1/df 0 2024-04-26T23:01:22.885 INFO:tasks.workunit.client.0.smithi027.stdout:4/108: dwrite da/df/f19 [0,4194304] 0 2024-04-26T23:01:22.886 INFO:tasks.workunit.client.0.smithi027.stdout:4/109: symlink da/dd/de/l1b 0 2024-04-26T23:01:22.887 INFO:tasks.workunit.client.0.smithi027.stdout:4/110: creat da/dd/de/f1c x:0 0 0 2024-04-26T23:01:22.888 INFO:tasks.workunit.client.0.smithi027.stdout:4/111: readlink da/dd/de/l1b 0 2024-04-26T23:01:22.889 INFO:tasks.workunit.client.0.smithi027.stdout:4/112: creat da/dd/de/d14/d17/f1d x:0 0 0 2024-04-26T23:01:22.891 INFO:tasks.workunit.client.0.smithi027.stdout:4/113: truncate f9 575620 0 2024-04-26T23:01:22.894 INFO:tasks.workunit.client.0.smithi027.stdout:3/129: dwrite d6/f8 [4194304,4194304] 0 2024-04-26T23:01:22.899 INFO:tasks.workunit.client.0.smithi027.stdout:3/130: rename f2 to d6/f2c 0 2024-04-26T23:01:22.899 INFO:tasks.workunit.client.0.smithi027.stdout:3/131: dread - d6/f29 zero size 2024-04-26T23:01:22.900 INFO:tasks.workunit.client.0.smithi027.stdout:8/65: dwrite f2 [0,4194304] 0 2024-04-26T23:01:22.901 INFO:tasks.workunit.client.0.smithi027.stdout:8/66: stat f7 0 2024-04-26T23:01:22.903 INFO:tasks.workunit.client.0.smithi027.stdout:8/67: unlink f1 0 2024-04-26T23:01:22.907 INFO:tasks.workunit.client.0.smithi027.stdout:8/68: read d9/f1b [111469,110533] 0 2024-04-26T23:01:22.910 INFO:tasks.workunit.client.0.smithi027.stdout:6/46: dwrite d1/f6 [0,4194304] 0 2024-04-26T23:01:22.911 INFO:tasks.workunit.client.0.smithi027.stdout:6/47: rename d1/df/c9 to d1/df/c10 0 2024-04-26T23:01:22.914 INFO:tasks.workunit.client.0.smithi027.stdout:6/48: rename d1/la to d1/l11 0 2024-04-26T23:01:22.917 INFO:tasks.workunit.client.0.smithi027.stdout:0/65: dwrite d0/d4/da/fb [0,4194304] 0 2024-04-26T23:01:22.920 INFO:tasks.workunit.client.0.smithi027.stdout:0/66: unlink d0/f8 0 2024-04-26T23:01:22.921 INFO:tasks.workunit.client.0.smithi027.stdout:0/67: chown d0/l10 0 1 2024-04-26T23:01:22.921 INFO:tasks.workunit.client.0.smithi027.stdout:0/68: write d0/d4/f5 [1085244,72203] 0 2024-04-26T23:01:22.921 INFO:tasks.workunit.client.0.smithi027.stdout:0/69: stat d0/d4/da/f14 0 2024-04-26T23:01:22.924 INFO:tasks.workunit.client.0.smithi027.stdout:4/114: dwrite da/df/f16 [0,4194304] 0 2024-04-26T23:01:22.924 INFO:tasks.workunit.client.0.smithi027.stdout:3/132: dread d6/dc/f25 [0,4194304] 0 2024-04-26T23:01:22.924 INFO:tasks.workunit.client.0.smithi027.stdout:4/115: chown da/dd/de/d14 3026167 1 2024-04-26T23:01:22.924 INFO:tasks.workunit.client.0.smithi027.stdout:0/70: mkdir d0/d4/da/d15 0 2024-04-26T23:01:22.928 INFO:tasks.workunit.client.0.smithi027.stdout:3/133: creat d6/f2d x:0 0 0 2024-04-26T23:01:22.929 INFO:tasks.workunit.client.0.smithi027.stdout:4/116: rmdir da/dd 39 2024-04-26T23:01:22.930 INFO:tasks.workunit.client.0.smithi027.stdout:4/117: symlink da/dd/de/d14/d17/l1e 0 2024-04-26T23:01:22.932 INFO:tasks.workunit.client.0.smithi027.stdout:9/112: dwrite d4/d17/f1d [0,4194304] 0 2024-04-26T23:01:22.932 INFO:tasks.workunit.client.0.smithi027.stdout:9/113: chown d4/d10/d25 1569756 1 2024-04-26T23:01:22.937 INFO:tasks.workunit.client.0.smithi027.stdout:9/114: mkdir d4/d10/d2c 0 2024-04-26T23:01:22.947 INFO:tasks.workunit.client.0.smithi027.stdout:8/69: dwrite f2 [0,4194304] 0 2024-04-26T23:01:22.948 INFO:tasks.workunit.client.0.smithi027.stdout:8/70: stat d9/de/f14 0 2024-04-26T23:01:22.950 INFO:tasks.workunit.client.0.smithi027.stdout:2/88: sync 2024-04-26T23:01:22.955 INFO:tasks.workunit.client.0.smithi027.stdout:5/81: sync 2024-04-26T23:01:22.962 INFO:tasks.workunit.client.0.smithi027.stdout:2/89: unlink d2/l4 0 2024-04-26T23:01:22.963 INFO:tasks.workunit.client.0.smithi027.stdout:2/90: truncate d2/d5/fa 3074610 0 2024-04-26T23:01:22.972 INFO:tasks.workunit.client.0.smithi027.stdout:6/49: dwrite d1/df/f8 [0,4194304] 0 2024-04-26T23:01:22.981 INFO:tasks.workunit.client.0.smithi027.stdout:8/71: dread d9/f17 [0,4194304] 0 2024-04-26T23:01:22.982 INFO:tasks.workunit.client.0.smithi027.stdout:8/72: fsync d9/de/f14 0 2024-04-26T23:01:22.982 INFO:tasks.workunit.client.0.smithi027.stdout:8/73: dread - d9/da/fd zero size 2024-04-26T23:01:22.982 INFO:tasks.workunit.client.0.smithi027.stdout:8/74: stat f3 0 2024-04-26T23:01:22.983 INFO:tasks.workunit.client.0.smithi027.stdout:5/82: read d4/db/f10 [4090042,85406] 0 2024-04-26T23:01:22.991 INFO:tasks.workunit.client.0.smithi027.stdout:0/71: dwrite d0/d4/da/fb [0,4194304] 0 2024-04-26T23:01:22.991 INFO:tasks.workunit.client.0.smithi027.stdout:0/72: chown d0/l10 155091 1 2024-04-26T23:01:22.991 INFO:tasks.workunit.client.0.smithi027.stdout:0/73: chown d0/d4/da/d15 97712687 1 2024-04-26T23:01:22.992 INFO:tasks.workunit.client.0.smithi027.stdout:0/74: creat d0/d4/da/d15/f16 x:0 0 0 2024-04-26T23:01:22.992 INFO:tasks.workunit.client.0.smithi027.stdout:0/75: write d0/d4/da/fb [4803322,83187] 0 2024-04-26T23:01:22.995 INFO:tasks.workunit.client.0.smithi027.stdout:0/76: mknod d0/d4/da/d15/c17 0 2024-04-26T23:01:22.995 INFO:tasks.workunit.client.0.smithi027.stdout:0/77: write d0/f9 [3681341,40972] 0 2024-04-26T23:01:22.996 INFO:tasks.workunit.client.0.smithi027.stdout:0/78: rmdir d0/d4/da/d15 39 2024-04-26T23:01:22.998 INFO:tasks.workunit.client.0.smithi027.stdout:0/79: getdents d0 0 2024-04-26T23:01:22.998 INFO:tasks.workunit.client.0.smithi027.stdout:9/115: dwrite d4/f11 [0,4194304] 0 2024-04-26T23:01:23.003 INFO:tasks.workunit.client.0.smithi027.stdout:0/80: creat d0/d4/f18 x:0 0 0 2024-04-26T23:01:23.003 INFO:tasks.workunit.client.0.smithi027.stdout:0/81: write d0/f12 [249107,42782] 0 2024-04-26T23:01:23.003 INFO:tasks.workunit.client.0.smithi027.stdout:9/116: link d4/d10/c23 d4/d17/c2d 0 2024-04-26T23:01:23.004 INFO:tasks.workunit.client.0.smithi027.stdout:9/117: mkdir d4/d17/d2e 0 2024-04-26T23:01:23.011 INFO:tasks.workunit.client.0.smithi027.stdout:6/50: dread d1/f2 [0,4194304] 0 2024-04-26T23:01:23.011 INFO:tasks.workunit.client.0.smithi027.stdout:6/51: dread - d1/df/fc zero size 2024-04-26T23:01:23.012 INFO:tasks.workunit.client.0.smithi027.stdout:6/52: getdents d1/df 0 2024-04-26T23:01:23.022 INFO:tasks.workunit.client.0.smithi027.stdout:6/53: mknod d1/df/c12 0 2024-04-26T23:01:23.022 INFO:tasks.workunit.client.0.smithi027.stdout:2/91: dwrite d2/d5/fa [0,4194304] 0 2024-04-26T23:01:23.023 INFO:tasks.workunit.client.0.smithi027.stdout:6/54: chown d1/df/fd 339206257 1 2024-04-26T23:01:23.028 INFO:tasks.workunit.client.0.smithi027.stdout:0/82: read d0/d4/da/fb [899771,31930] 0 2024-04-26T23:01:23.029 INFO:tasks.workunit.client.0.smithi027.stdout:8/75: dwrite d9/f1b [0,4194304] 0 2024-04-26T23:01:23.035 INFO:tasks.workunit.client.0.smithi027.stdout:0/83: creat d0/d4/da/f19 x:0 0 0 2024-04-26T23:01:23.035 INFO:tasks.workunit.client.0.smithi027.stdout:5/83: dwrite d4/f15 [0,4194304] 0 2024-04-26T23:01:23.035 INFO:tasks.workunit.client.0.smithi027.stdout:5/84: fdatasync d4/f8 0 2024-04-26T23:01:23.035 INFO:tasks.workunit.client.0.smithi027.stdout:8/76: symlink d9/da/l1c 0 2024-04-26T23:01:23.035 INFO:tasks.workunit.client.0.smithi027.stdout:8/77: chown d9/da/f13 251840 1 2024-04-26T23:01:23.043 INFO:tasks.workunit.client.0.smithi027.stdout:5/85: rmdir d4/db/dc 0 2024-04-26T23:01:23.043 INFO:tasks.workunit.client.0.smithi027.stdout:5/86: stat d4/ca 0 2024-04-26T23:01:23.044 INFO:tasks.workunit.client.0.smithi027.stdout:8/78: mknod d9/da/d15/c1d 0 2024-04-26T23:01:23.045 INFO:tasks.workunit.client.0.smithi027.stdout:2/92: write d2/d5/fa [2602924,17778] 0 2024-04-26T23:01:23.045 INFO:tasks.workunit.client.0.smithi027.stdout:5/87: mkdir d4/d1a 0 2024-04-26T23:01:23.045 INFO:tasks.workunit.client.0.smithi027.stdout:5/88: chown d4/db/d16 97464 1 2024-04-26T23:01:23.045 INFO:tasks.workunit.client.0.smithi027.stdout:5/89: write d4/d7/f17 [1285985,28326] 0 2024-04-26T23:01:23.071 INFO:tasks.workunit.client.0.smithi027.stdout:0/84: dwrite d0/d4/da/fb [0,4194304] 0 2024-04-26T23:01:23.073 INFO:tasks.workunit.client.0.smithi027.stdout:0/85: mkdir d0/d4/da/d1a 0 2024-04-26T23:01:23.073 INFO:tasks.workunit.client.0.smithi027.stdout:0/86: write d0/d4/da/f11 [325292,45611] 0 2024-04-26T23:01:23.083 INFO:tasks.workunit.client.0.smithi027.stdout:0/87: dread d0/d4/da/f11 [0,4194304] 0 2024-04-26T23:01:23.085 INFO:tasks.workunit.client.0.smithi027.stdout:0/88: creat d0/f1b x:0 0 0 2024-04-26T23:01:23.085 INFO:tasks.workunit.client.0.smithi027.stdout:0/89: chown d0/f13 28 1 2024-04-26T23:01:23.096 INFO:tasks.workunit.client.0.smithi027.stdout:6/55: dwrite d1/df/f8 [0,4194304] 0 2024-04-26T23:01:23.096 INFO:tasks.workunit.client.0.smithi027.stdout:6/56: fsync d1/df/f8 0 2024-04-26T23:01:23.098 INFO:tasks.workunit.client.0.smithi027.stdout:6/57: creat d1/de/f13 x:0 0 0 2024-04-26T23:01:23.098 INFO:tasks.workunit.client.0.smithi027.stdout:6/58: stat d1/df 0 2024-04-26T23:01:23.099 INFO:tasks.workunit.client.0.smithi027.stdout:6/59: truncate d1/f6 2067347 0 2024-04-26T23:01:23.100 INFO:tasks.workunit.client.0.smithi027.stdout:6/60: creat d1/df/f14 x:0 0 0 2024-04-26T23:01:23.100 INFO:tasks.workunit.client.0.smithi027.stdout:6/61: chown d1/f6 20753059 1 2024-04-26T23:01:23.100 INFO:tasks.workunit.client.0.smithi027.stdout:6/62: dread - d1/df/fd zero size 2024-04-26T23:01:23.101 INFO:tasks.workunit.client.0.smithi027.stdout:6/63: dread - d1/df/fc zero size 2024-04-26T23:01:23.103 INFO:tasks.workunit.client.0.smithi027.stdout:6/64: unlink d1/df/c12 0 2024-04-26T23:01:23.103 INFO:tasks.workunit.client.0.smithi027.stdout:6/65: write d1/df/fc [242426,52170] 0 2024-04-26T23:01:23.109 INFO:tasks.workunit.client.0.smithi027.stdout:0/90: write d0/d4/da/fb [1673641,118533] 0 2024-04-26T23:01:23.109 INFO:tasks.workunit.client.0.smithi027.stdout:0/91: write d0/d4/f7 [595175,9894] 0 2024-04-26T23:01:23.110 INFO:tasks.workunit.client.0.smithi027.stdout:2/93: dwrite d2/d5/db/f14 [0,4194304] 0 2024-04-26T23:01:23.111 INFO:tasks.workunit.client.0.smithi027.stdout:0/92: creat d0/d4/da/d1a/f1c x:0 0 0 2024-04-26T23:01:23.112 INFO:tasks.workunit.client.0.smithi027.stdout:2/94: creat d2/d5/db/f1a x:0 0 0 2024-04-26T23:01:23.115 INFO:tasks.workunit.client.0.smithi027.stdout:5/90: dwrite d4/f18 [0,4194304] 0 2024-04-26T23:01:23.115 INFO:tasks.workunit.client.0.smithi027.stdout:5/91: chown f1 15 1 2024-04-26T23:01:23.117 INFO:tasks.workunit.client.0.smithi027.stdout:2/95: mkdir d2/d5/db/d1b 0 2024-04-26T23:01:23.117 INFO:tasks.workunit.client.0.smithi027.stdout:0/93: rename d0/l6 to d0/d4/da/d1a/l1d 0 2024-04-26T23:01:23.117 INFO:tasks.workunit.client.0.smithi027.stdout:0/94: readlink d0/ld 0 2024-04-26T23:01:23.117 INFO:tasks.workunit.client.0.smithi027.stdout:0/95: chown d0/d4/f5 184794432 1 2024-04-26T23:01:23.117 INFO:tasks.workunit.client.0.smithi027.stdout:0/96: write d0/f9 [3774694,51369] 0 2024-04-26T23:01:23.117 INFO:tasks.workunit.client.0.smithi027.stdout:0/97: readlink d0/ld 0 2024-04-26T23:01:23.135 INFO:tasks.workunit.client.0.smithi027.stdout:7/52: sync 2024-04-26T23:01:23.135 INFO:tasks.workunit.client.0.smithi027.stdout:7/53: fdatasync d0/d3/ff 0 2024-04-26T23:01:23.135 INFO:tasks.workunit.client.0.smithi027.stdout:7/54: write d0/f11 [657591,97134] 0 2024-04-26T23:01:23.135 INFO:tasks.workunit.client.0.smithi027.stdout:7/55: chown d0/l7 68 1 2024-04-26T23:01:23.135 INFO:tasks.workunit.client.0.smithi027.stdout:5/92: dread d4/d7/f17 [0,4194304] 0 2024-04-26T23:01:23.145 INFO:tasks.workunit.client.0.smithi027.stdout:0/98: dread d0/f9 [0,4194304] 0 2024-04-26T23:01:23.149 INFO:tasks.workunit.client.0.smithi027.stdout:0/99: mkdir d0/d4/da/d1a/d1e 0 2024-04-26T23:01:23.150 INFO:tasks.workunit.client.0.smithi027.stdout:6/66: dwrite d1/de/f13 [0,4194304] 0 2024-04-26T23:01:23.151 INFO:tasks.workunit.client.0.smithi027.stdout:0/100: truncate d0/d4/f7 161345 0 2024-04-26T23:01:23.151 INFO:tasks.workunit.client.0.smithi027.stdout:0/101: chown d0/f9 113 1 2024-04-26T23:01:23.151 INFO:tasks.workunit.client.0.smithi027.stdout:0/102: getdents d0/df 0 2024-04-26T23:01:23.155 INFO:tasks.workunit.client.0.smithi027.stdout:6/67: mknod d1/df/c15 0 2024-04-26T23:01:23.155 INFO:tasks.workunit.client.0.smithi027.stdout:6/68: chown d1/df/c15 289 1 2024-04-26T23:01:23.155 INFO:tasks.workunit.client.0.smithi027.stdout:6/69: rename d1/de to d1/de/d16 22 2024-04-26T23:01:23.156 INFO:tasks.workunit.client.0.smithi027.stdout:6/70: symlink d1/l17 0 2024-04-26T23:01:23.161 INFO:tasks.workunit.client.0.smithi027.stdout:7/56: write d0/d3/ff [2010092,118891] 0 2024-04-26T23:01:23.163 INFO:tasks.workunit.client.0.smithi027.stdout:7/57: creat d0/f13 x:0 0 0 2024-04-26T23:01:23.164 INFO:tasks.workunit.client.0.smithi027.stdout:5/93: dread d4/db/fe [0,4194304] 0 2024-04-26T23:01:23.165 INFO:tasks.workunit.client.0.smithi027.stdout:5/94: creat d4/d7/f1b x:0 0 0 2024-04-26T23:01:23.176 INFO:tasks.workunit.client.0.smithi027.stdout:0/103: dread d0/f9 [0,4194304] 0 2024-04-26T23:01:23.176 INFO:tasks.workunit.client.0.smithi027.stdout:2/96: dwrite d2/d5/db/f1a [0,4194304] 0 2024-04-26T23:01:23.179 INFO:tasks.workunit.client.0.smithi027.stdout:0/104: creat d0/d4/da/f1f x:0 0 0 2024-04-26T23:01:23.181 INFO:tasks.workunit.client.0.smithi027.stdout:5/95: write d4/db/fe [3854127,112321] 0 2024-04-26T23:01:23.181 INFO:tasks.workunit.client.0.smithi027.stdout:5/96: truncate d4/f18 4627285 0 2024-04-26T23:01:23.181 INFO:tasks.workunit.client.0.smithi027.stdout:2/97: mkdir d2/d5/db/d1b/d1c 0 2024-04-26T23:01:23.183 INFO:tasks.workunit.client.0.smithi027.stdout:5/97: mknod d4/d7/c1c 0 2024-04-26T23:01:23.183 INFO:tasks.workunit.client.0.smithi027.stdout:5/98: chown d4/f15 700 1 2024-04-26T23:01:23.184 INFO:tasks.workunit.client.0.smithi027.stdout:5/99: creat d4/f1d x:0 0 0 2024-04-26T23:01:23.194 INFO:tasks.workunit.client.0.smithi027.stdout:7/58: dwrite d0/f11 [0,4194304] 0 2024-04-26T23:01:23.196 INFO:tasks.workunit.client.0.smithi027.stdout:2/98: rename d2/d5/f9 to d2/d5/db/f1d 0 2024-04-26T23:01:23.196 INFO:tasks.workunit.client.0.smithi027.stdout:5/100: creat d4/d1a/f1e x:0 0 0 2024-04-26T23:01:23.197 INFO:tasks.workunit.client.0.smithi027.stdout:7/59: mkdir d0/d3/d14 0 2024-04-26T23:01:23.199 INFO:tasks.workunit.client.0.smithi027.stdout:2/99: link d2/d5/db/df/fd d2/d5/db/df/d12/d19/f1e 0 2024-04-26T23:01:23.199 INFO:tasks.workunit.client.0.smithi027.stdout:2/100: rename d2/d5/db/de to d2/d5/db/de/d1f 22 2024-04-26T23:01:23.201 INFO:tasks.workunit.client.0.smithi027.stdout:7/60: link d0/d3/ff d0/d3/d14/f15 0 2024-04-26T23:01:23.201 INFO:tasks.workunit.client.0.smithi027.stdout:7/61: stat d0/l1 0 2024-04-26T23:01:23.203 INFO:tasks.workunit.client.0.smithi027.stdout:7/62: creat d0/d3/d14/f16 x:0 0 0 2024-04-26T23:01:23.203 INFO:tasks.workunit.client.0.smithi027.stdout:7/63: fdatasync d0/d3/ff 0 2024-04-26T23:01:23.218 INFO:tasks.workunit.client.0.smithi027.stdout:2/101: write d2/d5/db/df/d12/d19/f1e [266908,86260] 0 2024-04-26T23:01:23.222 INFO:tasks.workunit.client.0.smithi027.stdout:2/102: fdatasync d2/d5/db/df/fd 0 2024-04-26T23:01:23.227 INFO:tasks.workunit.client.0.smithi027.stdout:0/105: dwrite d0/d4/f5 [0,4194304] 0 2024-04-26T23:01:23.227 INFO:tasks.workunit.client.0.smithi027.stdout:0/106: read d0/d4/da/fb [1869352,81955] 0 2024-04-26T23:01:23.231 INFO:tasks.workunit.client.0.smithi027.stdout:0/107: link d0/f12 d0/d4/f20 0 2024-04-26T23:01:23.231 INFO:tasks.workunit.client.0.smithi027.stdout:0/108: chown d0/d4/da 3192010 1 2024-04-26T23:01:23.231 INFO:tasks.workunit.client.0.smithi027.stdout:0/109: truncate d0/f1b 277428 0 2024-04-26T23:01:23.231 INFO:tasks.workunit.client.0.smithi027.stdout:0/110: stat d0/d4/f7 0 2024-04-26T23:01:23.231 INFO:tasks.workunit.client.0.smithi027.stdout:0/111: dread - d0/d4/da/f19 zero size 2024-04-26T23:01:23.232 INFO:tasks.workunit.client.0.smithi027.stdout:0/112: chown d0 15 1 2024-04-26T23:01:23.232 INFO:tasks.workunit.client.0.smithi027.stdout:0/113: fsync d0/d4/da/f11 0 2024-04-26T23:01:23.232 INFO:tasks.workunit.client.0.smithi027.stdout:0/114: fsync d0/f1b 0 2024-04-26T23:01:23.232 INFO:tasks.workunit.client.0.smithi027.stdout:0/115: truncate d0/d4/f18 4992 0 2024-04-26T23:01:23.232 INFO:tasks.workunit.client.0.smithi027.stdout:0/116: dread - d0/d4/da/d1a/f1c zero size 2024-04-26T23:01:23.232 INFO:tasks.workunit.client.0.smithi027.stdout:0/117: dread - d0/d4/da/d15/f16 zero size 2024-04-26T23:01:23.233 INFO:tasks.workunit.client.0.smithi027.stdout:0/118: mknod d0/d4/da/d1a/d1e/c21 0 2024-04-26T23:01:23.233 INFO:tasks.workunit.client.0.smithi027.stdout:0/119: chown d0/d4/da/d1a 188106180 1 2024-04-26T23:01:23.233 INFO:tasks.workunit.client.0.smithi027.stdout:0/120: write d0/f1b [316677,20667] 0 2024-04-26T23:01:23.234 INFO:tasks.workunit.client.0.smithi027.stdout:0/121: rename d0/d4/da/d1a/l1d to d0/l22 0 2024-04-26T23:01:23.247 INFO:tasks.workunit.client.0.smithi027.stdout:7/64: dwrite d0/f11 [0,4194304] 0 2024-04-26T23:01:23.257 INFO:tasks.workunit.client.0.smithi027.stdout:2/103: dwrite d2/d5/db/df/f16 [0,4194304] 0 2024-04-26T23:01:23.260 INFO:tasks.workunit.client.0.smithi027.stdout:2/104: link d2/d5/db/f1a d2/d5/db/d1b/d1c/f20 0 2024-04-26T23:01:23.268 INFO:tasks.workunit.client.0.smithi027.stdout:0/122: dwrite d0/d4/f5 [4194304,4194304] 0 2024-04-26T23:01:23.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:22 smithi080 ceph-mon[32324]: pgmap v86: 129 pgs: 129 active+clean; 236 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 1.02k op/s 2024-04-26T23:01:23.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:22 smithi094 ceph-mon[32889]: pgmap v86: 129 pgs: 129 active+clean; 236 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 1.02k op/s 2024-04-26T23:01:23.313 INFO:tasks.workunit.client.0.smithi027.stdout:0/123: dwrite d0/d4/f5 [4194304,4194304] 0 2024-04-26T23:01:23.313 INFO:tasks.workunit.client.0.smithi027.stdout:0/124: truncate d0/d4/da/d15/f16 492541 0 2024-04-26T23:01:23.313 INFO:tasks.workunit.client.0.smithi027.stdout:0/125: chown d0/d4/da/d1a/d1e 581581 1 2024-04-26T23:01:23.314 INFO:tasks.workunit.client.0.smithi027.stdout:0/126: chown d0/d4/da 59029009 1 2024-04-26T23:01:23.325 INFO:tasks.workunit.client.0.smithi027.stdout:1/163: sync 2024-04-26T23:01:23.325 INFO:tasks.workunit.client.0.smithi027.stdout:1/164: chown d1/d5/d21/f22 102 1 2024-04-26T23:01:23.326 INFO:tasks.workunit.client.0.smithi027.stdout:3/134: sync 2024-04-26T23:01:23.326 INFO:tasks.workunit.client.0.smithi027.stdout:1/165: fdatasync d1/f3 0 2024-04-26T23:01:23.326 INFO:tasks.workunit.client.0.smithi027.stdout:3/135: chown d6/c27 27556870 1 2024-04-26T23:01:23.326 INFO:tasks.workunit.client.0.smithi027.stdout:1/166: write d1/dc/f1d [4927038,205] 0 2024-04-26T23:01:23.328 INFO:tasks.workunit.client.0.smithi027.stdout:1/167: mkdir d1/d26/d36 0 2024-04-26T23:01:23.359 INFO:tasks.workunit.client.0.smithi027.stdout:0/127: dwrite d0/d4/da/d15/f16 [0,4194304] 0 2024-04-26T23:01:23.362 INFO:tasks.workunit.client.0.smithi027.stdout:0/128: mkdir d0/d4/da/d1a/d1e/d23 0 2024-04-26T23:01:23.363 INFO:tasks.workunit.client.0.smithi027.stdout:0/129: write d0/d4/da/f1f [869401,84474] 0 2024-04-26T23:01:23.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:22 smithi027 ceph-mon[28034]: pgmap v86: 129 pgs: 129 active+clean; 236 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 1.02k op/s 2024-04-26T23:01:23.368 INFO:tasks.workunit.client.0.smithi027.stdout:1/168: dread d1/dc/f25 [0,4194304] 0 2024-04-26T23:01:23.370 INFO:tasks.workunit.client.0.smithi027.stdout:0/130: dread d0/f12 [0,4194304] 0 2024-04-26T23:01:23.415 INFO:tasks.workunit.client.0.smithi027.stdout:0/131: dwrite d0/d4/f5 [4194304,4194304] 0 2024-04-26T23:01:23.415 INFO:tasks.workunit.client.0.smithi027.stdout:0/132: readlink d0/d4/lc 0 2024-04-26T23:01:23.415 INFO:tasks.workunit.client.0.smithi027.stdout:0/133: fdatasync d0/d4/da/f1f 0 2024-04-26T23:01:23.427 INFO:tasks.workunit.client.0.smithi027.stdout:1/169: dwrite d1/dc/f16 [0,4194304] 0 2024-04-26T23:01:23.428 INFO:tasks.workunit.client.0.smithi027.stdout:0/134: write d0/f9 [523801,122974] 0 2024-04-26T23:01:23.429 INFO:tasks.workunit.client.0.smithi027.stdout:1/170: rename d1/d5/d21/f2c to d1/dc/f37 0 2024-04-26T23:01:23.429 INFO:tasks.workunit.client.0.smithi027.stdout:0/135: unlink d0/d4/da/d1a/f1c 0 2024-04-26T23:01:23.430 INFO:tasks.workunit.client.0.smithi027.stdout:1/171: symlink d1/d2b/d2d/l38 0 2024-04-26T23:01:23.431 INFO:tasks.workunit.client.0.smithi027.stdout:1/172: mknod d1/d5/d21/c39 0 2024-04-26T23:01:23.433 INFO:tasks.workunit.client.0.smithi027.stdout:1/173: link d1/d5/l1a d1/l3a 0 2024-04-26T23:01:23.435 INFO:tasks.workunit.client.0.smithi027.stdout:8/79: sync 2024-04-26T23:01:23.435 INFO:tasks.workunit.client.0.smithi027.stdout:9/118: sync 2024-04-26T23:01:23.435 INFO:tasks.workunit.client.0.smithi027.stdout:9/119: chown d4/d10/d25 202714123 1 2024-04-26T23:01:23.435 INFO:tasks.workunit.client.0.smithi027.stdout:9/120: chown d4/cf 77284 1 2024-04-26T23:01:23.435 INFO:tasks.workunit.client.0.smithi027.stdout:5/101: sync 2024-04-26T23:01:23.435 INFO:tasks.workunit.client.0.smithi027.stdout:9/121: chown d4/l5 178304220 1 2024-04-26T23:01:23.435 INFO:tasks.workunit.client.0.smithi027.stdout:5/102: truncate f1 4982621 0 2024-04-26T23:01:23.435 INFO:tasks.workunit.client.0.smithi027.stdout:7/65: sync 2024-04-26T23:01:23.436 INFO:tasks.workunit.client.0.smithi027.stdout:7/66: chown d0/l5 0 1 2024-04-26T23:01:23.436 INFO:tasks.workunit.client.0.smithi027.stdout:4/118: sync 2024-04-26T23:01:23.436 INFO:tasks.workunit.client.0.smithi027.stdout:6/71: sync 2024-04-26T23:01:23.436 INFO:tasks.workunit.client.0.smithi027.stdout:6/72: chown d1 85647 1 2024-04-26T23:01:23.436 INFO:tasks.workunit.client.0.smithi027.stdout:2/105: sync 2024-04-26T23:01:23.436 INFO:tasks.workunit.client.0.smithi027.stdout:9/122: getdents d4/d17 0 2024-04-26T23:01:23.437 INFO:tasks.workunit.client.0.smithi027.stdout:0/136: dread d0/d4/f20 [0,4194304] 0 2024-04-26T23:01:23.438 INFO:tasks.workunit.client.0.smithi027.stdout:5/103: rename d4/db/fe to d4/db/d16/f1f 0 2024-04-26T23:01:23.438 INFO:tasks.workunit.client.0.smithi027.stdout:5/104: dread - d4/f1d zero size 2024-04-26T23:01:23.438 INFO:tasks.workunit.client.0.smithi027.stdout:5/105: fsync d4/db/f10 0 2024-04-26T23:01:23.438 INFO:tasks.workunit.client.0.smithi027.stdout:5/106: chown d4/d7/f1b 1627886940 1 2024-04-26T23:01:23.439 INFO:tasks.workunit.client.0.smithi027.stdout:7/67: unlink d0/d3/ff 0 2024-04-26T23:01:23.439 INFO:tasks.workunit.client.0.smithi027.stdout:7/68: dread - d0/d3/d14/f16 zero size 2024-04-26T23:01:23.439 INFO:tasks.workunit.client.0.smithi027.stdout:7/69: chown d0/d3/d14 67716850 1 2024-04-26T23:01:23.440 INFO:tasks.workunit.client.0.smithi027.stdout:4/119: creat da/f1f x:0 0 0 2024-04-26T23:01:23.441 INFO:tasks.workunit.client.0.smithi027.stdout:0/137: creat d0/d4/f24 x:0 0 0 2024-04-26T23:01:23.442 INFO:tasks.workunit.client.0.smithi027.stdout:6/73: rename d1/df/c15 to d1/de/c18 0 2024-04-26T23:01:23.443 INFO:tasks.workunit.client.0.smithi027.stdout:7/70: mknod d0/c17 0 2024-04-26T23:01:23.444 INFO:tasks.workunit.client.0.smithi027.stdout:4/120: creat da/dd/de/d14/d17/f20 x:0 0 0 2024-04-26T23:01:23.445 INFO:tasks.workunit.client.0.smithi027.stdout:0/138: creat d0/d4/da/d1a/d1e/f25 x:0 0 0 2024-04-26T23:01:23.446 INFO:tasks.workunit.client.0.smithi027.stdout:2/106: rename d2/d5/db/df to d2/d5/db/de/d21 0 2024-04-26T23:01:23.446 INFO:tasks.workunit.client.0.smithi027.stdout:2/107: truncate d2/d5/db/f14 4352038 0 2024-04-26T23:01:23.446 INFO:tasks.workunit.client.0.smithi027.stdout:0/139: mkdir d0/d4/da/d15/d26 0 2024-04-26T23:01:23.452 INFO:tasks.workunit.client.0.smithi027.stdout:9/123: rename d4/fe to d4/d10/f2f 0 2024-04-26T23:01:23.452 INFO:tasks.workunit.client.0.smithi027.stdout:2/108: creat d2/d5/f22 x:0 0 0 2024-04-26T23:01:23.452 INFO:tasks.workunit.client.0.smithi027.stdout:2/109: mkdir d2/d5/db/de/d21/d12/d19/d23 0 2024-04-26T23:01:23.454 INFO:tasks.workunit.client.0.smithi027.stdout:2/110: rmdir d2/d5/db/d1b 39 2024-04-26T23:01:23.455 INFO:tasks.workunit.client.0.smithi027.stdout:2/111: fdatasync d2/d5/db/f1d 0 2024-04-26T23:01:23.458 INFO:tasks.workunit.client.0.smithi027.stdout:2/112: symlink d2/d5/db/de/l24 0 2024-04-26T23:01:23.461 INFO:tasks.workunit.client.0.smithi027.stdout:5/107: dread d4/f9 [0,4194304] 0 2024-04-26T23:01:23.480 INFO:tasks.workunit.client.0.smithi027.stdout:7/71: read d0/f11 [3946732,37975] 0 2024-04-26T23:01:23.481 INFO:tasks.workunit.client.0.smithi027.stdout:7/72: chown d0/l7 31284 1 2024-04-26T23:01:23.483 INFO:tasks.workunit.client.0.smithi027.stdout:6/74: dwrite d1/df/fc [0,4194304] 0 2024-04-26T23:01:23.483 INFO:tasks.workunit.client.0.smithi027.stdout:6/75: chown d1/de 6447757 1 2024-04-26T23:01:23.483 INFO:tasks.workunit.client.0.smithi027.stdout:2/113: rename d2/d5/db/de/d21/d12 to d2/d5/db/d1b/d1c/d25 0 2024-04-26T23:01:23.485 INFO:tasks.workunit.client.0.smithi027.stdout:5/108: dread f1 [0,4194304] 0 2024-04-26T23:01:23.485 INFO:tasks.workunit.client.0.smithi027.stdout:5/109: read - d4/f1d zero size 2024-04-26T23:01:23.485 INFO:tasks.workunit.client.0.smithi027.stdout:5/110: stat d4/db/f10 0 2024-04-26T23:01:23.487 INFO:tasks.workunit.client.0.smithi027.stdout:6/76: rename d1/df/f14 to d1/f19 0 2024-04-26T23:01:23.487 INFO:tasks.workunit.client.0.smithi027.stdout:2/114: rmdir d2 39 2024-04-26T23:01:23.487 INFO:tasks.workunit.client.0.smithi027.stdout:0/140: dwrite d0/d4/da/d1a/d1e/f25 [0,4194304] 0 2024-04-26T23:01:23.490 INFO:tasks.workunit.client.0.smithi027.stdout:0/141: write d0/f12 [471801,35201] 0 2024-04-26T23:01:23.490 INFO:tasks.workunit.client.0.smithi027.stdout:6/77: rename d1/df/fc to d1/de/f1a 0 2024-04-26T23:01:23.490 INFO:tasks.workunit.client.0.smithi027.stdout:9/124: dread d4/f14 [0,4194304] 0 2024-04-26T23:01:23.492 INFO:tasks.workunit.client.0.smithi027.stdout:9/125: mkdir d4/d10/d30 0 2024-04-26T23:01:23.494 INFO:tasks.workunit.client.0.smithi027.stdout:9/126: rmdir d4/d17 39 2024-04-26T23:01:23.494 INFO:tasks.workunit.client.0.smithi027.stdout:9/127: fsync d4/d10/f19 0 2024-04-26T23:01:23.497 INFO:tasks.workunit.client.0.smithi027.stdout:9/128: creat d4/d17/d2e/f31 x:0 0 0 2024-04-26T23:01:23.498 INFO:tasks.workunit.client.0.smithi027.stdout:9/129: mknod d4/d10/c32 0 2024-04-26T23:01:23.500 INFO:tasks.workunit.client.0.smithi027.stdout:9/130: creat d4/d10/d30/f33 x:0 0 0 2024-04-26T23:01:23.500 INFO:tasks.workunit.client.0.smithi027.stdout:9/131: stat d4/d10/f28 0 2024-04-26T23:01:23.500 INFO:tasks.workunit.client.0.smithi027.stdout:9/132: write d4/d10/f12 [378803,41068] 0 2024-04-26T23:01:23.502 INFO:tasks.workunit.client.0.smithi027.stdout:9/133: creat d4/d17/d2e/f34 x:0 0 0 2024-04-26T23:01:23.510 INFO:tasks.workunit.client.0.smithi027.stdout:8/80: dwrite f7 [0,4194304] 0 2024-04-26T23:01:23.513 INFO:tasks.workunit.client.0.smithi027.stdout:9/134: rmdir d4/d17 39 2024-04-26T23:01:23.513 INFO:tasks.workunit.client.0.smithi027.stdout:9/135: read - d4/d17/d2e/f34 zero size 2024-04-26T23:01:23.513 INFO:tasks.workunit.client.0.smithi027.stdout:2/115: dread d2/d5/f13 [0,4194304] 0 2024-04-26T23:01:23.514 INFO:tasks.workunit.client.0.smithi027.stdout:8/81: creat d9/da/d15/f1e x:0 0 0 2024-04-26T23:01:23.516 INFO:tasks.workunit.client.0.smithi027.stdout:2/116: mkdir d2/d5/db/de/d21/d26 0 2024-04-26T23:01:23.517 INFO:tasks.workunit.client.0.smithi027.stdout:8/82: rename f6 to d9/da/d15/f1f 0 2024-04-26T23:01:23.523 INFO:tasks.workunit.client.0.smithi027.stdout:2/117: creat d2/d5/db/d1b/d1c/d25/f27 x:0 0 0 2024-04-26T23:01:23.524 INFO:tasks.workunit.client.0.smithi027.stdout:4/121: dwrite f3 [0,4194304] 0 2024-04-26T23:01:23.524 INFO:tasks.workunit.client.0.smithi027.stdout:4/122: chown f4 457271 1 2024-04-26T23:01:23.524 INFO:tasks.workunit.client.0.smithi027.stdout:4/123: truncate da/dd/de/f15 1012016 0 2024-04-26T23:01:23.524 INFO:tasks.workunit.client.0.smithi027.stdout:4/124: rmdir da/df 39 2024-04-26T23:01:23.525 INFO:tasks.workunit.client.0.smithi027.stdout:9/136: getdents d4/d10/d30 0 2024-04-26T23:01:23.525 INFO:tasks.workunit.client.0.smithi027.stdout:9/137: readlink d4/d10/d25/l2a 0 2024-04-26T23:01:23.528 INFO:tasks.workunit.client.0.smithi027.stdout:9/138: creat d4/f35 x:0 0 0 2024-04-26T23:01:23.528 INFO:tasks.workunit.client.0.smithi027.stdout:9/139: chown d4/f35 17236052 1 2024-04-26T23:01:23.529 INFO:tasks.workunit.client.0.smithi027.stdout:9/140: write d4/d10/f19 [4371546,126810] 0 2024-04-26T23:01:23.530 INFO:tasks.workunit.client.0.smithi027.stdout:9/141: symlink d4/l36 0 2024-04-26T23:01:23.531 INFO:tasks.workunit.client.0.smithi027.stdout:6/78: write d1/df/f8 [3932607,7451] 0 2024-04-26T23:01:23.532 INFO:tasks.workunit.client.0.smithi027.stdout:9/142: link d4/d17/c27 d4/d17/d2e/c37 0 2024-04-26T23:01:23.536 INFO:tasks.workunit.client.0.smithi027.stdout:9/143: truncate d4/f2b 3919757 0 2024-04-26T23:01:23.538 INFO:tasks.workunit.client.0.smithi027.stdout:9/144: mknod d4/d10/c38 0 2024-04-26T23:01:23.540 INFO:tasks.workunit.client.0.smithi027.stdout:4/125: dread f4 [4194304,4194304] 0 2024-04-26T23:01:23.541 INFO:tasks.workunit.client.0.smithi027.stdout:4/126: rmdir da/df 39 2024-04-26T23:01:23.546 INFO:tasks.workunit.client.0.smithi027.stdout:4/127: rmdir da/dd/de 39 2024-04-26T23:01:23.557 INFO:tasks.workunit.client.0.smithi027.stdout:8/83: dread d9/f16 [0,4194304] 0 2024-04-26T23:01:23.557 INFO:tasks.workunit.client.0.smithi027.stdout:8/84: chown d9/da/cc 532215 1 2024-04-26T23:01:23.562 INFO:tasks.workunit.client.0.smithi027.stdout:6/79: dread d1/f2 [0,4194304] 0 2024-04-26T23:01:23.564 INFO:tasks.workunit.client.0.smithi027.stdout:0/142: dwrite d0/d4/da/d15/f16 [0,4194304] 0 2024-04-26T23:01:23.564 INFO:tasks.workunit.client.0.smithi027.stdout:6/80: creat d1/de/f1b x:0 0 0 2024-04-26T23:01:23.568 INFO:tasks.workunit.client.0.smithi027.stdout:6/81: mknod d1/de/c1c 0 2024-04-26T23:01:23.568 INFO:tasks.workunit.client.0.smithi027.stdout:6/82: fdatasync d1/df/fd 0 2024-04-26T23:01:23.569 INFO:tasks.workunit.client.0.smithi027.stdout:5/111: dwrite d4/db/f10 [4194304,4194304] 0 2024-04-26T23:01:23.569 INFO:tasks.workunit.client.0.smithi027.stdout:5/112: stat c3 0 2024-04-26T23:01:23.571 INFO:tasks.workunit.client.0.smithi027.stdout:2/118: dwrite d2/d5/db/d1b/d1c/d25/f27 [0,4194304] 0 2024-04-26T23:01:23.571 INFO:tasks.workunit.client.0.smithi027.stdout:2/119: fdatasync d2/d5/db/de/d21/f16 0 2024-04-26T23:01:23.571 INFO:tasks.workunit.client.0.smithi027.stdout:2/120: dread - d2/d5/f22 zero size 2024-04-26T23:01:23.571 INFO:tasks.workunit.client.0.smithi027.stdout:2/121: readlink d2/d5/db/de/l24 0 2024-04-26T23:01:23.572 INFO:tasks.workunit.client.0.smithi027.stdout:5/113: rename d4/d7/f17 to d4/d1a/f20 0 2024-04-26T23:01:23.576 INFO:tasks.workunit.client.0.smithi027.stdout:5/114: symlink d4/l21 0 2024-04-26T23:01:23.579 INFO:tasks.workunit.client.0.smithi027.stdout:5/115: mkdir d4/d7/d22 0 2024-04-26T23:01:23.579 INFO:tasks.workunit.client.0.smithi027.stdout:5/116: fsync d4/d7/ff 0 2024-04-26T23:01:23.579 INFO:tasks.workunit.client.0.smithi027.stdout:5/117: fdatasync d4/f15 0 2024-04-26T23:01:23.579 INFO:tasks.workunit.client.0.smithi027.stdout:5/118: write d4/f8 [674558,125750] 0 2024-04-26T23:01:23.588 INFO:tasks.workunit.client.0.smithi027.stdout:7/73: dwrite d0/d3/d14/f15 [0,4194304] 0 2024-04-26T23:01:23.588 INFO:tasks.workunit.client.0.smithi027.stdout:4/128: dread da/fc [0,4194304] 0 2024-04-26T23:01:23.588 INFO:tasks.workunit.client.0.smithi027.stdout:4/129: stat da/fc 0 2024-04-26T23:01:23.589 INFO:tasks.workunit.client.0.smithi027.stdout:4/130: chown c1 14875 1 2024-04-26T23:01:23.592 INFO:tasks.workunit.client.0.smithi027.stdout:7/74: creat d0/d3/f18 x:0 0 0 2024-04-26T23:01:23.592 INFO:tasks.workunit.client.0.smithi027.stdout:7/75: readlink d0/l1 0 2024-04-26T23:01:23.592 INFO:tasks.workunit.client.0.smithi027.stdout:7/76: fsync d0/f13 0 2024-04-26T23:01:23.592 INFO:tasks.workunit.client.0.smithi027.stdout:8/85: dread f2 [0,4194304] 0 2024-04-26T23:01:23.592 INFO:tasks.workunit.client.0.smithi027.stdout:8/86: fdatasync d9/de/f18 0 2024-04-26T23:01:23.593 INFO:tasks.workunit.client.0.smithi027.stdout:8/87: mkdir d9/de/d20 0 2024-04-26T23:01:23.609 INFO:tasks.workunit.client.0.smithi027.stdout:3/136: sync 2024-04-26T23:01:23.611 INFO:tasks.workunit.client.0.smithi027.stdout:8/88: rename d9/da/d15/l19 to d9/da/d15/l21 0 2024-04-26T23:01:23.611 INFO:tasks.workunit.client.0.smithi027.stdout:8/89: stat d9/da/fd 0 2024-04-26T23:01:23.611 INFO:tasks.workunit.client.0.smithi027.stdout:8/90: write f3 [3616925,78325] 0 2024-04-26T23:01:23.613 INFO:tasks.workunit.client.0.smithi027.stdout:8/91: mkdir d9/de/d22 0 2024-04-26T23:01:23.613 INFO:tasks.workunit.client.0.smithi027.stdout:8/92: fsync d9/de/f14 0 2024-04-26T23:01:23.617 INFO:tasks.workunit.client.0.smithi027.stdout:3/137: rename d6/dc/d1d to d6/d28/d2e 0 2024-04-26T23:01:23.619 INFO:tasks.workunit.client.0.smithi027.stdout:8/93: rename d9/da/l1c to d9/l23 0 2024-04-26T23:01:23.619 INFO:tasks.workunit.client.0.smithi027.stdout:8/94: chown d9/f1b 14665257 1 2024-04-26T23:01:23.619 INFO:tasks.workunit.client.0.smithi027.stdout:8/95: chown d9/da/d15/f1e 39 1 2024-04-26T23:01:23.620 INFO:tasks.workunit.client.0.smithi027.stdout:3/138: mkdir d6/dc/d2f 0 2024-04-26T23:01:23.621 INFO:tasks.workunit.client.0.smithi027.stdout:3/139: write d6/d28/d2e/f1e [718563,6845] 0 2024-04-26T23:01:23.622 INFO:tasks.workunit.client.0.smithi027.stdout:2/122: dwrite d2/d5/f22 [0,4194304] 0 2024-04-26T23:01:23.622 INFO:tasks.workunit.client.0.smithi027.stdout:2/123: stat d2/d5/db/de/l24 0 2024-04-26T23:01:23.625 INFO:tasks.workunit.client.0.smithi027.stdout:8/96: unlink d9/da/d15/f1e 0 2024-04-26T23:01:23.631 INFO:tasks.workunit.client.0.smithi027.stdout:8/97: creat d9/de/d22/f24 x:0 0 0 2024-04-26T23:01:23.637 INFO:tasks.workunit.client.0.smithi027.stdout:7/77: dread d0/f11 [0,4194304] 0 2024-04-26T23:01:23.640 INFO:tasks.workunit.client.0.smithi027.stdout:4/131: dwrite da/dd/de/d14/d17/f1d [0,4194304] 0 2024-04-26T23:01:23.640 INFO:tasks.workunit.client.0.smithi027.stdout:4/132: truncate da/df/f19 5030675 0 2024-04-26T23:01:23.640 INFO:tasks.workunit.client.0.smithi027.stdout:4/133: dread - da/dd/de/f1c zero size 2024-04-26T23:01:23.640 INFO:tasks.workunit.client.0.smithi027.stdout:4/134: write da/dd/de/f1c [881534,124963] 0 2024-04-26T23:01:23.662 INFO:tasks.workunit.client.0.smithi027.stdout:8/98: mknod d9/de/d22/c25 0 2024-04-26T23:01:23.663 INFO:tasks.workunit.client.0.smithi027.stdout:3/140: write d6/dc/f10 [3944842,63266] 0 2024-04-26T23:01:23.663 INFO:tasks.workunit.client.0.smithi027.stdout:5/119: dwrite d4/f9 [0,4194304] 0 2024-04-26T23:01:23.664 INFO:tasks.workunit.client.0.smithi027.stdout:8/99: creat d9/de/d20/f26 x:0 0 0 2024-04-26T23:01:23.664 INFO:tasks.workunit.client.0.smithi027.stdout:8/100: write d9/de/f14 [749691,109278] 0 2024-04-26T23:01:23.665 INFO:tasks.workunit.client.0.smithi027.stdout:5/120: creat d4/db/f23 x:0 0 0 2024-04-26T23:01:23.665 INFO:tasks.workunit.client.0.smithi027.stdout:8/101: rmdir d9/de/d20 39 2024-04-26T23:01:23.665 INFO:tasks.workunit.client.0.smithi027.stdout:8/102: fsync d9/de/f18 0 2024-04-26T23:01:23.665 INFO:tasks.workunit.client.0.smithi027.stdout:8/103: stat d9/da/d15/c1a 0 2024-04-26T23:01:23.666 INFO:tasks.workunit.client.0.smithi027.stdout:8/104: fsync d9/da/d15/f1f 0 2024-04-26T23:01:23.666 INFO:tasks.workunit.client.0.smithi027.stdout:5/121: rename d4/d7 to d4/d24 0 2024-04-26T23:01:23.668 INFO:tasks.workunit.client.0.smithi027.stdout:5/122: unlink d4/f8 0 2024-04-26T23:01:23.669 INFO:tasks.workunit.client.0.smithi027.stdout:5/123: mkdir d4/d24/d25 0 2024-04-26T23:01:23.673 INFO:tasks.workunit.client.0.smithi027.stdout:3/141: write d6/d28/d2e/f1e [333252,106191] 0 2024-04-26T23:01:23.681 INFO:tasks.workunit.client.0.smithi027.stdout:2/124: dwrite d2/d5/db/de/d21/f16 [0,4194304] 0 2024-04-26T23:01:23.681 INFO:tasks.workunit.client.0.smithi027.stdout:2/125: readlink d2/d5/db/de/l24 0 2024-04-26T23:01:23.689 INFO:tasks.workunit.client.0.smithi027.stdout:7/78: dwrite d0/d3/f18 [0,4194304] 0 2024-04-26T23:01:23.689 INFO:tasks.workunit.client.0.smithi027.stdout:7/79: fdatasync d0/d3/d14/f15 0 2024-04-26T23:01:23.691 INFO:tasks.workunit.client.0.smithi027.stdout:2/126: symlink d2/d5/db/de/d21/l28 0 2024-04-26T23:01:23.694 INFO:tasks.workunit.client.0.smithi027.stdout:4/135: dwrite da/dd/de/d14/d17/f1d [0,4194304] 0 2024-04-26T23:01:23.694 INFO:tasks.workunit.client.0.smithi027.stdout:4/136: chown c6 16431 1 2024-04-26T23:01:23.694 INFO:tasks.workunit.client.0.smithi027.stdout:2/127: rename d2/d5/db/de/d21/l28 to d2/d5/db/d1b/d1c/l29 0 2024-04-26T23:01:23.703 INFO:tasks.workunit.client.0.smithi027.stdout:1/174: sync 2024-04-26T23:01:23.704 INFO:tasks.workunit.client.0.smithi027.stdout:1/175: mknod d1/d26/d36/c3b 0 2024-04-26T23:01:23.705 INFO:tasks.workunit.client.0.smithi027.stdout:1/176: write d1/f6 [859320,52456] 0 2024-04-26T23:01:23.714 INFO:tasks.workunit.client.0.smithi027.stdout:3/142: dread d6/f7 [0,4194304] 0 2024-04-26T23:01:23.715 INFO:tasks.workunit.client.0.smithi027.stdout:3/143: symlink d6/l30 0 2024-04-26T23:01:23.746 INFO:tasks.workunit.client.0.smithi027.stdout:7/80: dwrite d0/f11 [4194304,4194304] 0 2024-04-26T23:01:23.749 INFO:tasks.workunit.client.0.smithi027.stdout:7/81: rename d0/c17 to d0/d3/d14/c19 0 2024-04-26T23:01:23.750 INFO:tasks.workunit.client.0.smithi027.stdout:7/82: rmdir d0/d3/d14 39 2024-04-26T23:01:23.752 INFO:tasks.workunit.client.0.smithi027.stdout:7/83: creat d0/f1a x:0 0 0 2024-04-26T23:01:23.770 INFO:tasks.workunit.client.0.smithi027.stdout:3/144: dwrite d6/f2d [0,4194304] 0 2024-04-26T23:01:23.770 INFO:tasks.workunit.client.0.smithi027.stdout:2/128: write d2/d5/db/f1d [131996,78661] 0 2024-04-26T23:01:23.770 INFO:tasks.workunit.client.0.smithi027.stdout:8/105: dwrite d9/de/d20/f26 [0,4194304] 0 2024-04-26T23:01:23.773 INFO:tasks.workunit.client.0.smithi027.stdout:7/84: write d0/d3/f18 [1832123,91194] 0 2024-04-26T23:01:23.774 INFO:tasks.workunit.client.0.smithi027.stdout:3/145: creat d6/d28/d2e/f31 x:0 0 0 2024-04-26T23:01:23.777 INFO:tasks.workunit.client.0.smithi027.stdout:5/124: dwrite d4/f1d [0,4194304] 0 2024-04-26T23:01:23.777 INFO:tasks.workunit.client.0.smithi027.stdout:8/106: truncate d9/de/f14 116071 0 2024-04-26T23:01:23.777 INFO:tasks.workunit.client.0.smithi027.stdout:8/107: readlink d9/da/l12 0 2024-04-26T23:01:23.777 INFO:tasks.workunit.client.0.smithi027.stdout:8/108: chown f2 755 1 2024-04-26T23:01:23.778 INFO:tasks.workunit.client.0.smithi027.stdout:7/85: write d0/d3/d14/f16 [104400,49964] 0 2024-04-26T23:01:23.778 INFO:tasks.workunit.client.0.smithi027.stdout:5/125: mknod d4/db/d16/c26 0 2024-04-26T23:01:23.779 INFO:tasks.workunit.client.0.smithi027.stdout:1/177: dwrite d1/f10 [0,4194304] 0 2024-04-26T23:01:23.780 INFO:tasks.workunit.client.0.smithi027.stdout:5/126: link d4/d24/c1c d4/db/c27 0 2024-04-26T23:01:23.784 INFO:tasks.workunit.client.0.smithi027.stdout:1/178: mkdir d1/d2a/d3c 0 2024-04-26T23:01:23.784 INFO:tasks.workunit.client.0.smithi027.stdout:5/127: mknod d4/db/d16/c28 0 2024-04-26T23:01:23.787 INFO:tasks.workunit.client.0.smithi027.stdout:7/86: link d0/l1 d0/d3/d14/l1b 0 2024-04-26T23:01:23.789 INFO:tasks.workunit.client.0.smithi027.stdout:7/87: link d0/l1 d0/l1c 0 2024-04-26T23:01:23.789 INFO:tasks.workunit.client.0.smithi027.stdout:7/88: chown d0/d3/f18 631 1 2024-04-26T23:01:23.789 INFO:tasks.workunit.client.0.smithi027.stdout:7/89: fdatasync d0/f1a 0 2024-04-26T23:01:23.791 INFO:tasks.workunit.client.0.smithi027.stdout:7/90: truncate d0/d3/d14/f15 1743347 0 2024-04-26T23:01:23.791 INFO:tasks.workunit.client.0.smithi027.stdout:7/91: rename d0/d3 to d0/d3/d14/d1d 22 2024-04-26T23:01:23.795 INFO:tasks.workunit.client.0.smithi027.stdout:7/92: link d0/d3/d14/c19 d0/d3/d14/c1e 0 2024-04-26T23:01:23.797 INFO:tasks.workunit.client.0.smithi027.stdout:7/93: rename d0/f1a to d0/f1f 0 2024-04-26T23:01:23.800 INFO:tasks.workunit.client.0.smithi027.stdout:7/94: rmdir d0 39 2024-04-26T23:01:23.803 INFO:tasks.workunit.client.0.smithi027.stdout:7/95: rename d0/c9 to d0/c20 0 2024-04-26T23:01:23.803 INFO:tasks.workunit.client.0.smithi027.stdout:7/96: fdatasync d0/f1f 0 2024-04-26T23:01:23.804 INFO:tasks.workunit.client.0.smithi027.stdout:3/146: read d6/dc/f25 [1639294,7623] 0 2024-04-26T23:01:23.804 INFO:tasks.workunit.client.0.smithi027.stdout:3/147: truncate d6/f18 4332214 0 2024-04-26T23:01:23.805 INFO:tasks.workunit.client.0.smithi027.stdout:4/137: dwrite da/f1f [0,4194304] 0 2024-04-26T23:01:23.816 INFO:tasks.workunit.client.0.smithi027.stdout:3/148: mknod d6/dc/d2f/c32 0 2024-04-26T23:01:23.817 INFO:tasks.workunit.client.0.smithi027.stdout:2/129: dwrite d2/d5/db/f1a [0,4194304] 0 2024-04-26T23:01:23.817 INFO:tasks.workunit.client.0.smithi027.stdout:2/130: chown d2/d5/db/de/d21/cc 527044538 1 2024-04-26T23:01:23.817 INFO:tasks.workunit.client.0.smithi027.stdout:2/131: chown d2/d5/db/d1b 35 1 2024-04-26T23:01:23.817 INFO:tasks.workunit.client.0.smithi027.stdout:2/132: chown d2 19 1 2024-04-26T23:01:23.821 INFO:tasks.workunit.client.0.smithi027.stdout:4/138: mkdir da/dd/de/d14/d21 0 2024-04-26T23:01:23.823 INFO:tasks.workunit.client.0.smithi027.stdout:4/139: write da/dd/de/f15 [842052,119381] 0 2024-04-26T23:01:23.823 INFO:tasks.workunit.client.0.smithi027.stdout:4/140: truncate da/f1f 4580249 0 2024-04-26T23:01:23.830 INFO:tasks.workunit.client.0.smithi027.stdout:3/149: rename d6/f13 to d6/dc/d17/f33 0 2024-04-26T23:01:23.831 INFO:tasks.workunit.client.0.smithi027.stdout:0/143: sync 2024-04-26T23:01:23.831 INFO:tasks.workunit.client.0.smithi027.stdout:6/83: sync 2024-04-26T23:01:23.831 INFO:tasks.workunit.client.0.smithi027.stdout:9/145: sync 2024-04-26T23:01:23.831 INFO:tasks.workunit.client.0.smithi027.stdout:0/144: chown d0/d4/da/d15/f16 2228527 1 2024-04-26T23:01:23.831 INFO:tasks.workunit.client.0.smithi027.stdout:6/84: fdatasync d1/f19 0 2024-04-26T23:01:23.831 INFO:tasks.workunit.client.0.smithi027.stdout:9/146: read - d4/f35 zero size 2024-04-26T23:01:23.832 INFO:tasks.workunit.client.0.smithi027.stdout:4/141: mknod da/c22 0 2024-04-26T23:01:23.833 INFO:tasks.workunit.client.0.smithi027.stdout:6/85: getdents d1 0 2024-04-26T23:01:23.835 INFO:tasks.workunit.client.0.smithi027.stdout:9/147: creat d4/d10/d25/f39 x:0 0 0 2024-04-26T23:01:23.835 INFO:tasks.workunit.client.0.smithi027.stdout:9/148: chown d4/d17/f29 7780502 1 2024-04-26T23:01:23.836 INFO:tasks.workunit.client.0.smithi027.stdout:8/109: dwrite d9/f16 [0,4194304] 0 2024-04-26T23:01:23.837 INFO:tasks.workunit.client.0.smithi027.stdout:6/86: creat d1/de/f1d x:0 0 0 2024-04-26T23:01:23.838 INFO:tasks.workunit.client.0.smithi027.stdout:0/145: read d0/d4/f20 [375041,38475] 0 2024-04-26T23:01:23.839 INFO:tasks.workunit.client.0.smithi027.stdout:0/146: chown d0/ld 7120 1 2024-04-26T23:01:23.839 INFO:tasks.workunit.client.0.smithi027.stdout:8/110: rmdir d9/de/d22 39 2024-04-26T23:01:23.840 INFO:tasks.workunit.client.0.smithi027.stdout:0/147: symlink d0/d4/l27 0 2024-04-26T23:01:23.841 INFO:tasks.workunit.client.0.smithi027.stdout:0/148: mknod d0/c28 0 2024-04-26T23:01:23.842 INFO:tasks.workunit.client.0.smithi027.stdout:0/149: write d0/d4/da/f11 [555018,31811] 0 2024-04-26T23:01:23.843 INFO:tasks.workunit.client.0.smithi027.stdout:0/150: symlink d0/d4/da/d15/l29 0 2024-04-26T23:01:23.844 INFO:tasks.workunit.client.0.smithi027.stdout:7/97: dwrite d0/f13 [0,4194304] 0 2024-04-26T23:01:23.844 INFO:tasks.workunit.client.0.smithi027.stdout:1/179: dwrite d1/dc/f16 [0,4194304] 0 2024-04-26T23:01:23.846 INFO:tasks.workunit.client.0.smithi027.stdout:3/150: rename d6/d28 to d6/dc/d34 0 2024-04-26T23:01:23.846 INFO:tasks.workunit.client.0.smithi027.stdout:0/151: mkdir d0/d4/da/d1a/d2a 0 2024-04-26T23:01:23.851 INFO:tasks.workunit.client.0.smithi027.stdout:7/98: getdents d0 0 2024-04-26T23:01:23.851 INFO:tasks.workunit.client.0.smithi027.stdout:1/180: symlink d1/d5/d21/l3d 0 2024-04-26T23:01:23.851 INFO:tasks.workunit.client.0.smithi027.stdout:7/99: truncate d0/d3/d14/f16 1155975 0 2024-04-26T23:01:23.851 INFO:tasks.workunit.client.0.smithi027.stdout:7/100: chown d0/d3/l12 0 1 2024-04-26T23:01:23.851 INFO:tasks.workunit.client.0.smithi027.stdout:7/101: stat d0/d3/d14/f15 0 2024-04-26T23:01:23.852 INFO:tasks.workunit.client.0.smithi027.stdout:4/142: dread da/df/f16 [0,4194304] 0 2024-04-26T23:01:23.855 INFO:tasks.workunit.client.0.smithi027.stdout:9/149: read d4/d17/f1d [1532966,55653] 0 2024-04-26T23:01:23.857 INFO:tasks.workunit.client.0.smithi027.stdout:3/151: creat d6/dc/f35 x:0 0 0 2024-04-26T23:01:23.857 INFO:tasks.workunit.client.0.smithi027.stdout:0/152: mknod d0/d4/da/d15/c2b 0 2024-04-26T23:01:23.857 INFO:tasks.workunit.client.0.smithi027.stdout:3/152: stat d6/f7 0 2024-04-26T23:01:23.857 INFO:tasks.workunit.client.0.smithi027.stdout:1/181: creat d1/d5/d21/f3e x:0 0 0 2024-04-26T23:01:23.857 INFO:tasks.workunit.client.0.smithi027.stdout:1/182: chown d1/d5/f19 3643 1 2024-04-26T23:01:23.857 INFO:tasks.workunit.client.0.smithi027.stdout:7/102: mknod d0/d3/c21 0 2024-04-26T23:01:23.857 INFO:tasks.workunit.client.0.smithi027.stdout:7/103: fdatasync d0/f1f 0 2024-04-26T23:01:23.858 INFO:tasks.workunit.client.0.smithi027.stdout:8/111: dread f2 [0,4194304] 0 2024-04-26T23:01:23.858 INFO:tasks.workunit.client.0.smithi027.stdout:8/112: fdatasync d9/de/d20/f26 0 2024-04-26T23:01:23.858 INFO:tasks.workunit.client.0.smithi027.stdout:4/143: creat da/dd/de/d14/d21/f23 x:0 0 0 2024-04-26T23:01:23.859 INFO:tasks.workunit.client.0.smithi027.stdout:9/150: unlink d4/d17/f21 0 2024-04-26T23:01:23.859 INFO:tasks.workunit.client.0.smithi027.stdout:9/151: truncate d4/d17/f29 993516 0 2024-04-26T23:01:23.859 INFO:tasks.workunit.client.0.smithi027.stdout:0/153: creat d0/d4/da/d1a/d1e/f2c x:0 0 0 2024-04-26T23:01:23.859 INFO:tasks.workunit.client.0.smithi027.stdout:0/154: stat d0/d4/da 0 2024-04-26T23:01:23.860 INFO:tasks.workunit.client.0.smithi027.stdout:1/183: getdents d1/d2b 0 2024-04-26T23:01:23.860 INFO:tasks.workunit.client.0.smithi027.stdout:1/184: chown d1/dc/f14 36817 1 2024-04-26T23:01:23.864 INFO:tasks.workunit.client.0.smithi027.stdout:7/104: mknod d0/c22 0 2024-04-26T23:01:23.864 INFO:tasks.workunit.client.0.smithi027.stdout:8/113: link d9/da/l12 d9/de/d22/l27 0 2024-04-26T23:01:23.864 INFO:tasks.workunit.client.0.smithi027.stdout:0/155: mkdir d0/d4/da/d2d 0 2024-04-26T23:01:23.866 INFO:tasks.workunit.client.0.smithi027.stdout:8/114: mkdir d9/d28 0 2024-04-26T23:01:23.866 INFO:tasks.workunit.client.0.smithi027.stdout:7/105: rmdir d0/d3 39 2024-04-26T23:01:23.868 INFO:tasks.workunit.client.0.smithi027.stdout:8/115: creat d9/de/d20/f29 x:0 0 0 2024-04-26T23:01:23.879 INFO:tasks.workunit.client.0.smithi027.stdout:1/185: dread d1/dc/f1d [0,4194304] 0 2024-04-26T23:01:23.881 INFO:tasks.workunit.client.0.smithi027.stdout:1/186: unlink d1/dc/l13 0 2024-04-26T23:01:23.902 INFO:tasks.workunit.client.0.smithi027.stdout:8/116: dread f7 [0,4194304] 0 2024-04-26T23:01:23.903 INFO:tasks.workunit.client.0.smithi027.stdout:7/106: write d0/f13 [2684853,122799] 0 2024-04-26T23:01:23.904 INFO:tasks.workunit.client.0.smithi027.stdout:4/144: dwrite da/df/f19 [0,4194304] 0 2024-04-26T23:01:23.904 INFO:tasks.workunit.client.0.smithi027.stdout:8/117: mkdir d9/da/d2a 0 2024-04-26T23:01:23.905 INFO:tasks.workunit.client.0.smithi027.stdout:8/118: fsync d9/de/d20/f26 0 2024-04-26T23:01:23.908 INFO:tasks.workunit.client.0.smithi027.stdout:8/119: mkdir d9/d28/d2b 0 2024-04-26T23:01:23.914 INFO:tasks.workunit.client.0.smithi027.stdout:0/156: dwrite d0/d4/f20 [0,4194304] 0 2024-04-26T23:01:23.915 INFO:tasks.workunit.client.0.smithi027.stdout:1/187: dwrite d1/dc/f16 [0,4194304] 0 2024-04-26T23:01:23.915 INFO:tasks.workunit.client.0.smithi027.stdout:1/188: stat d1/dc/f25 0 2024-04-26T23:01:23.915 INFO:tasks.workunit.client.0.smithi027.stdout:7/107: dread d0/d3/d14/f16 [0,4194304] 0 2024-04-26T23:01:23.915 INFO:tasks.workunit.client.0.smithi027.stdout:7/108: chown d0/c22 98537 1 2024-04-26T23:01:23.919 INFO:tasks.workunit.client.0.smithi027.stdout:3/153: dwrite d6/dc/d34/d2e/f1e [0,4194304] 0 2024-04-26T23:01:23.919 INFO:tasks.workunit.client.0.smithi027.stdout:3/154: write d6/dc/d17/f2b [775316,56014] 0 2024-04-26T23:01:23.920 INFO:tasks.workunit.client.0.smithi027.stdout:1/189: link d1/d2b/d2d/l33 d1/d5/l3f 0 2024-04-26T23:01:23.920 INFO:tasks.workunit.client.0.smithi027.stdout:1/190: chown d1/d5/c23 0 1 2024-04-26T23:01:23.922 INFO:tasks.workunit.client.0.smithi027.stdout:8/120: dread d9/f17 [0,4194304] 0 2024-04-26T23:01:23.922 INFO:tasks.workunit.client.0.smithi027.stdout:8/121: readlink d9/l23 0 2024-04-26T23:01:23.923 INFO:tasks.workunit.client.0.smithi027.stdout:7/109: write d0/d3/d14/f15 [814771,18406] 0 2024-04-26T23:01:23.923 INFO:tasks.workunit.client.0.smithi027.stdout:0/157: dread d0/d4/da/f11 [0,4194304] 0 2024-04-26T23:01:23.924 INFO:tasks.workunit.client.0.smithi027.stdout:3/155: creat d6/dc/d34/f36 x:0 0 0 2024-04-26T23:01:23.925 INFO:tasks.workunit.client.0.smithi027.stdout:7/110: fdatasync d0/f13 0 2024-04-26T23:01:23.926 INFO:tasks.workunit.client.0.smithi027.stdout:8/122: creat d9/de/f2c x:0 0 0 2024-04-26T23:01:23.927 INFO:tasks.workunit.client.0.smithi027.stdout:3/156: mkdir d6/dc/d2f/d37 0 2024-04-26T23:01:23.930 INFO:tasks.workunit.client.0.smithi027.stdout:0/158: dread d0/d4/da/f11 [0,4194304] 0 2024-04-26T23:01:23.931 INFO:tasks.workunit.client.0.smithi027.stdout:8/123: mknod d9/de/d22/c2d 0 2024-04-26T23:01:23.933 INFO:tasks.workunit.client.0.smithi027.stdout:3/157: symlink d6/l38 0 2024-04-26T23:01:23.933 INFO:tasks.workunit.client.0.smithi027.stdout:3/158: write d6/dc/d17/f20 [1117222,32601] 0 2024-04-26T23:01:23.934 INFO:tasks.workunit.client.0.smithi027.stdout:0/159: symlink d0/l2e 0 2024-04-26T23:01:23.935 INFO:tasks.workunit.client.0.smithi027.stdout:3/159: creat d6/dc/d2f/f39 x:0 0 0 2024-04-26T23:01:23.935 INFO:tasks.workunit.client.0.smithi027.stdout:3/160: chown d6/l30 7 1 2024-04-26T23:01:23.936 INFO:tasks.workunit.client.0.smithi027.stdout:3/161: symlink d6/dc/d2f/l3a 0 2024-04-26T23:01:23.938 INFO:tasks.workunit.client.0.smithi027.stdout:3/162: symlink d6/dc/l3b 0 2024-04-26T23:01:23.938 INFO:tasks.workunit.client.0.smithi027.stdout:3/163: fdatasync d6/dc/f10 0 2024-04-26T23:01:23.940 INFO:tasks.workunit.client.0.smithi027.stdout:0/160: dread d0/d4/f18 [0,4194304] 0 2024-04-26T23:01:23.942 INFO:tasks.workunit.client.0.smithi027.stdout:0/161: rmdir d0/d4/da/d1a/d1e/d23 0 2024-04-26T23:01:23.949 INFO:tasks.workunit.client.0.smithi027.stdout:1/191: rename d1/d2b to d1/d5/d40 0 2024-04-26T23:01:23.949 INFO:tasks.workunit.client.0.smithi027.stdout:1/192: readlink d1/d5/d40/d2d/l38 0 2024-04-26T23:01:23.949 INFO:tasks.workunit.client.0.smithi027.stdout:1/193: chown d1/dc/l17 2 1 2024-04-26T23:01:23.951 INFO:tasks.workunit.client.0.smithi027.stdout:8/124: rename d9/da/d15 to d9/de/d2e 0 2024-04-26T23:01:23.955 INFO:tasks.workunit.client.0.smithi027.stdout:0/162: rename d0/f9 to d0/d4/da/f2f 0 2024-04-26T23:01:23.955 INFO:tasks.workunit.client.0.smithi027.stdout:8/125: unlink d9/de/d20/f26 0 2024-04-26T23:01:23.957 INFO:tasks.workunit.client.0.smithi027.stdout:0/163: creat d0/d4/da/d1a/d2a/f30 x:0 0 0 2024-04-26T23:01:23.959 INFO:tasks.workunit.client.0.smithi027.stdout:8/126: unlink d9/de/d22/f24 0 2024-04-26T23:01:23.959 INFO:tasks.workunit.client.0.smithi027.stdout:8/127: chown f3 438 1 2024-04-26T23:01:23.960 INFO:tasks.workunit.client.0.smithi027.stdout:8/128: symlink d9/da/d2a/l2f 0 2024-04-26T23:01:23.962 INFO:tasks.workunit.client.0.smithi027.stdout:0/164: dread d0/d4/f18 [0,4194304] 0 2024-04-26T23:01:23.964 INFO:tasks.workunit.client.0.smithi027.stdout:0/165: creat d0/d4/da/d2d/f31 x:0 0 0 2024-04-26T23:01:23.968 INFO:tasks.workunit.client.0.smithi027.stdout:7/111: dwrite d0/d3/f18 [4194304,4194304] 0 2024-04-26T23:01:23.970 INFO:tasks.workunit.client.0.smithi027.stdout:8/129: dread d9/de/f14 [0,4194304] 0 2024-04-26T23:01:23.970 INFO:tasks.workunit.client.0.smithi027.stdout:8/130: chown d9/de/d20 1 1 2024-04-26T23:01:23.985 INFO:tasks.workunit.client.0.smithi027.stdout:8/131: truncate f3 655521 0 2024-04-26T23:01:23.988 INFO:tasks.workunit.client.0.smithi027.stdout:8/132: unlink d9/da/d2a/l2f 0 2024-04-26T23:01:23.988 INFO:tasks.workunit.client.0.smithi027.stdout:8/133: chown f2 123616 1 2024-04-26T23:01:23.989 INFO:tasks.workunit.client.0.smithi027.stdout:8/134: link c5 d9/de/d20/c30 0 2024-04-26T23:01:24.004 INFO:tasks.workunit.client.0.smithi027.stdout:7/112: dwrite d0/f1f [0,4194304] 0 2024-04-26T23:01:24.004 INFO:tasks.workunit.client.0.smithi027.stdout:7/113: write d0/d3/f18 [4127273,53419] 0 2024-04-26T23:01:24.006 INFO:tasks.workunit.client.0.smithi027.stdout:7/114: rmdir d0/d3/d14 39 2024-04-26T23:01:24.021 INFO:tasks.workunit.client.0.smithi027.stdout:1/194: dwrite d1/f1b [0,4194304] 0 2024-04-26T23:01:24.023 INFO:tasks.workunit.client.0.smithi027.stdout:1/195: truncate d1/d26/f29 1992677 0 2024-04-26T23:01:24.024 INFO:tasks.workunit.client.0.smithi027.stdout:1/196: mknod d1/dc/c41 0 2024-04-26T23:01:24.025 INFO:tasks.workunit.client.0.smithi027.stdout:1/197: getdents d1/d26 0 2024-04-26T23:01:24.026 INFO:tasks.workunit.client.0.smithi027.stdout:1/198: mkdir d1/d2a/d3c/d42 0 2024-04-26T23:01:24.026 INFO:tasks.workunit.client.0.smithi027.stdout:1/199: readlink d1/d5/l3f 0 2024-04-26T23:01:24.026 INFO:tasks.workunit.client.0.smithi027.stdout:1/200: readlink d1/l3a 0 2024-04-26T23:01:24.026 INFO:tasks.workunit.client.0.smithi027.stdout:1/201: dread - d1/d5/d21/f27 zero size 2024-04-26T23:01:24.026 INFO:tasks.workunit.client.0.smithi027.stdout:1/202: readlink d1/d5/d40/d2d/l38 0 2024-04-26T23:01:24.026 INFO:tasks.workunit.client.0.smithi027.stdout:1/203: stat d1/c28 0 2024-04-26T23:01:24.027 INFO:tasks.workunit.client.0.smithi027.stdout:1/204: mknod d1/d26/c43 0 2024-04-26T23:01:24.044 INFO:tasks.workunit.client.0.smithi027.stdout:8/135: dwrite d9/da/f13 [0,4194304] 0 2024-04-26T23:01:24.047 INFO:tasks.workunit.client.0.smithi027.stdout:8/136: mkdir d9/d28/d2b/d31 0 2024-04-26T23:01:24.047 INFO:tasks.workunit.client.0.smithi027.stdout:8/137: dread - d9/de/f2c zero size 2024-04-26T23:01:24.049 INFO:tasks.workunit.client.0.smithi027.stdout:8/138: link f7 d9/de/f32 0 2024-04-26T23:01:24.050 INFO:tasks.workunit.client.0.smithi027.stdout:8/139: mkdir d9/da/d33 0 2024-04-26T23:01:24.052 INFO:tasks.workunit.client.0.smithi027.stdout:8/140: rename d9/de/d22/c2d to d9/da/d2a/c34 0 2024-04-26T23:01:24.053 INFO:tasks.workunit.client.0.smithi027.stdout:8/141: truncate d9/f16 384603 0 2024-04-26T23:01:24.055 INFO:tasks.workunit.client.0.smithi027.stdout:8/142: mkdir d9/de/d20/d35 0 2024-04-26T23:01:24.069 INFO:tasks.workunit.client.0.smithi027.stdout:7/115: dwrite d0/d3/d14/f15 [0,4194304] 0 2024-04-26T23:01:24.070 INFO:tasks.workunit.client.0.smithi027.stdout:7/116: symlink d0/l23 0 2024-04-26T23:01:24.073 INFO:tasks.workunit.client.0.smithi027.stdout:5/128: sync 2024-04-26T23:01:24.075 INFO:tasks.workunit.client.0.smithi027.stdout:5/129: getdents d4/d24 0 2024-04-26T23:01:24.076 INFO:tasks.workunit.client.0.smithi027.stdout:5/130: mkdir d4/d24/d22/d29 0 2024-04-26T23:01:24.078 INFO:tasks.workunit.client.0.smithi027.stdout:5/131: mkdir d4/d24/d22/d29/d2a 0 2024-04-26T23:01:24.078 INFO:tasks.workunit.client.0.smithi027.stdout:5/132: chown d4/l12 117171323 1 2024-04-26T23:01:24.110 INFO:tasks.workunit.client.0.smithi027.stdout:5/133: dread d4/f18 [0,4194304] 0 2024-04-26T23:01:24.118 INFO:tasks.workunit.client.0.smithi027.stdout:1/205: dwrite d1/d5/f19 [0,4194304] 0 2024-04-26T23:01:24.120 INFO:tasks.workunit.client.0.smithi027.stdout:8/143: dwrite d9/de/f18 [0,4194304] 0 2024-04-26T23:01:24.120 INFO:tasks.workunit.client.0.smithi027.stdout:1/206: unlink d1/dc/l1c 0 2024-04-26T23:01:24.123 INFO:tasks.workunit.client.0.smithi027.stdout:1/207: unlink d1/dc/f25 0 2024-04-26T23:01:24.128 INFO:tasks.workunit.client.0.smithi027.stdout:1/208: creat d1/d5/d21/f44 x:0 0 0 2024-04-26T23:01:24.129 INFO:tasks.workunit.client.0.smithi027.stdout:7/117: dwrite d0/f1f [0,4194304] 0 2024-04-26T23:01:24.130 INFO:tasks.workunit.client.0.smithi027.stdout:1/209: symlink d1/l45 0 2024-04-26T23:01:24.131 INFO:tasks.workunit.client.0.smithi027.stdout:7/118: creat d0/d3/d14/f24 x:0 0 0 2024-04-26T23:01:24.131 INFO:tasks.workunit.client.0.smithi027.stdout:7/119: chown d0/l7 191 1 2024-04-26T23:01:24.132 INFO:tasks.workunit.client.0.smithi027.stdout:1/210: truncate d1/fb 2377666 0 2024-04-26T23:01:24.132 INFO:tasks.workunit.client.0.smithi027.stdout:5/134: dread d4/db/d16/f1f [0,4194304] 0 2024-04-26T23:01:24.132 INFO:tasks.workunit.client.0.smithi027.stdout:5/135: fdatasync d4/d1a/f1e 0 2024-04-26T23:01:24.133 INFO:tasks.workunit.client.0.smithi027.stdout:7/120: mknod d0/d3/d14/c25 0 2024-04-26T23:01:24.134 INFO:tasks.workunit.client.0.smithi027.stdout:5/136: mkdir d4/d1a/d2b 0 2024-04-26T23:01:24.135 INFO:tasks.workunit.client.0.smithi027.stdout:5/137: rename d4/f1d to d4/d24/d22/d29/d2a/f2c 0 2024-04-26T23:01:24.137 INFO:tasks.workunit.client.0.smithi027.stdout:7/121: mknod d0/d3/d14/c26 0 2024-04-26T23:01:24.141 INFO:tasks.workunit.client.0.smithi027.stdout:7/122: rmdir d0/d3/d14 39 2024-04-26T23:01:24.142 INFO:tasks.workunit.client.0.smithi027.stdout:8/144: dread d9/da/f13 [0,4194304] 0 2024-04-26T23:01:24.148 INFO:tasks.workunit.client.0.smithi027.stdout:2/133: sync 2024-04-26T23:01:24.151 INFO:tasks.workunit.client.0.smithi027.stdout:2/134: unlink d2/d5/f13 0 2024-04-26T23:01:24.151 INFO:tasks.workunit.client.0.smithi027.stdout:2/135: chown d2/d5/db/de/d21/fd 122 1 2024-04-26T23:01:24.152 INFO:tasks.workunit.client.0.smithi027.stdout:2/136: symlink d2/d5/db/de/l2a 0 2024-04-26T23:01:24.152 INFO:tasks.workunit.client.0.smithi027.stdout:2/137: fdatasync d2/d5/db/d1b/d1c/f20 0 2024-04-26T23:01:24.152 INFO:tasks.workunit.client.0.smithi027.stdout:1/211: dread d1/f10 [0,4194304] 0 2024-04-26T23:01:24.152 INFO:tasks.workunit.client.0.smithi027.stdout:1/212: chown d1/dc/f1d 358 1 2024-04-26T23:01:24.153 INFO:tasks.workunit.client.0.smithi027.stdout:1/213: truncate d1/f6 1288831 0 2024-04-26T23:01:24.155 INFO:tasks.workunit.client.0.smithi027.stdout:1/214: creat d1/d5/f46 x:0 0 0 2024-04-26T23:01:24.155 INFO:tasks.workunit.client.0.smithi027.stdout:1/215: write d1/f6 [2116401,24494] 0 2024-04-26T23:01:24.157 INFO:tasks.workunit.client.0.smithi027.stdout:1/216: rmdir d1/dc 39 2024-04-26T23:01:24.161 INFO:tasks.workunit.client.0.smithi027.stdout:5/138: dread d4/f15 [0,4194304] 0 2024-04-26T23:01:24.161 INFO:tasks.workunit.client.0.smithi027.stdout:5/139: stat d4/f9 0 2024-04-26T23:01:24.161 INFO:tasks.workunit.client.0.smithi027.stdout:5/140: fsync d4/db/f23 0 2024-04-26T23:01:24.166 INFO:tasks.workunit.client.0.smithi027.stdout:8/145: dread f2 [0,4194304] 0 2024-04-26T23:01:24.166 INFO:tasks.workunit.client.0.smithi027.stdout:8/146: chown f3 394057 1 2024-04-26T23:01:24.166 INFO:tasks.workunit.client.0.smithi027.stdout:8/147: write f3 [461669,117610] 0 2024-04-26T23:01:24.166 INFO:tasks.workunit.client.0.smithi027.stdout:1/217: rename d1/f10 to d1/d2a/f47 0 2024-04-26T23:01:24.167 INFO:tasks.workunit.client.0.smithi027.stdout:5/141: symlink d4/d24/d25/l2d 0 2024-04-26T23:01:24.170 INFO:tasks.workunit.client.0.smithi027.stdout:1/218: rename d1/d26/c43 to d1/d26/d36/c48 0 2024-04-26T23:01:24.176 INFO:tasks.workunit.client.0.smithi027.stdout:1/219: creat d1/d2a/f49 x:0 0 0 2024-04-26T23:01:24.176 INFO:tasks.workunit.client.0.smithi027.stdout:1/220: readlink d1/d5/la 0 2024-04-26T23:01:24.176 INFO:tasks.workunit.client.0.smithi027.stdout:1/221: chown d1/d2a/f49 53 1 2024-04-26T23:01:24.176 INFO:tasks.workunit.client.0.smithi027.stdout:1/222: chown d1/d2a 13 1 2024-04-26T23:01:24.185 INFO:tasks.workunit.client.0.smithi027.stdout:2/138: dread d2/d5/fa [0,4194304] 0 2024-04-26T23:01:24.191 INFO:tasks.workunit.client.0.smithi027.stdout:2/139: getdents d2/d5/db 0 2024-04-26T23:01:24.195 INFO:tasks.workunit.client.0.smithi027.stdout:5/142: dread d4/d24/d22/d29/d2a/f2c [0,4194304] 0 2024-04-26T23:01:24.229 INFO:tasks.workunit.client.0.smithi027.stdout:1/223: dwrite d1/dc/f37 [0,4194304] 0 2024-04-26T23:01:24.230 INFO:tasks.workunit.client.0.smithi027.stdout:1/224: stat d1/dc/f37 0 2024-04-26T23:01:24.230 INFO:tasks.workunit.client.0.smithi027.stdout:1/225: chown d1/d5/d40/d35 79805 1 2024-04-26T23:01:24.246 INFO:tasks.workunit.client.0.smithi027.stdout:5/143: dwrite f1 [0,4194304] 0 2024-04-26T23:01:24.248 INFO:tasks.workunit.client.0.smithi027.stdout:5/144: symlink d4/d24/d22/d29/l2e 0 2024-04-26T23:01:24.258 INFO:tasks.workunit.client.0.smithi027.stdout:2/140: dwrite d2/d5/db/d1b/d1c/d25/d19/f1e [4194304,4194304] 0 2024-04-26T23:01:24.259 INFO:tasks.workunit.client.0.smithi027.stdout:2/141: truncate d2/d5/db/de/d21/fd 9388999 0 2024-04-26T23:01:24.259 INFO:tasks.workunit.client.0.smithi027.stdout:2/142: chown d2/d5/f22 1949405 1 2024-04-26T23:01:24.259 INFO:tasks.workunit.client.0.smithi027.stdout:1/226: dread d1/f1b [0,4194304] 0 2024-04-26T23:01:24.260 INFO:tasks.workunit.client.0.smithi027.stdout:2/143: creat d2/d5/db/de/d21/d26/f2b x:0 0 0 2024-04-26T23:01:24.261 INFO:tasks.workunit.client.0.smithi027.stdout:1/227: mknod d1/d5/d21/c4a 0 2024-04-26T23:01:24.261 INFO:tasks.workunit.client.0.smithi027.stdout:1/228: write d1/d5/d21/f27 [763304,25706] 0 2024-04-26T23:01:24.261 INFO:tasks.workunit.client.0.smithi027.stdout:1/229: fdatasync d1/d2a/f47 0 2024-04-26T23:01:24.263 INFO:tasks.workunit.client.0.smithi027.stdout:1/230: symlink d1/d26/d36/l4b 0 2024-04-26T23:01:24.274 INFO:tasks.workunit.client.0.smithi027.stdout:6/87: sync 2024-04-26T23:01:24.274 INFO:tasks.workunit.client.0.smithi027.stdout:9/152: sync 2024-04-26T23:01:24.279 INFO:tasks.workunit.client.0.smithi027.stdout:5/145: dread d4/f15 [0,4194304] 0 2024-04-26T23:01:24.280 INFO:tasks.workunit.client.0.smithi027.stdout:5/146: symlink d4/db/l2f 0 2024-04-26T23:01:24.281 INFO:tasks.workunit.client.0.smithi027.stdout:5/147: chown d4/l21 7 1 2024-04-26T23:01:24.281 INFO:tasks.workunit.client.0.smithi027.stdout:5/148: fdatasync d4/d24/ff 0 2024-04-26T23:01:24.282 INFO:tasks.workunit.client.0.smithi027.stdout:5/149: getdents d4/d1a 0 2024-04-26T23:01:24.282 INFO:tasks.workunit.client.0.smithi027.stdout:5/150: write d4/d1a/f20 [1518154,66648] 0 2024-04-26T23:01:24.309 INFO:tasks.workunit.client.0.smithi027.stdout:1/231: dread d1/d5/d21/f27 [0,4194304] 0 2024-04-26T23:01:24.309 INFO:tasks.workunit.client.0.smithi027.stdout:1/232: fdatasync d1/f3 0 2024-04-26T23:01:24.326 INFO:tasks.workunit.client.0.smithi027.stdout:9/153: dwrite d4/d10/f19 [4194304,4194304] 0 2024-04-26T23:01:24.330 INFO:tasks.workunit.client.0.smithi027.stdout:9/154: symlink d4/d17/d2e/l3a 0 2024-04-26T23:01:24.331 INFO:tasks.workunit.client.0.smithi027.stdout:9/155: dread - d4/d10/d25/f39 zero size 2024-04-26T23:01:24.331 INFO:tasks.workunit.client.0.smithi027.stdout:9/156: chown d4/cf 5 1 2024-04-26T23:01:24.331 INFO:tasks.workunit.client.0.smithi027.stdout:9/157: fsync d4/d10/f12 0 2024-04-26T23:01:24.332 INFO:tasks.workunit.client.0.smithi027.stdout:5/151: dwrite f1 [0,4194304] 0 2024-04-26T23:01:24.333 INFO:tasks.workunit.client.0.smithi027.stdout:9/158: unlink d4/d10/d30/f33 0 2024-04-26T23:01:24.333 INFO:tasks.workunit.client.0.smithi027.stdout:9/159: fdatasync d4/f24 0 2024-04-26T23:01:24.333 INFO:tasks.workunit.client.0.smithi027.stdout:9/160: write d4/d10/f28 [556678,129524] 0 2024-04-26T23:01:24.336 INFO:tasks.workunit.client.0.smithi027.stdout:1/233: dread d1/dc/f37 [0,4194304] 0 2024-04-26T23:01:24.337 INFO:tasks.workunit.client.0.smithi027.stdout:5/152: unlink d4/d24/ff 0 2024-04-26T23:01:24.341 INFO:tasks.workunit.client.0.smithi027.stdout:5/153: mknod d4/db/d16/c30 0 2024-04-26T23:01:24.341 INFO:tasks.workunit.client.0.smithi027.stdout:4/145: sync 2024-04-26T23:01:24.358 INFO:tasks.workunit.client.0.smithi027.stdout:9/161: dread d4/d17/f1d [0,4194304] 0 2024-04-26T23:01:24.358 INFO:tasks.workunit.client.0.smithi027.stdout:9/162: readlink d4/la 0 2024-04-26T23:01:24.364 INFO:tasks.workunit.client.0.smithi027.stdout:9/163: fdatasync d4/d10/f28 0 2024-04-26T23:01:24.368 INFO:tasks.workunit.client.0.smithi027.stdout:9/164: creat d4/d10/d30/f3b x:0 0 0 2024-04-26T23:01:24.370 INFO:tasks.workunit.client.0.smithi027.stdout:9/165: link d4/l36 d4/d17/d2e/l3c 0 2024-04-26T23:01:24.370 INFO:tasks.workunit.client.0.smithi027.stdout:9/166: stat d4/d17/f29 0 2024-04-26T23:01:24.386 INFO:tasks.workunit.client.0.smithi027.stdout:1/234: dwrite d1/d5/d21/f22 [4194304,4194304] 0 2024-04-26T23:01:24.389 INFO:tasks.workunit.client.0.smithi027.stdout:5/154: dwrite d4/f18 [0,4194304] 0 2024-04-26T23:01:24.389 INFO:tasks.workunit.client.0.smithi027.stdout:5/155: chown d4/d24/d22/d29/d2a 11017 1 2024-04-26T23:01:24.389 INFO:tasks.workunit.client.0.smithi027.stdout:5/156: chown d4/d24 656922519 1 2024-04-26T23:01:24.389 INFO:tasks.workunit.client.0.smithi027.stdout:5/157: fsync d4/f18 0 2024-04-26T23:01:24.390 INFO:tasks.workunit.client.0.smithi027.stdout:5/158: rmdir d4/db/d16 39 2024-04-26T23:01:24.392 INFO:tasks.workunit.client.0.smithi027.stdout:5/159: rename d4/f18 to d4/db/f31 0 2024-04-26T23:01:24.406 INFO:tasks.workunit.client.0.smithi027.stdout:9/167: dwrite d4/f35 [0,4194304] 0 2024-04-26T23:01:24.408 INFO:tasks.workunit.client.0.smithi027.stdout:9/168: link d4/d17/d2e/f34 d4/d17/d2e/f3d 0 2024-04-26T23:01:24.409 INFO:tasks.workunit.client.0.smithi027.stdout:5/160: dread d4/db/f10 [0,4194304] 0 2024-04-26T23:01:24.410 INFO:tasks.workunit.client.0.smithi027.stdout:9/169: truncate d4/f14 3477895 0 2024-04-26T23:01:24.410 INFO:tasks.workunit.client.0.smithi027.stdout:9/170: chown d4/d10/c13 36925 1 2024-04-26T23:01:24.410 INFO:tasks.workunit.client.0.smithi027.stdout:5/161: getdents d4/d24/d22 0 2024-04-26T23:01:24.440 INFO:tasks.workunit.client.0.smithi027.stdout:5/162: dread d4/d24/d22/d29/d2a/f2c [0,4194304] 0 2024-04-26T23:01:24.456 INFO:tasks.workunit.client.0.smithi027.stdout:1/235: dwrite d1/f32 [0,4194304] 0 2024-04-26T23:01:24.458 INFO:tasks.workunit.client.0.smithi027.stdout:1/236: creat d1/d5/d40/d35/f4c x:0 0 0 2024-04-26T23:01:24.459 INFO:tasks.workunit.client.0.smithi027.stdout:5/163: write d4/db/f31 [3225300,23909] 0 2024-04-26T23:01:24.459 INFO:tasks.workunit.client.0.smithi027.stdout:5/164: fsync d4/d24/d22/d29/d2a/f2c 0 2024-04-26T23:01:24.459 INFO:tasks.workunit.client.0.smithi027.stdout:1/237: link d1/d5/d21/f34 d1/d2a/d3c/f4d 0 2024-04-26T23:01:24.462 INFO:tasks.workunit.client.0.smithi027.stdout:1/238: symlink d1/d5/d40/d2d/l4e 0 2024-04-26T23:01:24.469 INFO:tasks.workunit.client.0.smithi027.stdout:1/239: truncate d1/d5/d21/f22 3833444 0 2024-04-26T23:01:24.469 INFO:tasks.workunit.client.0.smithi027.stdout:1/240: dread - d1/d5/f2e zero size 2024-04-26T23:01:24.473 INFO:tasks.workunit.client.0.smithi027.stdout:1/241: symlink d1/d5/d40/d35/l4f 0 2024-04-26T23:01:24.473 INFO:tasks.workunit.client.0.smithi027.stdout:9/171: dwrite d4/d10/f19 [8388608,4194304] 0 2024-04-26T23:01:24.478 INFO:tasks.workunit.client.0.smithi027.stdout:7/123: sync 2024-04-26T23:01:24.478 INFO:tasks.workunit.client.0.smithi027.stdout:3/164: sync 2024-04-26T23:01:24.478 INFO:tasks.workunit.client.0.smithi027.stdout:8/148: sync 2024-04-26T23:01:24.478 INFO:tasks.workunit.client.0.smithi027.stdout:0/166: sync 2024-04-26T23:01:24.478 INFO:tasks.workunit.client.0.smithi027.stdout:6/88: sync 2024-04-26T23:01:24.479 INFO:tasks.workunit.client.0.smithi027.stdout:2/144: sync 2024-04-26T23:01:24.479 INFO:tasks.workunit.client.0.smithi027.stdout:4/146: sync 2024-04-26T23:01:24.479 INFO:tasks.workunit.client.0.smithi027.stdout:7/124: unlink d0/d3/d14/f24 0 2024-04-26T23:01:24.480 INFO:tasks.workunit.client.0.smithi027.stdout:3/165: truncate d6/dc/f11 998210 0 2024-04-26T23:01:24.480 INFO:tasks.workunit.client.0.smithi027.stdout:3/166: chown d6/dc/f11 217 1 2024-04-26T23:01:24.480 INFO:tasks.workunit.client.0.smithi027.stdout:3/167: getdents d6/dc/d2f/d37 0 2024-04-26T23:01:24.480 INFO:tasks.workunit.client.0.smithi027.stdout:8/149: mknod d9/de/d22/c36 0 2024-04-26T23:01:24.483 INFO:tasks.workunit.client.0.smithi027.stdout:2/145: getdents d2/d5/db/d1b/d1c/d25 0 2024-04-26T23:01:24.483 INFO:tasks.workunit.client.0.smithi027.stdout:6/89: creat d1/de/f1e x:0 0 0 2024-04-26T23:01:24.488 INFO:tasks.workunit.client.0.smithi027.stdout:3/168: creat d6/dc/d2f/d37/f3c x:0 0 0 2024-04-26T23:01:24.488 INFO:tasks.workunit.client.0.smithi027.stdout:4/147: rename da/df to da/dd/de/d14/d17/d24 0 2024-04-26T23:01:24.488 INFO:tasks.workunit.client.0.smithi027.stdout:8/150: symlink d9/de/d20/l37 0 2024-04-26T23:01:24.492 INFO:tasks.workunit.client.0.smithi027.stdout:2/146: link d2/d5/fa d2/d5/db/d1b/d1c/d25/d19/d23/f2c 0 2024-04-26T23:01:24.492 INFO:tasks.workunit.client.0.smithi027.stdout:6/90: creat d1/de/f1f x:0 0 0 2024-04-26T23:01:24.492 INFO:tasks.workunit.client.0.smithi027.stdout:3/169: truncate d6/dc/d34/d2e/f1e 47074 0 2024-04-26T23:01:24.492 INFO:tasks.workunit.client.0.smithi027.stdout:4/148: creat da/dd/de/d14/d21/f25 x:0 0 0 2024-04-26T23:01:24.492 INFO:tasks.workunit.client.0.smithi027.stdout:4/149: chown da/dd/de 0 1 2024-04-26T23:01:24.496 INFO:tasks.workunit.client.0.smithi027.stdout:8/151: creat d9/de/d2e/f38 x:0 0 0 2024-04-26T23:01:24.496 INFO:tasks.workunit.client.0.smithi027.stdout:8/152: chown d9/de/d22/c36 45 1 2024-04-26T23:01:24.496 INFO:tasks.workunit.client.0.smithi027.stdout:9/172: dread d4/f2b [0,4194304] 0 2024-04-26T23:01:24.496 INFO:tasks.workunit.client.0.smithi027.stdout:9/173: dread - d4/d17/d2e/f34 zero size 2024-04-26T23:01:24.496 INFO:tasks.workunit.client.0.smithi027.stdout:9/174: write d4/d10/f12 [3382780,43079] 0 2024-04-26T23:01:24.496 INFO:tasks.workunit.client.0.smithi027.stdout:7/125: rename d0/d3/d14/f15 to d0/f27 0 2024-04-26T23:01:24.496 INFO:tasks.workunit.client.0.smithi027.stdout:2/147: unlink d2/d5/db/de/d21/fd 0 2024-04-26T23:01:24.497 INFO:tasks.workunit.client.0.smithi027.stdout:2/148: chown d2/d5/db/d1b/d1c/d25 2 1 2024-04-26T23:01:24.497 INFO:tasks.workunit.client.0.smithi027.stdout:2/149: fdatasync d2/d5/f22 0 2024-04-26T23:01:24.497 INFO:tasks.workunit.client.0.smithi027.stdout:6/91: creat d1/de/f20 x:0 0 0 2024-04-26T23:01:24.501 INFO:tasks.workunit.client.0.smithi027.stdout:6/92: write d1/df/f8 [2770060,49926] 0 2024-04-26T23:01:24.501 INFO:tasks.workunit.client.0.smithi027.stdout:4/150: unlink da/lb 0 2024-04-26T23:01:24.501 INFO:tasks.workunit.client.0.smithi027.stdout:9/175: creat d4/d10/d25/f3e x:0 0 0 2024-04-26T23:01:24.502 INFO:tasks.workunit.client.0.smithi027.stdout:2/150: creat d2/d5/db/de/d21/d26/f2d x:0 0 0 2024-04-26T23:01:24.503 INFO:tasks.workunit.client.0.smithi027.stdout:5/165: read d4/f9 [173743,122939] 0 2024-04-26T23:01:24.503 INFO:tasks.workunit.client.0.smithi027.stdout:5/166: chown d4/d24/d22/d29/d2a 286 1 2024-04-26T23:01:24.511 INFO:tasks.workunit.client.0.smithi027.stdout:2/151: symlink d2/d5/db/d1b/l2e 0 2024-04-26T23:01:24.511 INFO:tasks.workunit.client.0.smithi027.stdout:2/152: getdents d2/d5/db/d1b/d1c/d25 0 2024-04-26T23:01:24.511 INFO:tasks.workunit.client.0.smithi027.stdout:9/176: rmdir d4/d17 39 2024-04-26T23:01:24.513 INFO:tasks.workunit.client.0.smithi027.stdout:5/167: symlink d4/d1a/l32 0 2024-04-26T23:01:24.513 INFO:tasks.workunit.client.0.smithi027.stdout:5/168: write d4/f9 [611519,114245] 0 2024-04-26T23:01:24.519 INFO:tasks.workunit.client.0.smithi027.stdout:9/177: rename d4/d17/c2d to d4/d10/c3f 0 2024-04-26T23:01:24.524 INFO:tasks.workunit.client.0.smithi027.stdout:5/169: unlink d4/d24/d22/d29/d2a/f2c 0 2024-04-26T23:01:24.524 INFO:tasks.workunit.client.0.smithi027.stdout:3/170: dread d6/dc/d34/d2e/f22 [0,4194304] 0 2024-04-26T23:01:24.530 INFO:tasks.workunit.client.0.smithi027.stdout:8/153: dread d9/de/f32 [0,4194304] 0 2024-04-26T23:01:24.533 INFO:tasks.workunit.client.0.smithi027.stdout:7/126: rename d0/d3 to d0/d28 0 2024-04-26T23:01:24.538 INFO:tasks.workunit.client.0.smithi027.stdout:5/170: rename d4/db/f31 to d4/d24/d22/d29/f33 0 2024-04-26T23:01:24.538 INFO:tasks.workunit.client.0.smithi027.stdout:8/154: mknod d9/de/d22/c39 0 2024-04-26T23:01:24.538 INFO:tasks.workunit.client.0.smithi027.stdout:8/155: fsync d9/da/f13 0 2024-04-26T23:01:24.538 INFO:tasks.workunit.client.0.smithi027.stdout:8/156: fdatasync d9/de/f14 0 2024-04-26T23:01:24.539 INFO:tasks.workunit.client.0.smithi027.stdout:7/127: creat d0/d28/d14/f29 x:0 0 0 2024-04-26T23:01:24.546 INFO:tasks.workunit.client.0.smithi027.stdout:5/171: mkdir d4/db/d16/d34 0 2024-04-26T23:01:24.546 INFO:tasks.workunit.client.0.smithi027.stdout:5/172: dread - d4/d1a/f1e zero size 2024-04-26T23:01:24.546 INFO:tasks.workunit.client.0.smithi027.stdout:2/153: dread d2/d5/db/d1b/d1c/d25/d19/f1e [0,4194304] 0 2024-04-26T23:01:24.546 INFO:tasks.workunit.client.0.smithi027.stdout:8/157: creat d9/da/d33/f3a x:0 0 0 2024-04-26T23:01:24.546 INFO:tasks.workunit.client.0.smithi027.stdout:6/93: write d1/de/f1a [3109607,37450] 0 2024-04-26T23:01:24.546 INFO:tasks.workunit.client.0.smithi027.stdout:5/173: symlink d4/db/l35 0 2024-04-26T23:01:24.546 INFO:tasks.workunit.client.0.smithi027.stdout:2/154: mkdir d2/d5/db/d1b/d1c/d25/d2f 0 2024-04-26T23:01:24.546 INFO:tasks.workunit.client.0.smithi027.stdout:2/155: fsync d2/d5/db/de/d21/d26/f2d 0 2024-04-26T23:01:24.546 INFO:tasks.workunit.client.0.smithi027.stdout:2/156: dread - d2/d5/db/de/d21/d26/f2b zero size 2024-04-26T23:01:24.546 INFO:tasks.workunit.client.0.smithi027.stdout:2/157: stat d2/d5/db/d1b/l2e 0 2024-04-26T23:01:24.547 INFO:tasks.workunit.client.0.smithi027.stdout:6/94: symlink d1/l21 0 2024-04-26T23:01:24.547 INFO:tasks.workunit.client.0.smithi027.stdout:5/174: creat d4/d24/d22/d29/f36 x:0 0 0 2024-04-26T23:01:24.548 INFO:tasks.workunit.client.0.smithi027.stdout:8/158: mkdir d9/d28/d2b/d3b 0 2024-04-26T23:01:24.549 INFO:tasks.workunit.client.0.smithi027.stdout:3/171: dread d6/dc/f10 [0,4194304] 0 2024-04-26T23:01:24.549 INFO:tasks.workunit.client.0.smithi027.stdout:3/172: chown d6/dc/f25 4126733 1 2024-04-26T23:01:24.551 INFO:tasks.workunit.client.0.smithi027.stdout:0/167: dwrite d0/f1b [0,4194304] 0 2024-04-26T23:01:24.551 INFO:tasks.workunit.client.0.smithi027.stdout:5/175: rmdir d4/d24/d22 39 2024-04-26T23:01:24.555 INFO:tasks.workunit.client.0.smithi027.stdout:3/173: creat d6/dc/d2f/f3d x:0 0 0 2024-04-26T23:01:24.555 INFO:tasks.workunit.client.0.smithi027.stdout:6/95: mknod d1/df/c22 0 2024-04-26T23:01:24.556 INFO:tasks.workunit.client.0.smithi027.stdout:0/168: mknod d0/d4/da/d1a/d2a/c32 0 2024-04-26T23:01:24.556 INFO:tasks.workunit.client.0.smithi027.stdout:0/169: fdatasync d0/d4/da/d1a/d2a/f30 0 2024-04-26T23:01:24.557 INFO:tasks.workunit.client.0.smithi027.stdout:5/176: link d4/d24/c11 d4/c37 0 2024-04-26T23:01:24.560 INFO:tasks.workunit.client.0.smithi027.stdout:6/96: mknod d1/c23 0 2024-04-26T23:01:24.563 INFO:tasks.workunit.client.0.smithi027.stdout:0/170: rename d0/d4/da/d1a/d1e/f2c to d0/d4/da/d15/f33 0 2024-04-26T23:01:24.563 INFO:tasks.workunit.client.0.smithi027.stdout:0/171: truncate d0/d4/da/d15/f33 676389 0 2024-04-26T23:01:24.564 INFO:tasks.workunit.client.0.smithi027.stdout:0/172: write d0/d4/f20 [4256783,56075] 0 2024-04-26T23:01:24.564 INFO:tasks.workunit.client.0.smithi027.stdout:0/173: chown d0/c28 15 1 2024-04-26T23:01:24.566 INFO:tasks.workunit.client.0.smithi027.stdout:5/177: link f1 d4/db/d16/f38 0 2024-04-26T23:01:24.566 INFO:tasks.workunit.client.0.smithi027.stdout:1/242: dwrite d1/d26/f29 [0,4194304] 0 2024-04-26T23:01:24.566 INFO:tasks.workunit.client.0.smithi027.stdout:1/243: write d1/d5/f2f [882244,54509] 0 2024-04-26T23:01:24.570 INFO:tasks.workunit.client.0.smithi027.stdout:6/97: unlink d1/de/c1c 0 2024-04-26T23:01:24.573 INFO:tasks.workunit.client.0.smithi027.stdout:0/174: rename d0/d4/da/d15/f16 to d0/d4/da/d15/f34 0 2024-04-26T23:01:24.577 INFO:tasks.workunit.client.0.smithi027.stdout:5/178: mkdir d4/d24/d22/d29/d39 0 2024-04-26T23:01:24.578 INFO:tasks.workunit.client.0.smithi027.stdout:4/151: dwrite f2 [0,4194304] 0 2024-04-26T23:01:24.582 INFO:tasks.workunit.client.0.smithi027.stdout:2/158: read d2/d5/f22 [1482608,57162] 0 2024-04-26T23:01:24.582 INFO:tasks.workunit.client.0.smithi027.stdout:2/159: dread - d2/d5/db/de/d21/d26/f2d zero size 2024-04-26T23:01:24.582 INFO:tasks.workunit.client.0.smithi027.stdout:9/178: dwrite d4/d17/f29 [0,4194304] 0 2024-04-26T23:01:24.582 INFO:tasks.workunit.client.0.smithi027.stdout:9/179: dread - d4/f24 zero size 2024-04-26T23:01:24.584 INFO:tasks.workunit.client.0.smithi027.stdout:6/98: unlink d1/de/c18 0 2024-04-26T23:01:24.584 INFO:tasks.workunit.client.0.smithi027.stdout:6/99: dread - d1/de/f1b zero size 2024-04-26T23:01:24.584 INFO:tasks.workunit.client.0.smithi027.stdout:0/175: creat d0/d4/da/d1a/d1e/f35 x:0 0 0 2024-04-26T23:01:24.584 INFO:tasks.workunit.client.0.smithi027.stdout:1/244: creat d1/d5/f50 x:0 0 0 2024-04-26T23:01:24.585 INFO:tasks.workunit.client.0.smithi027.stdout:5/179: mknod d4/c3a 0 2024-04-26T23:01:24.585 INFO:tasks.workunit.client.0.smithi027.stdout:5/180: stat c3 0 2024-04-26T23:01:24.585 INFO:tasks.workunit.client.0.smithi027.stdout:5/181: truncate d4/db/f23 952211 0 2024-04-26T23:01:24.587 INFO:tasks.workunit.client.0.smithi027.stdout:4/152: creat da/dd/de/d14/d17/d24/f26 x:0 0 0 2024-04-26T23:01:24.590 INFO:tasks.workunit.client.0.smithi027.stdout:9/180: mkdir d4/d10/d40 0 2024-04-26T23:01:24.590 INFO:tasks.workunit.client.0.smithi027.stdout:9/181: fdatasync d4/d10/f28 0 2024-04-26T23:01:24.591 INFO:tasks.workunit.client.0.smithi027.stdout:0/176: rmdir d0/d4/da/d1a/d1e 39 2024-04-26T23:01:24.591 INFO:tasks.workunit.client.0.smithi027.stdout:0/177: write d0/d4/da/d1a/d1e/f25 [4588992,60314] 0 2024-04-26T23:01:24.591 INFO:tasks.workunit.client.0.smithi027.stdout:0/178: write d0/d4/da/f2f [421154,45070] 0 2024-04-26T23:01:24.591 INFO:tasks.workunit.client.0.smithi027.stdout:0/179: stat d0/d4/da/d15/c2b 0 2024-04-26T23:01:24.592 INFO:tasks.workunit.client.0.smithi027.stdout:1/245: symlink d1/d5/d40/d35/l51 0 2024-04-26T23:01:24.593 INFO:tasks.workunit.client.0.smithi027.stdout:1/246: write d1/f6 [2949977,127876] 0 2024-04-26T23:01:24.596 INFO:tasks.workunit.client.0.smithi027.stdout:3/174: dwrite d6/dc/d17/f20 [0,4194304] 0 2024-04-26T23:01:24.596 INFO:tasks.workunit.client.0.smithi027.stdout:3/175: write d6/dc/f11 [1019751,87296] 0 2024-04-26T23:01:24.596 INFO:tasks.workunit.client.0.smithi027.stdout:9/182: creat d4/d10/d2c/f41 x:0 0 0 2024-04-26T23:01:24.597 INFO:tasks.workunit.client.0.smithi027.stdout:6/100: symlink d1/df/l24 0 2024-04-26T23:01:24.597 INFO:tasks.workunit.client.0.smithi027.stdout:6/101: chown d1/df 456 1 2024-04-26T23:01:24.599 INFO:tasks.workunit.client.0.smithi027.stdout:8/159: dwrite d9/f17 [0,4194304] 0 2024-04-26T23:01:24.599 INFO:tasks.workunit.client.0.smithi027.stdout:8/160: chown d9/de/d2e/c1a 57 1 2024-04-26T23:01:24.600 INFO:tasks.workunit.client.0.smithi027.stdout:3/176: symlink d6/dc/l3e 0 2024-04-26T23:01:24.600 INFO:tasks.workunit.client.0.smithi027.stdout:0/180: dread d0/d4/da/f11 [0,4194304] 0 2024-04-26T23:01:24.605 INFO:tasks.workunit.client.0.smithi027.stdout:9/183: unlink d4/lb 0 2024-04-26T23:01:24.606 INFO:tasks.workunit.client.0.smithi027.stdout:9/184: fsync d4/d10/d30/f3b 0 2024-04-26T23:01:24.606 INFO:tasks.workunit.client.0.smithi027.stdout:8/161: rmdir d9/de 39 2024-04-26T23:01:24.606 INFO:tasks.workunit.client.0.smithi027.stdout:8/162: readlink d9/de/d20/l37 0 2024-04-26T23:01:24.606 INFO:tasks.workunit.client.0.smithi027.stdout:8/163: creat d9/de/d20/d35/f3c x:0 0 0 2024-04-26T23:01:24.607 INFO:tasks.workunit.client.0.smithi027.stdout:8/164: rename d9/de/f32 to d9/d28/d2b/f3d 0 2024-04-26T23:01:24.607 INFO:tasks.workunit.client.0.smithi027.stdout:8/165: dread - d9/da/d33/f3a zero size 2024-04-26T23:01:24.608 INFO:tasks.workunit.client.0.smithi027.stdout:6/102: rename d1/df/c10 to d1/df/c25 0 2024-04-26T23:01:24.611 INFO:tasks.workunit.client.0.smithi027.stdout:4/153: write da/dd/de/d14/d17/f1d [467162,70485] 0 2024-04-26T23:01:24.612 INFO:tasks.workunit.client.0.smithi027.stdout:8/166: creat d9/de/d22/f3e x:0 0 0 2024-04-26T23:01:24.612 INFO:tasks.workunit.client.0.smithi027.stdout:6/103: symlink d1/l26 0 2024-04-26T23:01:24.613 INFO:tasks.workunit.client.0.smithi027.stdout:4/154: symlink da/dd/de/d14/d17/d24/l27 0 2024-04-26T23:01:24.614 INFO:tasks.workunit.client.0.smithi027.stdout:4/155: chown da/c10 2847373 1 2024-04-26T23:01:24.614 INFO:tasks.workunit.client.0.smithi027.stdout:4/156: chown da/dd/de/d14/d17/f1d 259577 1 2024-04-26T23:01:24.614 INFO:tasks.workunit.client.0.smithi027.stdout:8/167: rmdir d9/d28/d2b/d31 0 2024-04-26T23:01:24.614 INFO:tasks.workunit.client.0.smithi027.stdout:8/168: readlink d9/de/d22/l27 0 2024-04-26T23:01:24.615 INFO:tasks.workunit.client.0.smithi027.stdout:6/104: stat d1/df/lb 0 2024-04-26T23:01:24.617 INFO:tasks.workunit.client.0.smithi027.stdout:8/169: creat d9/de/d20/f3f x:0 0 0 2024-04-26T23:01:24.618 INFO:tasks.workunit.client.0.smithi027.stdout:6/105: creat d1/f27 x:0 0 0 2024-04-26T23:01:24.618 INFO:tasks.workunit.client.0.smithi027.stdout:6/106: dread - d1/de/f1d zero size 2024-04-26T23:01:24.619 INFO:tasks.workunit.client.0.smithi027.stdout:8/170: creat d9/de/d20/f40 x:0 0 0 2024-04-26T23:01:24.622 INFO:tasks.workunit.client.0.smithi027.stdout:6/107: unlink d1/c23 0 2024-04-26T23:01:24.622 INFO:tasks.workunit.client.0.smithi027.stdout:6/108: fsync d1/f2 0 2024-04-26T23:01:24.624 INFO:tasks.workunit.client.0.smithi027.stdout:6/109: unlink d1/df/c22 0 2024-04-26T23:01:24.629 INFO:tasks.workunit.client.0.smithi027.stdout:8/171: creat d9/da/d33/f41 x:0 0 0 2024-04-26T23:01:24.629 INFO:tasks.workunit.client.0.smithi027.stdout:6/110: write d1/f27 [583754,68257] 0 2024-04-26T23:01:24.629 INFO:tasks.workunit.client.0.smithi027.stdout:6/111: creat d1/de/f28 x:0 0 0 2024-04-26T23:01:24.629 INFO:tasks.workunit.client.0.smithi027.stdout:6/112: write d1/f19 [144741,86382] 0 2024-04-26T23:01:24.630 INFO:tasks.workunit.client.0.smithi027.stdout:0/181: dread d0/d4/da/f1f [0,4194304] 0 2024-04-26T23:01:24.636 INFO:tasks.workunit.client.0.smithi027.stdout:2/160: dwrite d2/d5/db/d1b/d1c/f20 [0,4194304] 0 2024-04-26T23:01:24.648 INFO:tasks.workunit.client.0.smithi027.stdout:4/157: dread f2 [0,4194304] 0 2024-04-26T23:01:24.660 INFO:tasks.workunit.client.0.smithi027.stdout:8/172: dread d9/da/f13 [0,4194304] 0 2024-04-26T23:01:24.661 INFO:tasks.workunit.client.0.smithi027.stdout:8/173: fsync d9/da/fd 0 2024-04-26T23:01:24.661 INFO:tasks.workunit.client.0.smithi027.stdout:8/174: stat d9/d28/d2b 0 2024-04-26T23:01:24.661 INFO:tasks.workunit.client.0.smithi027.stdout:8/175: read - d9/da/d33/f41 zero size 2024-04-26T23:01:24.661 INFO:tasks.workunit.client.0.smithi027.stdout:4/158: rename da/dd/de/d14/d21/f25 to da/dd/de/d14/d17/f28 0 2024-04-26T23:01:24.665 INFO:tasks.workunit.client.0.smithi027.stdout:4/159: rename f3 to da/dd/de/d14/d21/f29 0 2024-04-26T23:01:24.665 INFO:tasks.workunit.client.0.smithi027.stdout:8/176: mknod d9/de/c42 0 2024-04-26T23:01:24.665 INFO:tasks.workunit.client.0.smithi027.stdout:8/177: write d9/de/f2c [256915,87513] 0 2024-04-26T23:01:24.666 INFO:tasks.workunit.client.0.smithi027.stdout:4/160: getdents da 0 2024-04-26T23:01:24.668 INFO:tasks.workunit.client.0.smithi027.stdout:1/247: dwrite d1/d5/f50 [0,4194304] 0 2024-04-26T23:01:24.670 INFO:tasks.workunit.client.0.smithi027.stdout:0/182: dread d0/d4/da/d15/f34 [0,4194304] 0 2024-04-26T23:01:24.671 INFO:tasks.workunit.client.0.smithi027.stdout:5/182: dwrite d4/d24/f1b [0,4194304] 0 2024-04-26T23:01:24.673 INFO:tasks.workunit.client.0.smithi027.stdout:4/161: link da/f1f da/dd/de/d14/d17/f2a 0 2024-04-26T23:01:24.673 INFO:tasks.workunit.client.0.smithi027.stdout:4/162: readlink da/dd/de/d14/d17/d24/l27 0 2024-04-26T23:01:24.674 INFO:tasks.workunit.client.0.smithi027.stdout:6/113: dread d1/de/f13 [0,4194304] 0 2024-04-26T23:01:24.676 INFO:tasks.workunit.client.0.smithi027.stdout:0/183: symlink d0/d4/l36 0 2024-04-26T23:01:24.681 INFO:tasks.workunit.client.0.smithi027.stdout:5/183: mkdir d4/d24/d22/d29/d2a/d3b 0 2024-04-26T23:01:24.681 INFO:tasks.workunit.client.0.smithi027.stdout:3/177: dwrite d6/dc/d2f/f3d [0,4194304] 0 2024-04-26T23:01:24.685 INFO:tasks.workunit.client.0.smithi027.stdout:6/114: creat d1/df/f29 x:0 0 0 2024-04-26T23:01:24.685 INFO:tasks.workunit.client.0.smithi027.stdout:6/115: truncate d1/df/fd 1041071 0 2024-04-26T23:01:24.686 INFO:tasks.workunit.client.0.smithi027.stdout:0/184: unlink d0/d4/da/fb 0 2024-04-26T23:01:24.698 INFO:tasks.workunit.client.0.smithi027.stdout:0/185: unlink d0/ld 0 2024-04-26T23:01:24.698 INFO:tasks.workunit.client.0.smithi027.stdout:0/186: fsync d0/d4/f7 0 2024-04-26T23:01:24.698 INFO:tasks.workunit.client.0.smithi027.stdout:6/116: getdents d1 0 2024-04-26T23:01:24.698 INFO:tasks.workunit.client.0.smithi027.stdout:6/117: rename d1/de to d1/de/d2a 22 2024-04-26T23:01:24.699 INFO:tasks.workunit.client.0.smithi027.stdout:4/163: getdents da/dd/de/d14/d21 0 2024-04-26T23:01:24.700 INFO:tasks.workunit.client.0.smithi027.stdout:6/118: chown d1/f19 1 1 2024-04-26T23:01:24.701 INFO:tasks.workunit.client.0.smithi027.stdout:0/187: link d0/d4/da/f11 d0/f37 0 2024-04-26T23:01:24.701 INFO:tasks.workunit.client.0.smithi027.stdout:0/188: chown d0/d4/da/d1a/d2a/c32 461217 1 2024-04-26T23:01:24.704 INFO:tasks.workunit.client.0.smithi027.stdout:4/164: unlink f9 0 2024-04-26T23:01:24.712 INFO:tasks.workunit.client.0.smithi027.stdout:6/119: rename d1/de/f1d to d1/f2b 0 2024-04-26T23:01:24.712 INFO:tasks.workunit.client.0.smithi027.stdout:0/189: getdents d0/d4 0 2024-04-26T23:01:24.712 INFO:tasks.workunit.client.0.smithi027.stdout:0/190: creat d0/d4/da/d2d/f38 x:0 0 0 2024-04-26T23:01:24.716 INFO:tasks.workunit.client.0.smithi027.stdout:8/178: dwrite d9/da/f13 [4194304,4194304] 0 2024-04-26T23:01:24.716 INFO:tasks.workunit.client.0.smithi027.stdout:8/179: rename d9/da to d9/da/d43 22 2024-04-26T23:01:24.716 INFO:tasks.workunit.client.0.smithi027.stdout:8/180: fdatasync d9/da/d33/f3a 0 2024-04-26T23:01:24.716 INFO:tasks.workunit.client.0.smithi027.stdout:3/178: dread d6/f8 [4194304,4194304] 0 2024-04-26T23:01:24.716 INFO:tasks.workunit.client.0.smithi027.stdout:3/179: truncate d6/dc/f21 4417623 0 2024-04-26T23:01:24.716 INFO:tasks.workunit.client.0.smithi027.stdout:0/191: mknod d0/d4/da/d1a/d1e/c39 0 2024-04-26T23:01:24.716 INFO:tasks.workunit.client.0.smithi027.stdout:0/192: getdents d0/d4/da/d15/d26 0 2024-04-26T23:01:24.719 INFO:tasks.workunit.client.0.smithi027.stdout:8/181: getdents d9/de/d20 0 2024-04-26T23:01:24.719 INFO:tasks.workunit.client.0.smithi027.stdout:3/180: mknod d6/dc/d34/d2e/c3f 0 2024-04-26T23:01:24.719 INFO:tasks.workunit.client.0.smithi027.stdout:8/182: chown d9/de/d22/c39 17396 1 2024-04-26T23:01:24.719 INFO:tasks.workunit.client.0.smithi027.stdout:3/181: chown d6/dc/d34/d2e/f1e 104856794 1 2024-04-26T23:01:24.725 INFO:tasks.workunit.client.0.smithi027.stdout:8/183: rename d9/da/cc to d9/de/d2e/c44 0 2024-04-26T23:01:24.725 INFO:tasks.workunit.client.0.smithi027.stdout:6/120: write d1/f6 [1218726,83226] 0 2024-04-26T23:01:24.738 INFO:tasks.workunit.client.0.smithi027.stdout:1/248: dwrite d1/f3 [0,4194304] 0 2024-04-26T23:01:24.741 INFO:tasks.workunit.client.0.smithi027.stdout:8/184: mknod d9/d28/d2b/d3b/c45 0 2024-04-26T23:01:24.746 INFO:tasks.workunit.client.0.smithi027.stdout:6/121: mknod d1/de/c2c 0 2024-04-26T23:01:24.746 INFO:tasks.workunit.client.0.smithi027.stdout:5/184: dwrite d4/d24/f1b [0,4194304] 0 2024-04-26T23:01:24.746 INFO:tasks.workunit.client.0.smithi027.stdout:5/185: write d4/d1a/f1e [1025789,113128] 0 2024-04-26T23:01:24.746 INFO:tasks.workunit.client.0.smithi027.stdout:5/186: fdatasync d4/db/f10 0 2024-04-26T23:01:24.746 INFO:tasks.workunit.client.0.smithi027.stdout:5/187: chown d4/db/f10 1918 1 2024-04-26T23:01:24.746 INFO:tasks.workunit.client.0.smithi027.stdout:5/188: write d4/f9 [4672502,111733] 0 2024-04-26T23:01:24.746 INFO:tasks.workunit.client.0.smithi027.stdout:5/189: chown d4 7 1 2024-04-26T23:01:24.757 INFO:tasks.workunit.client.0.smithi027.stdout:0/193: dwrite d0/f1b [0,4194304] 0 2024-04-26T23:01:24.759 INFO:tasks.workunit.client.0.smithi027.stdout:0/194: mknod d0/d4/da/d15/d26/c3a 0 2024-04-26T23:01:24.761 INFO:tasks.workunit.client.0.smithi027.stdout:4/165: dwrite da/fc [0,4194304] 0 2024-04-26T23:01:24.762 INFO:tasks.workunit.client.0.smithi027.stdout:1/249: write d1/d5/f19 [2708750,67520] 0 2024-04-26T23:01:24.763 INFO:tasks.workunit.client.0.smithi027.stdout:2/161: dwrite d2/d5/fa [0,4194304] 0 2024-04-26T23:01:24.765 INFO:tasks.workunit.client.0.smithi027.stdout:1/250: link d1/d5/d21/f22 d1/d5/d40/f52 0 2024-04-26T23:01:24.775 INFO:tasks.workunit.client.0.smithi027.stdout:7/128: sync 2024-04-26T23:01:24.776 INFO:tasks.workunit.client.0.smithi027.stdout:7/129: creat d0/f2a x:0 0 0 2024-04-26T23:01:24.776 INFO:tasks.workunit.client.0.smithi027.stdout:7/130: chown d0/d28/f18 5801 1 2024-04-26T23:01:24.778 INFO:tasks.workunit.client.0.smithi027.stdout:7/131: unlink d0/d28/d14/c1e 0 2024-04-26T23:01:24.778 INFO:tasks.workunit.client.0.smithi027.stdout:7/132: chown d0/f1f 117078884 1 2024-04-26T23:01:24.779 INFO:tasks.workunit.client.0.smithi027.stdout:7/133: truncate d0/f1f 4141 0 2024-04-26T23:01:24.779 INFO:tasks.workunit.client.0.smithi027.stdout:7/134: dread - d0/f2a zero size 2024-04-26T23:01:24.780 INFO:tasks.workunit.client.0.smithi027.stdout:7/135: mkdir d0/d28/d2b 0 2024-04-26T23:01:24.788 INFO:tasks.workunit.client.0.smithi027.stdout:4/166: dread da/dd/de/d14/d17/d24/f16 [0,4194304] 0 2024-04-26T23:01:24.788 INFO:tasks.workunit.client.0.smithi027.stdout:4/167: chown da/dd/de/d14/d21 49592036 1 2024-04-26T23:01:24.788 INFO:tasks.workunit.client.0.smithi027.stdout:4/168: dread - da/dd/de/d14/d17/d24/f26 zero size 2024-04-26T23:01:24.790 INFO:tasks.workunit.client.0.smithi027.stdout:3/182: dwrite d6/dc/d34/f36 [0,4194304] 0 2024-04-26T23:01:24.791 INFO:tasks.workunit.client.0.smithi027.stdout:4/169: truncate da/dd/de/d14/d17/d24/f16 3285703 0 2024-04-26T23:01:24.795 INFO:tasks.workunit.client.0.smithi027.stdout:3/183: getdents d6/dc/d34 0 2024-04-26T23:01:24.795 INFO:tasks.workunit.client.0.smithi027.stdout:3/184: write d6/dc/d17/f1c [4620416,13314] 0 2024-04-26T23:01:24.797 INFO:tasks.workunit.client.0.smithi027.stdout:4/170: creat da/dd/f2b x:0 0 0 2024-04-26T23:01:24.798 INFO:tasks.workunit.client.0.smithi027.stdout:0/195: write d0/d4/da/d15/f34 [2314258,52716] 0 2024-04-26T23:01:24.802 INFO:tasks.workunit.client.0.smithi027.stdout:0/196: rmdir d0/d4/da/d2d 39 2024-04-26T23:01:24.802 INFO:tasks.workunit.client.0.smithi027.stdout:3/185: mkdir d6/dc/d17/d40 0 2024-04-26T23:01:24.802 INFO:tasks.workunit.client.0.smithi027.stdout:7/136: write d0/f27 [3533063,118382] 0 2024-04-26T23:01:24.803 INFO:tasks.workunit.client.0.smithi027.stdout:2/162: dread d2/d5/db/de/d21/f16 [0,4194304] 0 2024-04-26T23:01:24.804 INFO:tasks.workunit.client.0.smithi027.stdout:2/163: write d2/d5/db/f14 [1317165,4701] 0 2024-04-26T23:01:24.804 INFO:tasks.workunit.client.0.smithi027.stdout:3/186: mknod d6/dc/c41 0 2024-04-26T23:01:24.806 INFO:tasks.workunit.client.0.smithi027.stdout:9/185: sync 2024-04-26T23:01:24.809 INFO:tasks.workunit.client.0.smithi027.stdout:9/186: truncate d4/d17/f29 663989 0 2024-04-26T23:01:24.809 INFO:tasks.workunit.client.0.smithi027.stdout:3/187: mknod d6/c42 0 2024-04-26T23:01:24.809 INFO:tasks.workunit.client.0.smithi027.stdout:3/188: read d6/dc/d34/d2e/f22 [2634992,99205] 0 2024-04-26T23:01:24.809 INFO:tasks.workunit.client.0.smithi027.stdout:3/189: dread - d6/dc/d2f/f39 zero size 2024-04-26T23:01:24.809 INFO:tasks.workunit.client.0.smithi027.stdout:3/190: chown d6/dc/d34/d2e/c3f 4516836 1 2024-04-26T23:01:24.840 INFO:tasks.workunit.client.0.smithi027.stdout:6/122: dwrite d1/df/f29 [0,4194304] 0 2024-04-26T23:01:24.857 INFO:tasks.workunit.client.0.smithi027.stdout:8/185: dwrite d9/de/d22/f3e [0,4194304] 0 2024-04-26T23:01:24.857 INFO:tasks.workunit.client.0.smithi027.stdout:1/251: dwrite d1/d5/d21/f44 [0,4194304] 0 2024-04-26T23:01:24.857 INFO:tasks.workunit.client.0.smithi027.stdout:5/190: dwrite f1 [0,4194304] 0 2024-04-26T23:01:24.863 INFO:tasks.workunit.client.0.smithi027.stdout:5/191: symlink d4/d24/d22/l3c 0 2024-04-26T23:01:24.864 INFO:tasks.workunit.client.0.smithi027.stdout:5/192: chown d4/db/d16/c28 1 1 2024-04-26T23:01:24.864 INFO:tasks.workunit.client.0.smithi027.stdout:0/197: dwrite d0/d4/da/d1a/d1e/f25 [0,4194304] 0 2024-04-26T23:01:24.868 INFO:tasks.workunit.client.0.smithi027.stdout:5/193: stat d4/l12 0 2024-04-26T23:01:24.868 INFO:tasks.workunit.client.0.smithi027.stdout:5/194: chown d4/db/f23 6059672 1 2024-04-26T23:01:24.889 INFO:tasks.workunit.client.0.smithi027.stdout:9/187: dwrite d4/d17/d2e/f34 [0,4194304] 0 2024-04-26T23:01:24.889 INFO:tasks.workunit.client.0.smithi027.stdout:9/188: chown d4/d10/c32 66 1 2024-04-26T23:01:24.891 INFO:tasks.workunit.client.0.smithi027.stdout:9/189: symlink d4/d10/d30/l42 0 2024-04-26T23:01:24.893 INFO:tasks.workunit.client.0.smithi027.stdout:9/190: creat d4/f43 x:0 0 0 2024-04-26T23:01:24.895 INFO:tasks.workunit.client.0.smithi027.stdout:9/191: unlink d4/f35 0 2024-04-26T23:01:24.897 INFO:tasks.workunit.client.0.smithi027.stdout:3/191: dwrite d6/dc/d17/f2b [0,4194304] 0 2024-04-26T23:01:24.901 INFO:tasks.workunit.client.0.smithi027.stdout:4/171: dwrite da/dd/de/d14/d17/f28 [0,4194304] 0 2024-04-26T23:01:24.903 INFO:tasks.workunit.client.0.smithi027.stdout:6/123: dwrite d1/f27 [0,4194304] 0 2024-04-26T23:01:24.904 INFO:tasks.workunit.client.0.smithi027.stdout:1/252: dread d1/f3 [0,4194304] 0 2024-04-26T23:01:24.904 INFO:tasks.workunit.client.0.smithi027.stdout:1/253: chown d1/dc/f37 973372 1 2024-04-26T23:01:24.905 INFO:tasks.workunit.client.0.smithi027.stdout:7/137: dwrite d0/d28/d14/f16 [0,4194304] 0 2024-04-26T23:01:24.909 INFO:tasks.workunit.client.0.smithi027.stdout:5/195: dread d4/db/d16/f1f [0,4194304] 0 2024-04-26T23:01:24.910 INFO:tasks.workunit.client.0.smithi027.stdout:0/198: write d0/d4/f20 [1306566,118186] 0 2024-04-26T23:01:24.910 INFO:tasks.workunit.client.0.smithi027.stdout:3/192: symlink d6/dc/d2f/d37/l43 0 2024-04-26T23:01:24.914 INFO:tasks.workunit.client.0.smithi027.stdout:6/124: symlink d1/l2d 0 2024-04-26T23:01:24.916 INFO:tasks.workunit.client.0.smithi027.stdout:1/254: symlink d1/d5/l53 0 2024-04-26T23:01:24.921 INFO:tasks.workunit.client.0.smithi027.stdout:5/196: link d4/db/f23 d4/d24/d22/d29/d2a/d3b/f3d 0 2024-04-26T23:01:24.923 INFO:tasks.workunit.client.0.smithi027.stdout:6/125: creat d1/df/f2e x:0 0 0 2024-04-26T23:01:24.926 INFO:tasks.workunit.client.0.smithi027.stdout:5/197: creat d4/d24/d22/d29/d2a/f3e x:0 0 0 2024-04-26T23:01:24.928 INFO:tasks.workunit.client.0.smithi027.stdout:5/198: creat d4/d1a/d2b/f3f x:0 0 0 2024-04-26T23:01:24.931 INFO:tasks.workunit.client.0.smithi027.stdout:8/186: dwrite d9/de/d20/f3f [0,4194304] 0 2024-04-26T23:01:24.932 INFO:tasks.workunit.client.0.smithi027.stdout:9/192: dread d4/d10/f19 [4194304,4194304] 0 2024-04-26T23:01:24.933 INFO:tasks.workunit.client.0.smithi027.stdout:9/193: mknod d4/d10/d30/c44 0 2024-04-26T23:01:24.933 INFO:tasks.workunit.client.0.smithi027.stdout:9/194: chown d4/d10/c20 4128778 1 2024-04-26T23:01:24.933 INFO:tasks.workunit.client.0.smithi027.stdout:9/195: chown d4/d10/d30/c44 34605652 1 2024-04-26T23:01:24.935 INFO:tasks.workunit.client.0.smithi027.stdout:9/196: rmdir d4/d10/d2c 39 2024-04-26T23:01:24.937 INFO:tasks.workunit.client.0.smithi027.stdout:9/197: creat d4/d17/f45 x:0 0 0 2024-04-26T23:01:24.937 INFO:tasks.workunit.client.0.smithi027.stdout:9/198: dread - d4/f43 zero size 2024-04-26T23:01:24.937 INFO:tasks.workunit.client.0.smithi027.stdout:9/199: dread - d4/f43 zero size 2024-04-26T23:01:24.937 INFO:tasks.workunit.client.0.smithi027.stdout:9/200: write d4/d17/d2e/f31 [927866,69995] 0 2024-04-26T23:01:24.939 INFO:tasks.workunit.client.0.smithi027.stdout:0/199: write d0/d4/f5 [2090871,52637] 0 2024-04-26T23:01:24.941 INFO:tasks.workunit.client.0.smithi027.stdout:0/200: creat d0/d4/da/d1a/f3b x:0 0 0 2024-04-26T23:01:24.943 INFO:tasks.workunit.client.0.smithi027.stdout:0/201: symlink d0/d4/da/d15/d26/l3c 0 2024-04-26T23:01:24.943 INFO:tasks.workunit.client.0.smithi027.stdout:0/202: write d0/d4/da/d2d/f38 [709966,33762] 0 2024-04-26T23:01:24.945 INFO:tasks.workunit.client.0.smithi027.stdout:0/203: creat d0/d4/f3d x:0 0 0 2024-04-26T23:01:24.947 INFO:tasks.workunit.client.0.smithi027.stdout:3/193: write d6/dc/d34/f36 [231908,127263] 0 2024-04-26T23:01:24.949 INFO:tasks.workunit.client.0.smithi027.stdout:3/194: rename d6/dc/l3e to d6/dc/d34/l44 0 2024-04-26T23:01:24.954 INFO:tasks.workunit.client.0.smithi027.stdout:3/195: write d6/f29 [829529,46938] 0 2024-04-26T23:01:24.955 INFO:tasks.workunit.client.0.smithi027.stdout:9/201: dread d4/d17/d2e/f31 [0,4194304] 0 2024-04-26T23:01:24.955 INFO:tasks.workunit.client.0.smithi027.stdout:9/202: truncate d4/d10/d25/f39 298622 0 2024-04-26T23:01:24.966 INFO:tasks.workunit.client.0.smithi027.stdout:9/203: mkdir d4/d17/d2e/d46 0 2024-04-26T23:01:24.966 INFO:tasks.workunit.client.0.smithi027.stdout:9/204: readlink d4/l18 0 2024-04-26T23:01:24.970 INFO:tasks.workunit.client.0.smithi027.stdout:4/172: dwrite f2 [0,4194304] 0 2024-04-26T23:01:24.980 INFO:tasks.workunit.client.0.smithi027.stdout:7/138: dread d0/d28/f18 [0,4194304] 0 2024-04-26T23:01:24.984 INFO:tasks.workunit.client.0.smithi027.stdout:9/205: dread d4/d10/f28 [0,4194304] 0 2024-04-26T23:01:24.984 INFO:tasks.workunit.client.0.smithi027.stdout:9/206: chown d4/l15 4 1 2024-04-26T23:01:24.989 INFO:tasks.workunit.client.0.smithi027.stdout:9/207: creat d4/d10/d30/f47 x:0 0 0 2024-04-26T23:01:24.989 INFO:tasks.workunit.client.0.smithi027.stdout:9/208: stat d4/l5 0 2024-04-26T23:01:24.995 INFO:tasks.workunit.client.0.smithi027.stdout:5/199: dwrite d4/f15 [0,4194304] 0 2024-04-26T23:01:24.995 INFO:tasks.workunit.client.0.smithi027.stdout:5/200: fsync d4/d24/d22/d29/f33 0 2024-04-26T23:01:24.996 INFO:tasks.workunit.client.0.smithi027.stdout:5/201: mkdir d4/d24/d25/d40 0 2024-04-26T23:01:24.996 INFO:tasks.workunit.client.0.smithi027.stdout:5/202: chown d4/ca 14 1 2024-04-26T23:01:25.002 INFO:tasks.workunit.client.0.smithi027.stdout:0/204: dwrite d0/d4/f3d [0,4194304] 0 2024-04-26T23:01:25.008 INFO:tasks.workunit.client.0.smithi027.stdout:6/126: dwrite d1/f27 [0,4194304] 0 2024-04-26T23:01:25.009 INFO:tasks.workunit.client.0.smithi027.stdout:6/127: symlink d1/de/l2f 0 2024-04-26T23:01:25.034 INFO:tasks.workunit.client.0.smithi027.stdout:4/173: dwrite f2 [0,4194304] 0 2024-04-26T23:01:25.034 INFO:tasks.workunit.client.0.smithi027.stdout:4/174: stat da/dd/de/d14/d17/d24/f26 0 2024-04-26T23:01:25.040 INFO:tasks.workunit.client.0.smithi027.stdout:4/175: mknod da/dd/de/c2c 0 2024-04-26T23:01:25.040 INFO:tasks.workunit.client.0.smithi027.stdout:4/176: write da/dd/de/d14/d17/f20 [802314,11975] 0 2024-04-26T23:01:25.040 INFO:tasks.workunit.client.0.smithi027.stdout:4/177: chown da/c22 198166 1 2024-04-26T23:01:25.053 INFO:tasks.workunit.client.0.smithi027.stdout:7/139: dwrite d0/f13 [4194304,4194304] 0 2024-04-26T23:01:25.057 INFO:tasks.workunit.client.0.smithi027.stdout:9/209: dwrite d4/d10/f28 [0,4194304] 0 2024-04-26T23:01:25.057 INFO:tasks.workunit.client.0.smithi027.stdout:7/140: rename d0/f1f to d0/d28/f2c 0 2024-04-26T23:01:25.057 INFO:tasks.workunit.client.0.smithi027.stdout:7/141: dread - d0/f2a zero size 2024-04-26T23:01:25.059 INFO:tasks.workunit.client.0.smithi027.stdout:9/210: creat d4/d10/d40/f48 x:0 0 0 2024-04-26T23:01:25.064 INFO:tasks.workunit.client.0.smithi027.stdout:6/128: dwrite d1/f2b [0,4194304] 0 2024-04-26T23:01:25.064 INFO:tasks.workunit.client.0.smithi027.stdout:6/129: chown d1/de/f20 71 1 2024-04-26T23:01:25.064 INFO:tasks.workunit.client.0.smithi027.stdout:6/130: chown d1 47 1 2024-04-26T23:01:25.064 INFO:tasks.workunit.client.0.smithi027.stdout:6/131: dread - d1/df/f2e zero size 2024-04-26T23:01:25.064 INFO:tasks.workunit.client.0.smithi027.stdout:3/196: dwrite d6/dc/d34/f36 [0,4194304] 0 2024-04-26T23:01:25.064 INFO:tasks.workunit.client.0.smithi027.stdout:7/142: mkdir d0/d28/d2b/d2d 0 2024-04-26T23:01:25.067 INFO:tasks.workunit.client.0.smithi027.stdout:3/197: truncate d6/dc/d17/f33 2215033 0 2024-04-26T23:01:25.067 INFO:tasks.workunit.client.0.smithi027.stdout:3/198: fsync d6/dc/d17/f20 0 2024-04-26T23:01:25.068 INFO:tasks.workunit.client.0.smithi027.stdout:3/199: truncate d6/f7 4896174 0 2024-04-26T23:01:25.068 INFO:tasks.workunit.client.0.smithi027.stdout:3/200: stat d6/f18 0 2024-04-26T23:01:25.069 INFO:tasks.workunit.client.0.smithi027.stdout:3/201: truncate d6/dc/d2f/f39 986425 0 2024-04-26T23:01:25.069 INFO:tasks.workunit.client.0.smithi027.stdout:7/143: creat d0/f2e x:0 0 0 2024-04-26T23:01:25.071 INFO:tasks.workunit.client.0.smithi027.stdout:3/202: creat d6/dc/d34/d2e/f45 x:0 0 0 2024-04-26T23:01:25.073 INFO:tasks.workunit.client.0.smithi027.stdout:7/144: rmdir d0/d28/d2b/d2d 0 2024-04-26T23:01:25.075 INFO:tasks.workunit.client.0.smithi027.stdout:0/205: dwrite d0/d4/da/d1a/f3b [0,4194304] 0 2024-04-26T23:01:25.076 INFO:tasks.workunit.client.0.smithi027.stdout:3/203: rename d6/dc/d2f/f3d to d6/dc/f46 0 2024-04-26T23:01:25.077 INFO:tasks.workunit.client.0.smithi027.stdout:7/145: mkdir d0/d2f 0 2024-04-26T23:01:25.080 INFO:tasks.workunit.client.0.smithi027.stdout:0/206: rename d0/d4/da/d15/d26/l3c to d0/d4/da/d1a/l3e 0 2024-04-26T23:01:25.096 INFO:tasks.workunit.client.0.smithi027.stdout:7/146: dread d0/f11 [4194304,4194304] 0 2024-04-26T23:01:25.096 INFO:tasks.workunit.client.0.smithi027.stdout:7/147: fdatasync d0/d28/f2c 0 2024-04-26T23:01:25.096 INFO:tasks.workunit.client.0.smithi027.stdout:7/148: readlink d0/l23 0 2024-04-26T23:01:25.096 INFO:tasks.workunit.client.0.smithi027.stdout:7/149: write d0/d28/f2c [273595,23793] 0 2024-04-26T23:01:25.107 INFO:tasks.workunit.client.0.smithi027.stdout:7/150: rmdir d0 39 2024-04-26T23:01:25.107 INFO:tasks.workunit.client.0.smithi027.stdout:2/164: sync 2024-04-26T23:01:25.108 INFO:tasks.workunit.client.0.smithi027.stdout:2/165: write d2/d5/db/f14 [1702477,72018] 0 2024-04-26T23:01:25.108 INFO:tasks.workunit.client.0.smithi027.stdout:2/166: write d2/d5/db/f14 [9575926,82210] 0 2024-04-26T23:01:25.108 INFO:tasks.workunit.client.0.smithi027.stdout:7/151: creat d0/f30 x:0 0 0 2024-04-26T23:01:25.108 INFO:tasks.workunit.client.0.smithi027.stdout:7/152: dread - d0/f30 zero size 2024-04-26T23:01:25.114 INFO:tasks.workunit.client.0.smithi027.stdout:6/132: dwrite d1/f2 [0,4194304] 0 2024-04-26T23:01:25.117 INFO:tasks.workunit.client.0.smithi027.stdout:6/133: mkdir d1/d30 0 2024-04-26T23:01:25.123 INFO:tasks.workunit.client.0.smithi027.stdout:4/178: dwrite da/dd/de/d14/d17/d24/f19 [4194304,4194304] 0 2024-04-26T23:01:25.129 INFO:tasks.workunit.client.0.smithi027.stdout:0/207: dread d0/f13 [0,4194304] 0 2024-04-26T23:01:25.129 INFO:tasks.workunit.client.0.smithi027.stdout:0/208: write d0/d4/da/d1a/d1e/f35 [341579,18668] 0 2024-04-26T23:01:25.129 INFO:tasks.workunit.client.0.smithi027.stdout:3/204: dwrite d6/dc/f11 [0,4194304] 0 2024-04-26T23:01:25.133 INFO:tasks.workunit.client.0.smithi027.stdout:0/209: mkdir d0/df/d3f 0 2024-04-26T23:01:25.133 INFO:tasks.workunit.client.0.smithi027.stdout:0/210: truncate d0/d4/da/f11 1037617 0 2024-04-26T23:01:25.133 INFO:tasks.workunit.client.0.smithi027.stdout:0/211: chown d0/d4/f20 1320829 1 2024-04-26T23:01:25.134 INFO:tasks.workunit.client.0.smithi027.stdout:0/212: getdents d0/d4/da/d1a/d2a 0 2024-04-26T23:01:25.134 INFO:tasks.workunit.client.0.smithi027.stdout:3/205: getdents d6/dc/d34 0 2024-04-26T23:01:25.134 INFO:tasks.workunit.client.0.smithi027.stdout:3/206: getdents d6/dc/d17/d40 0 2024-04-26T23:01:25.134 INFO:tasks.workunit.client.0.smithi027.stdout:3/207: chown d6/dc/d34/d2e/c3f 9 1 2024-04-26T23:01:25.135 INFO:tasks.workunit.client.0.smithi027.stdout:0/213: creat d0/d4/da/d1a/f40 x:0 0 0 2024-04-26T23:01:25.141 INFO:tasks.workunit.client.0.smithi027.stdout:7/153: truncate d0/d28/f2c 150149 0 2024-04-26T23:01:25.142 INFO:tasks.workunit.client.0.smithi027.stdout:9/211: dwrite d4/d10/d40/f48 [0,4194304] 0 2024-04-26T23:01:25.144 INFO:tasks.workunit.client.0.smithi027.stdout:7/154: rename d0/f2e to d0/d2f/f31 0 2024-04-26T23:01:25.144 INFO:tasks.workunit.client.0.smithi027.stdout:6/134: dread d1/f2 [0,4194304] 0 2024-04-26T23:01:25.152 INFO:tasks.workunit.client.0.smithi027.stdout:3/208: write d6/dc/f21 [1447407,119094] 0 2024-04-26T23:01:25.152 INFO:tasks.workunit.client.0.smithi027.stdout:9/212: rename d4/d10/d30/f3b to d4/d10/d40/f49 0 2024-04-26T23:01:25.152 INFO:tasks.workunit.client.0.smithi027.stdout:1/255: sync 2024-04-26T23:01:25.152 INFO:tasks.workunit.client.0.smithi027.stdout:5/203: sync 2024-04-26T23:01:25.152 INFO:tasks.workunit.client.0.smithi027.stdout:8/187: sync 2024-04-26T23:01:25.152 INFO:tasks.workunit.client.0.smithi027.stdout:5/204: stat d4/d24/d22/d29/d2a/d3b/f3d 0 2024-04-26T23:01:25.152 INFO:tasks.workunit.client.0.smithi027.stdout:5/205: chown d4/d24/d22/d29/d2a/d3b 500266490 1 2024-04-26T23:01:25.152 INFO:tasks.workunit.client.0.smithi027.stdout:5/206: readlink d4/d24/d22/d29/l2e 0 2024-04-26T23:01:25.156 INFO:tasks.workunit.client.0.smithi027.stdout:9/213: creat d4/d10/f4a x:0 0 0 2024-04-26T23:01:25.156 INFO:tasks.workunit.client.0.smithi027.stdout:1/256: symlink d1/d5/d40/d35/l54 0 2024-04-26T23:01:25.156 INFO:tasks.workunit.client.0.smithi027.stdout:5/207: mknod d4/d24/c41 0 2024-04-26T23:01:25.157 INFO:tasks.workunit.client.0.smithi027.stdout:6/135: rename d1/f6 to d1/df/f31 0 2024-04-26T23:01:25.157 INFO:tasks.workunit.client.0.smithi027.stdout:6/136: fdatasync d1/de/f28 0 2024-04-26T23:01:25.161 INFO:tasks.workunit.client.0.smithi027.stdout:5/208: creat d4/d24/f42 x:0 0 0 2024-04-26T23:01:25.161 INFO:tasks.workunit.client.0.smithi027.stdout:5/209: write d4/f9 [2109915,107906] 0 2024-04-26T23:01:25.161 INFO:tasks.workunit.client.0.smithi027.stdout:9/214: creat d4/d17/d2e/f4b x:0 0 0 2024-04-26T23:01:25.161 INFO:tasks.workunit.client.0.smithi027.stdout:6/137: mknod d1/c32 0 2024-04-26T23:01:25.163 INFO:tasks.workunit.client.0.smithi027.stdout:4/179: dread da/dd/de/d14/d21/f29 [0,4194304] 0 2024-04-26T23:01:25.163 INFO:tasks.workunit.client.0.smithi027.stdout:4/180: stat da/dd 0 2024-04-26T23:01:25.163 INFO:tasks.workunit.client.0.smithi027.stdout:4/181: read - da/dd/de/d14/d21/f23 zero size 2024-04-26T23:01:25.167 INFO:tasks.workunit.client.0.smithi027.stdout:3/209: rename d6/dc/d2f/d37 to d6/dc/d34/d47 0 2024-04-26T23:01:25.168 INFO:tasks.workunit.client.0.smithi027.stdout:3/210: write d6/dc/d34/d47/f3c [864748,64511] 0 2024-04-26T23:01:25.168 INFO:tasks.workunit.client.0.smithi027.stdout:3/211: write d6/f7 [4843855,61459] 0 2024-04-26T23:01:25.171 INFO:tasks.workunit.client.0.smithi027.stdout:5/210: link d4/db/d16/c26 d4/d24/d22/d29/c43 0 2024-04-26T23:01:25.172 INFO:tasks.workunit.client.0.smithi027.stdout:6/138: creat d1/f33 x:0 0 0 2024-04-26T23:01:25.172 INFO:tasks.workunit.client.0.smithi027.stdout:6/139: stat d1/de/f1f 0 2024-04-26T23:01:25.176 INFO:tasks.workunit.client.0.smithi027.stdout:3/212: link d6/c42 d6/dc/d17/c48 0 2024-04-26T23:01:25.187 INFO:tasks.workunit.client.0.smithi027.stdout:5/211: dread d4/d1a/f20 [0,4194304] 0 2024-04-26T23:01:25.187 INFO:tasks.workunit.client.0.smithi027.stdout:5/212: chown d4/d24/f42 155425201 1 2024-04-26T23:01:25.198 INFO:tasks.workunit.client.0.smithi027.stdout:1/257: dwrite d1/d5/d40/f52 [0,4194304] 0 2024-04-26T23:01:25.198 INFO:tasks.workunit.client.0.smithi027.stdout:0/214: dwrite d0/d4/f20 [0,4194304] 0 2024-04-26T23:01:25.204 INFO:tasks.workunit.client.0.smithi027.stdout:4/182: dread da/dd/de/f1c [0,4194304] 0 2024-04-26T23:01:25.211 INFO:tasks.workunit.client.0.smithi027.stdout:1/258: truncate d1/d5/d40/d2d/f30 3761109 0 2024-04-26T23:01:25.211 INFO:tasks.workunit.client.0.smithi027.stdout:1/259: fsync d1/d2a/f49 0 2024-04-26T23:01:25.214 INFO:tasks.workunit.client.0.smithi027.stdout:1/260: creat d1/f55 x:0 0 0 2024-04-26T23:01:25.216 INFO:tasks.workunit.client.0.smithi027.stdout:7/155: dwrite d0/d2f/f31 [0,4194304] 0 2024-04-26T23:01:25.216 INFO:tasks.workunit.client.0.smithi027.stdout:7/156: truncate d0/f30 216661 0 2024-04-26T23:01:25.216 INFO:tasks.workunit.client.0.smithi027.stdout:7/157: chown d0/d2f 635695229 1 2024-04-26T23:01:25.218 INFO:tasks.workunit.client.0.smithi027.stdout:6/140: write d1/de/f13 [1069036,96370] 0 2024-04-26T23:01:25.219 INFO:tasks.workunit.client.0.smithi027.stdout:1/261: mkdir d1/d26/d36/d56 0 2024-04-26T23:01:25.220 INFO:tasks.workunit.client.0.smithi027.stdout:1/262: read - d1/f55 zero size 2024-04-26T23:01:25.224 INFO:tasks.workunit.client.0.smithi027.stdout:7/158: creat d0/d2f/f32 x:0 0 0 2024-04-26T23:01:25.224 INFO:tasks.workunit.client.0.smithi027.stdout:6/141: getdents d1/de 0 2024-04-26T23:01:25.224 INFO:tasks.workunit.client.0.smithi027.stdout:6/142: stat d1/df/lb 0 2024-04-26T23:01:25.238 INFO:tasks.workunit.client.0.smithi027.stdout:9/215: dwrite d4/d17/d2e/f34 [0,4194304] 0 2024-04-26T23:01:25.241 INFO:tasks.workunit.client.0.smithi027.stdout:8/188: dwrite d9/da/d33/f3a [0,4194304] 0 2024-04-26T23:01:25.249 INFO:tasks.workunit.client.0.smithi027.stdout:9/216: symlink d4/d17/l4c 0 2024-04-26T23:01:25.249 INFO:tasks.workunit.client.0.smithi027.stdout:9/217: readlink d4/d17/l4c 0 2024-04-26T23:01:25.254 INFO:tasks.workunit.client.0.smithi027.stdout:8/189: mkdir d9/d28/d46 0 2024-04-26T23:01:25.254 INFO:tasks.workunit.client.0.smithi027.stdout:8/190: readlink d9/de/d20/l37 0 2024-04-26T23:01:25.255 INFO:tasks.workunit.client.0.smithi027.stdout:1/263: dread d1/d5/d21/f27 [0,4194304] 0 2024-04-26T23:01:25.256 INFO:tasks.workunit.client.0.smithi027.stdout:1/264: chown d1/d26/f29 15 1 2024-04-26T23:01:25.257 INFO:tasks.workunit.client.0.smithi027.stdout:1/265: write d1/d5/f50 [5041187,50421] 0 2024-04-26T23:01:25.257 INFO:tasks.workunit.client.0.smithi027.stdout:1/266: chown d1/d5/d40/d35/l4f 9021664 1 2024-04-26T23:01:25.265 INFO:tasks.workunit.client.0.smithi027.stdout:4/183: dwrite da/dd/de/d14/d17/d24/f16 [0,4194304] 0 2024-04-26T23:01:25.266 INFO:tasks.workunit.client.0.smithi027.stdout:0/215: dwrite d0/d4/da/f1f [0,4194304] 0 2024-04-26T23:01:25.268 INFO:tasks.workunit.client.0.smithi027.stdout:0/216: creat d0/f41 x:0 0 0 2024-04-26T23:01:25.268 INFO:tasks.workunit.client.0.smithi027.stdout:5/213: dwrite d4/d1a/f20 [0,4194304] 0 2024-04-26T23:01:25.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:24 smithi080 ceph-mon[32324]: pgmap v87: 129 pgs: 129 active+clean; 255 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 19 MiB/s wr, 986 op/s 2024-04-26T23:01:25.280 INFO:tasks.workunit.client.0.smithi027.stdout:5/214: symlink d4/db/l44 0 2024-04-26T23:01:25.281 INFO:tasks.workunit.client.0.smithi027.stdout:9/218: dread d4/d17/d2e/f34 [0,4194304] 0 2024-04-26T23:01:25.281 INFO:tasks.workunit.client.0.smithi027.stdout:9/219: write d4/f14 [3302224,7732] 0 2024-04-26T23:01:25.282 INFO:tasks.workunit.client.0.smithi027.stdout:5/215: creat d4/d24/d22/d29/d2a/f45 x:0 0 0 2024-04-26T23:01:25.282 INFO:tasks.workunit.client.0.smithi027.stdout:9/220: write d4/d17/f1d [897237,34706] 0 2024-04-26T23:01:25.282 INFO:tasks.workunit.client.0.smithi027.stdout:5/216: chown d4/d24/d25/d40 229 1 2024-04-26T23:01:25.282 INFO:tasks.workunit.client.0.smithi027.stdout:5/217: mknod d4/d1a/d2b/c46 0 2024-04-26T23:01:25.284 INFO:tasks.workunit.client.0.smithi027.stdout:5/218: write d4/f9 [5348607,59293] 0 2024-04-26T23:01:25.284 INFO:tasks.workunit.client.0.smithi027.stdout:9/221: symlink d4/d10/l4d 0 2024-04-26T23:01:25.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:24 smithi094 ceph-mon[32889]: pgmap v87: 129 pgs: 129 active+clean; 255 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 19 MiB/s wr, 986 op/s 2024-04-26T23:01:25.285 INFO:tasks.workunit.client.0.smithi027.stdout:5/219: rename d4/db/c27 to d4/db/d16/c47 0 2024-04-26T23:01:25.285 INFO:tasks.workunit.client.0.smithi027.stdout:5/220: rename d4/d24/d22 to d4/d24/d22/d29/d2a/d48 22 2024-04-26T23:01:25.285 INFO:tasks.workunit.client.0.smithi027.stdout:5/221: fsync d4/d24/d22/d29/f33 0 2024-04-26T23:01:25.286 INFO:tasks.workunit.client.0.smithi027.stdout:9/222: getdents d4/d10/d40 0 2024-04-26T23:01:25.286 INFO:tasks.workunit.client.0.smithi027.stdout:5/222: rename d4/l12 to d4/d24/d22/d29/l49 0 2024-04-26T23:01:25.287 INFO:tasks.workunit.client.0.smithi027.stdout:4/184: write f4 [3222582,44180] 0 2024-04-26T23:01:25.288 INFO:tasks.workunit.client.0.smithi027.stdout:9/223: rename d4/d10/d30/l42 to d4/d17/l4e 0 2024-04-26T23:01:25.289 INFO:tasks.workunit.client.0.smithi027.stdout:4/185: mknod da/c2d 0 2024-04-26T23:01:25.289 INFO:tasks.workunit.client.0.smithi027.stdout:3/213: dwrite d6/dc/d34/d47/f3c [0,4194304] 0 2024-04-26T23:01:25.291 INFO:tasks.workunit.client.0.smithi027.stdout:9/224: mkdir d4/d10/d2c/d4f 0 2024-04-26T23:01:25.291 INFO:tasks.workunit.client.0.smithi027.stdout:9/225: chown d4/d10/d25/f39 3407 1 2024-04-26T23:01:25.291 INFO:tasks.workunit.client.0.smithi027.stdout:7/159: dwrite d0/d2f/f32 [0,4194304] 0 2024-04-26T23:01:25.291 INFO:tasks.workunit.client.0.smithi027.stdout:7/160: write d0/d28/f2c [362667,40291] 0 2024-04-26T23:01:25.294 INFO:tasks.workunit.client.0.smithi027.stdout:4/186: link da/c2d da/c2e 0 2024-04-26T23:01:25.295 INFO:tasks.workunit.client.0.smithi027.stdout:5/223: getdents d4/db/d16 0 2024-04-26T23:01:25.295 INFO:tasks.workunit.client.0.smithi027.stdout:3/214: creat d6/dc/d34/d2e/f49 x:0 0 0 2024-04-26T23:01:25.295 INFO:tasks.workunit.client.0.smithi027.stdout:3/215: write d6/f2c [264123,16121] 0 2024-04-26T23:01:25.298 INFO:tasks.workunit.client.0.smithi027.stdout:9/226: mknod d4/d10/c50 0 2024-04-26T23:01:25.300 INFO:tasks.workunit.client.0.smithi027.stdout:4/187: unlink da/c2d 0 2024-04-26T23:01:25.301 INFO:tasks.workunit.client.0.smithi027.stdout:5/224: truncate d4/d24/d22/d29/d2a/d3b/f3d 367761 0 2024-04-26T23:01:25.306 INFO:tasks.workunit.client.0.smithi027.stdout:8/191: dwrite d9/da/d33/f41 [0,4194304] 0 2024-04-26T23:01:25.308 INFO:tasks.workunit.client.0.smithi027.stdout:3/216: symlink d6/dc/d2f/l4a 0 2024-04-26T23:01:25.308 INFO:tasks.workunit.client.0.smithi027.stdout:3/217: fsync d6/f18 0 2024-04-26T23:01:25.310 INFO:tasks.workunit.client.0.smithi027.stdout:1/267: write d1/d2a/f47 [1301186,63586] 0 2024-04-26T23:01:25.310 INFO:tasks.workunit.client.0.smithi027.stdout:1/268: stat d1/d5 0 2024-04-26T23:01:25.314 INFO:tasks.workunit.client.0.smithi027.stdout:9/227: rename d4/cf to d4/d17/d2e/d46/c51 0 2024-04-26T23:01:25.314 INFO:tasks.workunit.client.0.smithi027.stdout:4/188: mknod da/dd/de/d14/c2f 0 2024-04-26T23:01:25.318 INFO:tasks.workunit.client.0.smithi027.stdout:5/225: rmdir d4/d24/d22 39 2024-04-26T23:01:25.320 INFO:tasks.workunit.client.0.smithi027.stdout:0/217: dwrite d0/f37 [0,4194304] 0 2024-04-26T23:01:25.320 INFO:tasks.workunit.client.0.smithi027.stdout:0/218: write d0/d4/da/f19 [794639,12057] 0 2024-04-26T23:01:25.321 INFO:tasks.workunit.client.0.smithi027.stdout:1/269: link d1/d5/d40/d2d/l4e d1/d26/d36/l57 0 2024-04-26T23:01:25.322 INFO:tasks.workunit.client.0.smithi027.stdout:7/161: dwrite d0/d28/d14/f29 [0,4194304] 0 2024-04-26T23:01:25.324 INFO:tasks.workunit.client.0.smithi027.stdout:8/192: rename d9/de/c42 to d9/d28/d2b/d3b/c47 0 2024-04-26T23:01:25.326 INFO:tasks.workunit.client.0.smithi027.stdout:2/167: sync 2024-04-26T23:01:25.327 INFO:tasks.workunit.client.0.smithi027.stdout:0/219: mknod d0/d4/da/d1a/c42 0 2024-04-26T23:01:25.327 INFO:tasks.workunit.client.0.smithi027.stdout:0/220: chown d0/f12 55568205 1 2024-04-26T23:01:25.331 INFO:tasks.workunit.client.0.smithi027.stdout:3/218: rename d6/le to d6/dc/d34/d47/l4b 0 2024-04-26T23:01:25.332 INFO:tasks.workunit.client.0.smithi027.stdout:1/270: mknod d1/c58 0 2024-04-26T23:01:25.334 INFO:tasks.workunit.client.0.smithi027.stdout:9/228: rename d4/d17/d2e/f4b to d4/f52 0 2024-04-26T23:01:25.335 INFO:tasks.workunit.client.0.smithi027.stdout:3/219: mkdir d6/dc/d4c 0 2024-04-26T23:01:25.337 INFO:tasks.workunit.client.0.smithi027.stdout:6/143: dwrite d1/de/f1b [0,4194304] 0 2024-04-26T23:01:25.337 INFO:tasks.workunit.client.0.smithi027.stdout:9/229: creat d4/d17/d2e/f53 x:0 0 0 2024-04-26T23:01:25.338 INFO:tasks.workunit.client.0.smithi027.stdout:3/220: creat d6/dc/d17/d40/f4d x:0 0 0 2024-04-26T23:01:25.339 INFO:tasks.workunit.client.0.smithi027.stdout:6/144: link d1/c32 d1/c34 0 2024-04-26T23:01:25.339 INFO:tasks.workunit.client.0.smithi027.stdout:6/145: chown d1/f2b 120943186 1 2024-04-26T23:01:25.339 INFO:tasks.workunit.client.0.smithi027.stdout:1/271: mknod d1/d5/d40/d35/c59 0 2024-04-26T23:01:25.339 INFO:tasks.workunit.client.0.smithi027.stdout:1/272: chown d1/d26 228394 1 2024-04-26T23:01:25.341 INFO:tasks.workunit.client.0.smithi027.stdout:3/221: rename d6/dc/d2f/l4a to d6/dc/d17/l4e 0 2024-04-26T23:01:25.343 INFO:tasks.workunit.client.0.smithi027.stdout:8/193: dread d9/de/f14 [4194304,4194304] 0 2024-04-26T23:01:25.343 INFO:tasks.workunit.client.0.smithi027.stdout:8/194: fdatasync d9/d28/d2b/f3d 0 2024-04-26T23:01:25.343 INFO:tasks.workunit.client.0.smithi027.stdout:8/195: stat d9/de/d2e/c1a 0 2024-04-26T23:01:25.344 INFO:tasks.workunit.client.0.smithi027.stdout:8/196: creat d9/da/d2a/f48 x:0 0 0 2024-04-26T23:01:25.351 INFO:tasks.workunit.client.0.smithi027.stdout:8/197: creat d9/de/d2e/f49 x:0 0 0 2024-04-26T23:01:25.351 INFO:tasks.workunit.client.0.smithi027.stdout:6/146: rmdir d1 39 2024-04-26T23:01:25.354 INFO:tasks.workunit.client.0.smithi027.stdout:8/198: rename d9/de/d22/c25 to d9/de/c4a 0 2024-04-26T23:01:25.356 INFO:tasks.workunit.client.0.smithi027.stdout:8/199: symlink d9/de/d2e/l4b 0 2024-04-26T23:01:25.356 INFO:tasks.workunit.client.0.smithi027.stdout:7/162: dread d0/d28/d14/f29 [0,4194304] 0 2024-04-26T23:01:25.361 INFO:tasks.workunit.client.0.smithi027.stdout:7/163: getdents d0/d28/d14 0 2024-04-26T23:01:25.361 INFO:tasks.workunit.client.0.smithi027.stdout:7/164: write d0/f27 [634201,111185] 0 2024-04-26T23:01:25.364 INFO:tasks.workunit.client.0.smithi027.stdout:4/189: write da/dd/de/d14/d17/d24/f16 [33252,33489] 0 2024-04-26T23:01:25.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:24 smithi027 ceph-mon[28034]: pgmap v87: 129 pgs: 129 active+clean; 255 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 19 MiB/s wr, 986 op/s 2024-04-26T23:01:25.368 INFO:tasks.workunit.client.0.smithi027.stdout:3/222: dread d6/dc/fd [4194304,4194304] 0 2024-04-26T23:01:25.368 INFO:tasks.workunit.client.0.smithi027.stdout:3/223: write d6/dc/d17/d40/f4d [778342,65488] 0 2024-04-26T23:01:25.373 INFO:tasks.workunit.client.0.smithi027.stdout:1/273: dread d1/dc/f11 [0,4194304] 0 2024-04-26T23:01:25.373 INFO:tasks.workunit.client.0.smithi027.stdout:1/274: stat d1/f6 0 2024-04-26T23:01:25.373 INFO:tasks.workunit.client.0.smithi027.stdout:1/275: fdatasync d1/d5/d21/f22 0 2024-04-26T23:01:25.373 INFO:tasks.workunit.client.0.smithi027.stdout:1/276: write d1/dc/f11 [942311,49928] 0 2024-04-26T23:01:25.373 INFO:tasks.workunit.client.0.smithi027.stdout:1/277: write d1/d5/d40/d2d/f30 [4345265,24884] 0 2024-04-26T23:01:25.378 INFO:tasks.workunit.client.0.smithi027.stdout:5/226: dwrite d4/d24/d22/d29/f33 [0,4194304] 0 2024-04-26T23:01:25.395 INFO:tasks.workunit.client.0.smithi027.stdout:2/168: dwrite d2/d5/db/d1b/d1c/d25/d19/f1e [0,4194304] 0 2024-04-26T23:01:25.406 INFO:tasks.workunit.client.0.smithi027.stdout:4/190: write da/dd/de/d14/d17/f28 [1549380,28335] 0 2024-04-26T23:01:25.406 INFO:tasks.workunit.client.0.smithi027.stdout:2/169: write d2/d5/db/d1b/d1c/d25/d19/d23/f2c [4744387,63744] 0 2024-04-26T23:01:25.406 INFO:tasks.workunit.client.0.smithi027.stdout:2/170: chown d2/d5/db/de/d21 4 1 2024-04-26T23:01:25.408 INFO:tasks.workunit.client.0.smithi027.stdout:4/191: creat da/dd/f30 x:0 0 0 2024-04-26T23:01:25.408 INFO:tasks.workunit.client.0.smithi027.stdout:4/192: truncate da/dd/de/d14/d21/f23 784656 0 2024-04-26T23:01:25.409 INFO:tasks.workunit.client.0.smithi027.stdout:4/193: mknod da/c31 0 2024-04-26T23:01:25.412 INFO:tasks.workunit.client.0.smithi027.stdout:1/278: write d1/d26/f29 [127014,26232] 0 2024-04-26T23:01:25.413 INFO:tasks.workunit.client.0.smithi027.stdout:1/279: creat d1/d2a/d3c/f5a x:0 0 0 2024-04-26T23:01:25.420 INFO:tasks.workunit.client.0.smithi027.stdout:5/227: dwrite d4/d24/d22/d29/f36 [0,4194304] 0 2024-04-26T23:01:25.420 INFO:tasks.workunit.client.0.smithi027.stdout:8/200: write d9/de/d20/f3f [1077935,113430] 0 2024-04-26T23:01:25.420 INFO:tasks.workunit.client.0.smithi027.stdout:8/201: fsync d9/da/d33/f41 0 2024-04-26T23:01:25.421 INFO:tasks.workunit.client.0.smithi027.stdout:8/202: chown d9/da/d33 0 1 2024-04-26T23:01:25.422 INFO:tasks.workunit.client.0.smithi027.stdout:5/228: mkdir d4/d1a/d4a 0 2024-04-26T23:01:25.427 INFO:tasks.workunit.client.0.smithi027.stdout:8/203: creat d9/d28/d2b/d3b/f4c x:0 0 0 2024-04-26T23:01:25.427 INFO:tasks.workunit.client.0.smithi027.stdout:8/204: fdatasync d9/da/fd 0 2024-04-26T23:01:25.427 INFO:tasks.workunit.client.0.smithi027.stdout:4/194: dread da/f1f [4194304,4194304] 0 2024-04-26T23:01:25.427 INFO:tasks.workunit.client.0.smithi027.stdout:3/224: dwrite d6/dc/f25 [4194304,4194304] 0 2024-04-26T23:01:25.429 INFO:tasks.workunit.client.0.smithi027.stdout:5/229: truncate d4/db/d16/f38 3280372 0 2024-04-26T23:01:25.437 INFO:tasks.workunit.client.0.smithi027.stdout:4/195: symlink da/dd/de/l32 0 2024-04-26T23:01:25.439 INFO:tasks.workunit.client.0.smithi027.stdout:3/225: creat d6/f4f x:0 0 0 2024-04-26T23:01:25.440 INFO:tasks.workunit.client.0.smithi027.stdout:9/230: dwrite d4/f14 [0,4194304] 0 2024-04-26T23:01:25.441 INFO:tasks.workunit.client.0.smithi027.stdout:5/230: mkdir d4/db/d16/d4b 0 2024-04-26T23:01:25.444 INFO:tasks.workunit.client.0.smithi027.stdout:4/196: creat da/dd/de/d14/d17/f33 x:0 0 0 2024-04-26T23:01:25.446 INFO:tasks.workunit.client.0.smithi027.stdout:3/226: truncate d6/f7 535704 0 2024-04-26T23:01:25.448 INFO:tasks.workunit.client.0.smithi027.stdout:2/171: dread d2/d5/db/f14 [4194304,4194304] 0 2024-04-26T23:01:25.448 INFO:tasks.workunit.client.0.smithi027.stdout:7/165: dwrite d0/d28/f2c [0,4194304] 0 2024-04-26T23:01:25.452 INFO:tasks.workunit.client.0.smithi027.stdout:8/205: dwrite f2 [0,4194304] 0 2024-04-26T23:01:25.455 INFO:tasks.workunit.client.0.smithi027.stdout:5/231: rename d4/d1a/d2b/f3f to d4/d24/f4c 0 2024-04-26T23:01:25.457 INFO:tasks.workunit.client.0.smithi027.stdout:4/197: unlink da/c31 0 2024-04-26T23:01:25.460 INFO:tasks.workunit.client.0.smithi027.stdout:2/172: creat d2/d5/db/de/f30 x:0 0 0 2024-04-26T23:01:25.460 INFO:tasks.workunit.client.0.smithi027.stdout:2/173: dread - d2/d5/db/de/f30 zero size 2024-04-26T23:01:25.461 INFO:tasks.workunit.client.0.smithi027.stdout:3/227: rmdir d6 39 2024-04-26T23:01:25.463 INFO:tasks.workunit.client.0.smithi027.stdout:7/166: truncate d0/d28/f18 369391 0 2024-04-26T23:01:25.463 INFO:tasks.workunit.client.0.smithi027.stdout:8/206: rename d9/de/d20/d35/f3c to d9/de/d22/f4d 0 2024-04-26T23:01:25.465 INFO:tasks.workunit.client.0.smithi027.stdout:5/232: symlink d4/d24/d25/d40/l4d 0 2024-04-26T23:01:25.465 INFO:tasks.workunit.client.0.smithi027.stdout:5/233: chown c0 0 1 2024-04-26T23:01:25.465 INFO:tasks.workunit.client.0.smithi027.stdout:5/234: chown d4/db/cd 113161 1 2024-04-26T23:01:25.465 INFO:tasks.workunit.client.0.smithi027.stdout:4/198: fsync da/fc 0 2024-04-26T23:01:25.467 INFO:tasks.workunit.client.0.smithi027.stdout:8/207: mknod d9/de/d2e/c4e 0 2024-04-26T23:01:25.469 INFO:tasks.workunit.client.0.smithi027.stdout:4/199: symlink da/dd/de/d14/l34 0 2024-04-26T23:01:25.469 INFO:tasks.workunit.client.0.smithi027.stdout:4/200: dread - da/dd/f2b zero size 2024-04-26T23:01:25.469 INFO:tasks.workunit.client.0.smithi027.stdout:8/208: mkdir d9/de/d20/d4f 0 2024-04-26T23:01:25.470 INFO:tasks.workunit.client.0.smithi027.stdout:8/209: write d9/de/d22/f4d [160984,100842] 0 2024-04-26T23:01:25.482 INFO:tasks.workunit.client.0.smithi027.stdout:2/174: dread d2/d5/db/d1b/d1c/d25/f27 [0,4194304] 0 2024-04-26T23:01:25.482 INFO:tasks.workunit.client.0.smithi027.stdout:2/175: chown d2/d5/db/f1d 18279940 1 2024-04-26T23:01:25.482 INFO:tasks.workunit.client.0.smithi027.stdout:2/176: fsync d2/d5/db/d1b/d1c/f20 0 2024-04-26T23:01:25.482 INFO:tasks.workunit.client.0.smithi027.stdout:2/177: chown d2 396 1 2024-04-26T23:01:25.483 INFO:tasks.workunit.client.0.smithi027.stdout:2/178: fdatasync d2/d5/db/de/d21/d26/f2b 0 2024-04-26T23:01:25.483 INFO:tasks.workunit.client.0.smithi027.stdout:2/179: fdatasync d2/d5/db/de/d21/f16 0 2024-04-26T23:01:25.483 INFO:tasks.workunit.client.0.smithi027.stdout:2/180: readlink d2/d5/db/de/d21/l18 0 2024-04-26T23:01:25.485 INFO:tasks.workunit.client.0.smithi027.stdout:2/181: creat d2/d5/db/d1b/d1c/d25/d19/d23/f31 x:0 0 0 2024-04-26T23:01:25.486 INFO:tasks.workunit.client.0.smithi027.stdout:2/182: rmdir d2/d5/db/de/d21/d26 39 2024-04-26T23:01:25.492 INFO:tasks.workunit.client.0.smithi027.stdout:8/210: dread d9/de/f14 [0,4194304] 0 2024-04-26T23:01:25.493 INFO:tasks.workunit.client.0.smithi027.stdout:8/211: mkdir d9/da/d50 0 2024-04-26T23:01:25.499 INFO:tasks.workunit.client.0.smithi027.stdout:8/212: symlink d9/de/d22/l51 0 2024-04-26T23:01:25.505 INFO:tasks.workunit.client.0.smithi027.stdout:3/228: dwrite d6/dc/d34/d2e/f1e [0,4194304] 0 2024-04-26T23:01:25.507 INFO:tasks.workunit.client.0.smithi027.stdout:3/229: mkdir d6/dc/d34/d2e/d50 0 2024-04-26T23:01:25.509 INFO:tasks.workunit.client.0.smithi027.stdout:3/230: truncate d6/f18 85049 0 2024-04-26T23:01:25.516 INFO:tasks.workunit.client.0.smithi027.stdout:2/183: dread d2/d5/db/d1b/d1c/d25/f27 [0,4194304] 0 2024-04-26T23:01:25.516 INFO:tasks.workunit.client.0.smithi027.stdout:7/167: dwrite d0/f11 [0,4194304] 0 2024-04-26T23:01:25.516 INFO:tasks.workunit.client.0.smithi027.stdout:7/168: dread - d0/f2a zero size 2024-04-26T23:01:25.516 INFO:tasks.workunit.client.0.smithi027.stdout:7/169: fsync d0/d2f/f32 0 2024-04-26T23:01:25.518 INFO:tasks.workunit.client.0.smithi027.stdout:3/231: rename d6/dc/c41 to d6/c51 0 2024-04-26T23:01:25.518 INFO:tasks.workunit.client.0.smithi027.stdout:3/232: write d6/dc/d34/d2e/f22 [989934,4312] 0 2024-04-26T23:01:25.518 INFO:tasks.workunit.client.0.smithi027.stdout:3/233: truncate d6/dc/d17/f33 2863087 0 2024-04-26T23:01:25.519 INFO:tasks.workunit.client.0.smithi027.stdout:2/184: symlink d2/d5/db/d1b/l32 0 2024-04-26T23:01:25.520 INFO:tasks.workunit.client.0.smithi027.stdout:2/185: read - d2/d5/db/de/d21/d26/f2d zero size 2024-04-26T23:01:25.520 INFO:tasks.workunit.client.0.smithi027.stdout:2/186: fdatasync d2/d5/db/f1a 0 2024-04-26T23:01:25.520 INFO:tasks.workunit.client.0.smithi027.stdout:3/234: mkdir d6/dc/d17/d52 0 2024-04-26T23:01:25.525 INFO:tasks.workunit.client.0.smithi027.stdout:2/187: rename d2/d5/db/d1b/l32 to d2/d5/l33 0 2024-04-26T23:01:25.528 INFO:tasks.workunit.client.0.smithi027.stdout:2/188: creat d2/d5/f34 x:0 0 0 2024-04-26T23:01:25.528 INFO:tasks.workunit.client.0.smithi027.stdout:2/189: fdatasync d2/d5/db/d1b/d1c/d25/d19/f1e 0 2024-04-26T23:01:25.545 INFO:tasks.workunit.client.0.smithi027.stdout:4/201: dwrite da/dd/de/f1c [0,4194304] 0 2024-04-26T23:01:25.546 INFO:tasks.workunit.client.0.smithi027.stdout:8/213: write d9/da/d33/f3a [1785070,58846] 0 2024-04-26T23:01:25.547 INFO:tasks.workunit.client.0.smithi027.stdout:5/235: dwrite f1 [0,4194304] 0 2024-04-26T23:01:25.547 INFO:tasks.workunit.client.0.smithi027.stdout:5/236: write d4/d24/d22/d29/d2a/d3b/f3d [943317,18591] 0 2024-04-26T23:01:25.551 INFO:tasks.workunit.client.0.smithi027.stdout:5/237: creat d4/f4e x:0 0 0 2024-04-26T23:01:25.553 INFO:tasks.workunit.client.0.smithi027.stdout:5/238: mknod d4/d24/d22/d29/d2a/c4f 0 2024-04-26T23:01:25.553 INFO:tasks.workunit.client.0.smithi027.stdout:5/239: write d4/d24/f42 [338723,39755] 0 2024-04-26T23:01:25.566 INFO:tasks.workunit.client.0.smithi027.stdout:3/235: dwrite d6/dc/d17/f1c [0,4194304] 0 2024-04-26T23:01:25.568 INFO:tasks.workunit.client.0.smithi027.stdout:3/236: link d6/dc/d34/d2e/c24 d6/dc/d34/d2e/d50/c53 0 2024-04-26T23:01:25.573 INFO:tasks.workunit.client.0.smithi027.stdout:5/240: read d4/db/d16/f1f [421025,86751] 0 2024-04-26T23:01:25.586 INFO:tasks.workunit.client.0.smithi027.stdout:2/190: dwrite d2/d5/f34 [0,4194304] 0 2024-04-26T23:01:25.601 INFO:tasks.workunit.client.0.smithi027.stdout:4/202: dwrite da/dd/de/f1c [0,4194304] 0 2024-04-26T23:01:25.602 INFO:tasks.workunit.client.0.smithi027.stdout:4/203: getdents da/dd 0 2024-04-26T23:01:25.603 INFO:tasks.workunit.client.0.smithi027.stdout:4/204: creat da/dd/de/d14/d17/d24/f35 x:0 0 0 2024-04-26T23:01:25.604 INFO:tasks.workunit.client.0.smithi027.stdout:8/214: dwrite d9/de/f2c [0,4194304] 0 2024-04-26T23:01:25.605 INFO:tasks.workunit.client.0.smithi027.stdout:4/205: symlink da/dd/de/d14/d17/d24/l36 0 2024-04-26T23:01:25.609 INFO:tasks.workunit.client.0.smithi027.stdout:8/215: rename d9/da/d50 to d9/d28/d2b/d52 0 2024-04-26T23:01:25.609 INFO:tasks.workunit.client.0.smithi027.stdout:8/216: dread - d9/de/d20/f29 zero size 2024-04-26T23:01:25.609 INFO:tasks.workunit.client.0.smithi027.stdout:8/217: fdatasync d9/da/d2a/f48 0 2024-04-26T23:01:25.611 INFO:tasks.workunit.client.0.smithi027.stdout:4/206: mkdir da/dd/de/d14/d17/d24/d37 0 2024-04-26T23:01:25.611 INFO:tasks.workunit.client.0.smithi027.stdout:4/207: readlink da/dd/de/d14/l34 0 2024-04-26T23:01:25.611 INFO:tasks.workunit.client.0.smithi027.stdout:4/208: readlink da/dd/de/l1b 0 2024-04-26T23:01:25.611 INFO:tasks.workunit.client.0.smithi027.stdout:4/209: chown da/dd/de/f15 468107524 1 2024-04-26T23:01:25.611 INFO:tasks.workunit.client.0.smithi027.stdout:4/210: dread - da/dd/de/d14/d17/f33 zero size 2024-04-26T23:01:25.616 INFO:tasks.workunit.client.0.smithi027.stdout:4/211: rmdir da/dd/de/d14/d21 39 2024-04-26T23:01:25.616 INFO:tasks.workunit.client.0.smithi027.stdout:4/212: write f4 [3034755,40773] 0 2024-04-26T23:01:25.616 INFO:tasks.workunit.client.0.smithi027.stdout:4/213: fsync da/dd/de/d14/d17/d24/f26 0 2024-04-26T23:01:25.619 INFO:tasks.workunit.client.0.smithi027.stdout:4/214: rename da/dd/de/d14/d21/f29 to da/f38 0 2024-04-26T23:01:25.620 INFO:tasks.workunit.client.0.smithi027.stdout:2/191: write d2/d5/db/d1b/d1c/d25/d19/f1e [6320297,52091] 0 2024-04-26T23:01:25.620 INFO:tasks.workunit.client.0.smithi027.stdout:4/215: truncate da/f38 215188 0 2024-04-26T23:01:25.624 INFO:tasks.workunit.client.0.smithi027.stdout:5/241: write d4/f15 [3102347,28964] 0 2024-04-26T23:01:25.624 INFO:tasks.workunit.client.0.smithi027.stdout:2/192: rename d2/d5/db/l10 to d2/l35 0 2024-04-26T23:01:25.624 INFO:tasks.workunit.client.0.smithi027.stdout:2/193: chown d2/d5/fa 0 1 2024-04-26T23:01:25.625 INFO:tasks.workunit.client.0.smithi027.stdout:3/237: dwrite d6/dc/f10 [0,4194304] 0 2024-04-26T23:01:25.625 INFO:tasks.workunit.client.0.smithi027.stdout:8/218: dread d9/de/d22/f4d [0,4194304] 0 2024-04-26T23:01:25.627 INFO:tasks.workunit.client.0.smithi027.stdout:5/242: unlink d4/d24/d22/l3c 0 2024-04-26T23:01:25.630 INFO:tasks.workunit.client.0.smithi027.stdout:2/194: unlink d2/d5/db/d1b/d1c/d25/d19/f1e 0 2024-04-26T23:01:25.631 INFO:tasks.workunit.client.0.smithi027.stdout:3/238: mknod d6/dc/d17/d40/c54 0 2024-04-26T23:01:25.635 INFO:tasks.workunit.client.0.smithi027.stdout:8/219: creat d9/de/d2e/f53 x:0 0 0 2024-04-26T23:01:25.639 INFO:tasks.workunit.client.0.smithi027.stdout:2/195: mkdir d2/d5/db/d1b/d1c/d25/d19/d36 0 2024-04-26T23:01:25.639 INFO:tasks.workunit.client.0.smithi027.stdout:8/220: rmdir d9/da/d33 39 2024-04-26T23:01:25.643 INFO:tasks.workunit.client.0.smithi027.stdout:2/196: write d2/d5/db/f1d [4671500,110927] 0 2024-04-26T23:01:25.643 INFO:tasks.workunit.client.0.smithi027.stdout:4/216: dread f4 [4194304,4194304] 0 2024-04-26T23:01:25.643 INFO:tasks.workunit.client.0.smithi027.stdout:8/221: mknod d9/d28/d2b/d3b/c54 0 2024-04-26T23:01:25.643 INFO:tasks.workunit.client.0.smithi027.stdout:8/222: dread - d9/da/d2a/f48 zero size 2024-04-26T23:01:25.645 INFO:tasks.workunit.client.0.smithi027.stdout:4/217: mknod da/dd/de/d14/d17/d24/d37/c39 0 2024-04-26T23:01:25.646 INFO:tasks.workunit.client.0.smithi027.stdout:8/223: getdents d9/da/d2a 0 2024-04-26T23:01:25.647 INFO:tasks.workunit.client.0.smithi027.stdout:2/197: creat d2/d5/db/d1b/d1c/f37 x:0 0 0 2024-04-26T23:01:25.648 INFO:tasks.workunit.client.0.smithi027.stdout:4/218: creat da/dd/de/d14/d17/d24/f3a x:0 0 0 2024-04-26T23:01:25.650 INFO:tasks.workunit.client.0.smithi027.stdout:8/224: mknod d9/de/d22/c55 0 2024-04-26T23:01:25.650 INFO:tasks.workunit.client.0.smithi027.stdout:3/239: write d6/dc/d17/f1c [928699,60771] 0 2024-04-26T23:01:25.651 INFO:tasks.workunit.client.0.smithi027.stdout:2/198: creat d2/d5/db/d1b/f38 x:0 0 0 2024-04-26T23:01:25.651 INFO:tasks.workunit.client.0.smithi027.stdout:2/199: truncate d2/d5/db/d1b/d1c/d25/f27 4552089 0 2024-04-26T23:01:25.652 INFO:tasks.workunit.client.0.smithi027.stdout:4/219: creat da/dd/f3b x:0 0 0 2024-04-26T23:01:25.652 INFO:tasks.workunit.client.0.smithi027.stdout:4/220: chown c6 7265111 1 2024-04-26T23:01:25.653 INFO:tasks.workunit.client.0.smithi027.stdout:8/225: creat d9/de/d20/d35/f56 x:0 0 0 2024-04-26T23:01:25.656 INFO:tasks.workunit.client.0.smithi027.stdout:2/200: mkdir d2/d5/db/d1b/d1c/d25/d19/d36/d39 0 2024-04-26T23:01:25.656 INFO:tasks.workunit.client.0.smithi027.stdout:4/221: creat da/dd/de/f3c x:0 0 0 2024-04-26T23:01:25.656 INFO:tasks.workunit.client.0.smithi027.stdout:4/222: readlink da/dd/de/d14/d17/d24/l27 0 2024-04-26T23:01:25.672 INFO:tasks.workunit.client.0.smithi027.stdout:4/223: rename da/c10 to da/dd/de/d14/c3d 0 2024-04-26T23:01:25.673 INFO:tasks.workunit.client.0.smithi027.stdout:4/224: creat da/dd/de/d14/d17/d24/f3e x:0 0 0 2024-04-26T23:01:25.675 INFO:tasks.workunit.client.0.smithi027.stdout:8/226: dread d9/de/f14 [0,4194304] 0 2024-04-26T23:01:25.679 INFO:tasks.workunit.client.0.smithi027.stdout:8/227: link d9/de/d20/f3f d9/d28/f57 0 2024-04-26T23:01:25.679 INFO:tasks.workunit.client.0.smithi027.stdout:8/228: stat d9/d28/d2b/f3d 0 2024-04-26T23:01:25.679 INFO:tasks.workunit.client.0.smithi027.stdout:8/229: chown d9/f16 600326 1 2024-04-26T23:01:25.680 INFO:tasks.workunit.client.0.smithi027.stdout:8/230: symlink d9/de/d20/d35/l58 0 2024-04-26T23:01:25.681 INFO:tasks.workunit.client.0.smithi027.stdout:8/231: readlink d9/de/d2e/l4b 0 2024-04-26T23:01:25.681 INFO:tasks.workunit.client.0.smithi027.stdout:8/232: truncate d9/d28/d2b/d3b/f4c 226467 0 2024-04-26T23:01:25.681 INFO:tasks.workunit.client.0.smithi027.stdout:8/233: write d9/da/d33/f41 [4281806,121100] 0 2024-04-26T23:01:25.681 INFO:tasks.workunit.client.0.smithi027.stdout:8/234: stat d9/de/d20/f3f 0 2024-04-26T23:01:25.687 INFO:tasks.workunit.client.0.smithi027.stdout:3/240: dwrite d6/dc/d34/d2e/f31 [0,4194304] 0 2024-04-26T23:01:25.702 INFO:tasks.workunit.client.0.smithi027.stdout:2/201: dwrite d2/d5/db/d1b/d1c/f37 [0,4194304] 0 2024-04-26T23:01:25.702 INFO:tasks.workunit.client.0.smithi027.stdout:4/225: write f2 [3391493,11494] 0 2024-04-26T23:01:25.702 INFO:tasks.workunit.client.0.smithi027.stdout:4/226: getdents da/dd/de/d14/d17 0 2024-04-26T23:01:25.703 INFO:tasks.workunit.client.0.smithi027.stdout:4/227: fsync da/dd/de/d14/d17/f33 0 2024-04-26T23:01:25.703 INFO:tasks.workunit.client.0.smithi027.stdout:4/228: dread - da/dd/de/f3c zero size 2024-04-26T23:01:25.731 INFO:tasks.workunit.client.0.smithi027.stdout:4/229: dread f2 [0,4194304] 0 2024-04-26T23:01:25.734 INFO:tasks.workunit.client.0.smithi027.stdout:4/230: rename da/fc to da/dd/de/f3f 0 2024-04-26T23:01:25.734 INFO:tasks.workunit.client.0.smithi027.stdout:4/231: chown da/dd/de/c2c 503408529 1 2024-04-26T23:01:25.741 INFO:tasks.workunit.client.0.smithi027.stdout:4/232: mkdir da/d40 0 2024-04-26T23:01:25.746 INFO:tasks.workunit.client.0.smithi027.stdout:4/233: write da/dd/de/d14/d21/f23 [950338,118341] 0 2024-04-26T23:01:25.749 INFO:tasks.workunit.client.0.smithi027.stdout:8/235: dwrite d9/de/d20/d35/f56 [0,4194304] 0 2024-04-26T23:01:25.749 INFO:tasks.workunit.client.0.smithi027.stdout:3/241: dwrite d6/dc/d17/d40/f4d [0,4194304] 0 2024-04-26T23:01:25.750 INFO:tasks.workunit.client.0.smithi027.stdout:2/202: dwrite d2/d5/db/f1d [4194304,4194304] 0 2024-04-26T23:01:25.752 INFO:tasks.workunit.client.0.smithi027.stdout:8/236: creat d9/d28/f59 x:0 0 0 2024-04-26T23:01:25.753 INFO:tasks.workunit.client.0.smithi027.stdout:3/242: truncate d6/dc/d34/d47/f3c 1410150 0 2024-04-26T23:01:25.754 INFO:tasks.workunit.client.0.smithi027.stdout:3/243: creat d6/dc/f55 x:0 0 0 2024-04-26T23:01:25.771 INFO:tasks.workunit.client.0.smithi027.stdout:2/203: write d2/d5/db/d1b/d1c/d25/f27 [511402,90607] 0 2024-04-26T23:01:25.771 INFO:tasks.workunit.client.0.smithi027.stdout:2/204: write d2/d5/db/f1d [3524577,116947] 0 2024-04-26T23:01:25.791 INFO:tasks.workunit.client.0.smithi027.stdout:8/237: dwrite d9/de/f18 [0,4194304] 0 2024-04-26T23:01:25.792 INFO:tasks.workunit.client.0.smithi027.stdout:8/238: symlink d9/d28/d2b/d52/l5a 0 2024-04-26T23:01:25.792 INFO:tasks.workunit.client.0.smithi027.stdout:8/239: dread - d9/de/d2e/f53 zero size 2024-04-26T23:01:25.794 INFO:tasks.workunit.client.0.smithi027.stdout:8/240: getdents d9/d28/d2b/d3b 0 2024-04-26T23:01:25.806 INFO:tasks.workunit.client.0.smithi027.stdout:3/244: dwrite d6/f8 [8388608,4194304] 0 2024-04-26T23:01:25.806 INFO:tasks.workunit.client.0.smithi027.stdout:3/245: chown d6/dc/d34/d2e/d50 20170 1 2024-04-26T23:01:25.806 INFO:tasks.workunit.client.0.smithi027.stdout:4/234: dwrite da/dd/de/d14/d17/f2a [0,4194304] 0 2024-04-26T23:01:25.806 INFO:tasks.workunit.client.0.smithi027.stdout:4/235: dread - da/dd/de/d14/d17/d24/f26 zero size 2024-04-26T23:01:25.807 INFO:tasks.workunit.client.0.smithi027.stdout:3/246: mknod d6/dc/d2f/c56 0 2024-04-26T23:01:25.808 INFO:tasks.workunit.client.0.smithi027.stdout:4/236: mkdir da/dd/de/d14/d21/d41 0 2024-04-26T23:01:25.808 INFO:tasks.workunit.client.0.smithi027.stdout:4/237: chown da 504136161 1 2024-04-26T23:01:25.810 INFO:tasks.workunit.client.0.smithi027.stdout:3/247: creat d6/dc/d17/d40/f57 x:0 0 0 2024-04-26T23:01:25.811 INFO:tasks.workunit.client.0.smithi027.stdout:6/147: sync 2024-04-26T23:01:25.812 INFO:tasks.workunit.client.0.smithi027.stdout:4/238: write da/dd/de/f15 [430767,99571] 0 2024-04-26T23:01:25.812 INFO:tasks.workunit.client.0.smithi027.stdout:3/248: creat d6/dc/d34/f58 x:0 0 0 2024-04-26T23:01:25.812 INFO:tasks.workunit.client.0.smithi027.stdout:0/221: sync 2024-04-26T23:01:25.813 INFO:tasks.workunit.client.0.smithi027.stdout:3/249: dread - d6/f4f zero size 2024-04-26T23:01:25.813 INFO:tasks.workunit.client.0.smithi027.stdout:1/280: sync 2024-04-26T23:01:25.813 INFO:tasks.workunit.client.0.smithi027.stdout:0/222: write d0/d4/da/d1a/f40 [26460,98505] 0 2024-04-26T23:01:25.813 INFO:tasks.workunit.client.0.smithi027.stdout:0/223: write d0/d4/f24 [603518,89189] 0 2024-04-26T23:01:25.813 INFO:tasks.workunit.client.0.smithi027.stdout:0/224: dread - d0/f41 zero size 2024-04-26T23:01:25.813 INFO:tasks.workunit.client.0.smithi027.stdout:4/239: symlink da/dd/de/d14/d21/d41/l42 0 2024-04-26T23:01:25.813 INFO:tasks.workunit.client.0.smithi027.stdout:4/240: chown da/dd/f30 2929960 1 2024-04-26T23:01:25.816 INFO:tasks.workunit.client.0.smithi027.stdout:0/225: rename d0/d4/f18 to d0/d4/da/d1a/d2a/f43 0 2024-04-26T23:01:25.816 INFO:tasks.workunit.client.0.smithi027.stdout:4/241: creat da/dd/de/d14/f43 x:0 0 0 2024-04-26T23:01:25.817 INFO:tasks.workunit.client.0.smithi027.stdout:0/226: creat d0/d4/da/d15/f44 x:0 0 0 2024-04-26T23:01:25.821 INFO:tasks.workunit.client.0.smithi027.stdout:8/241: read d9/de/d20/d35/f56 [1605792,123933] 0 2024-04-26T23:01:25.823 INFO:tasks.workunit.client.0.smithi027.stdout:8/242: mknod d9/da/d33/c5b 0 2024-04-26T23:01:25.825 INFO:tasks.workunit.client.0.smithi027.stdout:8/243: symlink d9/de/d20/l5c 0 2024-04-26T23:01:25.826 INFO:tasks.workunit.client.0.smithi027.stdout:8/244: mkdir d9/de/d20/d5d 0 2024-04-26T23:01:25.826 INFO:tasks.workunit.client.0.smithi027.stdout:8/245: truncate f3 779584 0 2024-04-26T23:01:25.838 INFO:tasks.workunit.client.0.smithi027.stdout:3/250: dread d6/dc/f11 [0,4194304] 0 2024-04-26T23:01:25.840 INFO:tasks.workunit.client.0.smithi027.stdout:3/251: mkdir d6/dc/d17/d59 0 2024-04-26T23:01:25.841 INFO:tasks.workunit.client.0.smithi027.stdout:4/242: dread da/dd/de/f1c [0,4194304] 0 2024-04-26T23:01:25.841 INFO:tasks.workunit.client.0.smithi027.stdout:4/243: write da/dd/de/d14/d17/f28 [2643425,93620] 0 2024-04-26T23:01:25.842 INFO:tasks.workunit.client.0.smithi027.stdout:3/252: unlink d6/dc/d17/f20 0 2024-04-26T23:01:25.842 INFO:tasks.workunit.client.0.smithi027.stdout:3/253: fsync d6/dc/f21 0 2024-04-26T23:01:25.845 INFO:tasks.workunit.client.0.smithi027.stdout:4/244: rename f4 to da/dd/de/d14/f44 0 2024-04-26T23:01:25.846 INFO:tasks.workunit.client.0.smithi027.stdout:3/254: unlink d6/dc/d34/d2e/f1e 0 2024-04-26T23:01:25.846 INFO:tasks.workunit.client.0.smithi027.stdout:3/255: readlink d6/l38 0 2024-04-26T23:01:25.846 INFO:tasks.workunit.client.0.smithi027.stdout:3/256: fsync d6/dc/f11 0 2024-04-26T23:01:25.847 INFO:tasks.workunit.client.0.smithi027.stdout:4/245: rename da/f38 to da/dd/de/d14/d17/f45 0 2024-04-26T23:01:25.848 INFO:tasks.workunit.client.0.smithi027.stdout:3/257: mkdir d6/dc/d17/d52/d5a 0 2024-04-26T23:01:25.849 INFO:tasks.workunit.client.0.smithi027.stdout:3/258: symlink d6/dc/d34/l5b 0 2024-04-26T23:01:25.849 INFO:tasks.workunit.client.0.smithi027.stdout:3/259: fdatasync d6/dc/d34/d2e/f49 0 2024-04-26T23:01:25.855 INFO:tasks.workunit.client.0.smithi027.stdout:1/281: dwrite d1/f1f [0,4194304] 0 2024-04-26T23:01:25.857 INFO:tasks.workunit.client.0.smithi027.stdout:1/282: unlink d1/d2a/f47 0 2024-04-26T23:01:25.857 INFO:tasks.workunit.client.0.smithi027.stdout:1/283: fsync d1/d5/d21/f22 0 2024-04-26T23:01:25.859 INFO:tasks.workunit.client.0.smithi027.stdout:1/284: unlink d1/d5/d21/c39 0 2024-04-26T23:01:25.859 INFO:tasks.workunit.client.0.smithi027.stdout:1/285: chown d1/d5/d21/f34 450758176 1 2024-04-26T23:01:25.860 INFO:tasks.workunit.client.0.smithi027.stdout:1/286: creat d1/dc/f5b x:0 0 0 2024-04-26T23:01:25.860 INFO:tasks.workunit.client.0.smithi027.stdout:1/287: read d1/d26/f29 [2202116,31283] 0 2024-04-26T23:01:25.861 INFO:tasks.workunit.client.0.smithi027.stdout:1/288: chown d1/d2a 29843 1 2024-04-26T23:01:25.862 INFO:tasks.workunit.client.0.smithi027.stdout:1/289: truncate d1/f1b 2263638 0 2024-04-26T23:01:25.862 INFO:tasks.workunit.client.0.smithi027.stdout:1/290: write d1/d5/f2e [223305,86753] 0 2024-04-26T23:01:25.868 INFO:tasks.workunit.client.0.smithi027.stdout:3/260: getdents d6/dc/d34 0 2024-04-26T23:01:25.879 INFO:tasks.workunit.client.0.smithi027.stdout:8/246: dwrite d9/f1b [0,4194304] 0 2024-04-26T23:01:25.881 INFO:tasks.workunit.client.0.smithi027.stdout:4/246: read da/dd/de/d14/d17/f20 [510031,127667] 0 2024-04-26T23:01:25.881 INFO:tasks.workunit.client.0.smithi027.stdout:4/247: truncate da/dd/de/d14/d17/d24/f3a 401772 0 2024-04-26T23:01:25.881 INFO:tasks.workunit.client.0.smithi027.stdout:4/248: chown da/dd/de/d14/d17/f45 6477461 1 2024-04-26T23:01:25.887 INFO:tasks.workunit.client.0.smithi027.stdout:8/247: rename d9/d28 to d9/da/d2a/d5e 0 2024-04-26T23:01:25.888 INFO:tasks.workunit.client.0.smithi027.stdout:8/248: creat d9/da/d2a/d5e/d2b/d3b/f5f x:0 0 0 2024-04-26T23:01:25.908 INFO:tasks.workunit.client.0.smithi027.stdout:8/249: write d9/da/d33/f41 [1671151,85519] 0 2024-04-26T23:01:25.911 INFO:tasks.workunit.client.0.smithi027.stdout:3/261: dwrite d6/dc/f11 [0,4194304] 0 2024-04-26T23:01:25.914 INFO:tasks.workunit.client.0.smithi027.stdout:6/148: dwrite d1/f33 [0,4194304] 0 2024-04-26T23:01:25.914 INFO:tasks.workunit.client.0.smithi027.stdout:6/149: fsync d1/f27 0 2024-04-26T23:01:25.914 INFO:tasks.workunit.client.0.smithi027.stdout:6/150: fsync d1/de/f1b 0 2024-04-26T23:01:25.914 INFO:tasks.workunit.client.0.smithi027.stdout:3/262: creat d6/f5c x:0 0 0 2024-04-26T23:01:25.925 INFO:tasks.workunit.client.0.smithi027.stdout:3/263: unlink d6/c27 0 2024-04-26T23:01:25.930 INFO:tasks.workunit.client.0.smithi027.stdout:6/151: creat d1/f35 x:0 0 0 2024-04-26T23:01:25.930 INFO:tasks.workunit.client.0.smithi027.stdout:3/264: write d6/f7 [627123,55986] 0 2024-04-26T23:01:25.931 INFO:tasks.workunit.client.0.smithi027.stdout:3/265: chown d6/dc/d34/d47/l43 61427807 1 2024-04-26T23:01:25.931 INFO:tasks.workunit.client.0.smithi027.stdout:3/266: chown d6/dc/d17/d40/f4d 740240658 1 2024-04-26T23:01:25.931 INFO:tasks.workunit.client.0.smithi027.stdout:3/267: fsync d6/f8 0 2024-04-26T23:01:25.952 INFO:tasks.workunit.client.0.smithi027.stdout:6/152: write d1/df/f29 [1492930,125581] 0 2024-04-26T23:01:25.953 INFO:tasks.workunit.client.0.smithi027.stdout:6/153: creat d1/d30/f36 x:0 0 0 2024-04-26T23:01:25.954 INFO:tasks.workunit.client.0.smithi027.stdout:6/154: fdatasync d1/de/f20 0 2024-04-26T23:01:25.955 INFO:tasks.workunit.client.0.smithi027.stdout:3/268: dread d6/dc/f21 [0,4194304] 0 2024-04-26T23:01:25.955 INFO:tasks.workunit.client.0.smithi027.stdout:6/155: getdents d1 0 2024-04-26T23:01:25.955 INFO:tasks.workunit.client.0.smithi027.stdout:6/156: chown d1/df/f2e 3 1 2024-04-26T23:01:25.955 INFO:tasks.workunit.client.0.smithi027.stdout:6/157: write d1/de/f28 [581154,46901] 0 2024-04-26T23:01:25.956 INFO:tasks.workunit.client.0.smithi027.stdout:3/269: creat d6/dc/d34/f5d x:0 0 0 2024-04-26T23:01:25.957 INFO:tasks.workunit.client.0.smithi027.stdout:6/158: symlink d1/d30/l37 0 2024-04-26T23:01:25.958 INFO:tasks.workunit.client.0.smithi027.stdout:8/250: dwrite d9/de/d22/f4d [0,4194304] 0 2024-04-26T23:01:25.958 INFO:tasks.workunit.client.0.smithi027.stdout:4/249: write da/dd/de/d14/d17/f2a [3776147,127012] 0 2024-04-26T23:01:25.959 INFO:tasks.workunit.client.0.smithi027.stdout:0/227: dwrite d0/f12 [0,4194304] 0 2024-04-26T23:01:25.959 INFO:tasks.workunit.client.0.smithi027.stdout:6/159: symlink d1/d30/l38 0 2024-04-26T23:01:25.960 INFO:tasks.workunit.client.0.smithi027.stdout:6/160: fsync d1/f2 0 2024-04-26T23:01:25.961 INFO:tasks.workunit.client.0.smithi027.stdout:8/251: creat d9/da/d33/f60 x:0 0 0 2024-04-26T23:01:25.966 INFO:tasks.workunit.client.0.smithi027.stdout:0/228: truncate d0/f37 644623 0 2024-04-26T23:01:25.966 INFO:tasks.workunit.client.0.smithi027.stdout:0/229: rename d0 to d0/d4/da/d15/d45 22 2024-04-26T23:01:25.966 INFO:tasks.workunit.client.0.smithi027.stdout:8/252: symlink d9/da/d2a/d5e/d46/l61 0 2024-04-26T23:01:25.966 INFO:tasks.workunit.client.0.smithi027.stdout:6/161: link d1/f35 d1/df/f39 0 2024-04-26T23:01:25.969 INFO:tasks.workunit.client.0.smithi027.stdout:6/162: creat d1/df/f3a x:0 0 0 2024-04-26T23:01:25.972 INFO:tasks.workunit.client.0.smithi027.stdout:8/253: write f3 [158588,26621] 0 2024-04-26T23:01:25.975 INFO:tasks.workunit.client.0.smithi027.stdout:8/254: getdents d9/de 0 2024-04-26T23:01:25.976 INFO:tasks.workunit.client.0.smithi027.stdout:8/255: symlink d9/da/d2a/d5e/d2b/l62 0 2024-04-26T23:01:25.977 INFO:tasks.workunit.client.0.smithi027.stdout:8/256: write d9/f1b [4732864,74185] 0 2024-04-26T23:01:25.977 INFO:tasks.workunit.client.0.smithi027.stdout:8/257: fdatasync d9/f17 0 2024-04-26T23:01:25.982 INFO:tasks.workunit.client.0.smithi027.stdout:8/258: rmdir d9/de 39 2024-04-26T23:01:25.983 INFO:tasks.workunit.client.0.smithi027.stdout:8/259: mknod d9/de/d20/c63 0 2024-04-26T23:01:25.983 INFO:tasks.workunit.client.0.smithi027.stdout:8/260: fdatasync d9/de/f2c 0 2024-04-26T23:01:25.984 INFO:tasks.workunit.client.0.smithi027.stdout:8/261: link d9/de/d22/f3e d9/da/d2a/d5e/f64 0 2024-04-26T23:01:25.985 INFO:tasks.workunit.client.0.smithi027.stdout:8/262: mknod d9/da/d2a/d5e/d2b/d3b/c65 0 2024-04-26T23:01:25.986 INFO:tasks.workunit.client.0.smithi027.stdout:8/263: creat d9/da/d2a/d5e/d2b/f66 x:0 0 0 2024-04-26T23:01:25.988 INFO:tasks.workunit.client.0.smithi027.stdout:8/264: symlink d9/da/d33/l67 0 2024-04-26T23:01:25.989 INFO:tasks.workunit.client.0.smithi027.stdout:8/265: link f2 d9/de/d20/f68 0 2024-04-26T23:01:26.000 INFO:tasks.workunit.client.0.smithi027.stdout:6/163: dwrite d1/df/f2e [0,4194304] 0 2024-04-26T23:01:26.003 INFO:tasks.workunit.client.0.smithi027.stdout:6/164: symlink d1/df/l3b 0 2024-04-26T23:01:26.005 INFO:tasks.workunit.client.0.smithi027.stdout:6/165: rmdir d1/df 39 2024-04-26T23:01:26.007 INFO:tasks.workunit.client.0.smithi027.stdout:6/166: mkdir d1/df/d3c 0 2024-04-26T23:01:26.009 INFO:tasks.workunit.client.0.smithi027.stdout:8/266: write d9/da/d2a/d5e/d2b/f3d [2373940,61742] 0 2024-04-26T23:01:26.009 INFO:tasks.workunit.client.0.smithi027.stdout:3/270: dwrite d6/dc/d34/d47/f3c [0,4194304] 0 2024-04-26T23:01:26.009 INFO:tasks.workunit.client.0.smithi027.stdout:0/230: dread d0/d4/f5 [4194304,4194304] 0 2024-04-26T23:01:26.009 INFO:tasks.workunit.client.0.smithi027.stdout:0/231: stat d0/d4/da/f1f 0 2024-04-26T23:01:26.011 INFO:tasks.workunit.client.0.smithi027.stdout:8/267: rename d9/de/d20/l37 to d9/de/d20/l69 0 2024-04-26T23:01:26.011 INFO:tasks.workunit.client.0.smithi027.stdout:8/268: write d9/da/fd [199147,70416] 0 2024-04-26T23:01:26.011 INFO:tasks.workunit.client.0.smithi027.stdout:3/271: mkdir d6/d5e 0 2024-04-26T23:01:26.012 INFO:tasks.workunit.client.0.smithi027.stdout:0/232: symlink d0/d4/da/d1a/d1e/l46 0 2024-04-26T23:01:26.013 INFO:tasks.workunit.client.0.smithi027.stdout:8/269: mknod d9/da/d2a/d5e/d2b/d52/c6a 0 2024-04-26T23:01:26.013 INFO:tasks.workunit.client.0.smithi027.stdout:8/270: chown d9/de/d2e/c4e 46352 1 2024-04-26T23:01:26.014 INFO:tasks.workunit.client.0.smithi027.stdout:8/271: truncate d9/da/d33/f3a 3913058 0 2024-04-26T23:01:26.014 INFO:tasks.workunit.client.0.smithi027.stdout:3/272: rmdir d6/dc/d17 39 2024-04-26T23:01:26.014 INFO:tasks.workunit.client.0.smithi027.stdout:3/273: readlink d6/dc/l12 0 2024-04-26T23:01:26.015 INFO:tasks.workunit.client.0.smithi027.stdout:3/274: mkdir d6/dc/d34/d2e/d5f 0 2024-04-26T23:01:26.015 INFO:tasks.workunit.client.0.smithi027.stdout:3/275: write d6/f5c [722909,46952] 0 2024-04-26T23:01:26.032 INFO:tasks.workunit.client.0.smithi027.stdout:8/272: dread d9/da/f13 [0,4194304] 0 2024-04-26T23:01:26.033 INFO:tasks.workunit.client.0.smithi027.stdout:7/170: sync 2024-04-26T23:01:26.033 INFO:tasks.workunit.client.0.smithi027.stdout:9/231: sync 2024-04-26T23:01:26.033 INFO:tasks.workunit.client.0.smithi027.stdout:5/243: sync 2024-04-26T23:01:26.033 INFO:tasks.workunit.client.0.smithi027.stdout:9/232: chown d4/d10/d2c/f41 9 1 2024-04-26T23:01:26.034 INFO:tasks.workunit.client.0.smithi027.stdout:5/244: fsync d4/db/f23 0 2024-04-26T23:01:26.035 INFO:tasks.workunit.client.0.smithi027.stdout:5/245: rename d4/db/d16/d34 to d4/d24/d22/d29/d2a/d3b/d50 0 2024-04-26T23:01:26.036 INFO:tasks.workunit.client.0.smithi027.stdout:5/246: mknod d4/db/d16/c51 0 2024-04-26T23:01:26.045 INFO:tasks.workunit.client.0.smithi027.stdout:9/233: write d4/d17/f29 [10168,34165] 0 2024-04-26T23:01:26.053 INFO:tasks.workunit.client.0.smithi027.stdout:6/167: dwrite d1/df/f39 [0,4194304] 0 2024-04-26T23:01:26.061 INFO:tasks.workunit.client.0.smithi027.stdout:6/168: symlink d1/df/l3d 0 2024-04-26T23:01:26.063 INFO:tasks.workunit.client.0.smithi027.stdout:6/169: mknod d1/d30/c3e 0 2024-04-26T23:01:26.065 INFO:tasks.workunit.client.0.smithi027.stdout:6/170: rename d1/c34 to d1/de/c3f 0 2024-04-26T23:01:26.068 INFO:tasks.workunit.client.0.smithi027.stdout:0/233: dwrite d0/d4/da/d15/f34 [0,4194304] 0 2024-04-26T23:01:26.068 INFO:tasks.workunit.client.0.smithi027.stdout:0/234: chown d0/d4/lc 37939326 1 2024-04-26T23:01:26.071 INFO:tasks.workunit.client.0.smithi027.stdout:0/235: rmdir d0/d4/da/d1a/d1e 39 2024-04-26T23:01:26.075 INFO:tasks.workunit.client.0.smithi027.stdout:3/276: dwrite d6/dc/d17/f33 [0,4194304] 0 2024-04-26T23:01:26.085 INFO:tasks.workunit.client.0.smithi027.stdout:7/171: dwrite d0/d28/f2c [0,4194304] 0 2024-04-26T23:01:26.087 INFO:tasks.workunit.client.0.smithi027.stdout:7/172: symlink d0/l33 0 2024-04-26T23:01:26.087 INFO:tasks.workunit.client.0.smithi027.stdout:7/173: fsync d0/f27 0 2024-04-26T23:01:26.096 INFO:tasks.workunit.client.0.smithi027.stdout:5/247: dwrite f1 [4194304,4194304] 0 2024-04-26T23:01:26.098 INFO:tasks.workunit.client.0.smithi027.stdout:5/248: link d4/d1a/l32 d4/d24/d22/d29/d39/l52 0 2024-04-26T23:01:26.101 INFO:tasks.workunit.client.0.smithi027.stdout:5/249: symlink d4/d24/d25/l53 0 2024-04-26T23:01:26.104 INFO:tasks.workunit.client.0.smithi027.stdout:3/277: dread d6/dc/fd [0,4194304] 0 2024-04-26T23:01:26.104 INFO:tasks.workunit.client.0.smithi027.stdout:5/250: mkdir d4/d24/d25/d40/d54 0 2024-04-26T23:01:26.105 INFO:tasks.workunit.client.0.smithi027.stdout:3/278: getdents d6/dc/d34/d2e/d50 0 2024-04-26T23:01:26.109 INFO:tasks.workunit.client.0.smithi027.stdout:5/251: stat d4/d1a/l32 0 2024-04-26T23:01:26.109 INFO:tasks.workunit.client.0.smithi027.stdout:9/234: dread d4/d10/f28 [0,4194304] 0 2024-04-26T23:01:26.110 INFO:tasks.workunit.client.0.smithi027.stdout:3/279: getdents d6/dc/d34/d47 0 2024-04-26T23:01:26.110 INFO:tasks.workunit.client.0.smithi027.stdout:8/273: dwrite d9/da/d2a/d5e/d2b/f66 [0,4194304] 0 2024-04-26T23:01:26.110 INFO:tasks.workunit.client.0.smithi027.stdout:8/274: readlink d9/de/d22/l27 0 2024-04-26T23:01:26.111 INFO:tasks.workunit.client.0.smithi027.stdout:7/174: write d0/f13 [6055592,121692] 0 2024-04-26T23:01:26.114 INFO:tasks.workunit.client.0.smithi027.stdout:6/171: dwrite d1/df/fd [0,4194304] 0 2024-04-26T23:01:26.118 INFO:tasks.workunit.client.0.smithi027.stdout:9/235: rmdir d4/d10/d2c/d4f 0 2024-04-26T23:01:26.124 INFO:tasks.workunit.client.0.smithi027.stdout:3/280: rename d6/c19 to d6/dc/d17/d59/c60 0 2024-04-26T23:01:26.125 INFO:tasks.workunit.client.0.smithi027.stdout:1/291: sync 2024-04-26T23:01:26.125 INFO:tasks.workunit.client.0.smithi027.stdout:4/250: sync 2024-04-26T23:01:26.125 INFO:tasks.workunit.client.0.smithi027.stdout:4/251: readlink da/dd/de/d14/d17/d24/l36 0 2024-04-26T23:01:26.126 INFO:tasks.workunit.client.0.smithi027.stdout:6/172: mkdir d1/d30/d40 0 2024-04-26T23:01:26.127 INFO:tasks.workunit.client.0.smithi027.stdout:8/275: write d9/da/d2a/d5e/f64 [601584,53759] 0 2024-04-26T23:01:26.127 INFO:tasks.workunit.client.0.smithi027.stdout:8/276: write d9/da/d2a/d5e/d2b/f3d [1025476,31934] 0 2024-04-26T23:01:26.127 INFO:tasks.workunit.client.0.smithi027.stdout:2/205: sync 2024-04-26T23:01:26.127 INFO:tasks.workunit.client.0.smithi027.stdout:2/206: read d2/d5/f22 [2896124,75551] 0 2024-04-26T23:01:26.127 INFO:tasks.workunit.client.0.smithi027.stdout:3/281: mkdir d6/dc/d17/d59/d61 0 2024-04-26T23:01:26.128 INFO:tasks.workunit.client.0.smithi027.stdout:1/292: truncate d1/d5/d40/f52 940157 0 2024-04-26T23:01:26.129 INFO:tasks.workunit.client.0.smithi027.stdout:8/277: mkdir d9/da/d2a/d6b 0 2024-04-26T23:01:26.129 INFO:tasks.workunit.client.0.smithi027.stdout:8/278: stat d9/da 0 2024-04-26T23:01:26.129 INFO:tasks.workunit.client.0.smithi027.stdout:2/207: stat d2/d5/l33 0 2024-04-26T23:01:26.130 INFO:tasks.workunit.client.0.smithi027.stdout:7/175: dread d0/d2f/f31 [0,4194304] 0 2024-04-26T23:01:26.131 INFO:tasks.workunit.client.0.smithi027.stdout:6/173: link d1/df/l3d d1/d30/l41 0 2024-04-26T23:01:26.131 INFO:tasks.workunit.client.0.smithi027.stdout:6/174: chown d1/c32 729137507 1 2024-04-26T23:01:26.132 INFO:tasks.workunit.client.0.smithi027.stdout:1/293: getdents d1/d5/d21 0 2024-04-26T23:01:26.135 INFO:tasks.workunit.client.0.smithi027.stdout:8/279: mkdir d9/de/d20/d5d/d6c 0 2024-04-26T23:01:26.142 INFO:tasks.workunit.client.0.smithi027.stdout:7/176: creat d0/d28/d2b/f34 x:0 0 0 2024-04-26T23:01:26.142 INFO:tasks.workunit.client.0.smithi027.stdout:7/177: fdatasync d0/f2a 0 2024-04-26T23:01:26.142 INFO:tasks.workunit.client.0.smithi027.stdout:8/280: rename d9/da/d2a/d5e/f59 to d9/da/f6d 0 2024-04-26T23:01:26.142 INFO:tasks.workunit.client.0.smithi027.stdout:6/175: creat d1/de/f42 x:0 0 0 2024-04-26T23:01:26.142 INFO:tasks.workunit.client.0.smithi027.stdout:8/281: readlink d9/da/d33/l67 0 2024-04-26T23:01:26.143 INFO:tasks.workunit.client.0.smithi027.stdout:4/252: dread da/dd/de/d14/d17/d24/f16 [0,4194304] 0 2024-04-26T23:01:26.144 INFO:tasks.workunit.client.0.smithi027.stdout:7/178: truncate d0/f11 5640498 0 2024-04-26T23:01:26.149 INFO:tasks.workunit.client.0.smithi027.stdout:3/282: dread d6/dc/f11 [0,4194304] 0 2024-04-26T23:01:26.154 INFO:tasks.workunit.client.0.smithi027.stdout:7/179: rename d0/c22 to d0/d2f/c35 0 2024-04-26T23:01:26.154 INFO:tasks.workunit.client.0.smithi027.stdout:7/180: chown d0/d28/d14/f16 1714 1 2024-04-26T23:01:26.154 INFO:tasks.workunit.client.0.smithi027.stdout:1/294: write d1/f3 [3591032,87668] 0 2024-04-26T23:01:26.154 INFO:tasks.workunit.client.0.smithi027.stdout:6/176: mkdir d1/df/d43 0 2024-04-26T23:01:26.158 INFO:tasks.workunit.client.0.smithi027.stdout:7/181: creat d0/d28/d14/f36 x:0 0 0 2024-04-26T23:01:26.162 INFO:tasks.workunit.client.0.smithi027.stdout:7/182: fdatasync d0/f13 0 2024-04-26T23:01:26.168 INFO:tasks.workunit.client.0.smithi027.stdout:4/253: write da/dd/de/f3f [1159220,290] 0 2024-04-26T23:01:26.169 INFO:tasks.workunit.client.0.smithi027.stdout:4/254: write da/dd/de/d14/d17/d24/f3a [608161,129989] 0 2024-04-26T23:01:26.170 INFO:tasks.workunit.client.0.smithi027.stdout:3/283: dread d6/dc/f46 [0,4194304] 0 2024-04-26T23:01:26.176 INFO:tasks.workunit.client.0.smithi027.stdout:5/252: dwrite d4/d1a/f20 [0,4194304] 0 2024-04-26T23:01:26.178 INFO:tasks.workunit.client.0.smithi027.stdout:3/284: mkdir d6/dc/d62 0 2024-04-26T23:01:26.178 INFO:tasks.workunit.client.0.smithi027.stdout:3/285: chown d6/dc/l12 764699 1 2024-04-26T23:01:26.178 INFO:tasks.workunit.client.0.smithi027.stdout:4/255: creat da/dd/de/d14/d21/f46 x:0 0 0 2024-04-26T23:01:26.179 INFO:tasks.workunit.client.0.smithi027.stdout:4/256: write da/dd/de/d14/d17/f20 [1483604,96499] 0 2024-04-26T23:01:26.179 INFO:tasks.workunit.client.0.smithi027.stdout:3/286: mkdir d6/dc/d17/d59/d61/d63 0 2024-04-26T23:01:26.181 INFO:tasks.workunit.client.0.smithi027.stdout:6/177: dread d1/df/f8 [0,4194304] 0 2024-04-26T23:01:26.182 INFO:tasks.workunit.client.0.smithi027.stdout:4/257: unlink da/dd/de/d14/d17/l1e 0 2024-04-26T23:01:26.183 INFO:tasks.workunit.client.0.smithi027.stdout:3/287: symlink d6/dc/d17/d59/d61/d63/l64 0 2024-04-26T23:01:26.184 INFO:tasks.workunit.client.0.smithi027.stdout:4/258: creat da/f47 x:0 0 0 2024-04-26T23:01:26.184 INFO:tasks.workunit.client.0.smithi027.stdout:3/288: creat d6/dc/d34/d47/f65 x:0 0 0 2024-04-26T23:01:26.187 INFO:tasks.workunit.client.0.smithi027.stdout:3/289: rename d6/dc/f35 to d6/dc/d34/d2e/f66 0 2024-04-26T23:01:26.195 INFO:tasks.workunit.client.0.smithi027.stdout:8/282: dwrite d9/de/d2e/f53 [0,4194304] 0 2024-04-26T23:01:26.205 INFO:tasks.workunit.client.0.smithi027.stdout:8/283: getdents d9/da 0 2024-04-26T23:01:26.206 INFO:tasks.workunit.client.0.smithi027.stdout:7/183: dwrite d0/f2a [0,4194304] 0 2024-04-26T23:01:26.208 INFO:tasks.workunit.client.0.smithi027.stdout:1/295: dwrite d1/d5/d21/f34 [0,4194304] 0 2024-04-26T23:01:26.208 INFO:tasks.workunit.client.0.smithi027.stdout:2/208: dwrite d2/d5/db/f14 [4194304,4194304] 0 2024-04-26T23:01:26.208 INFO:tasks.workunit.client.0.smithi027.stdout:1/296: fdatasync d1/dc/f16 0 2024-04-26T23:01:26.213 INFO:tasks.workunit.client.0.smithi027.stdout:2/209: mknod d2/d5/db/de/d21/d26/c3a 0 2024-04-26T23:01:26.216 INFO:tasks.workunit.client.0.smithi027.stdout:5/253: dread d4/db/f10 [4194304,4194304] 0 2024-04-26T23:01:26.216 INFO:tasks.workunit.client.0.smithi027.stdout:5/254: getdents d4/d24/d22/d29/d2a/d3b/d50 0 2024-04-26T23:01:26.216 INFO:tasks.workunit.client.0.smithi027.stdout:5/255: chown d4/d1a/f1e 5715 1 2024-04-26T23:01:26.216 INFO:tasks.workunit.client.0.smithi027.stdout:5/256: write d4/f15 [3083571,28668] 0 2024-04-26T23:01:26.222 INFO:tasks.workunit.client.0.smithi027.stdout:5/257: mkdir d4/d24/d55 0 2024-04-26T23:01:26.222 INFO:tasks.workunit.client.0.smithi027.stdout:5/258: write d4/db/d16/f38 [1604667,71626] 0 2024-04-26T23:01:26.226 INFO:tasks.workunit.client.0.smithi027.stdout:1/297: read d1/f1b [1369284,2365] 0 2024-04-26T23:01:26.231 INFO:tasks.workunit.client.0.smithi027.stdout:1/298: write d1/d5/d21/f27 [731059,121714] 0 2024-04-26T23:01:26.233 INFO:tasks.workunit.client.0.smithi027.stdout:6/178: dwrite d1/df/f8 [0,4194304] 0 2024-04-26T23:01:26.236 INFO:tasks.workunit.client.0.smithi027.stdout:3/290: dwrite d6/dc/d17/f33 [0,4194304] 0 2024-04-26T23:01:26.237 INFO:tasks.workunit.client.0.smithi027.stdout:3/291: truncate d6/f5c 1660573 0 2024-04-26T23:01:26.237 INFO:tasks.workunit.client.0.smithi027.stdout:6/179: creat d1/df/f44 x:0 0 0 2024-04-26T23:01:26.239 INFO:tasks.workunit.client.0.smithi027.stdout:3/292: creat d6/dc/d34/d2e/f67 x:0 0 0 2024-04-26T23:01:26.239 INFO:tasks.workunit.client.0.smithi027.stdout:3/293: mknod d6/dc/d17/d59/d61/c68 0 2024-04-26T23:01:26.240 INFO:tasks.workunit.client.0.smithi027.stdout:7/184: dread d0/d2f/f32 [0,4194304] 0 2024-04-26T23:01:26.240 INFO:tasks.workunit.client.0.smithi027.stdout:7/185: read d0/f27 [802714,103733] 0 2024-04-26T23:01:26.242 INFO:tasks.workunit.client.0.smithi027.stdout:3/294: creat d6/dc/d34/d47/f69 x:0 0 0 2024-04-26T23:01:26.242 INFO:tasks.workunit.client.0.smithi027.stdout:3/295: write d6/dc/d34/d2e/f31 [4312484,11714] 0 2024-04-26T23:01:26.243 INFO:tasks.workunit.client.0.smithi027.stdout:3/296: chown d6/f7 0 1 2024-04-26T23:01:26.243 INFO:tasks.workunit.client.0.smithi027.stdout:3/297: truncate d6/dc/d34/d2e/f45 362944 0 2024-04-26T23:01:26.243 INFO:tasks.workunit.client.0.smithi027.stdout:3/298: chown d6/dc 4962 1 2024-04-26T23:01:26.243 INFO:tasks.workunit.client.0.smithi027.stdout:7/186: creat d0/d28/f37 x:0 0 0 2024-04-26T23:01:26.244 INFO:tasks.workunit.client.0.smithi027.stdout:3/299: rename d6/f29 to d6/dc/d2f/f6a 0 2024-04-26T23:01:26.245 INFO:tasks.workunit.client.0.smithi027.stdout:3/300: mkdir d6/dc/d2f/d6b 0 2024-04-26T23:01:26.249 INFO:tasks.workunit.client.0.smithi027.stdout:3/301: rename d6/dc/d17/d59/c60 to d6/dc/d34/d2e/d5f/c6c 0 2024-04-26T23:01:26.265 INFO:tasks.workunit.client.0.smithi027.stdout:7/187: dread d0/f11 [4194304,4194304] 0 2024-04-26T23:01:26.265 INFO:tasks.workunit.client.0.smithi027.stdout:7/188: chown d0/d28/f37 54 1 2024-04-26T23:01:26.275 INFO:tasks.workunit.client.0.smithi027.stdout:8/284: dwrite d9/da/d33/f3a [0,4194304] 0 2024-04-26T23:01:26.279 INFO:tasks.workunit.client.0.smithi027.stdout:7/189: mkdir d0/d2f/d38 0 2024-04-26T23:01:26.280 INFO:tasks.workunit.client.0.smithi027.stdout:7/190: chown d0/l23 183873 1 2024-04-26T23:01:26.282 INFO:tasks.workunit.client.0.smithi027.stdout:8/285: link d9/da/d2a/d5e/d2b/d3b/f4c d9/da/d2a/d5e/f6e 0 2024-04-26T23:01:26.283 INFO:tasks.workunit.client.0.smithi027.stdout:7/191: rmdir d0/d2f/d38 0 2024-04-26T23:01:26.284 INFO:tasks.workunit.client.0.smithi027.stdout:8/286: creat d9/da/d2a/d5e/d46/f6f x:0 0 0 2024-04-26T23:01:26.285 INFO:tasks.workunit.client.0.smithi027.stdout:7/192: creat d0/d28/d14/f39 x:0 0 0 2024-04-26T23:01:26.285 INFO:tasks.workunit.client.0.smithi027.stdout:8/287: mknod d9/da/d2a/d5e/d2b/c70 0 2024-04-26T23:01:26.286 INFO:tasks.workunit.client.0.smithi027.stdout:8/288: chown d9/da/d2a/d5e/d2b/d52/l5a 60 1 2024-04-26T23:01:26.286 INFO:tasks.workunit.client.0.smithi027.stdout:8/289: chown d9/da/d2a 2069 1 2024-04-26T23:01:26.286 INFO:tasks.workunit.client.0.smithi027.stdout:7/193: mkdir d0/d28/d2b/d3a 0 2024-04-26T23:01:26.287 INFO:tasks.workunit.client.0.smithi027.stdout:7/194: symlink d0/d28/d2b/l3b 0 2024-04-26T23:01:26.288 INFO:tasks.workunit.client.0.smithi027.stdout:7/195: creat d0/d28/d14/f3c x:0 0 0 2024-04-26T23:01:26.288 INFO:tasks.workunit.client.0.smithi027.stdout:7/196: fsync d0/d28/f18 0 2024-04-26T23:01:26.289 INFO:tasks.workunit.client.0.smithi027.stdout:7/197: mkdir d0/d3d 0 2024-04-26T23:01:26.289 INFO:tasks.workunit.client.0.smithi027.stdout:7/198: truncate d0/d2f/f32 4925575 0 2024-04-26T23:01:26.289 INFO:tasks.workunit.client.0.smithi027.stdout:7/199: truncate d0/d28/d14/f36 97986 0 2024-04-26T23:01:26.290 INFO:tasks.workunit.client.0.smithi027.stdout:7/200: creat d0/d28/d14/f3e x:0 0 0 2024-04-26T23:01:26.290 INFO:tasks.workunit.client.0.smithi027.stdout:7/201: fsync d0/f13 0 2024-04-26T23:01:26.296 INFO:tasks.workunit.client.0.smithi027.stdout:4/259: dwrite da/f1f [0,4194304] 0 2024-04-26T23:01:26.296 INFO:tasks.workunit.client.0.smithi027.stdout:2/210: dwrite d2/d5/db/d1b/d1c/d25/d19/d23/f31 [0,4194304] 0 2024-04-26T23:01:26.296 INFO:tasks.workunit.client.0.smithi027.stdout:2/211: write d2/d5/db/de/f30 [141939,6267] 0 2024-04-26T23:01:26.297 INFO:tasks.workunit.client.0.smithi027.stdout:4/260: unlink da/dd/de/d14/d17/d24/l36 0 2024-04-26T23:01:26.298 INFO:tasks.workunit.client.0.smithi027.stdout:2/212: rmdir d2/d5/db/d1b/d1c/d25/d19/d23 39 2024-04-26T23:01:26.301 INFO:tasks.workunit.client.0.smithi027.stdout:2/213: truncate d2/d5/f22 264265 0 2024-04-26T23:01:26.301 INFO:tasks.workunit.client.0.smithi027.stdout:4/261: link da/dd/f3b da/f48 0 2024-04-26T23:01:26.303 INFO:tasks.workunit.client.0.smithi027.stdout:2/214: fsync d2/d5/db/f1a 0 2024-04-26T23:01:26.303 INFO:tasks.workunit.client.0.smithi027.stdout:2/215: chown d2/d5/db/d1b/d1c/d25/d19/d23/f31 90668 1 2024-04-26T23:01:26.315 INFO:tasks.workunit.client.0.smithi027.stdout:6/180: dwrite d1/df/f31 [0,4194304] 0 2024-04-26T23:01:26.315 INFO:tasks.workunit.client.0.smithi027.stdout:6/181: dread - d1/de/f20 zero size 2024-04-26T23:01:26.329 INFO:tasks.workunit.client.0.smithi027.stdout:0/236: sync 2024-04-26T23:01:26.334 INFO:tasks.workunit.client.0.smithi027.stdout:5/259: dwrite d4/d1a/f1e [0,4194304] 0 2024-04-26T23:01:26.334 INFO:tasks.workunit.client.0.smithi027.stdout:5/260: readlink d4/d24/d22/d29/l49 0 2024-04-26T23:01:26.338 INFO:tasks.workunit.client.0.smithi027.stdout:5/261: link d4/d24/d22/d29/d2a/f3e d4/d24/d22/d29/d2a/d3b/f56 0 2024-04-26T23:01:26.338 INFO:tasks.workunit.client.0.smithi027.stdout:5/262: write d4/d24/d22/d29/d2a/d3b/f3d [1360615,65035] 0 2024-04-26T23:01:26.338 INFO:tasks.workunit.client.0.smithi027.stdout:6/182: dread d1/de/f13 [0,4194304] 0 2024-04-26T23:01:26.339 INFO:tasks.workunit.client.0.smithi027.stdout:6/183: creat d1/df/d43/f45 x:0 0 0 2024-04-26T23:01:26.341 INFO:tasks.workunit.client.0.smithi027.stdout:6/184: link d1/d30/l41 d1/de/l46 0 2024-04-26T23:01:26.341 INFO:tasks.workunit.client.0.smithi027.stdout:6/185: chown d1 5675 1 2024-04-26T23:01:26.343 INFO:tasks.workunit.client.0.smithi027.stdout:4/262: dwrite da/dd/de/d14/d17/f1d [0,4194304] 0 2024-04-26T23:01:26.343 INFO:tasks.workunit.client.0.smithi027.stdout:4/263: chown da/dd/de/d14 43528745 1 2024-04-26T23:01:26.343 INFO:tasks.workunit.client.0.smithi027.stdout:4/264: chown da/dd/de/d14/d17/f20 2790875 1 2024-04-26T23:01:26.343 INFO:tasks.workunit.client.0.smithi027.stdout:4/265: write da/dd/de/f3f [996371,129419] 0 2024-04-26T23:01:26.343 INFO:tasks.workunit.client.0.smithi027.stdout:4/266: readlink da/dd/de/d14/d17/d24/l27 0 2024-04-26T23:01:26.343 INFO:tasks.workunit.client.0.smithi027.stdout:6/186: truncate d1/df/fd 1633255 0 2024-04-26T23:01:26.344 INFO:tasks.workunit.client.0.smithi027.stdout:4/267: creat da/dd/de/d14/d21/d41/f49 x:0 0 0 2024-04-26T23:01:26.352 INFO:tasks.workunit.client.0.smithi027.stdout:0/237: dread d0/d4/da/f14 [0,4194304] 0 2024-04-26T23:01:26.357 INFO:tasks.workunit.client.0.smithi027.stdout:1/299: dwrite d1/d5/d21/f27 [0,4194304] 0 2024-04-26T23:01:26.357 INFO:tasks.workunit.client.0.smithi027.stdout:1/300: write d1/d2a/d3c/f5a [373763,16257] 0 2024-04-26T23:01:26.363 INFO:tasks.workunit.client.0.smithi027.stdout:6/187: write d1/df/fd [68177,74674] 0 2024-04-26T23:01:26.368 INFO:tasks.workunit.client.0.smithi027.stdout:6/188: symlink d1/df/d43/l47 0 2024-04-26T23:01:26.369 INFO:tasks.workunit.client.0.smithi027.stdout:6/189: creat d1/de/f48 x:0 0 0 2024-04-26T23:01:26.371 INFO:tasks.workunit.client.0.smithi027.stdout:0/238: dread d0/d4/da/d15/f34 [0,4194304] 0 2024-04-26T23:01:26.373 INFO:tasks.workunit.client.0.smithi027.stdout:8/290: dwrite d9/de/d20/f40 [0,4194304] 0 2024-04-26T23:01:26.373 INFO:tasks.workunit.client.0.smithi027.stdout:6/190: mkdir d1/df/d49 0 2024-04-26T23:01:26.374 INFO:tasks.workunit.client.0.smithi027.stdout:6/191: chown d1/f27 132 1 2024-04-26T23:01:26.378 INFO:tasks.workunit.client.0.smithi027.stdout:2/216: write d2/d5/fa [3967624,40579] 0 2024-04-26T23:01:26.380 INFO:tasks.workunit.client.0.smithi027.stdout:8/291: mkdir d9/da/d33/d71 0 2024-04-26T23:01:26.380 INFO:tasks.workunit.client.0.smithi027.stdout:6/192: mknod d1/c4a 0 2024-04-26T23:01:26.380 INFO:tasks.workunit.client.0.smithi027.stdout:6/193: rename d1 to d1/df/d3c/d4b 22 2024-04-26T23:01:26.380 INFO:tasks.workunit.client.0.smithi027.stdout:2/217: mknod d2/d5/db/d1b/d1c/d25/d2f/c3b 0 2024-04-26T23:01:26.382 INFO:tasks.workunit.client.0.smithi027.stdout:5/263: dwrite d4/d1a/f1e [4194304,4194304] 0 2024-04-26T23:01:26.385 INFO:tasks.workunit.client.0.smithi027.stdout:6/194: mknod d1/df/d43/c4c 0 2024-04-26T23:01:26.385 INFO:tasks.workunit.client.0.smithi027.stdout:8/292: mkdir d9/da/d2a/d5e/d72 0 2024-04-26T23:01:26.385 INFO:tasks.workunit.client.0.smithi027.stdout:8/293: truncate d9/da/f6d 904825 0 2024-04-26T23:01:26.394 INFO:tasks.workunit.client.0.smithi027.stdout:7/202: dwrite d0/d28/d14/f36 [0,4194304] 0 2024-04-26T23:01:26.397 INFO:tasks.workunit.client.0.smithi027.stdout:1/301: dread d1/f1f [0,4194304] 0 2024-04-26T23:01:26.406 INFO:tasks.workunit.client.0.smithi027.stdout:2/218: dread d2/d5/f22 [0,4194304] 0 2024-04-26T23:01:26.417 INFO:tasks.workunit.client.0.smithi027.stdout:0/239: dwrite d0/d4/da/d2d/f31 [0,4194304] 0 2024-04-26T23:01:26.418 INFO:tasks.workunit.client.0.smithi027.stdout:4/268: dwrite da/dd/de/d14/d17/d24/f3e [0,4194304] 0 2024-04-26T23:01:26.418 INFO:tasks.workunit.client.0.smithi027.stdout:0/240: mknod d0/d4/c47 0 2024-04-26T23:01:26.435 INFO:tasks.workunit.client.0.smithi027.stdout:1/302: write d1/d5/d40/d2d/f30 [1138541,50067] 0 2024-04-26T23:01:26.436 INFO:tasks.workunit.client.0.smithi027.stdout:8/294: dwrite d9/de/d20/d35/f56 [0,4194304] 0 2024-04-26T23:01:26.438 INFO:tasks.workunit.client.0.smithi027.stdout:8/295: mknod d9/da/d2a/c73 0 2024-04-26T23:01:26.441 INFO:tasks.workunit.client.0.smithi027.stdout:8/296: rename d9/da/d2a/d5e/d2b/f66 to d9/f74 0 2024-04-26T23:01:26.442 INFO:tasks.workunit.client.0.smithi027.stdout:8/297: rmdir d9/da/d33 39 2024-04-26T23:01:26.442 INFO:tasks.workunit.client.0.smithi027.stdout:2/219: write d2/d5/db/f1a [1671204,116504] 0 2024-04-26T23:01:26.442 INFO:tasks.workunit.client.0.smithi027.stdout:8/298: chown d9/da/d2a/d5e/d46/l61 2 1 2024-04-26T23:01:26.442 INFO:tasks.workunit.client.0.smithi027.stdout:8/299: write d9/da/d2a/d5e/d2b/d3b/f5f [78253,49967] 0 2024-04-26T23:01:26.443 INFO:tasks.workunit.client.0.smithi027.stdout:6/195: dwrite d1/f2b [0,4194304] 0 2024-04-26T23:01:26.446 INFO:tasks.workunit.client.0.smithi027.stdout:8/300: mknod d9/de/d22/c75 0 2024-04-26T23:01:26.446 INFO:tasks.workunit.client.0.smithi027.stdout:8/301: getdents d9/da/d2a/d5e/d2b/d3b 0 2024-04-26T23:01:26.465 INFO:tasks.workunit.client.0.smithi027.stdout:7/203: write d0/f11 [3242672,53832] 0 2024-04-26T23:01:26.465 INFO:tasks.workunit.client.0.smithi027.stdout:7/204: fsync d0/d28/d14/f36 0 2024-04-26T23:01:26.466 INFO:tasks.workunit.client.0.smithi027.stdout:7/205: creat d0/d28/d2b/d3a/f3f x:0 0 0 2024-04-26T23:01:26.466 INFO:tasks.workunit.client.0.smithi027.stdout:7/206: chown d0/d28/c10 4571 1 2024-04-26T23:01:26.467 INFO:tasks.workunit.client.0.smithi027.stdout:7/207: truncate d0/d28/d14/f29 1097669 0 2024-04-26T23:01:26.467 INFO:tasks.workunit.client.0.smithi027.stdout:7/208: write d0/d2f/f32 [5938558,51343] 0 2024-04-26T23:01:26.469 INFO:tasks.workunit.client.0.smithi027.stdout:1/303: dread d1/d5/d21/f44 [0,4194304] 0 2024-04-26T23:01:26.470 INFO:tasks.workunit.client.0.smithi027.stdout:5/264: dwrite d4/f9 [0,4194304] 0 2024-04-26T23:01:26.475 INFO:tasks.workunit.client.0.smithi027.stdout:8/302: dread d9/f74 [0,4194304] 0 2024-04-26T23:01:26.477 INFO:tasks.workunit.client.0.smithi027.stdout:4/269: dread da/dd/de/d14/d17/f28 [0,4194304] 0 2024-04-26T23:01:26.477 INFO:tasks.workunit.client.0.smithi027.stdout:4/270: fsync da/dd/de/d14/d17/f33 0 2024-04-26T23:01:26.477 INFO:tasks.workunit.client.0.smithi027.stdout:4/271: stat da/dd/de/d14/d17/f28 0 2024-04-26T23:01:26.478 INFO:tasks.workunit.client.0.smithi027.stdout:0/241: dwrite d0/d4/f3d [0,4194304] 0 2024-04-26T23:01:26.482 INFO:tasks.workunit.client.0.smithi027.stdout:5/265: creat d4/d1a/f57 x:0 0 0 2024-04-26T23:01:26.487 INFO:tasks.workunit.client.0.smithi027.stdout:8/303: truncate d9/da/d2a/d5e/d2b/f3d 2210627 0 2024-04-26T23:01:26.487 INFO:tasks.workunit.client.0.smithi027.stdout:3/302: sync 2024-04-26T23:01:26.488 INFO:tasks.workunit.client.0.smithi027.stdout:3/303: creat d6/d5e/f6d x:0 0 0 2024-04-26T23:01:26.489 INFO:tasks.workunit.client.0.smithi027.stdout:1/304: dread d1/d5/d40/f52 [0,4194304] 0 2024-04-26T23:01:26.489 INFO:tasks.workunit.client.0.smithi027.stdout:1/305: readlink d1/d5/d40/d2d/l31 0 2024-04-26T23:01:26.494 INFO:tasks.workunit.client.0.smithi027.stdout:3/304: unlink d6/f5c 0 2024-04-26T23:01:26.494 INFO:tasks.workunit.client.0.smithi027.stdout:2/220: dwrite d2/d5/db/d1b/d1c/f37 [0,4194304] 0 2024-04-26T23:01:26.494 INFO:tasks.workunit.client.0.smithi027.stdout:2/221: truncate d2/d5/db/de/d21/d26/f2b 635949 0 2024-04-26T23:01:26.494 INFO:tasks.workunit.client.0.smithi027.stdout:1/306: mkdir d1/d5/d40/d35/d5c 0 2024-04-26T23:01:26.496 INFO:tasks.workunit.client.0.smithi027.stdout:9/236: sync 2024-04-26T23:01:26.496 INFO:tasks.workunit.client.0.smithi027.stdout:9/237: write d4/f52 [131314,2766] 0 2024-04-26T23:01:26.500 INFO:tasks.workunit.client.0.smithi027.stdout:1/307: read d1/d5/f2f [178318,96718] 0 2024-04-26T23:01:26.503 INFO:tasks.workunit.client.0.smithi027.stdout:9/238: rename d4/d17/d2e/d46/c51 to d4/d17/d2e/c54 0 2024-04-26T23:01:26.503 INFO:tasks.workunit.client.0.smithi027.stdout:9/239: stat d4/l6 0 2024-04-26T23:01:26.503 INFO:tasks.workunit.client.0.smithi027.stdout:9/240: write d4/d17/d2e/f53 [190577,9184] 0 2024-04-26T23:01:26.504 INFO:tasks.workunit.client.0.smithi027.stdout:9/241: truncate d4/f14 2366637 0 2024-04-26T23:01:26.517 INFO:tasks.workunit.client.0.smithi027.stdout:3/305: write d6/dc/f46 [2620604,129364] 0 2024-04-26T23:01:26.522 INFO:tasks.workunit.client.0.smithi027.stdout:3/306: creat d6/dc/d17/d59/d61/d63/f6e x:0 0 0 2024-04-26T23:01:26.524 INFO:tasks.workunit.client.0.smithi027.stdout:2/222: write d2/d5/db/f1d [7366982,28072] 0 2024-04-26T23:01:26.540 INFO:tasks.workunit.client.0.smithi027.stdout:7/209: dwrite d0/d2f/f31 [0,4194304] 0 2024-04-26T23:01:26.544 INFO:tasks.workunit.client.0.smithi027.stdout:3/307: write d6/dc/f25 [8222242,95208] 0 2024-04-26T23:01:26.544 INFO:tasks.workunit.client.0.smithi027.stdout:3/308: fsync d6/f4f 0 2024-04-26T23:01:26.544 INFO:tasks.workunit.client.0.smithi027.stdout:3/309: stat d6/dc/d34/d47/l4b 0 2024-04-26T23:01:26.544 INFO:tasks.workunit.client.0.smithi027.stdout:3/310: chown d6/dc/d34/d47/l43 0 1 2024-04-26T23:01:26.544 INFO:tasks.workunit.client.0.smithi027.stdout:3/311: write d6/dc/f46 [4884235,56319] 0 2024-04-26T23:01:26.545 INFO:tasks.workunit.client.0.smithi027.stdout:7/210: creat d0/d28/f40 x:0 0 0 2024-04-26T23:01:26.545 INFO:tasks.workunit.client.0.smithi027.stdout:7/211: write d0/d28/d14/f3e [1021431,127120] 0 2024-04-26T23:01:26.545 INFO:tasks.workunit.client.0.smithi027.stdout:7/212: fdatasync d0/f2a 0 2024-04-26T23:01:26.545 INFO:tasks.workunit.client.0.smithi027.stdout:6/196: dwrite d1/f2b [4194304,4194304] 0 2024-04-26T23:01:26.547 INFO:tasks.workunit.client.0.smithi027.stdout:3/312: symlink d6/dc/l6f 0 2024-04-26T23:01:26.547 INFO:tasks.workunit.client.0.smithi027.stdout:3/313: stat d6/dc/d34/d2e/f49 0 2024-04-26T23:01:26.549 INFO:tasks.workunit.client.0.smithi027.stdout:7/213: mknod d0/d28/d14/c41 0 2024-04-26T23:01:26.549 INFO:tasks.workunit.client.0.smithi027.stdout:7/214: chown d0/d28/d2b/d3a/f3f 34 1 2024-04-26T23:01:26.549 INFO:tasks.workunit.client.0.smithi027.stdout:7/215: chown d0/f13 21048 1 2024-04-26T23:01:26.549 INFO:tasks.workunit.client.0.smithi027.stdout:6/197: creat d1/df/d49/f4d x:0 0 0 2024-04-26T23:01:26.549 INFO:tasks.workunit.client.0.smithi027.stdout:6/198: truncate d1/df/f44 605791 0 2024-04-26T23:01:26.549 INFO:tasks.workunit.client.0.smithi027.stdout:6/199: chown d1/de/f1a 8484533 1 2024-04-26T23:01:26.549 INFO:tasks.workunit.client.0.smithi027.stdout:6/200: getdents d1/df/d3c 0 2024-04-26T23:01:26.549 INFO:tasks.workunit.client.0.smithi027.stdout:3/314: mkdir d6/dc/d34/d2e/d70 0 2024-04-26T23:01:26.550 INFO:tasks.workunit.client.0.smithi027.stdout:7/216: getdents d0/d28 0 2024-04-26T23:01:26.552 INFO:tasks.workunit.client.0.smithi027.stdout:8/304: dwrite d9/de/d22/f4d [4194304,4194304] 0 2024-04-26T23:01:26.552 INFO:tasks.workunit.client.0.smithi027.stdout:6/201: link d1/de/f42 d1/f4e 0 2024-04-26T23:01:26.553 INFO:tasks.workunit.client.0.smithi027.stdout:3/315: mkdir d6/dc/d34/d2e/d70/d71 0 2024-04-26T23:01:26.553 INFO:tasks.workunit.client.0.smithi027.stdout:2/223: dread d2/d5/f34 [0,4194304] 0 2024-04-26T23:01:26.553 INFO:tasks.workunit.client.0.smithi027.stdout:2/224: truncate d2/d5/db/d1b/d1c/f37 4290483 0 2024-04-26T23:01:26.553 INFO:tasks.workunit.client.0.smithi027.stdout:2/225: truncate d2/d5/db/de/d21/d26/f2d 383590 0 2024-04-26T23:01:26.554 INFO:tasks.workunit.client.0.smithi027.stdout:0/242: dwrite d0/d4/f24 [0,4194304] 0 2024-04-26T23:01:26.554 INFO:tasks.workunit.client.0.smithi027.stdout:7/217: creat d0/f42 x:0 0 0 2024-04-26T23:01:26.555 INFO:tasks.workunit.client.0.smithi027.stdout:8/305: truncate d9/da/d33/f3a 2239485 0 2024-04-26T23:01:26.558 INFO:tasks.workunit.client.0.smithi027.stdout:5/266: dread d4/db/f23 [0,4194304] 0 2024-04-26T23:01:26.558 INFO:tasks.workunit.client.0.smithi027.stdout:5/267: readlink d4/l21 0 2024-04-26T23:01:26.558 INFO:tasks.workunit.client.0.smithi027.stdout:5/268: readlink d4/d24/d22/d29/d39/l52 0 2024-04-26T23:01:26.562 INFO:tasks.workunit.client.0.smithi027.stdout:3/316: creat d6/dc/f72 x:0 0 0 2024-04-26T23:01:26.565 INFO:tasks.workunit.client.0.smithi027.stdout:2/226: rmdir d2/d5/db/de/d21/d26 39 2024-04-26T23:01:26.567 INFO:tasks.workunit.client.0.smithi027.stdout:7/218: creat d0/d3d/f43 x:0 0 0 2024-04-26T23:01:26.569 INFO:tasks.workunit.client.0.smithi027.stdout:8/306: link d9/da/f13 d9/da/d2a/d5e/d2b/f76 0 2024-04-26T23:01:26.570 INFO:tasks.workunit.client.0.smithi027.stdout:5/269: creat d4/d24/d22/d29/f58 x:0 0 0 2024-04-26T23:01:26.571 INFO:tasks.workunit.client.0.smithi027.stdout:0/243: write d0/d4/f3d [258398,79306] 0 2024-04-26T23:01:26.572 INFO:tasks.workunit.client.0.smithi027.stdout:0/244: write d0/f41 [826176,40565] 0 2024-04-26T23:01:26.572 INFO:tasks.workunit.client.0.smithi027.stdout:3/317: rename d6/dc/d2f/c56 to d6/dc/d2f/d6b/c73 0 2024-04-26T23:01:26.572 INFO:tasks.workunit.client.0.smithi027.stdout:3/318: dread - d6/dc/f72 zero size 2024-04-26T23:01:26.572 INFO:tasks.workunit.client.0.smithi027.stdout:3/319: stat d6/dc/d2f/f39 0 2024-04-26T23:01:26.573 INFO:tasks.workunit.client.0.smithi027.stdout:7/219: unlink d0/d28/d14/f36 0 2024-04-26T23:01:26.574 INFO:tasks.workunit.client.0.smithi027.stdout:8/307: truncate d9/de/d20/f3f 2496263 0 2024-04-26T23:01:26.575 INFO:tasks.workunit.client.0.smithi027.stdout:5/270: unlink d4/d24/d22/d29/c43 0 2024-04-26T23:01:26.578 INFO:tasks.workunit.client.0.smithi027.stdout:2/227: rename d2/d5/db/d1b/d1c/d25/f27 to d2/d5/db/d1b/d1c/d25/d19/d36/d39/f3c 0 2024-04-26T23:01:26.579 INFO:tasks.workunit.client.0.smithi027.stdout:3/320: mkdir d6/dc/d2f/d6b/d74 0 2024-04-26T23:01:26.581 INFO:tasks.workunit.client.0.smithi027.stdout:7/220: creat d0/d3d/f44 x:0 0 0 2024-04-26T23:01:26.581 INFO:tasks.workunit.client.0.smithi027.stdout:7/221: stat d0/cd 0 2024-04-26T23:01:26.581 INFO:tasks.workunit.client.0.smithi027.stdout:7/222: chown d0/l23 25270 1 2024-04-26T23:01:26.583 INFO:tasks.workunit.client.0.smithi027.stdout:5/271: mkdir d4/db/d16/d4b/d59 0 2024-04-26T23:01:26.584 INFO:tasks.workunit.client.0.smithi027.stdout:5/272: truncate d4/d24/d22/d29/f58 1038112 0 2024-04-26T23:01:26.584 INFO:tasks.workunit.client.0.smithi027.stdout:5/273: write d4/d24/d22/d29/d2a/d3b/f56 [410127,102689] 0 2024-04-26T23:01:26.588 INFO:tasks.workunit.client.0.smithi027.stdout:8/308: rmdir d9/da/d2a/d5e/d2b 39 2024-04-26T23:01:26.589 INFO:tasks.workunit.client.0.smithi027.stdout:2/228: creat d2/d5/db/d1b/d1c/d25/f3d x:0 0 0 2024-04-26T23:01:26.592 INFO:tasks.workunit.client.0.smithi027.stdout:3/321: mknod d6/dc/d2f/c75 0 2024-04-26T23:01:26.592 INFO:tasks.workunit.client.0.smithi027.stdout:8/309: mknod d9/de/d22/c77 0 2024-04-26T23:01:26.594 INFO:tasks.workunit.client.0.smithi027.stdout:2/229: truncate d2/d5/db/d1b/d1c/d25/d19/d23/f31 348587 0 2024-04-26T23:01:26.597 INFO:tasks.workunit.client.0.smithi027.stdout:8/310: rename d9/de/d20/f68 to d9/da/d2a/f78 0 2024-04-26T23:01:26.600 INFO:tasks.workunit.client.0.smithi027.stdout:0/245: write d0/d4/da/d2d/f31 [1149167,70686] 0 2024-04-26T23:01:26.607 INFO:tasks.workunit.client.0.smithi027.stdout:2/230: unlink d2/d5/db/de/d21/l18 0 2024-04-26T23:01:26.607 INFO:tasks.workunit.client.0.smithi027.stdout:7/223: dread d0/f2a [0,4194304] 0 2024-04-26T23:01:26.607 INFO:tasks.workunit.client.0.smithi027.stdout:7/224: fdatasync d0/d28/d14/f3c 0 2024-04-26T23:01:26.607 INFO:tasks.workunit.client.0.smithi027.stdout:7/225: write d0/f42 [252628,72705] 0 2024-04-26T23:01:26.607 INFO:tasks.workunit.client.0.smithi027.stdout:8/311: getdents d9 0 2024-04-26T23:01:26.607 INFO:tasks.workunit.client.0.smithi027.stdout:3/322: rmdir d6/dc/d34/d2e/d50 39 2024-04-26T23:01:26.608 INFO:tasks.workunit.client.0.smithi027.stdout:5/274: write d4/d24/d22/d29/f36 [2250844,26870] 0 2024-04-26T23:01:26.609 INFO:tasks.workunit.client.0.smithi027.stdout:8/312: creat d9/de/d20/d4f/f79 x:0 0 0 2024-04-26T23:01:26.611 INFO:tasks.workunit.client.0.smithi027.stdout:3/323: unlink d6/dc/d17/f1c 0 2024-04-26T23:01:26.614 INFO:tasks.workunit.client.0.smithi027.stdout:6/202: dwrite d1/de/f20 [0,4194304] 0 2024-04-26T23:01:26.614 INFO:tasks.workunit.client.0.smithi027.stdout:6/203: chown d1/de/f1f 37100678 1 2024-04-26T23:01:26.617 INFO:tasks.workunit.client.0.smithi027.stdout:5/275: creat d4/d24/d25/f5a x:0 0 0 2024-04-26T23:01:26.617 INFO:tasks.workunit.client.0.smithi027.stdout:3/324: mknod d6/dc/d2f/d6b/d74/c76 0 2024-04-26T23:01:26.619 INFO:tasks.workunit.client.0.smithi027.stdout:6/204: truncate d1/f27 1024448 0 2024-04-26T23:01:26.620 INFO:tasks.workunit.client.0.smithi027.stdout:6/205: stat d1/de/f28 0 2024-04-26T23:01:26.626 INFO:tasks.workunit.client.0.smithi027.stdout:7/226: read d0/d28/d14/f16 [2256596,81674] 0 2024-04-26T23:01:26.627 INFO:tasks.workunit.client.0.smithi027.stdout:7/227: dread - d0/d28/d2b/f34 zero size 2024-04-26T23:01:26.644 INFO:tasks.workunit.client.0.smithi027.stdout:6/206: write d1/f2 [3809782,1401] 0 2024-04-26T23:01:26.644 INFO:tasks.workunit.client.0.smithi027.stdout:2/231: dread d2/d5/db/d1b/d1c/f20 [0,4194304] 0 2024-04-26T23:01:26.644 INFO:tasks.workunit.client.0.smithi027.stdout:5/276: write f1 [6335543,10673] 0 2024-04-26T23:01:26.644 INFO:tasks.workunit.client.0.smithi027.stdout:5/277: write d4/d24/f42 [1423924,101532] 0 2024-04-26T23:01:26.644 INFO:tasks.workunit.client.0.smithi027.stdout:5/278: stat d4/d1a/f57 0 2024-04-26T23:01:26.644 INFO:tasks.workunit.client.0.smithi027.stdout:5/279: stat d4/d1a 0 2024-04-26T23:01:26.644 INFO:tasks.workunit.client.0.smithi027.stdout:3/325: write d6/dc/d34/f36 [3275069,17623] 0 2024-04-26T23:01:26.645 INFO:tasks.workunit.client.0.smithi027.stdout:3/326: readlink d6/l23 0 2024-04-26T23:01:26.645 INFO:tasks.workunit.client.0.smithi027.stdout:2/232: mknod d2/d5/db/d1b/d1c/d25/d2f/c3e 0 2024-04-26T23:01:26.645 INFO:tasks.workunit.client.0.smithi027.stdout:3/327: mknod d6/dc/d17/d59/d61/c77 0 2024-04-26T23:01:26.647 INFO:tasks.workunit.client.0.smithi027.stdout:3/328: mkdir d6/dc/d34/d78 0 2024-04-26T23:01:26.654 INFO:tasks.workunit.client.0.smithi027.stdout:3/329: fdatasync d6/dc/d34/f36 0 2024-04-26T23:01:26.655 INFO:tasks.workunit.client.0.smithi027.stdout:5/280: dread d4/d24/d22/d29/f58 [0,4194304] 0 2024-04-26T23:01:26.655 INFO:tasks.workunit.client.0.smithi027.stdout:5/281: fdatasync d4/d1a/f20 0 2024-04-26T23:01:26.656 INFO:tasks.workunit.client.0.smithi027.stdout:3/330: creat d6/dc/d62/f79 x:0 0 0 2024-04-26T23:01:26.656 INFO:tasks.workunit.client.0.smithi027.stdout:5/282: link d4/db/d16/c28 d4/d1a/d2b/c5b 0 2024-04-26T23:01:26.659 INFO:tasks.workunit.client.0.smithi027.stdout:5/283: link d4/db/d16/c28 d4/db/c5c 0 2024-04-26T23:01:26.660 INFO:tasks.workunit.client.0.smithi027.stdout:5/284: creat d4/d24/f5d x:0 0 0 2024-04-26T23:01:26.660 INFO:tasks.workunit.client.0.smithi027.stdout:5/285: chown d4/d24/d25/d40/l4d 209 1 2024-04-26T23:01:26.661 INFO:tasks.workunit.client.0.smithi027.stdout:7/228: dread d0/f27 [0,4194304] 0 2024-04-26T23:01:26.669 INFO:tasks.workunit.client.0.smithi027.stdout:3/331: dread d6/dc/f11 [0,4194304] 0 2024-04-26T23:01:26.669 INFO:tasks.workunit.client.0.smithi027.stdout:3/332: truncate d6/f2c 1726393 0 2024-04-26T23:01:26.672 INFO:tasks.workunit.client.0.smithi027.stdout:3/333: link d6/dc/d17/d59/d61/d63/l64 d6/dc/d34/d2e/d5f/l7a 0 2024-04-26T23:01:26.686 INFO:tasks.workunit.client.0.smithi027.stdout:7/229: dread d0/d2f/f32 [0,4194304] 0 2024-04-26T23:01:26.688 INFO:tasks.workunit.client.0.smithi027.stdout:7/230: symlink d0/d2f/l45 0 2024-04-26T23:01:26.689 INFO:tasks.workunit.client.0.smithi027.stdout:7/231: mknod d0/d28/c46 0 2024-04-26T23:01:26.689 INFO:tasks.workunit.client.0.smithi027.stdout:7/232: truncate d0/f42 1197316 0 2024-04-26T23:01:26.693 INFO:tasks.workunit.client.0.smithi027.stdout:0/246: dwrite d0/f12 [4194304,4194304] 0 2024-04-26T23:01:26.694 INFO:tasks.workunit.client.0.smithi027.stdout:8/313: dwrite d9/da/d2a/d5e/d2b/f76 [0,4194304] 0 2024-04-26T23:01:26.694 INFO:tasks.workunit.client.0.smithi027.stdout:8/314: chown d9/da/d33 8678808 1 2024-04-26T23:01:26.694 INFO:tasks.workunit.client.0.smithi027.stdout:0/247: unlink d0/f41 0 2024-04-26T23:01:26.694 INFO:tasks.workunit.client.0.smithi027.stdout:7/233: creat d0/d2f/f47 x:0 0 0 2024-04-26T23:01:26.695 INFO:tasks.workunit.client.0.smithi027.stdout:7/234: mkdir d0/d28/d2b/d3a/d48 0 2024-04-26T23:01:26.695 INFO:tasks.workunit.client.0.smithi027.stdout:7/235: read d0/f11 [519996,101814] 0 2024-04-26T23:01:26.695 INFO:tasks.workunit.client.0.smithi027.stdout:7/236: stat d0/d2f/f47 0 2024-04-26T23:01:26.695 INFO:tasks.workunit.client.0.smithi027.stdout:7/237: readlink d0/l1c 0 2024-04-26T23:01:26.695 INFO:tasks.workunit.client.0.smithi027.stdout:7/238: write d0/d28/d14/f3e [1958916,3036] 0 2024-04-26T23:01:26.696 INFO:tasks.workunit.client.0.smithi027.stdout:7/239: symlink d0/d3d/l49 0 2024-04-26T23:01:26.696 INFO:tasks.workunit.client.0.smithi027.stdout:7/240: chown d0/d28/f2c 34818463 1 2024-04-26T23:01:26.705 INFO:tasks.workunit.client.0.smithi027.stdout:7/241: write d0/d28/d14/f3e [896863,15149] 0 2024-04-26T23:01:26.705 INFO:tasks.workunit.client.0.smithi027.stdout:7/242: fdatasync d0/d28/d14/f16 0 2024-04-26T23:01:26.705 INFO:tasks.workunit.client.0.smithi027.stdout:7/243: chown d0/d28/d14/f3c 417251 1 2024-04-26T23:01:26.713 INFO:tasks.workunit.client.0.smithi027.stdout:7/244: symlink d0/d28/d2b/l4a 0 2024-04-26T23:01:26.713 INFO:tasks.workunit.client.0.smithi027.stdout:5/286: dwrite d4/d24/d25/f5a [0,4194304] 0 2024-04-26T23:01:26.713 INFO:tasks.workunit.client.0.smithi027.stdout:7/245: write d0/d28/f37 [488145,76520] 0 2024-04-26T23:01:26.719 INFO:tasks.workunit.client.0.smithi027.stdout:6/207: dwrite d1/d30/f36 [0,4194304] 0 2024-04-26T23:01:26.724 INFO:tasks.workunit.client.0.smithi027.stdout:5/287: dread d4/d24/d22/d29/d2a/d3b/f56 [0,4194304] 0 2024-04-26T23:01:26.734 INFO:tasks.workunit.client.0.smithi027.stdout:7/246: write d0/d2f/f32 [4197619,49524] 0 2024-04-26T23:01:26.740 INFO:tasks.workunit.client.0.smithi027.stdout:8/315: dwrite d9/de/d20/f3f [0,4194304] 0 2024-04-26T23:01:26.743 INFO:tasks.workunit.client.0.smithi027.stdout:8/316: unlink c5 0 2024-04-26T23:01:26.743 INFO:tasks.workunit.client.0.smithi027.stdout:8/317: chown d9/de/f18 379 1 2024-04-26T23:01:26.745 INFO:tasks.workunit.client.0.smithi027.stdout:8/318: link f7 d9/da/d2a/d5e/d2b/d52/f7a 0 2024-04-26T23:01:26.748 INFO:tasks.workunit.client.0.smithi027.stdout:6/208: write d1/f2b [1774276,2116] 0 2024-04-26T23:01:26.748 INFO:tasks.workunit.client.0.smithi027.stdout:6/209: readlink d1/l21 0 2024-04-26T23:01:26.748 INFO:tasks.workunit.client.0.smithi027.stdout:3/334: dwrite d6/dc/d34/d47/f65 [0,4194304] 0 2024-04-26T23:01:26.748 INFO:tasks.workunit.client.0.smithi027.stdout:3/335: truncate d6/dc/d62/f79 443046 0 2024-04-26T23:01:26.748 INFO:tasks.workunit.client.0.smithi027.stdout:8/319: truncate d9/de/d22/f3e 1435796 0 2024-04-26T23:01:26.749 INFO:tasks.workunit.client.0.smithi027.stdout:0/248: dwrite d0/d4/da/f11 [0,4194304] 0 2024-04-26T23:01:26.750 INFO:tasks.workunit.client.0.smithi027.stdout:0/249: chown d0/d4/da/d1a/d1e/c39 670664676 1 2024-04-26T23:01:26.750 INFO:tasks.workunit.client.0.smithi027.stdout:6/210: rename d1/de/f1f to d1/df/d49/f4f 0 2024-04-26T23:01:26.750 INFO:tasks.workunit.client.0.smithi027.stdout:3/336: mkdir d6/dc/d34/d47/d7b 0 2024-04-26T23:01:26.751 INFO:tasks.workunit.client.0.smithi027.stdout:0/250: rmdir d0/d4/da/d15/d26 39 2024-04-26T23:01:26.754 INFO:tasks.workunit.client.0.smithi027.stdout:6/211: mknod d1/df/d43/c50 0 2024-04-26T23:01:26.756 INFO:tasks.workunit.client.0.smithi027.stdout:0/251: dread d0/d4/da/d1a/d2a/f43 [0,4194304] 0 2024-04-26T23:01:26.757 INFO:tasks.workunit.client.0.smithi027.stdout:3/337: creat d6/dc/d62/f7c x:0 0 0 2024-04-26T23:01:26.758 INFO:tasks.workunit.client.0.smithi027.stdout:7/247: write d0/f2a [1773589,104075] 0 2024-04-26T23:01:26.758 INFO:tasks.workunit.client.0.smithi027.stdout:7/248: write d0/d28/d14/f3e [2031824,7591] 0 2024-04-26T23:01:26.760 INFO:tasks.workunit.client.0.smithi027.stdout:5/288: dwrite d4/d1a/f20 [0,4194304] 0 2024-04-26T23:01:26.761 INFO:tasks.workunit.client.0.smithi027.stdout:0/252: creat d0/d4/da/d15/f48 x:0 0 0 2024-04-26T23:01:26.767 INFO:tasks.workunit.client.0.smithi027.stdout:8/320: dread d9/de/f2c [0,4194304] 0 2024-04-26T23:01:26.769 INFO:tasks.workunit.client.0.smithi027.stdout:1/308: sync 2024-04-26T23:01:26.776 INFO:tasks.workunit.client.0.smithi027.stdout:6/212: mknod d1/df/c51 0 2024-04-26T23:01:26.776 INFO:tasks.workunit.client.0.smithi027.stdout:3/338: creat d6/dc/d34/d2e/d70/f7d x:0 0 0 2024-04-26T23:01:26.776 INFO:tasks.workunit.client.0.smithi027.stdout:5/289: link d4/c6 d4/d24/d22/d29/d2a/c5e 0 2024-04-26T23:01:26.777 INFO:tasks.workunit.client.0.smithi027.stdout:0/253: link d0/d4/l36 d0/d4/da/d15/l49 0 2024-04-26T23:01:26.778 INFO:tasks.workunit.client.0.smithi027.stdout:8/321: mknod d9/da/d2a/d5e/d2b/c7b 0 2024-04-26T23:01:26.778 INFO:tasks.workunit.client.0.smithi027.stdout:1/309: stat d1/d26/d36/c48 0 2024-04-26T23:01:26.778 INFO:tasks.workunit.client.0.smithi027.stdout:1/310: fsync d1/d2a/f49 0 2024-04-26T23:01:26.779 INFO:tasks.workunit.client.0.smithi027.stdout:3/339: getdents d6/dc/d2f/d6b 0 2024-04-26T23:01:26.780 INFO:tasks.workunit.client.0.smithi027.stdout:6/213: creat d1/df/d49/f52 x:0 0 0 2024-04-26T23:01:26.780 INFO:tasks.workunit.client.0.smithi027.stdout:0/254: rmdir d0/d4/da/d15 39 2024-04-26T23:01:26.780 INFO:tasks.workunit.client.0.smithi027.stdout:0/255: fdatasync d0/d4/da/f2f 0 2024-04-26T23:01:26.780 INFO:tasks.workunit.client.0.smithi027.stdout:0/256: chown d0/d4/da/f14 1 1 2024-04-26T23:01:26.781 INFO:tasks.workunit.client.0.smithi027.stdout:1/311: mknod d1/d26/c5d 0 2024-04-26T23:01:26.783 INFO:tasks.workunit.client.0.smithi027.stdout:3/340: rename d6/dc/d17/d59/d61/d63 to d6/dc/d2f/d7e 0 2024-04-26T23:01:26.785 INFO:tasks.workunit.client.0.smithi027.stdout:0/257: truncate d0/d4/da/d2d/f38 353850 0 2024-04-26T23:01:26.787 INFO:tasks.workunit.client.0.smithi027.stdout:1/312: creat d1/d26/d36/d56/f5e x:0 0 0 2024-04-26T23:01:26.789 INFO:tasks.workunit.client.0.smithi027.stdout:6/214: symlink d1/df/l53 0 2024-04-26T23:01:26.794 INFO:tasks.workunit.client.0.smithi027.stdout:0/258: mkdir d0/df/d3f/d4a 0 2024-04-26T23:01:26.794 INFO:tasks.workunit.client.0.smithi027.stdout:0/259: readlink d0/d4/da/d1a/d1e/l46 0 2024-04-26T23:01:26.796 INFO:tasks.workunit.client.0.smithi027.stdout:1/313: symlink d1/d26/l5f 0 2024-04-26T23:01:26.798 INFO:tasks.workunit.client.0.smithi027.stdout:1/314: symlink d1/d5/l60 0 2024-04-26T23:01:26.801 INFO:tasks.workunit.client.0.smithi027.stdout:5/290: dread d4/f15 [0,4194304] 0 2024-04-26T23:01:26.810 INFO:tasks.workunit.client.0.smithi027.stdout:3/341: write d6/dc/d17/d40/f4d [1550311,33390] 0 2024-04-26T23:01:26.810 INFO:tasks.workunit.client.0.smithi027.stdout:3/342: chown d6/l30 664 1 2024-04-26T23:01:26.810 INFO:tasks.workunit.client.0.smithi027.stdout:6/215: link d1/d30/l38 d1/df/l54 0 2024-04-26T23:01:26.810 INFO:tasks.workunit.client.0.smithi027.stdout:6/216: chown d1/f19 8 1 2024-04-26T23:01:26.810 INFO:tasks.workunit.client.0.smithi027.stdout:1/315: mknod d1/d5/d21/c61 0 2024-04-26T23:01:26.810 INFO:tasks.workunit.client.0.smithi027.stdout:6/217: fdatasync d1/de/f42 0 2024-04-26T23:01:26.810 INFO:tasks.workunit.client.0.smithi027.stdout:7/249: dwrite d0/d28/f40 [0,4194304] 0 2024-04-26T23:01:26.816 INFO:tasks.workunit.client.0.smithi027.stdout:6/218: getdents d1/d30 0 2024-04-26T23:01:26.816 INFO:tasks.workunit.client.0.smithi027.stdout:1/316: creat d1/d2a/f62 x:0 0 0 2024-04-26T23:01:26.816 INFO:tasks.workunit.client.0.smithi027.stdout:0/260: dread d0/d4/da/d2d/f31 [0,4194304] 0 2024-04-26T23:01:26.817 INFO:tasks.workunit.client.0.smithi027.stdout:0/261: write d0/d4/da/d15/f48 [59770,122826] 0 2024-04-26T23:01:26.819 INFO:tasks.workunit.client.0.smithi027.stdout:7/250: mknod d0/d28/d14/c4b 0 2024-04-26T23:01:26.819 INFO:tasks.workunit.client.0.smithi027.stdout:6/219: creat d1/df/d43/f55 x:0 0 0 2024-04-26T23:01:26.819 INFO:tasks.workunit.client.0.smithi027.stdout:5/291: dread d4/db/d16/f1f [4194304,4194304] 0 2024-04-26T23:01:26.820 INFO:tasks.workunit.client.0.smithi027.stdout:0/262: symlink d0/d4/da/d2d/l4b 0 2024-04-26T23:01:26.821 INFO:tasks.workunit.client.0.smithi027.stdout:7/251: link d0/d28/f18 d0/d28/d2b/d3a/f4c 0 2024-04-26T23:01:26.821 INFO:tasks.workunit.client.0.smithi027.stdout:5/292: unlink c3 0 2024-04-26T23:01:26.822 INFO:tasks.workunit.client.0.smithi027.stdout:0/263: mknod d0/c4c 0 2024-04-26T23:01:26.823 INFO:tasks.workunit.client.0.smithi027.stdout:7/252: mkdir d0/d2f/d4d 0 2024-04-26T23:01:26.824 INFO:tasks.workunit.client.0.smithi027.stdout:5/293: mkdir d4/d24/d5f 0 2024-04-26T23:01:26.824 INFO:tasks.workunit.client.0.smithi027.stdout:5/294: write d4/f15 [34341,60812] 0 2024-04-26T23:01:26.825 INFO:tasks.workunit.client.0.smithi027.stdout:0/264: rename d0/d4/f5 to d0/df/d3f/d4a/f4d 0 2024-04-26T23:01:26.825 INFO:tasks.workunit.client.0.smithi027.stdout:0/265: fdatasync d0/f37 0 2024-04-26T23:01:26.826 INFO:tasks.workunit.client.0.smithi027.stdout:5/295: symlink d4/d24/d25/l60 0 2024-04-26T23:01:26.826 INFO:tasks.workunit.client.0.smithi027.stdout:5/296: chown d4/d24/d22/d29/l2e 2088262 1 2024-04-26T23:01:26.827 INFO:tasks.workunit.client.0.smithi027.stdout:5/297: write d4/db/d16/f38 [1802405,24265] 0 2024-04-26T23:01:26.830 INFO:tasks.workunit.client.0.smithi027.stdout:7/253: rename d0/d2f/f32 to d0/d28/d2b/f4e 0 2024-04-26T23:01:26.830 INFO:tasks.workunit.client.0.smithi027.stdout:7/254: write d0/f27 [2735049,41513] 0 2024-04-26T23:01:26.832 INFO:tasks.workunit.client.0.smithi027.stdout:7/255: mknod d0/c4f 0 2024-04-26T23:01:26.836 INFO:tasks.workunit.client.0.smithi027.stdout:0/266: dread d0/d4/da/d2d/f38 [0,4194304] 0 2024-04-26T23:01:26.836 INFO:tasks.workunit.client.0.smithi027.stdout:7/256: symlink d0/d28/d14/l50 0 2024-04-26T23:01:26.837 INFO:tasks.workunit.client.0.smithi027.stdout:0/267: creat d0/df/f4e x:0 0 0 2024-04-26T23:01:26.838 INFO:tasks.workunit.client.0.smithi027.stdout:6/220: dread d1/de/f1a [0,4194304] 0 2024-04-26T23:01:26.838 INFO:tasks.workunit.client.0.smithi027.stdout:6/221: stat d1/de/f1b 0 2024-04-26T23:01:26.840 INFO:tasks.workunit.client.0.smithi027.stdout:6/222: mkdir d1/df/d43/d56 0 2024-04-26T23:01:26.841 INFO:tasks.workunit.client.0.smithi027.stdout:6/223: write d1/df/d49/f4d [140074,129274] 0 2024-04-26T23:01:26.842 INFO:tasks.workunit.client.0.smithi027.stdout:0/268: mknod d0/d4/c4f 0 2024-04-26T23:01:26.842 INFO:tasks.workunit.client.0.smithi027.stdout:0/269: readlink d0/d4/lc 0 2024-04-26T23:01:26.842 INFO:tasks.workunit.client.0.smithi027.stdout:6/224: unlink d1/l11 0 2024-04-26T23:01:26.847 INFO:tasks.workunit.client.0.smithi027.stdout:0/270: symlink d0/d4/da/d15/d26/l50 0 2024-04-26T23:01:26.847 INFO:tasks.workunit.client.0.smithi027.stdout:6/225: mkdir d1/de/d57 0 2024-04-26T23:01:26.847 INFO:tasks.workunit.client.0.smithi027.stdout:3/343: dwrite d6/dc/d17/f2b [0,4194304] 0 2024-04-26T23:01:26.847 INFO:tasks.workunit.client.0.smithi027.stdout:3/344: write d6/dc/d17/d40/f57 [18824,50124] 0 2024-04-26T23:01:26.847 INFO:tasks.workunit.client.0.smithi027.stdout:3/345: fdatasync d6/dc/d2f/f39 0 2024-04-26T23:01:26.847 INFO:tasks.workunit.client.0.smithi027.stdout:3/346: write d6/dc/d62/f7c [151990,58866] 0 2024-04-26T23:01:26.847 INFO:tasks.workunit.client.0.smithi027.stdout:3/347: getdents d6/dc/d4c 0 2024-04-26T23:01:26.847 INFO:tasks.workunit.client.0.smithi027.stdout:3/348: fsync d6/dc/f72 0 2024-04-26T23:01:26.848 INFO:tasks.workunit.client.0.smithi027.stdout:3/349: symlink d6/dc/d17/d59/l7f 0 2024-04-26T23:01:26.851 INFO:tasks.workunit.client.0.smithi027.stdout:3/350: rmdir d6/dc/d17/d52 39 2024-04-26T23:01:26.853 INFO:tasks.workunit.client.0.smithi027.stdout:3/351: symlink d6/dc/d17/l80 0 2024-04-26T23:01:26.855 INFO:tasks.workunit.client.0.smithi027.stdout:1/317: dwrite d1/d26/d36/d56/f5e [0,4194304] 0 2024-04-26T23:01:26.855 INFO:tasks.workunit.client.0.smithi027.stdout:1/318: readlink d1/l20 0 2024-04-26T23:01:26.855 INFO:tasks.workunit.client.0.smithi027.stdout:1/319: stat d1/d26/d36 0 2024-04-26T23:01:26.856 INFO:tasks.workunit.client.0.smithi027.stdout:3/352: rename d6/l9 to d6/dc/d34/d78/l81 0 2024-04-26T23:01:26.856 INFO:tasks.workunit.client.0.smithi027.stdout:3/353: chown d6/f18 230355 1 2024-04-26T23:01:26.858 INFO:tasks.workunit.client.0.smithi027.stdout:1/320: symlink d1/d5/d40/d35/d5c/l63 0 2024-04-26T23:01:26.858 INFO:tasks.workunit.client.0.smithi027.stdout:1/321: fsync d1/d5/d21/f22 0 2024-04-26T23:01:26.859 INFO:tasks.workunit.client.0.smithi027.stdout:3/354: creat d6/f82 x:0 0 0 2024-04-26T23:01:26.859 INFO:tasks.workunit.client.0.smithi027.stdout:3/355: read d6/dc/d17/d40/f57 [61077,45373] 0 2024-04-26T23:01:26.860 INFO:tasks.workunit.client.0.smithi027.stdout:1/322: rmdir d1/d5/d40/d35/d5c 39 2024-04-26T23:01:26.861 INFO:tasks.workunit.client.0.smithi027.stdout:1/323: chown d1/dc/l17 7 1 2024-04-26T23:01:26.861 INFO:tasks.workunit.client.0.smithi027.stdout:3/356: getdents d6/dc/d17/d59/d61 0 2024-04-26T23:01:26.862 INFO:tasks.workunit.client.0.smithi027.stdout:9/242: sync 2024-04-26T23:01:26.862 INFO:tasks.workunit.client.0.smithi027.stdout:4/272: sync 2024-04-26T23:01:26.862 INFO:tasks.workunit.client.0.smithi027.stdout:9/243: write d4/d17/f1d [1199177,53763] 0 2024-04-26T23:01:26.866 INFO:tasks.workunit.client.0.smithi027.stdout:1/324: mknod d1/d2a/d3c/c64 0 2024-04-26T23:01:26.866 INFO:tasks.workunit.client.0.smithi027.stdout:1/325: chown d1/dc/f5b 172525 1 2024-04-26T23:01:26.867 INFO:tasks.workunit.client.0.smithi027.stdout:3/357: rename d6/dc/d17/d40/c54 to d6/dc/d4c/c83 0 2024-04-26T23:01:26.868 INFO:tasks.workunit.client.0.smithi027.stdout:9/244: creat d4/d17/d2e/d46/f55 x:0 0 0 2024-04-26T23:01:26.868 INFO:tasks.workunit.client.0.smithi027.stdout:9/245: fsync d4/d10/f28 0 2024-04-26T23:01:26.869 INFO:tasks.workunit.client.0.smithi027.stdout:9/246: write d4/f2b [2896062,48529] 0 2024-04-26T23:01:26.871 INFO:tasks.workunit.client.0.smithi027.stdout:1/326: rename d1/d5/d40/d2d/l4e to d1/d26/d36/l65 0 2024-04-26T23:01:26.871 INFO:tasks.workunit.client.0.smithi027.stdout:1/327: stat d1/d2a/d3c/d42 0 2024-04-26T23:01:26.875 INFO:tasks.workunit.client.0.smithi027.stdout:9/247: creat d4/d10/f56 x:0 0 0 2024-04-26T23:01:26.876 INFO:tasks.workunit.client.0.smithi027.stdout:9/248: chown d4/d10/d40 0 1 2024-04-26T23:01:26.876 INFO:tasks.workunit.client.0.smithi027.stdout:6/226: read d1/df/f8 [2432410,98634] 0 2024-04-26T23:01:26.881 INFO:tasks.workunit.client.0.smithi027.stdout:0/271: dwrite d0/d4/da/d1a/d1e/f35 [0,4194304] 0 2024-04-26T23:01:26.882 INFO:tasks.workunit.client.0.smithi027.stdout:7/257: dread d0/f27 [0,4194304] 0 2024-04-26T23:01:26.887 INFO:tasks.workunit.client.0.smithi027.stdout:0/272: dread d0/d4/da/d2d/f38 [0,4194304] 0 2024-04-26T23:01:26.888 INFO:tasks.workunit.client.0.smithi027.stdout:1/328: mkdir d1/d26/d66 0 2024-04-26T23:01:26.888 INFO:tasks.workunit.client.0.smithi027.stdout:9/249: creat d4/d10/d2c/f57 x:0 0 0 2024-04-26T23:01:26.888 INFO:tasks.workunit.client.0.smithi027.stdout:6/227: creat d1/df/f58 x:0 0 0 2024-04-26T23:01:26.892 INFO:tasks.workunit.client.0.smithi027.stdout:5/298: dwrite d4/d1a/f20 [0,4194304] 0 2024-04-26T23:01:26.895 INFO:tasks.workunit.client.0.smithi027.stdout:7/258: rename d0/d28/c46 to d0/d28/d2b/d3a/d48/c51 0 2024-04-26T23:01:26.899 INFO:tasks.workunit.client.0.smithi027.stdout:0/273: rmdir d0/d4/da 39 2024-04-26T23:01:26.899 INFO:tasks.workunit.client.0.smithi027.stdout:1/329: getdents d1/d5/d40/d2d 0 2024-04-26T23:01:26.899 INFO:tasks.workunit.client.0.smithi027.stdout:1/330: truncate d1/d5/f46 835643 0 2024-04-26T23:01:26.900 INFO:tasks.workunit.client.0.smithi027.stdout:9/250: unlink d4/d10/c20 0 2024-04-26T23:01:26.901 INFO:tasks.workunit.client.0.smithi027.stdout:9/251: truncate d4/f24 274831 0 2024-04-26T23:01:26.905 INFO:tasks.workunit.client.0.smithi027.stdout:7/259: dread d0/d28/f18 [0,4194304] 0 2024-04-26T23:01:26.908 INFO:tasks.workunit.client.0.smithi027.stdout:9/252: mkdir d4/d10/d40/d58 0 2024-04-26T23:01:26.914 INFO:tasks.workunit.client.0.smithi027.stdout:7/260: creat d0/d28/d2b/d3a/f52 x:0 0 0 2024-04-26T23:01:26.914 INFO:tasks.workunit.client.0.smithi027.stdout:5/299: dread f1 [0,4194304] 0 2024-04-26T23:01:26.915 INFO:tasks.workunit.client.0.smithi027.stdout:5/300: mkdir d4/d24/d22/d29/d39/d61 0 2024-04-26T23:01:26.916 INFO:tasks.workunit.client.0.smithi027.stdout:5/301: rename d4/d24/d25 to d4/d24/d22/d29/d2a/d3b/d62 0 2024-04-26T23:01:26.921 INFO:tasks.workunit.client.0.smithi027.stdout:5/302: mkdir d4/d24/d55/d63 0 2024-04-26T23:01:26.924 INFO:tasks.workunit.client.0.smithi027.stdout:1/331: write d1/d26/d36/d56/f5e [2588051,23830] 0 2024-04-26T23:01:26.924 INFO:tasks.workunit.client.0.smithi027.stdout:1/332: fsync d1/d5/d21/f34 0 2024-04-26T23:01:26.925 INFO:tasks.workunit.client.0.smithi027.stdout:0/274: write d0/f1b [4184246,35149] 0 2024-04-26T23:01:26.927 INFO:tasks.workunit.client.0.smithi027.stdout:1/333: link d1/c58 d1/d5/d40/d35/c67 0 2024-04-26T23:01:26.929 INFO:tasks.workunit.client.0.smithi027.stdout:0/275: unlink d0/d4/da/d15/d26/l50 0 2024-04-26T23:01:26.929 INFO:tasks.workunit.client.0.smithi027.stdout:0/276: read d0/d4/da/d1a/f40 [118497,75756] 0 2024-04-26T23:01:26.931 INFO:tasks.workunit.client.0.smithi027.stdout:1/334: truncate d1/f1f 2135716 0 2024-04-26T23:01:26.931 INFO:tasks.workunit.client.0.smithi027.stdout:1/335: dread - d1/d2a/f49 zero size 2024-04-26T23:01:26.933 INFO:tasks.workunit.client.0.smithi027.stdout:0/277: mknod d0/d4/da/d1a/c51 0 2024-04-26T23:01:26.952 INFO:tasks.workunit.client.0.smithi027.stdout:4/273: dwrite da/dd/de/d14/d17/f33 [0,4194304] 0 2024-04-26T23:01:26.952 INFO:tasks.workunit.client.0.smithi027.stdout:4/274: truncate da/f48 508926 0 2024-04-26T23:01:26.952 INFO:tasks.workunit.client.0.smithi027.stdout:4/275: chown da/dd/de/d14/d17/d24/l27 2331067 1 2024-04-26T23:01:26.952 INFO:tasks.workunit.client.0.smithi027.stdout:4/276: fdatasync da/dd/de/d14/d17/d24/f3e 0 2024-04-26T23:01:26.952 INFO:tasks.workunit.client.0.smithi027.stdout:7/261: dwrite d0/d28/d2b/d3a/f3f [0,4194304] 0 2024-04-26T23:01:26.962 INFO:tasks.workunit.client.0.smithi027.stdout:7/262: dread d0/f42 [0,4194304] 0 2024-04-26T23:01:26.963 INFO:tasks.workunit.client.0.smithi027.stdout:9/253: dwrite d4/d10/d40/f48 [0,4194304] 0 2024-04-26T23:01:26.964 INFO:tasks.workunit.client.0.smithi027.stdout:9/254: write d4/d10/f2f [8887281,12918] 0 2024-04-26T23:01:26.964 INFO:tasks.workunit.client.0.smithi027.stdout:9/255: read - d4/d10/d2c/f57 zero size 2024-04-26T23:01:26.964 INFO:tasks.workunit.client.0.smithi027.stdout:1/336: dwrite d1/dc/f16 [0,4194304] 0 2024-04-26T23:01:26.968 INFO:tasks.workunit.client.0.smithi027.stdout:7/263: link d0/d28/d14/c41 d0/c53 0 2024-04-26T23:01:26.968 INFO:tasks.workunit.client.0.smithi027.stdout:1/337: symlink d1/d26/d66/l68 0 2024-04-26T23:01:26.972 INFO:tasks.workunit.client.0.smithi027.stdout:2/233: sync 2024-04-26T23:01:26.972 INFO:tasks.workunit.client.0.smithi027.stdout:2/234: fsync d2/d5/db/d1b/d1c/d25/f3d 0 2024-04-26T23:01:26.972 INFO:tasks.workunit.client.0.smithi027.stdout:2/235: write d2/d5/f22 [790525,27086] 0 2024-04-26T23:01:26.974 INFO:tasks.workunit.client.0.smithi027.stdout:2/236: creat d2/d5/db/d1b/d1c/d25/d19/d36/d39/f3f x:0 0 0 2024-04-26T23:01:26.978 INFO:tasks.workunit.client.0.smithi027.stdout:4/277: write da/dd/de/d14/d17/d24/f3e [3713976,117012] 0 2024-04-26T23:01:26.978 INFO:tasks.workunit.client.0.smithi027.stdout:4/278: dread - da/dd/de/f3c zero size 2024-04-26T23:01:26.986 INFO:tasks.workunit.client.0.smithi027.stdout:7/264: dread d0/d28/d2b/f4e [0,4194304] 0 2024-04-26T23:01:26.988 INFO:tasks.workunit.client.0.smithi027.stdout:7/265: creat d0/d3d/f54 x:0 0 0 2024-04-26T23:01:26.988 INFO:tasks.workunit.client.0.smithi027.stdout:7/266: dread - d0/d3d/f44 zero size 2024-04-26T23:01:26.988 INFO:tasks.workunit.client.0.smithi027.stdout:7/267: readlink d0/d2f/l45 0 2024-04-26T23:01:26.988 INFO:tasks.workunit.client.0.smithi027.stdout:7/268: stat d0 0 2024-04-26T23:01:26.989 INFO:tasks.workunit.client.0.smithi027.stdout:7/269: mkdir d0/d28/d2b/d55 0 2024-04-26T23:01:26.990 INFO:tasks.workunit.client.0.smithi027.stdout:7/270: mkdir d0/d28/d2b/d3a/d48/d56 0 2024-04-26T23:01:26.990 INFO:tasks.workunit.client.0.smithi027.stdout:7/271: fdatasync d0/f42 0 2024-04-26T23:01:26.990 INFO:tasks.workunit.client.0.smithi027.stdout:7/272: stat d0/d3d/l49 0 2024-04-26T23:01:27.000 INFO:tasks.workunit.client.0.smithi027.stdout:0/278: dwrite d0/d4/da/d15/f34 [0,4194304] 0 2024-04-26T23:01:27.004 INFO:tasks.workunit.client.0.smithi027.stdout:0/279: mkdir d0/d4/da/d1a/d1e/d52 0 2024-04-26T23:01:27.004 INFO:tasks.workunit.client.0.smithi027.stdout:1/338: dwrite d1/fb [0,4194304] 0 2024-04-26T23:01:27.004 INFO:tasks.workunit.client.0.smithi027.stdout:1/339: symlink d1/d2a/d3c/d42/l69 0 2024-04-26T23:01:27.004 INFO:tasks.workunit.client.0.smithi027.stdout:1/340: mknod d1/dc/c6a 0 2024-04-26T23:01:27.007 INFO:tasks.workunit.client.0.smithi027.stdout:7/273: dread d0/f2a [0,4194304] 0 2024-04-26T23:01:27.008 INFO:tasks.workunit.client.0.smithi027.stdout:7/274: chown d0/d28/d14/l1b 2029674 1 2024-04-26T23:01:27.008 INFO:tasks.workunit.client.0.smithi027.stdout:7/275: write d0/f2a [1277754,87051] 0 2024-04-26T23:01:27.009 INFO:tasks.workunit.client.0.smithi027.stdout:7/276: mknod d0/d3d/c57 0 2024-04-26T23:01:27.010 INFO:tasks.workunit.client.0.smithi027.stdout:7/277: unlink d0/l5 0 2024-04-26T23:01:27.012 INFO:tasks.workunit.client.0.smithi027.stdout:7/278: symlink d0/d28/d14/l58 0 2024-04-26T23:01:27.013 INFO:tasks.workunit.client.0.smithi027.stdout:7/279: mkdir d0/d28/d2b/d55/d59 0 2024-04-26T23:01:27.013 INFO:tasks.workunit.client.0.smithi027.stdout:7/280: truncate d0/f30 242371 0 2024-04-26T23:01:27.014 INFO:tasks.workunit.client.0.smithi027.stdout:7/281: creat d0/f5a x:0 0 0 2024-04-26T23:01:27.017 INFO:tasks.workunit.client.0.smithi027.stdout:7/282: unlink d0/d28/d14/c26 0 2024-04-26T23:01:27.018 INFO:tasks.workunit.client.0.smithi027.stdout:0/280: write d0/d4/da/f11 [1287303,25977] 0 2024-04-26T23:01:27.018 INFO:tasks.workunit.client.0.smithi027.stdout:0/281: write d0/d4/da/d2d/f31 [1182391,104503] 0 2024-04-26T23:01:27.020 INFO:tasks.workunit.client.0.smithi027.stdout:7/283: creat d0/d28/d2b/d3a/d48/d56/f5b x:0 0 0 2024-04-26T23:01:27.020 INFO:tasks.workunit.client.0.smithi027.stdout:7/284: write d0/d28/f18 [1208610,90647] 0 2024-04-26T23:01:27.026 INFO:tasks.workunit.client.0.smithi027.stdout:2/237: dwrite d2/d5/db/d1b/f38 [0,4194304] 0 2024-04-26T23:01:27.026 INFO:tasks.workunit.client.0.smithi027.stdout:1/341: rename d1/d2a/d3c to d1/d26/d6b 0 2024-04-26T23:01:27.026 INFO:tasks.workunit.client.0.smithi027.stdout:1/342: write d1/dc/f14 [2437306,50278] 0 2024-04-26T23:01:27.026 INFO:tasks.workunit.client.0.smithi027.stdout:0/282: dread d0/f13 [0,4194304] 0 2024-04-26T23:01:27.027 INFO:tasks.workunit.client.0.smithi027.stdout:1/343: rename d1/d5/l60 to d1/d5/l6c 0 2024-04-26T23:01:27.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/344: rename d1/d5/d40/d35/l54 to d1/d26/d6b/d42/l6d 0 2024-04-26T23:01:27.038 INFO:tasks.workunit.client.0.smithi027.stdout:4/279: dwrite da/dd/de/d14/f44 [0,4194304] 0 2024-04-26T23:01:27.043 INFO:tasks.workunit.client.0.smithi027.stdout:1/345: creat d1/d26/d36/d56/f6e x:0 0 0 2024-04-26T23:01:27.043 INFO:tasks.workunit.client.0.smithi027.stdout:1/346: readlink d1/d5/d40/d2d/l38 0 2024-04-26T23:01:27.043 INFO:tasks.workunit.client.0.smithi027.stdout:1/347: write d1/dc/f16 [5004160,22974] 0 2024-04-26T23:01:27.043 INFO:tasks.workunit.client.0.smithi027.stdout:1/348: readlink d1/d5/d40/d35/l51 0 2024-04-26T23:01:27.054 INFO:tasks.workunit.client.0.smithi027.stdout:2/238: dread d2/d5/db/d1b/d1c/d25/d19/d23/f2c [0,4194304] 0 2024-04-26T23:01:27.054 INFO:tasks.workunit.client.0.smithi027.stdout:2/239: write d2/d5/fa [3846246,86483] 0 2024-04-26T23:01:27.056 INFO:tasks.workunit.client.0.smithi027.stdout:0/283: dread d0/d4/f20 [4194304,4194304] 0 2024-04-26T23:01:27.058 INFO:tasks.workunit.client.0.smithi027.stdout:2/240: unlink d2/d5/db/d1b/d1c/d25/d19/d23/f31 0 2024-04-26T23:01:27.059 INFO:tasks.workunit.client.0.smithi027.stdout:0/284: truncate d0/d4/da/d1a/d1e/f25 2069086 0 2024-04-26T23:01:27.064 INFO:tasks.workunit.client.0.smithi027.stdout:0/285: rename d0/d4/da/d15/f34 to d0/d4/f53 0 2024-04-26T23:01:27.064 INFO:tasks.workunit.client.0.smithi027.stdout:2/241: mknod d2/d5/db/c40 0 2024-04-26T23:01:27.075 INFO:tasks.workunit.client.0.smithi027.stdout:1/349: dread d1/d5/d21/f27 [0,4194304] 0 2024-04-26T23:01:27.078 INFO:tasks.workunit.client.0.smithi027.stdout:7/285: dwrite d0/f2a [0,4194304] 0 2024-04-26T23:01:27.088 INFO:tasks.workunit.client.0.smithi027.stdout:4/280: dwrite da/dd/de/d14/d17/d24/f35 [0,4194304] 0 2024-04-26T23:01:27.088 INFO:tasks.workunit.client.0.smithi027.stdout:4/281: write da/dd/de/d14/d17/d24/f3e [1486784,112031] 0 2024-04-26T23:01:27.093 INFO:tasks.workunit.client.0.smithi027.stdout:8/322: sync 2024-04-26T23:01:27.093 INFO:tasks.workunit.client.0.smithi027.stdout:8/323: fdatasync d9/de/d2e/f49 0 2024-04-26T23:01:27.096 INFO:tasks.workunit.client.0.smithi027.stdout:8/324: rename f7 to d9/de/d20/d35/f7c 0 2024-04-26T23:01:27.096 INFO:tasks.workunit.client.0.smithi027.stdout:1/350: read d1/fb [154749,2820] 0 2024-04-26T23:01:27.098 INFO:tasks.workunit.client.0.smithi027.stdout:1/351: symlink d1/d5/d21/l6f 0 2024-04-26T23:01:27.100 INFO:tasks.workunit.client.0.smithi027.stdout:8/325: rename d9/da/f6d to d9/de/d20/d5d/f7d 0 2024-04-26T23:01:27.101 INFO:tasks.workunit.client.0.smithi027.stdout:8/326: creat d9/da/d2a/d5e/d72/f7e x:0 0 0 2024-04-26T23:01:27.103 INFO:tasks.workunit.client.0.smithi027.stdout:8/327: symlink d9/de/d20/d4f/l7f 0 2024-04-26T23:01:27.103 INFO:tasks.workunit.client.0.smithi027.stdout:8/328: write d9/f17 [4673087,100049] 0 2024-04-26T23:01:27.105 INFO:tasks.workunit.client.0.smithi027.stdout:8/329: truncate d9/f16 533964 0 2024-04-26T23:01:27.105 INFO:tasks.workunit.client.0.smithi027.stdout:8/330: chown d9/de/d22 1323261 1 2024-04-26T23:01:27.115 INFO:tasks.workunit.client.0.smithi027.stdout:1/352: dread d1/d5/d21/f34 [0,4194304] 0 2024-04-26T23:01:27.115 INFO:tasks.workunit.client.0.smithi027.stdout:2/242: dwrite d2/d5/db/d1b/f38 [4194304,4194304] 0 2024-04-26T23:01:27.120 INFO:tasks.workunit.client.0.smithi027.stdout:1/353: truncate d1/dc/f16 175968 0 2024-04-26T23:01:27.120 INFO:tasks.workunit.client.0.smithi027.stdout:1/354: fdatasync d1/dc/f14 0 2024-04-26T23:01:27.120 INFO:tasks.workunit.client.0.smithi027.stdout:1/355: fdatasync d1/d26/d6b/f5a 0 2024-04-26T23:01:27.120 INFO:tasks.workunit.client.0.smithi027.stdout:1/356: truncate d1/d5/d21/f3e 40569 0 2024-04-26T23:01:27.121 INFO:tasks.workunit.client.0.smithi027.stdout:1/357: getdents d1/d2a 0 2024-04-26T23:01:27.132 INFO:tasks.workunit.client.0.smithi027.stdout:7/286: dwrite d0/f13 [0,4194304] 0 2024-04-26T23:01:27.143 INFO:tasks.workunit.client.0.smithi027.stdout:7/287: creat d0/d28/d2b/f5c x:0 0 0 2024-04-26T23:01:27.145 INFO:tasks.workunit.client.0.smithi027.stdout:7/288: unlink d0/d2f/f47 0 2024-04-26T23:01:27.146 INFO:tasks.workunit.client.0.smithi027.stdout:7/289: read d0/f27 [4007656,40434] 0 2024-04-26T23:01:27.147 INFO:tasks.workunit.client.0.smithi027.stdout:7/290: creat d0/d28/d2b/f5d x:0 0 0 2024-04-26T23:01:27.157 INFO:tasks.workunit.client.0.smithi027.stdout:2/243: dwrite d2/d5/db/f1a [0,4194304] 0 2024-04-26T23:01:27.158 INFO:tasks.workunit.client.0.smithi027.stdout:2/244: dread - d2/d5/db/d1b/d1c/d25/d19/d36/d39/f3f zero size 2024-04-26T23:01:27.161 INFO:tasks.workunit.client.0.smithi027.stdout:8/331: write d9/de/d20/d35/f56 [3854257,57085] 0 2024-04-26T23:01:27.161 INFO:tasks.workunit.client.0.smithi027.stdout:2/245: creat d2/d5/db/de/f41 x:0 0 0 2024-04-26T23:01:27.162 INFO:tasks.workunit.client.0.smithi027.stdout:4/282: dwrite da/dd/f2b [0,4194304] 0 2024-04-26T23:01:27.164 INFO:tasks.workunit.client.0.smithi027.stdout:1/358: dwrite d1/d26/d36/d56/f6e [0,4194304] 0 2024-04-26T23:01:27.165 INFO:tasks.workunit.client.0.smithi027.stdout:4/283: mkdir da/dd/de/d14/d21/d4a 0 2024-04-26T23:01:27.166 INFO:tasks.workunit.client.0.smithi027.stdout:4/284: mknod da/d40/c4b 0 2024-04-26T23:01:27.168 INFO:tasks.workunit.client.0.smithi027.stdout:4/285: symlink da/dd/de/d14/d21/d4a/l4c 0 2024-04-26T23:01:27.170 INFO:tasks.workunit.client.0.smithi027.stdout:4/286: symlink da/dd/de/d14/d21/d41/l4d 0 2024-04-26T23:01:27.170 INFO:tasks.workunit.client.0.smithi027.stdout:4/287: stat da/dd/de/c2c 0 2024-04-26T23:01:27.170 INFO:tasks.workunit.client.0.smithi027.stdout:4/288: chown da/dd/de/d14/d17 60006465 1 2024-04-26T23:01:27.171 INFO:tasks.workunit.client.0.smithi027.stdout:4/289: mknod da/dd/de/d14/d17/d24/c4e 0 2024-04-26T23:01:27.174 INFO:tasks.workunit.client.0.smithi027.stdout:4/290: creat da/dd/de/d14/d21/d4a/f4f x:0 0 0 2024-04-26T23:01:27.177 INFO:tasks.workunit.client.0.smithi027.stdout:2/246: dread d2/d5/db/d1b/d1c/d25/d19/d23/f2c [0,4194304] 0 2024-04-26T23:01:27.178 INFO:tasks.workunit.client.0.smithi027.stdout:4/291: creat da/dd/de/d14/d21/d4a/f50 x:0 0 0 2024-04-26T23:01:27.179 INFO:tasks.workunit.client.0.smithi027.stdout:2/247: truncate d2/d5/f34 3653513 0 2024-04-26T23:01:27.180 INFO:tasks.workunit.client.0.smithi027.stdout:2/248: creat d2/d5/db/de/f42 x:0 0 0 2024-04-26T23:01:27.181 INFO:tasks.workunit.client.0.smithi027.stdout:1/359: dread d1/d26/d36/d56/f5e [0,4194304] 0 2024-04-26T23:01:27.182 INFO:tasks.workunit.client.0.smithi027.stdout:1/360: mkdir d1/d26/d6b/d42/d70 0 2024-04-26T23:01:27.184 INFO:tasks.workunit.client.0.smithi027.stdout:1/361: read d1/dc/f16 [112043,96954] 0 2024-04-26T23:01:27.184 INFO:tasks.workunit.client.0.smithi027.stdout:1/362: readlink d1/d26/d6b/d42/l69 0 2024-04-26T23:01:27.184 INFO:tasks.workunit.client.0.smithi027.stdout:1/363: write d1/dc/f5b [840392,82722] 0 2024-04-26T23:01:27.186 INFO:tasks.workunit.client.0.smithi027.stdout:1/364: link d1/d5/d40/d2d/l38 d1/d26/d36/d56/l71 0 2024-04-26T23:01:27.198 INFO:tasks.workunit.client.0.smithi027.stdout:1/365: write d1/dc/f37 [1751342,129894] 0 2024-04-26T23:01:27.200 INFO:tasks.workunit.client.0.smithi027.stdout:1/366: unlink d1/c58 0 2024-04-26T23:01:27.202 INFO:tasks.workunit.client.0.smithi027.stdout:1/367: getdents d1/d5/d40 0 2024-04-26T23:01:27.202 INFO:tasks.workunit.client.0.smithi027.stdout:1/368: truncate d1/d2a/f49 297888 0 2024-04-26T23:01:27.211 INFO:tasks.workunit.client.0.smithi027.stdout:8/332: dwrite d9/da/d33/f60 [0,4194304] 0 2024-04-26T23:01:27.217 INFO:tasks.workunit.client.0.smithi027.stdout:8/333: unlink d9/de/d20/l69 0 2024-04-26T23:01:27.219 INFO:tasks.workunit.client.0.smithi027.stdout:1/369: dread d1/fb [0,4194304] 0 2024-04-26T23:01:27.220 INFO:tasks.workunit.client.0.smithi027.stdout:1/370: truncate d1/d26/d36/d56/f5e 3121847 0 2024-04-26T23:01:27.221 INFO:tasks.workunit.client.0.smithi027.stdout:1/371: truncate d1/d5/f46 335979 0 2024-04-26T23:01:27.224 INFO:tasks.workunit.client.0.smithi027.stdout:8/334: dread f3 [0,4194304] 0 2024-04-26T23:01:27.225 INFO:tasks.workunit.client.0.smithi027.stdout:8/335: rename d9/de to d9/de/d20/d4f/d80 22 2024-04-26T23:01:27.235 INFO:tasks.workunit.client.0.smithi027.stdout:2/249: dwrite d2/d5/db/de/f41 [0,4194304] 0 2024-04-26T23:01:27.236 INFO:tasks.workunit.client.0.smithi027.stdout:1/372: dread d1/d26/f29 [0,4194304] 0 2024-04-26T23:01:27.237 INFO:tasks.workunit.client.0.smithi027.stdout:2/250: creat d2/d5/db/d1b/d1c/d25/d19/d36/f43 x:0 0 0 2024-04-26T23:01:27.238 INFO:tasks.workunit.client.0.smithi027.stdout:2/251: stat d2/d5/db/de/d21/d26/f2d 0 2024-04-26T23:01:27.243 INFO:tasks.workunit.client.0.smithi027.stdout:3/358: sync 2024-04-26T23:01:27.243 INFO:tasks.workunit.client.0.smithi027.stdout:6/228: sync 2024-04-26T23:01:27.243 INFO:tasks.workunit.client.0.smithi027.stdout:6/229: stat d1/df/d49/f4d 0 2024-04-26T23:01:27.243 INFO:tasks.workunit.client.0.smithi027.stdout:3/359: chown d6/dc/d34/l44 28 1 2024-04-26T23:01:27.245 INFO:tasks.workunit.client.0.smithi027.stdout:3/360: link d6/dc/f55 d6/dc/d34/d47/f84 0 2024-04-26T23:01:27.257 INFO:tasks.workunit.client.0.smithi027.stdout:3/361: dread d6/f2c [0,4194304] 0 2024-04-26T23:01:27.260 INFO:tasks.workunit.client.0.smithi027.stdout:3/362: rename d6/dc/d34/d47/f84 to d6/dc/f85 0 2024-04-26T23:01:27.261 INFO:tasks.workunit.client.0.smithi027.stdout:8/336: dwrite d9/da/d2a/d5e/f57 [4194304,4194304] 0 2024-04-26T23:01:27.261 INFO:tasks.workunit.client.0.smithi027.stdout:8/337: write d9/da/d2a/d5e/f57 [8781379,103720] 0 2024-04-26T23:01:27.265 INFO:tasks.workunit.client.0.smithi027.stdout:8/338: creat d9/da/d2a/d6b/f81 x:0 0 0 2024-04-26T23:01:27.266 INFO:tasks.workunit.client.0.smithi027.stdout:8/339: getdents d9/da/d2a/d5e/d46 0 2024-04-26T23:01:27.268 INFO:tasks.workunit.client.0.smithi027.stdout:4/292: dwrite da/dd/de/d14/d17/d24/f3a [0,4194304] 0 2024-04-26T23:01:27.268 INFO:tasks.workunit.client.0.smithi027.stdout:4/293: chown da/dd/de/d14/d17/d24/f26 1775468 1 2024-04-26T23:01:27.269 INFO:tasks.workunit.client.0.smithi027.stdout:8/340: mknod d9/da/d2a/d5e/d2b/d3b/c82 0 2024-04-26T23:01:27.269 INFO:tasks.workunit.client.0.smithi027.stdout:8/341: chown d9/da/d2a/d5e/d2b/d3b/f5f 179 1 2024-04-26T23:01:27.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:26 smithi080 ceph-mon[32324]: pgmap v88: 129 pgs: 129 active+clean; 255 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 18 MiB/s wr, 970 op/s 2024-04-26T23:01:27.273 INFO:tasks.workunit.client.0.smithi027.stdout:4/294: rmdir da/dd/de/d14/d17 39 2024-04-26T23:01:27.273 INFO:tasks.workunit.client.0.smithi027.stdout:6/230: write d1/f27 [32320,868] 0 2024-04-26T23:01:27.276 INFO:tasks.workunit.client.0.smithi027.stdout:4/295: link da/dd/de/d14/d21/d41/l4d da/d40/l51 0 2024-04-26T23:01:27.277 INFO:tasks.workunit.client.0.smithi027.stdout:4/296: read da/dd/de/d14/d17/f20 [167319,123342] 0 2024-04-26T23:01:27.277 INFO:tasks.workunit.client.0.smithi027.stdout:4/297: dread - da/dd/de/d14/d21/f46 zero size 2024-04-26T23:01:27.280 INFO:tasks.workunit.client.0.smithi027.stdout:2/252: dwrite d2/d5/db/d1b/d1c/d25/f3d [0,4194304] 0 2024-04-26T23:01:27.280 INFO:tasks.workunit.client.0.smithi027.stdout:2/253: readlink d2/d5/db/de/l24 0 2024-04-26T23:01:27.281 INFO:tasks.workunit.client.0.smithi027.stdout:1/373: dwrite d1/fb [4194304,4194304] 0 2024-04-26T23:01:27.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:26 smithi094 ceph-mon[32889]: pgmap v88: 129 pgs: 129 active+clean; 255 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 18 MiB/s wr, 970 op/s 2024-04-26T23:01:27.287 INFO:tasks.workunit.client.0.smithi027.stdout:6/231: truncate d1/df/d49/f4d 144304 0 2024-04-26T23:01:27.288 INFO:tasks.workunit.client.0.smithi027.stdout:4/298: creat da/dd/de/d14/d21/d41/f52 x:0 0 0 2024-04-26T23:01:27.291 INFO:tasks.workunit.client.0.smithi027.stdout:1/374: link d1/d5/c23 d1/d26/d6b/d42/d70/c72 0 2024-04-26T23:01:27.293 INFO:tasks.workunit.client.0.smithi027.stdout:6/232: creat d1/f59 x:0 0 0 2024-04-26T23:01:27.294 INFO:tasks.workunit.client.0.smithi027.stdout:4/299: creat da/dd/de/d14/f53 x:0 0 0 2024-04-26T23:01:27.296 INFO:tasks.workunit.client.0.smithi027.stdout:2/254: mkdir d2/d5/db/d1b/d1c/d25/d19/d44 0 2024-04-26T23:01:27.298 INFO:tasks.workunit.client.0.smithi027.stdout:4/300: symlink da/d40/l54 0 2024-04-26T23:01:27.301 INFO:tasks.workunit.client.0.smithi027.stdout:2/255: link d2/d5/db/de/f42 d2/d5/db/d1b/d1c/d25/d19/f45 0 2024-04-26T23:01:27.301 INFO:tasks.workunit.client.0.smithi027.stdout:2/256: write d2/d5/fa [3078330,39037] 0 2024-04-26T23:01:27.301 INFO:tasks.workunit.client.0.smithi027.stdout:2/257: chown d2/d5/db/d1b/d1c/f20 110385157 1 2024-04-26T23:01:27.301 INFO:tasks.workunit.client.0.smithi027.stdout:4/301: rmdir da 39 2024-04-26T23:01:27.301 INFO:tasks.workunit.client.0.smithi027.stdout:4/302: chown da/dd/de/d14/f43 71 1 2024-04-26T23:01:27.303 INFO:tasks.workunit.client.0.smithi027.stdout:2/258: rename d2/d5/db/f1d to d2/d5/db/d1b/f46 0 2024-04-26T23:01:27.303 INFO:tasks.workunit.client.0.smithi027.stdout:2/259: write d2/d5/db/de/f42 [415332,33888] 0 2024-04-26T23:01:27.303 INFO:tasks.workunit.client.0.smithi027.stdout:4/303: getdents da/dd/de 0 2024-04-26T23:01:27.304 INFO:tasks.workunit.client.0.smithi027.stdout:4/304: dread - da/dd/de/d14/f43 zero size 2024-04-26T23:01:27.305 INFO:tasks.workunit.client.0.smithi027.stdout:2/260: creat d2/d5/db/d1b/d1c/d25/d19/d23/f47 x:0 0 0 2024-04-26T23:01:27.309 INFO:tasks.workunit.client.0.smithi027.stdout:0/286: sync 2024-04-26T23:01:27.309 INFO:tasks.workunit.client.0.smithi027.stdout:9/256: sync 2024-04-26T23:01:27.310 INFO:tasks.workunit.client.0.smithi027.stdout:5/303: sync 2024-04-26T23:01:27.310 INFO:tasks.workunit.client.0.smithi027.stdout:8/342: dwrite d9/da/d2a/d5e/f64 [0,4194304] 0 2024-04-26T23:01:27.311 INFO:tasks.workunit.client.0.smithi027.stdout:9/257: getdents d4/d17 0 2024-04-26T23:01:27.313 INFO:tasks.workunit.client.0.smithi027.stdout:0/287: mknod d0/df/c54 0 2024-04-26T23:01:27.313 INFO:tasks.workunit.client.0.smithi027.stdout:0/288: truncate d0/d4/da/d1a/f40 1167516 0 2024-04-26T23:01:27.313 INFO:tasks.workunit.client.0.smithi027.stdout:5/304: rename d4/d24/d22/d29/d2a/f3e to d4/f64 0 2024-04-26T23:01:27.313 INFO:tasks.workunit.client.0.smithi027.stdout:5/305: stat d4/d1a 0 2024-04-26T23:01:27.313 INFO:tasks.workunit.client.0.smithi027.stdout:5/306: readlink d4/d24/d22/d29/d2a/d3b/d62/l53 0 2024-04-26T23:01:27.313 INFO:tasks.workunit.client.0.smithi027.stdout:8/343: chown d9/de/d20/f3f 30 1 2024-04-26T23:01:27.314 INFO:tasks.workunit.client.0.smithi027.stdout:8/344: read - d9/da/d2a/d5e/d46/f6f zero size 2024-04-26T23:01:27.314 INFO:tasks.workunit.client.0.smithi027.stdout:8/345: chown d9/de/d2e/c44 16 1 2024-04-26T23:01:27.323 INFO:tasks.workunit.client.0.smithi027.stdout:8/346: link d9/da/d2a/d5e/d46/l61 d9/l83 0 2024-04-26T23:01:27.324 INFO:tasks.workunit.client.0.smithi027.stdout:9/258: symlink d4/d17/d2e/l59 0 2024-04-26T23:01:27.324 INFO:tasks.workunit.client.0.smithi027.stdout:5/307: write d4/f9 [4548254,102753] 0 2024-04-26T23:01:27.328 INFO:tasks.workunit.client.0.smithi027.stdout:6/233: write d1/f33 [3292290,72131] 0 2024-04-26T23:01:27.329 INFO:tasks.workunit.client.0.smithi027.stdout:8/347: rename d9/de/d20 to d9/da/d2a/d5e/d72/d84 0 2024-04-26T23:01:27.331 INFO:tasks.workunit.client.0.smithi027.stdout:6/234: symlink d1/df/d43/d56/l5a 0 2024-04-26T23:01:27.331 INFO:tasks.workunit.client.0.smithi027.stdout:6/235: chown d1/l21 6474473 1 2024-04-26T23:01:27.331 INFO:tasks.workunit.client.0.smithi027.stdout:6/236: chown d1/f4e 51 1 2024-04-26T23:01:27.331 INFO:tasks.workunit.client.0.smithi027.stdout:5/308: rename d4/d1a/d2b/c5b to d4/db/d16/c65 0 2024-04-26T23:01:27.332 INFO:tasks.workunit.client.0.smithi027.stdout:5/309: rmdir d4/d24/d22 39 2024-04-26T23:01:27.332 INFO:tasks.workunit.client.0.smithi027.stdout:8/348: symlink d9/da/d33/d71/l85 0 2024-04-26T23:01:27.337 INFO:tasks.workunit.client.0.smithi027.stdout:8/349: mkdir d9/da/d2a/d5e/d86 0 2024-04-26T23:01:27.340 INFO:tasks.workunit.client.0.smithi027.stdout:8/350: mknod d9/da/d2a/d5e/d72/d84/d4f/c87 0 2024-04-26T23:01:27.344 INFO:tasks.workunit.client.0.smithi027.stdout:8/351: write d9/f17 [4680007,116712] 0 2024-04-26T23:01:27.345 INFO:tasks.workunit.client.0.smithi027.stdout:8/352: chown d9/da/d2a/d5e/f64 1906 1 2024-04-26T23:01:27.345 INFO:tasks.workunit.client.0.smithi027.stdout:0/289: dread d0/d4/da/f2f [0,4194304] 0 2024-04-26T23:01:27.345 INFO:tasks.workunit.client.0.smithi027.stdout:0/290: fdatasync d0/d4/da/d1a/f40 0 2024-04-26T23:01:27.345 INFO:tasks.workunit.client.0.smithi027.stdout:0/291: mkdir d0/d4/da/d15/d55 0 2024-04-26T23:01:27.346 INFO:tasks.workunit.client.0.smithi027.stdout:0/292: mknod d0/d4/da/d15/d26/c56 0 2024-04-26T23:01:27.348 INFO:tasks.workunit.client.0.smithi027.stdout:0/293: symlink d0/l57 0 2024-04-26T23:01:27.350 INFO:tasks.workunit.client.0.smithi027.stdout:5/310: getdents d4/db/d16 0 2024-04-26T23:01:27.350 INFO:tasks.workunit.client.0.smithi027.stdout:5/311: dread - d4/d24/d22/d29/d2a/f45 zero size 2024-04-26T23:01:27.350 INFO:tasks.workunit.client.0.smithi027.stdout:5/312: write d4/f64 [798424,33309] 0 2024-04-26T23:01:27.351 INFO:tasks.workunit.client.0.smithi027.stdout:5/313: write d4/d24/d22/d29/f36 [2216067,16209] 0 2024-04-26T23:01:27.351 INFO:tasks.workunit.client.0.smithi027.stdout:5/314: stat d4/d24/f5d 0 2024-04-26T23:01:27.351 INFO:tasks.workunit.client.0.smithi027.stdout:5/315: stat d4/db/d16 0 2024-04-26T23:01:27.353 INFO:tasks.workunit.client.0.smithi027.stdout:0/294: rename d0/f13 to d0/d4/da/d15/d55/f58 0 2024-04-26T23:01:27.353 INFO:tasks.workunit.client.0.smithi027.stdout:0/295: chown d0/d4 365627689 1 2024-04-26T23:01:27.355 INFO:tasks.workunit.client.0.smithi027.stdout:5/316: link d4/d24/d22/d29/d2a/d3b/d62/d40/l4d d4/d24/d22/l66 0 2024-04-26T23:01:27.357 INFO:tasks.workunit.client.0.smithi027.stdout:4/305: dwrite da/dd/de/d14/d21/d41/f49 [0,4194304] 0 2024-04-26T23:01:27.357 INFO:tasks.workunit.client.0.smithi027.stdout:4/306: chown da/dd/de/d14/d17/f33 34285281 1 2024-04-26T23:01:27.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:26 smithi027 ceph-mon[28034]: pgmap v88: 129 pgs: 129 active+clean; 255 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 18 MiB/s wr, 970 op/s 2024-04-26T23:01:27.365 INFO:tasks.workunit.client.0.smithi027.stdout:4/307: symlink da/dd/de/d14/l55 0 2024-04-26T23:01:27.369 INFO:tasks.workunit.client.0.smithi027.stdout:4/308: rename da/dd/f3b to da/dd/f56 0 2024-04-26T23:01:27.372 INFO:tasks.workunit.client.0.smithi027.stdout:4/309: unlink da/dd/de/f15 0 2024-04-26T23:01:27.376 INFO:tasks.workunit.client.0.smithi027.stdout:2/261: dwrite d2/d5/db/de/d21/d26/f2d [0,4194304] 0 2024-04-26T23:01:27.376 INFO:tasks.workunit.client.0.smithi027.stdout:4/310: symlink da/dd/l57 0 2024-04-26T23:01:27.376 INFO:tasks.workunit.client.0.smithi027.stdout:4/311: chown da/dd/de/d14/l34 7 1 2024-04-26T23:01:27.376 INFO:tasks.workunit.client.0.smithi027.stdout:1/375: dwrite d1/d5/d40/d35/f4c [0,4194304] 0 2024-04-26T23:01:27.377 INFO:tasks.workunit.client.0.smithi027.stdout:1/376: dread - d1/f55 zero size 2024-04-26T23:01:27.379 INFO:tasks.workunit.client.0.smithi027.stdout:2/262: truncate d2/d5/db/d1b/d1c/d25/d19/d36/d39/f3c 2403486 0 2024-04-26T23:01:27.380 INFO:tasks.workunit.client.0.smithi027.stdout:4/312: creat da/dd/de/d14/d17/d24/d37/f58 x:0 0 0 2024-04-26T23:01:27.380 INFO:tasks.workunit.client.0.smithi027.stdout:4/313: write da/f1f [5088605,20941] 0 2024-04-26T23:01:27.382 INFO:tasks.workunit.client.0.smithi027.stdout:4/314: mkdir da/dd/de/d14/d21/d41/d59 0 2024-04-26T23:01:27.390 INFO:tasks.workunit.client.0.smithi027.stdout:8/353: dwrite d9/da/f13 [4194304,4194304] 0 2024-04-26T23:01:27.392 INFO:tasks.workunit.client.0.smithi027.stdout:8/354: creat d9/da/d33/f88 x:0 0 0 2024-04-26T23:01:27.394 INFO:tasks.workunit.client.0.smithi027.stdout:8/355: unlink d9/da/d2a/d5e/d72/d84/d35/l58 0 2024-04-26T23:01:27.394 INFO:tasks.workunit.client.0.smithi027.stdout:8/356: chown d9/da/d33/l67 0 1 2024-04-26T23:01:27.395 INFO:tasks.workunit.client.0.smithi027.stdout:8/357: creat d9/da/d2a/f89 x:0 0 0 2024-04-26T23:01:27.402 INFO:tasks.workunit.client.0.smithi027.stdout:1/377: write d1/d5/d21/f34 [1469644,52781] 0 2024-04-26T23:01:27.402 INFO:tasks.workunit.client.0.smithi027.stdout:1/378: write d1/f3 [2493032,122930] 0 2024-04-26T23:01:27.404 INFO:tasks.workunit.client.0.smithi027.stdout:0/296: dread d0/d4/da/d15/d55/f58 [0,4194304] 0 2024-04-26T23:01:27.406 INFO:tasks.workunit.client.0.smithi027.stdout:6/237: dwrite d1/df/f8 [0,4194304] 0 2024-04-26T23:01:27.406 INFO:tasks.workunit.client.0.smithi027.stdout:6/238: readlink d1/df/l3d 0 2024-04-26T23:01:27.406 INFO:tasks.workunit.client.0.smithi027.stdout:6/239: chown d1 14 1 2024-04-26T23:01:27.406 INFO:tasks.workunit.client.0.smithi027.stdout:4/315: dread da/dd/de/d14/d17/f1d [0,4194304] 0 2024-04-26T23:01:27.407 INFO:tasks.workunit.client.0.smithi027.stdout:4/316: write da/dd/de/f3f [5097771,106431] 0 2024-04-26T23:01:27.407 INFO:tasks.workunit.client.0.smithi027.stdout:4/317: write da/dd/de/d14/f43 [100779,97553] 0 2024-04-26T23:01:27.407 INFO:tasks.workunit.client.0.smithi027.stdout:4/318: fdatasync da/dd/de/d14/d21/d41/f52 0 2024-04-26T23:01:27.409 INFO:tasks.workunit.client.0.smithi027.stdout:6/240: mknod d1/df/d49/c5b 0 2024-04-26T23:01:27.409 INFO:tasks.workunit.client.0.smithi027.stdout:1/379: mkdir d1/d26/d66/d73 0 2024-04-26T23:01:27.409 INFO:tasks.workunit.client.0.smithi027.stdout:1/380: stat d1/d26/d36/c48 0 2024-04-26T23:01:27.410 INFO:tasks.workunit.client.0.smithi027.stdout:4/319: creat da/dd/de/d14/d17/d24/d37/f5a x:0 0 0 2024-04-26T23:01:27.415 INFO:tasks.workunit.client.0.smithi027.stdout:1/381: creat d1/d26/d66/f74 x:0 0 0 2024-04-26T23:01:27.417 INFO:tasks.workunit.client.0.smithi027.stdout:4/320: mknod da/dd/de/d14/d21/d4a/c5b 0 2024-04-26T23:01:27.417 INFO:tasks.workunit.client.0.smithi027.stdout:4/321: write da/dd/de/d14/f53 [310432,58844] 0 2024-04-26T23:01:27.418 INFO:tasks.workunit.client.0.smithi027.stdout:8/358: dread d9/da/d2a/d5e/d72/d84/f3f [0,4194304] 0 2024-04-26T23:01:27.418 INFO:tasks.workunit.client.0.smithi027.stdout:8/359: write d9/da/fd [451650,28025] 0 2024-04-26T23:01:27.418 INFO:tasks.workunit.client.0.smithi027.stdout:4/322: creat da/dd/f5c x:0 0 0 2024-04-26T23:01:27.419 INFO:tasks.workunit.client.0.smithi027.stdout:8/360: creat d9/da/d2a/d5e/d72/f8a x:0 0 0 2024-04-26T23:01:27.419 INFO:tasks.workunit.client.0.smithi027.stdout:8/361: read d9/da/d2a/d5e/d2b/f3d [459434,66829] 0 2024-04-26T23:01:27.421 INFO:tasks.workunit.client.0.smithi027.stdout:8/362: mkdir d9/da/d2a/d5e/d72/d84/d5d/d8b 0 2024-04-26T23:01:27.421 INFO:tasks.workunit.client.0.smithi027.stdout:5/317: dwrite d4/f9 [0,4194304] 0 2024-04-26T23:01:27.423 INFO:tasks.workunit.client.0.smithi027.stdout:5/318: creat d4/db/d16/f67 x:0 0 0 2024-04-26T23:01:27.438 INFO:tasks.workunit.client.0.smithi027.stdout:1/382: dread d1/d5/f19 [0,4194304] 0 2024-04-26T23:01:27.442 INFO:tasks.workunit.client.0.smithi027.stdout:5/319: truncate d4/d24/d22/d29/d2a/d3b/f56 472609 0 2024-04-26T23:01:27.445 INFO:tasks.workunit.client.0.smithi027.stdout:5/320: rename d4/d24/d22/d29/d2a/d3b/d62 to d4/d1a/d4a/d68 0 2024-04-26T23:01:27.447 INFO:tasks.workunit.client.0.smithi027.stdout:2/263: dwrite d2/d5/db/de/f42 [0,4194304] 0 2024-04-26T23:01:27.447 INFO:tasks.workunit.client.0.smithi027.stdout:2/264: read d2/d5/db/d1b/d1c/d25/d19/d36/d39/f3c [849000,57133] 0 2024-04-26T23:01:27.448 INFO:tasks.workunit.client.0.smithi027.stdout:2/265: chown d2/d5/db/de/f41 1 1 2024-04-26T23:01:27.448 INFO:tasks.workunit.client.0.smithi027.stdout:5/321: mknod d4/d24/d22/d29/d39/c69 0 2024-04-26T23:01:27.452 INFO:tasks.workunit.client.0.smithi027.stdout:0/297: dwrite d0/d4/da/d2d/f31 [0,4194304] 0 2024-04-26T23:01:27.452 INFO:tasks.workunit.client.0.smithi027.stdout:0/298: write d0/f37 [2651067,121526] 0 2024-04-26T23:01:27.452 INFO:tasks.workunit.client.0.smithi027.stdout:0/299: fdatasync d0/f1b 0 2024-04-26T23:01:27.458 INFO:tasks.workunit.client.0.smithi027.stdout:2/266: symlink d2/d5/db/d1b/d1c/l48 0 2024-04-26T23:01:27.458 INFO:tasks.workunit.client.0.smithi027.stdout:2/267: fsync d2/d5/db/d1b/d1c/f20 0 2024-04-26T23:01:27.458 INFO:tasks.workunit.client.0.smithi027.stdout:5/322: mknod d4/d24/d22/c6a 0 2024-04-26T23:01:27.460 INFO:tasks.workunit.client.0.smithi027.stdout:0/300: mkdir d0/d4/da/d1a/d59 0 2024-04-26T23:01:27.460 INFO:tasks.workunit.client.0.smithi027.stdout:0/301: chown d0/d4/da/d1a/l3e 1041120819 1 2024-04-26T23:01:27.460 INFO:tasks.workunit.client.0.smithi027.stdout:0/302: fsync d0/d4/da/f1f 0 2024-04-26T23:01:27.460 INFO:tasks.workunit.client.0.smithi027.stdout:0/303: write d0/d4/f3d [1957078,596] 0 2024-04-26T23:01:27.460 INFO:tasks.workunit.client.0.smithi027.stdout:2/268: getdents d2/d5/db/d1b/d1c/d25/d19 0 2024-04-26T23:01:27.461 INFO:tasks.workunit.client.0.smithi027.stdout:7/291: sync 2024-04-26T23:01:27.461 INFO:tasks.workunit.client.0.smithi027.stdout:0/304: creat d0/d4/da/d15/f5a x:0 0 0 2024-04-26T23:01:27.461 INFO:tasks.workunit.client.0.smithi027.stdout:2/269: rmdir d2/d5/db/de/d21/d26 39 2024-04-26T23:01:27.463 INFO:tasks.workunit.client.0.smithi027.stdout:0/305: link d0/d4/da/d15/d26/c56 d0/d4/c5b 0 2024-04-26T23:01:27.463 INFO:tasks.workunit.client.0.smithi027.stdout:0/306: stat d0/c4c 0 2024-04-26T23:01:27.464 INFO:tasks.workunit.client.0.smithi027.stdout:2/270: creat d2/f49 x:0 0 0 2024-04-26T23:01:27.470 INFO:tasks.workunit.client.0.smithi027.stdout:5/323: dread d4/d24/d22/d29/d2a/d3b/f56 [0,4194304] 0 2024-04-26T23:01:27.472 INFO:tasks.workunit.client.0.smithi027.stdout:7/292: symlink d0/l5e 0 2024-04-26T23:01:27.472 INFO:tasks.workunit.client.0.smithi027.stdout:0/307: rmdir d0/d4/da/d1a/d1e 39 2024-04-26T23:01:27.473 INFO:tasks.workunit.client.0.smithi027.stdout:5/324: rename d4/db/c5c to d4/d24/d5f/c6b 0 2024-04-26T23:01:27.474 INFO:tasks.workunit.client.0.smithi027.stdout:4/323: dwrite da/f47 [0,4194304] 0 2024-04-26T23:01:27.474 INFO:tasks.workunit.client.0.smithi027.stdout:4/324: chown da/dd/de/d14/d17/f1d 2889 1 2024-04-26T23:01:27.474 INFO:tasks.workunit.client.0.smithi027.stdout:4/325: fdatasync da/f47 0 2024-04-26T23:01:27.474 INFO:tasks.workunit.client.0.smithi027.stdout:7/293: symlink d0/d28/d2b/d3a/l5f 0 2024-04-26T23:01:27.475 INFO:tasks.workunit.client.0.smithi027.stdout:7/294: dread - d0/d28/d2b/f34 zero size 2024-04-26T23:01:27.476 INFO:tasks.workunit.client.0.smithi027.stdout:0/308: link d0/d4/da/d15/d26/c3a d0/d4/da/d15/d26/c5c 0 2024-04-26T23:01:27.481 INFO:tasks.workunit.client.0.smithi027.stdout:5/325: creat d4/d1a/d4a/d68/d40/f6c x:0 0 0 2024-04-26T23:01:27.481 INFO:tasks.workunit.client.0.smithi027.stdout:5/326: write d4/f4e [770984,2716] 0 2024-04-26T23:01:27.481 INFO:tasks.workunit.client.0.smithi027.stdout:5/327: read - d4/d1a/f57 zero size 2024-04-26T23:01:27.481 INFO:tasks.workunit.client.0.smithi027.stdout:5/328: fdatasync d4/d24/f4c 0 2024-04-26T23:01:27.481 INFO:tasks.workunit.client.0.smithi027.stdout:5/329: fsync d4/d24/f4c 0 2024-04-26T23:01:27.482 INFO:tasks.workunit.client.0.smithi027.stdout:4/326: creat da/dd/f5d x:0 0 0 2024-04-26T23:01:27.482 INFO:tasks.workunit.client.0.smithi027.stdout:4/327: fdatasync da/dd/de/d14/d21/d4a/f4f 0 2024-04-26T23:01:27.482 INFO:tasks.workunit.client.0.smithi027.stdout:7/295: creat d0/d28/f60 x:0 0 0 2024-04-26T23:01:27.483 INFO:tasks.workunit.client.0.smithi027.stdout:1/383: dwrite d1/d5/d40/d35/f4c [4194304,4194304] 0 2024-04-26T23:01:27.483 INFO:tasks.workunit.client.0.smithi027.stdout:1/384: stat d1/dc/f11 0 2024-04-26T23:01:27.484 INFO:tasks.workunit.client.0.smithi027.stdout:0/309: rename d0/d4/da/d15/c17 to d0/d4/da/d1a/d59/c5d 0 2024-04-26T23:01:27.484 INFO:tasks.workunit.client.0.smithi027.stdout:0/310: write d0/d4/da/d2d/f31 [4650028,61925] 0 2024-04-26T23:01:27.487 INFO:tasks.workunit.client.0.smithi027.stdout:1/385: link d1/d5/d21/c61 d1/d26/d36/d56/c75 0 2024-04-26T23:01:27.490 INFO:tasks.workunit.client.0.smithi027.stdout:1/386: creat d1/d5/d40/d2d/f76 x:0 0 0 2024-04-26T23:01:27.501 INFO:tasks.workunit.client.0.smithi027.stdout:4/328: dread da/dd/de/d14/d17/d24/f35 [0,4194304] 0 2024-04-26T23:01:27.505 INFO:tasks.workunit.client.0.smithi027.stdout:4/329: mkdir da/dd/d5e 0 2024-04-26T23:01:27.505 INFO:tasks.workunit.client.0.smithi027.stdout:4/330: fsync da/dd/de/d14/f53 0 2024-04-26T23:01:27.507 INFO:tasks.workunit.client.0.smithi027.stdout:8/363: dwrite d9/da/fd [0,4194304] 0 2024-04-26T23:01:27.507 INFO:tasks.workunit.client.0.smithi027.stdout:8/364: chown d9/da/d2a/d5e/d86 14 1 2024-04-26T23:01:27.509 INFO:tasks.workunit.client.0.smithi027.stdout:1/387: dread d1/d26/f29 [0,4194304] 0 2024-04-26T23:01:27.510 INFO:tasks.workunit.client.0.smithi027.stdout:4/331: getdents da/dd/de/d14/d17/d24 0 2024-04-26T23:01:27.510 INFO:tasks.workunit.client.0.smithi027.stdout:0/311: dread d0/d4/da/f1f [0,4194304] 0 2024-04-26T23:01:27.513 INFO:tasks.workunit.client.0.smithi027.stdout:8/365: getdents d9/de 0 2024-04-26T23:01:27.515 INFO:tasks.workunit.client.0.smithi027.stdout:1/388: creat d1/d26/d66/f77 x:0 0 0 2024-04-26T23:01:27.517 INFO:tasks.workunit.client.0.smithi027.stdout:4/332: mkdir da/dd/d5f 0 2024-04-26T23:01:27.521 INFO:tasks.workunit.client.0.smithi027.stdout:1/389: rename d1/d5/d21/f3e to d1/d5/d40/f78 0 2024-04-26T23:01:27.521 INFO:tasks.workunit.client.0.smithi027.stdout:1/390: fsync d1/f32 0 2024-04-26T23:01:27.523 INFO:tasks.workunit.client.0.smithi027.stdout:4/333: unlink da/dd/de/d14/c3d 0 2024-04-26T23:01:27.523 INFO:tasks.workunit.client.0.smithi027.stdout:4/334: chown f2 17497 1 2024-04-26T23:01:27.523 INFO:tasks.workunit.client.0.smithi027.stdout:1/391: rmdir d1/d26/d6b/d42 39 2024-04-26T23:01:27.524 INFO:tasks.workunit.client.0.smithi027.stdout:4/335: getdents da/dd/de/d14/d21/d41 0 2024-04-26T23:01:27.525 INFO:tasks.workunit.client.0.smithi027.stdout:4/336: write da/dd/de/d14/d21/f46 [869631,119863] 0 2024-04-26T23:01:27.525 INFO:tasks.workunit.client.0.smithi027.stdout:1/392: mkdir d1/d26/d6b/d42/d79 0 2024-04-26T23:01:27.525 INFO:tasks.workunit.client.0.smithi027.stdout:1/393: chown d1/d26/c5d 2850 1 2024-04-26T23:01:27.525 INFO:tasks.workunit.client.0.smithi027.stdout:1/394: readlink d1/d26/d36/l4b 0 2024-04-26T23:01:27.527 INFO:tasks.workunit.client.0.smithi027.stdout:4/337: creat da/d40/f60 x:0 0 0 2024-04-26T23:01:27.527 INFO:tasks.workunit.client.0.smithi027.stdout:4/338: write da/dd/de/d14/d21/d41/f52 [618502,83116] 0 2024-04-26T23:01:27.527 INFO:tasks.workunit.client.0.smithi027.stdout:4/339: getdents da/dd/d5e 0 2024-04-26T23:01:27.527 INFO:tasks.workunit.client.0.smithi027.stdout:1/395: symlink d1/d26/d6b/d42/d70/l7a 0 2024-04-26T23:01:27.528 INFO:tasks.workunit.client.0.smithi027.stdout:1/396: symlink d1/d26/l7b 0 2024-04-26T23:01:27.529 INFO:tasks.workunit.client.0.smithi027.stdout:8/366: dread d9/da/d33/f3a [0,4194304] 0 2024-04-26T23:01:27.529 INFO:tasks.workunit.client.0.smithi027.stdout:8/367: fsync d9/f16 0 2024-04-26T23:01:27.532 INFO:tasks.workunit.client.0.smithi027.stdout:4/340: symlink da/l61 0 2024-04-26T23:01:27.532 INFO:tasks.workunit.client.0.smithi027.stdout:4/341: write da/dd/de/d14/d21/d4a/f4f [573719,54908] 0 2024-04-26T23:01:27.532 INFO:tasks.workunit.client.0.smithi027.stdout:4/342: write da/dd/f30 [1000676,17618] 0 2024-04-26T23:01:27.534 INFO:tasks.workunit.client.0.smithi027.stdout:4/343: mknod da/dd/de/d14/d17/d24/c62 0 2024-04-26T23:01:27.534 INFO:tasks.workunit.client.0.smithi027.stdout:4/344: read - da/dd/de/d14/d17/d24/f26 zero size 2024-04-26T23:01:27.538 INFO:tasks.workunit.client.0.smithi027.stdout:4/345: symlink da/l63 0 2024-04-26T23:01:27.541 INFO:tasks.workunit.client.0.smithi027.stdout:3/363: sync 2024-04-26T23:01:27.541 INFO:tasks.workunit.client.0.smithi027.stdout:3/364: dread - d6/dc/f72 zero size 2024-04-26T23:01:27.546 INFO:tasks.workunit.client.0.smithi027.stdout:2/271: dwrite d2/d5/db/f1a [0,4194304] 0 2024-04-26T23:01:27.546 INFO:tasks.workunit.client.0.smithi027.stdout:2/272: write d2/d5/db/f1a [4435665,100115] 0 2024-04-26T23:01:27.546 INFO:tasks.workunit.client.0.smithi027.stdout:2/273: write d2/d5/db/de/f41 [4350408,83965] 0 2024-04-26T23:01:27.559 INFO:tasks.workunit.client.0.smithi027.stdout:0/312: write d0/f12 [6498664,43265] 0 2024-04-26T23:01:27.559 INFO:tasks.workunit.client.0.smithi027.stdout:0/313: stat d0/d4/f7 0 2024-04-26T23:01:27.559 INFO:tasks.workunit.client.0.smithi027.stdout:0/314: write d0/d4/f3d [4134407,41370] 0 2024-04-26T23:01:27.559 INFO:tasks.workunit.client.0.smithi027.stdout:7/296: dwrite d0/d28/f37 [0,4194304] 0 2024-04-26T23:01:27.563 INFO:tasks.workunit.client.0.smithi027.stdout:3/365: write d6/dc/d34/d2e/f31 [967410,84581] 0 2024-04-26T23:01:27.568 INFO:tasks.workunit.client.0.smithi027.stdout:0/315: creat d0/d4/da/d2d/f5e x:0 0 0 2024-04-26T23:01:27.568 INFO:tasks.workunit.client.0.smithi027.stdout:0/316: write d0/d4/da/d15/d55/f58 [1018978,32368] 0 2024-04-26T23:01:27.569 INFO:tasks.workunit.client.0.smithi027.stdout:3/366: unlink d6/dc/d62/f79 0 2024-04-26T23:01:27.572 INFO:tasks.workunit.client.0.smithi027.stdout:1/397: dread d1/d5/d21/f34 [0,4194304] 0 2024-04-26T23:01:27.576 INFO:tasks.workunit.client.0.smithi027.stdout:2/274: read d2/d5/db/d1b/d1c/f20 [83486,40167] 0 2024-04-26T23:01:27.576 INFO:tasks.workunit.client.0.smithi027.stdout:2/275: chown d2/d5/db/de/d21/f16 110 1 2024-04-26T23:01:27.576 INFO:tasks.workunit.client.0.smithi027.stdout:5/330: dwrite d4/d1a/f1e [0,4194304] 0 2024-04-26T23:01:27.578 INFO:tasks.workunit.client.0.smithi027.stdout:3/367: creat d6/dc/d17/d52/f86 x:0 0 0 2024-04-26T23:01:27.578 INFO:tasks.workunit.client.0.smithi027.stdout:3/368: fsync d6/f82 0 2024-04-26T23:01:27.580 INFO:tasks.workunit.client.0.smithi027.stdout:1/398: truncate d1/dc/f14 358259 0 2024-04-26T23:01:27.580 INFO:tasks.workunit.client.0.smithi027.stdout:1/399: truncate d1/dc/f1d 5497667 0 2024-04-26T23:01:27.583 INFO:tasks.workunit.client.0.smithi027.stdout:2/276: rename d2/d5/db/de/d21/d26/f2b to d2/d5/db/d1b/d1c/d25/f4a 0 2024-04-26T23:01:27.584 INFO:tasks.workunit.client.0.smithi027.stdout:5/331: creat d4/d24/d22/d29/d39/f6d x:0 0 0 2024-04-26T23:01:27.584 INFO:tasks.workunit.client.0.smithi027.stdout:5/332: read - d4/d24/f4c zero size 2024-04-26T23:01:27.584 INFO:tasks.workunit.client.0.smithi027.stdout:5/333: dread - d4/d1a/f57 zero size 2024-04-26T23:01:27.586 INFO:tasks.workunit.client.0.smithi027.stdout:5/334: rename d4/d1a/d4a/d68/l60 to d4/d1a/d4a/d68/d40/l6e 0 2024-04-26T23:01:27.587 INFO:tasks.workunit.client.0.smithi027.stdout:7/297: dread d0/d2f/f31 [0,4194304] 0 2024-04-26T23:01:27.588 INFO:tasks.workunit.client.0.smithi027.stdout:5/335: rmdir d4/d1a/d4a/d68 39 2024-04-26T23:01:27.589 INFO:tasks.workunit.client.0.smithi027.stdout:7/298: rename d0/d28/d14/l50 to d0/d28/d14/l61 0 2024-04-26T23:01:27.590 INFO:tasks.workunit.client.0.smithi027.stdout:5/336: mknod d4/d24/d22/d29/d2a/d3b/c6f 0 2024-04-26T23:01:27.592 INFO:tasks.workunit.client.0.smithi027.stdout:7/299: creat d0/f62 x:0 0 0 2024-04-26T23:01:27.592 INFO:tasks.workunit.client.0.smithi027.stdout:7/300: write d0/d28/d2b/f34 [450192,59690] 0 2024-04-26T23:01:27.592 INFO:tasks.workunit.client.0.smithi027.stdout:5/337: link d4/d1a/d4a/d68/d40/f6c d4/d24/d22/d29/d2a/d3b/d50/f70 0 2024-04-26T23:01:27.594 INFO:tasks.workunit.client.0.smithi027.stdout:4/346: dwrite da/dd/de/d14/d21/f46 [0,4194304] 0 2024-04-26T23:01:27.594 INFO:tasks.workunit.client.0.smithi027.stdout:5/338: rename d4/d24/l19 to d4/d24/d22/d29/d2a/d3b/d50/l71 0 2024-04-26T23:01:27.598 INFO:tasks.workunit.client.0.smithi027.stdout:4/347: creat da/dd/de/d14/d21/d41/f64 x:0 0 0 2024-04-26T23:01:27.601 INFO:tasks.workunit.client.0.smithi027.stdout:5/339: link d4/db/l44 d4/d24/d22/d29/d2a/l72 0 2024-04-26T23:01:27.602 INFO:tasks.workunit.client.0.smithi027.stdout:4/348: creat da/dd/de/d14/d17/f65 x:0 0 0 2024-04-26T23:01:27.605 INFO:tasks.workunit.client.0.smithi027.stdout:5/340: unlink d4/d24/d22/l66 0 2024-04-26T23:01:27.605 INFO:tasks.workunit.client.0.smithi027.stdout:5/341: readlink d4/d24/d22/d29/d2a/d3b/d50/l71 0 2024-04-26T23:01:27.605 INFO:tasks.workunit.client.0.smithi027.stdout:5/342: fsync d4/f15 0 2024-04-26T23:01:27.605 INFO:tasks.workunit.client.0.smithi027.stdout:3/369: read d6/dc/d17/f33 [1862767,77269] 0 2024-04-26T23:01:27.607 INFO:tasks.workunit.client.0.smithi027.stdout:5/343: truncate d4/f64 210346 0 2024-04-26T23:01:27.607 INFO:tasks.workunit.client.0.smithi027.stdout:5/344: fdatasync d4/d24/d22/d29/d2a/d3b/f56 0 2024-04-26T23:01:27.609 INFO:tasks.workunit.client.0.smithi027.stdout:5/345: creat d4/d1a/d4a/d68/d40/d54/f73 x:0 0 0 2024-04-26T23:01:27.614 INFO:tasks.workunit.client.0.smithi027.stdout:5/346: mknod d4/d1a/d4a/d68/d40/c74 0 2024-04-26T23:01:27.614 INFO:tasks.workunit.client.0.smithi027.stdout:5/347: write d4/db/d16/f1f [3517604,83850] 0 2024-04-26T23:01:27.620 INFO:tasks.workunit.client.0.smithi027.stdout:7/301: dread d0/d28/d2b/d3a/f3f [0,4194304] 0 2024-04-26T23:01:27.627 INFO:tasks.workunit.client.0.smithi027.stdout:2/277: dwrite d2/d5/db/f1a [0,4194304] 0 2024-04-26T23:01:27.627 INFO:tasks.workunit.client.0.smithi027.stdout:2/278: chown d2/d5/db/f14 15 1 2024-04-26T23:01:27.633 INFO:tasks.workunit.client.0.smithi027.stdout:1/400: dwrite d1/dc/f16 [0,4194304] 0 2024-04-26T23:01:27.634 INFO:tasks.workunit.client.0.smithi027.stdout:1/401: stat d1/d5/d40/d2d/l33 0 2024-04-26T23:01:27.636 INFO:tasks.workunit.client.0.smithi027.stdout:2/279: getdents d2/d5/db/d1b/d1c 0 2024-04-26T23:01:27.638 INFO:tasks.workunit.client.0.smithi027.stdout:1/402: getdents d1/d26/d66 0 2024-04-26T23:01:27.640 INFO:tasks.workunit.client.0.smithi027.stdout:1/403: rmdir d1 39 2024-04-26T23:01:27.641 INFO:tasks.workunit.client.0.smithi027.stdout:1/404: creat d1/d26/d66/d73/f7c x:0 0 0 2024-04-26T23:01:27.641 INFO:tasks.workunit.client.0.smithi027.stdout:1/405: chown d1/d26/d36 26921571 1 2024-04-26T23:01:27.641 INFO:tasks.workunit.client.0.smithi027.stdout:1/406: fdatasync d1/d5/d40/d35/f4c 0 2024-04-26T23:01:27.643 INFO:tasks.workunit.client.0.smithi027.stdout:3/370: dwrite d6/dc/d34/d2e/f67 [0,4194304] 0 2024-04-26T23:01:27.645 INFO:tasks.workunit.client.0.smithi027.stdout:3/371: unlink d6/dc/d2f/f6a 0 2024-04-26T23:01:27.647 INFO:tasks.workunit.client.0.smithi027.stdout:9/259: sync 2024-04-26T23:01:27.650 INFO:tasks.workunit.client.0.smithi027.stdout:3/372: write d6/f18 [71862,38367] 0 2024-04-26T23:01:27.650 INFO:tasks.workunit.client.0.smithi027.stdout:9/260: truncate d4/d10/f1a 4545360 0 2024-04-26T23:01:27.655 INFO:tasks.workunit.client.0.smithi027.stdout:1/407: rename d1/d5/d40/d2d to d1/d26/d7d 0 2024-04-26T23:01:27.657 INFO:tasks.workunit.client.0.smithi027.stdout:9/261: rename d4/d10/f12 to d4/d10/d40/d58/f5a 0 2024-04-26T23:01:27.659 INFO:tasks.workunit.client.0.smithi027.stdout:9/262: symlink d4/d10/d40/d58/l5b 0 2024-04-26T23:01:27.660 INFO:tasks.workunit.client.0.smithi027.stdout:7/302: dwrite d0/d28/f18 [0,4194304] 0 2024-04-26T23:01:27.661 INFO:tasks.workunit.client.0.smithi027.stdout:9/263: creat d4/d10/d2c/f5c x:0 0 0 2024-04-26T23:01:27.662 INFO:tasks.workunit.client.0.smithi027.stdout:4/349: dwrite da/dd/de/d14/d17/d24/f16 [0,4194304] 0 2024-04-26T23:01:27.662 INFO:tasks.workunit.client.0.smithi027.stdout:4/350: fdatasync da/dd/de/d14/d21/d41/f49 0 2024-04-26T23:01:27.663 INFO:tasks.workunit.client.0.smithi027.stdout:7/303: unlink d0/d28/lb 0 2024-04-26T23:01:27.664 INFO:tasks.workunit.client.0.smithi027.stdout:9/264: creat d4/d10/d40/f5d x:0 0 0 2024-04-26T23:01:27.668 INFO:tasks.workunit.client.0.smithi027.stdout:4/351: creat da/d40/f66 x:0 0 0 2024-04-26T23:01:27.668 INFO:tasks.workunit.client.0.smithi027.stdout:4/352: write da/dd/de/d14/d17/d24/f3e [656397,77638] 0 2024-04-26T23:01:27.668 INFO:tasks.workunit.client.0.smithi027.stdout:7/304: creat d0/d28/d2b/d3a/d48/f63 x:0 0 0 2024-04-26T23:01:27.669 INFO:tasks.workunit.client.0.smithi027.stdout:7/305: rmdir d0/d28/d14 39 2024-04-26T23:01:27.671 INFO:tasks.workunit.client.0.smithi027.stdout:7/306: creat d0/d2f/d4d/f64 x:0 0 0 2024-04-26T23:01:27.674 INFO:tasks.workunit.client.0.smithi027.stdout:7/307: rmdir d0/d28 39 2024-04-26T23:01:27.674 INFO:tasks.workunit.client.0.smithi027.stdout:4/353: write da/dd/de/d14/d17/f20 [1214403,101152] 0 2024-04-26T23:01:27.674 INFO:tasks.workunit.client.0.smithi027.stdout:7/308: fsync d0/f42 0 2024-04-26T23:01:27.675 INFO:tasks.workunit.client.0.smithi027.stdout:4/354: mkdir da/dd/de/d14/d21/d67 0 2024-04-26T23:01:27.675 INFO:tasks.workunit.client.0.smithi027.stdout:4/355: write da/dd/de/f3f [1823759,90406] 0 2024-04-26T23:01:27.679 INFO:tasks.workunit.client.0.smithi027.stdout:7/309: rmdir d0/d28/d2b/d55 39 2024-04-26T23:01:27.680 INFO:tasks.workunit.client.0.smithi027.stdout:5/348: dwrite d4/d1a/f57 [0,4194304] 0 2024-04-26T23:01:27.684 INFO:tasks.workunit.client.0.smithi027.stdout:5/349: unlink d4/db/d16/c26 0 2024-04-26T23:01:27.687 INFO:tasks.workunit.client.0.smithi027.stdout:2/280: write d2/d5/db/d1b/d1c/d25/d19/f45 [2526541,91297] 0 2024-04-26T23:01:27.687 INFO:tasks.workunit.client.0.smithi027.stdout:2/281: chown d2/d5/db/c40 0 1 2024-04-26T23:01:27.687 INFO:tasks.workunit.client.0.smithi027.stdout:2/282: dread - d2/d5/db/d1b/d1c/d25/d19/d23/f47 zero size 2024-04-26T23:01:27.689 INFO:tasks.workunit.client.0.smithi027.stdout:2/283: mkdir d2/d5/db/d1b/d1c/d25/d19/d44/d4b 0 2024-04-26T23:01:27.699 INFO:tasks.workunit.client.0.smithi027.stdout:2/284: creat d2/d5/db/d1b/d1c/d25/d2f/f4c x:0 0 0 2024-04-26T23:01:27.699 INFO:tasks.workunit.client.0.smithi027.stdout:2/285: dread - d2/d5/db/d1b/d1c/d25/d2f/f4c zero size 2024-04-26T23:01:27.699 INFO:tasks.workunit.client.0.smithi027.stdout:6/241: sync 2024-04-26T23:01:27.699 INFO:tasks.workunit.client.0.smithi027.stdout:8/368: sync 2024-04-26T23:01:27.699 INFO:tasks.workunit.client.0.smithi027.stdout:8/369: write d9/de/f18 [5033608,85464] 0 2024-04-26T23:01:27.708 INFO:tasks.workunit.client.0.smithi027.stdout:2/286: dread d2/d5/db/de/f30 [0,4194304] 0 2024-04-26T23:01:27.710 INFO:tasks.workunit.client.0.smithi027.stdout:1/408: dread d1/d5/d21/f34 [0,4194304] 0 2024-04-26T23:01:27.710 INFO:tasks.workunit.client.0.smithi027.stdout:1/409: write d1/d5/d21/f34 [5238255,43427] 0 2024-04-26T23:01:27.712 INFO:tasks.workunit.client.0.smithi027.stdout:1/410: symlink d1/d26/d6b/d42/d70/l7e 0 2024-04-26T23:01:27.712 INFO:tasks.workunit.client.0.smithi027.stdout:7/310: write d0/d2f/f31 [261233,32920] 0 2024-04-26T23:01:27.714 INFO:tasks.workunit.client.0.smithi027.stdout:1/411: truncate d1/d5/d21/f27 3984940 0 2024-04-26T23:01:27.715 INFO:tasks.workunit.client.0.smithi027.stdout:1/412: creat d1/d26/d6b/f7f x:0 0 0 2024-04-26T23:01:27.717 INFO:tasks.workunit.client.0.smithi027.stdout:9/265: dwrite d4/d17/f1d [4194304,4194304] 0 2024-04-26T23:01:27.720 INFO:tasks.workunit.client.0.smithi027.stdout:9/266: unlink d4/d17/d2e/l3a 0 2024-04-26T23:01:27.721 INFO:tasks.workunit.client.0.smithi027.stdout:7/311: readlink d0/l1 0 2024-04-26T23:01:27.721 INFO:tasks.workunit.client.0.smithi027.stdout:7/312: readlink d0/d28/d2b/l4a 0 2024-04-26T23:01:27.724 INFO:tasks.workunit.client.0.smithi027.stdout:8/370: write d9/da/d2a/d5e/d72/d84/f40 [1436448,97001] 0 2024-04-26T23:01:27.725 INFO:tasks.workunit.client.0.smithi027.stdout:7/313: truncate d0/f13 5771510 0 2024-04-26T23:01:27.729 INFO:tasks.workunit.client.0.smithi027.stdout:8/371: truncate d9/de/d22/f4d 4131932 0 2024-04-26T23:01:27.731 INFO:tasks.workunit.client.0.smithi027.stdout:8/372: creat d9/da/d2a/d5e/d86/f8c x:0 0 0 2024-04-26T23:01:27.736 INFO:tasks.workunit.client.0.smithi027.stdout:3/373: dwrite d6/dc/d2f/f39 [0,4194304] 0 2024-04-26T23:01:27.736 INFO:tasks.workunit.client.0.smithi027.stdout:3/374: stat d6/dc/d34/d47/l43 0 2024-04-26T23:01:27.736 INFO:tasks.workunit.client.0.smithi027.stdout:3/375: readlink d6/dc/l14 0 2024-04-26T23:01:27.736 INFO:tasks.workunit.client.0.smithi027.stdout:8/373: mkdir d9/da/d2a/d5e/d72/d84/d35/d8d 0 2024-04-26T23:01:27.736 INFO:tasks.workunit.client.0.smithi027.stdout:8/374: stat d9/da/d2a 0 2024-04-26T23:01:27.738 INFO:tasks.workunit.client.0.smithi027.stdout:6/242: dread d1/f27 [0,4194304] 0 2024-04-26T23:01:27.738 INFO:tasks.workunit.client.0.smithi027.stdout:6/243: read - d1/de/f42 zero size 2024-04-26T23:01:27.742 INFO:tasks.workunit.client.0.smithi027.stdout:6/244: link d1/c32 d1/de/d57/c5c 0 2024-04-26T23:01:27.747 INFO:tasks.workunit.client.0.smithi027.stdout:0/317: sync 2024-04-26T23:01:27.749 INFO:tasks.workunit.client.0.smithi027.stdout:0/318: write d0/d4/da/d15/d55/f58 [1359705,60703] 0 2024-04-26T23:01:27.751 INFO:tasks.workunit.client.0.smithi027.stdout:7/314: dread d0/d28/d2b/d3a/f3f [0,4194304] 0 2024-04-26T23:01:27.754 INFO:tasks.workunit.client.0.smithi027.stdout:0/319: symlink d0/d4/da/d1a/l5f 0 2024-04-26T23:01:27.758 INFO:tasks.workunit.client.0.smithi027.stdout:7/315: chown d0/l23 3294 1 2024-04-26T23:01:27.771 INFO:tasks.workunit.client.0.smithi027.stdout:4/356: dwrite da/dd/f5d [0,4194304] 0 2024-04-26T23:01:27.774 INFO:tasks.workunit.client.0.smithi027.stdout:2/287: dwrite d2/d5/db/de/d21/f16 [0,4194304] 0 2024-04-26T23:01:27.780 INFO:tasks.workunit.client.0.smithi027.stdout:2/288: mknod d2/d5/db/de/d21/d26/c4d 0 2024-04-26T23:01:27.780 INFO:tasks.workunit.client.0.smithi027.stdout:2/289: read - d2/d5/db/d1b/d1c/d25/d19/d23/f47 zero size 2024-04-26T23:01:27.782 INFO:tasks.workunit.client.0.smithi027.stdout:8/375: dwrite d9/da/d33/f60 [0,4194304] 0 2024-04-26T23:01:27.782 INFO:tasks.workunit.client.0.smithi027.stdout:8/376: stat d9/da/d2a/d5e/f64 0 2024-04-26T23:01:27.786 INFO:tasks.workunit.client.0.smithi027.stdout:5/350: dwrite d4/d1a/f1e [8388608,4194304] 0 2024-04-26T23:01:27.786 INFO:tasks.workunit.client.0.smithi027.stdout:0/320: dread d0/d4/da/f14 [0,4194304] 0 2024-04-26T23:01:27.788 INFO:tasks.workunit.client.0.smithi027.stdout:9/267: dwrite d4/d10/d2c/f41 [0,4194304] 0 2024-04-26T23:01:27.788 INFO:tasks.workunit.client.0.smithi027.stdout:8/377: link d9/da/d2a/d5e/d72/d84/l5c d9/da/d2a/l8e 0 2024-04-26T23:01:27.788 INFO:tasks.workunit.client.0.smithi027.stdout:8/378: chown d9/da/d33/c5b 175556 1 2024-04-26T23:01:27.788 INFO:tasks.workunit.client.0.smithi027.stdout:8/379: chown d9/de/f2c 231 1 2024-04-26T23:01:27.789 INFO:tasks.workunit.client.0.smithi027.stdout:8/380: chown d9/de/d2e/c1d 0 1 2024-04-26T23:01:27.789 INFO:tasks.workunit.client.0.smithi027.stdout:3/376: dwrite d6/dc/d34/d2e/f67 [0,4194304] 0 2024-04-26T23:01:27.789 INFO:tasks.workunit.client.0.smithi027.stdout:5/351: creat d4/d1a/d4a/d68/d40/d54/f75 x:0 0 0 2024-04-26T23:01:27.790 INFO:tasks.workunit.client.0.smithi027.stdout:0/321: creat d0/df/d3f/d4a/f60 x:0 0 0 2024-04-26T23:01:27.793 INFO:tasks.workunit.client.0.smithi027.stdout:9/268: rename d4/d10/d2c/f5c to d4/d10/f5e 0 2024-04-26T23:01:27.794 INFO:tasks.workunit.client.0.smithi027.stdout:3/377: getdents d6/dc/d17 0 2024-04-26T23:01:27.795 INFO:tasks.workunit.client.0.smithi027.stdout:5/352: getdents d4/d24/d55 0 2024-04-26T23:01:27.795 INFO:tasks.workunit.client.0.smithi027.stdout:0/322: getdents d0 0 2024-04-26T23:01:27.799 INFO:tasks.workunit.client.0.smithi027.stdout:2/290: write d2/d5/db/de/d21/f16 [1988915,111652] 0 2024-04-26T23:01:27.800 INFO:tasks.workunit.client.0.smithi027.stdout:7/316: dread d0/d28/d2b/d3a/f4c [0,4194304] 0 2024-04-26T23:01:27.801 INFO:tasks.workunit.client.0.smithi027.stdout:9/269: creat d4/d17/d2e/d46/f5f x:0 0 0 2024-04-26T23:01:27.801 INFO:tasks.workunit.client.0.smithi027.stdout:5/353: rename d4/d24/d22/d29/d2a/d3b/f56 to d4/d1a/d4a/f76 0 2024-04-26T23:01:27.801 INFO:tasks.workunit.client.0.smithi027.stdout:3/378: truncate d6/dc/d34/d47/f65 3715288 0 2024-04-26T23:01:27.801 INFO:tasks.workunit.client.0.smithi027.stdout:4/357: dread da/dd/de/d14/d17/d24/f3a [0,4194304] 0 2024-04-26T23:01:27.803 INFO:tasks.workunit.client.0.smithi027.stdout:2/291: rename d2/d5/db/d1b/d1c/d25/d2f/c3b to d2/d5/db/d1b/d1c/d25/d19/d44/d4b/c4e 0 2024-04-26T23:01:27.803 INFO:tasks.workunit.client.0.smithi027.stdout:2/292: chown d2/d5/db/de/d21/d26/c3a 30 1 2024-04-26T23:01:27.805 INFO:tasks.workunit.client.0.smithi027.stdout:4/358: link da/dd/de/d14/d21/f23 da/dd/de/d14/d17/f68 0 2024-04-26T23:01:27.806 INFO:tasks.workunit.client.0.smithi027.stdout:4/359: write da/dd/de/f3c [795652,58603] 0 2024-04-26T23:01:27.810 INFO:tasks.workunit.client.0.smithi027.stdout:0/323: dread d0/d4/f7 [0,4194304] 0 2024-04-26T23:01:27.810 INFO:tasks.workunit.client.0.smithi027.stdout:0/324: chown d0/f12 103995 1 2024-04-26T23:01:27.822 INFO:tasks.workunit.client.0.smithi027.stdout:7/317: write d0/d28/d2b/d3a/f4c [976829,18277] 0 2024-04-26T23:01:27.823 INFO:tasks.workunit.client.0.smithi027.stdout:0/325: creat d0/df/d3f/f61 x:0 0 0 2024-04-26T23:01:27.841 INFO:tasks.workunit.client.0.smithi027.stdout:5/354: dwrite d4/d24/d22/d29/f33 [0,4194304] 0 2024-04-26T23:01:27.841 INFO:tasks.workunit.client.0.smithi027.stdout:4/360: dread da/dd/de/d14/d21/f23 [0,4194304] 0 2024-04-26T23:01:27.849 INFO:tasks.workunit.client.0.smithi027.stdout:5/355: rmdir d4/d24/d22/d29/d2a/d3b 39 2024-04-26T23:01:27.849 INFO:tasks.workunit.client.0.smithi027.stdout:5/356: write d4/d1a/d4a/d68/d40/d54/f75 [1042210,31978] 0 2024-04-26T23:01:27.850 INFO:tasks.workunit.client.0.smithi027.stdout:5/357: link d4/d24/f42 d4/db/d16/d4b/f77 0 2024-04-26T23:01:27.855 INFO:tasks.workunit.client.0.smithi027.stdout:2/293: dwrite d2/d5/db/de/f30 [0,4194304] 0 2024-04-26T23:01:27.855 INFO:tasks.workunit.client.0.smithi027.stdout:4/361: truncate da/dd/de/d14/d21/f46 3358753 0 2024-04-26T23:01:27.855 INFO:tasks.workunit.client.0.smithi027.stdout:4/362: getdents da/dd/d5f 0 2024-04-26T23:01:27.861 INFO:tasks.workunit.client.0.smithi027.stdout:5/358: getdents d4/d1a/d2b 0 2024-04-26T23:01:27.861 INFO:tasks.workunit.client.0.smithi027.stdout:5/359: write d4/d24/d22/d29/d2a/d3b/d50/f70 [338839,48998] 0 2024-04-26T23:01:27.861 INFO:tasks.workunit.client.0.smithi027.stdout:5/360: chown d4/d24/d22/d29/d39/l52 1085749 1 2024-04-26T23:01:27.861 INFO:tasks.workunit.client.0.smithi027.stdout:5/361: readlink d4/db/l2f 0 2024-04-26T23:01:27.861 INFO:tasks.workunit.client.0.smithi027.stdout:6/245: dwrite d1/df/f44 [0,4194304] 0 2024-04-26T23:01:27.865 INFO:tasks.workunit.client.0.smithi027.stdout:6/246: creat d1/df/d43/f5d x:0 0 0 2024-04-26T23:01:27.866 INFO:tasks.workunit.client.0.smithi027.stdout:8/381: dwrite d9/da/d2a/f89 [0,4194304] 0 2024-04-26T23:01:27.867 INFO:tasks.workunit.client.0.smithi027.stdout:8/382: mknod d9/de/d2e/c8f 0 2024-04-26T23:01:27.872 INFO:tasks.workunit.client.0.smithi027.stdout:8/383: mknod d9/da/d2a/d5e/d72/d84/c90 0 2024-04-26T23:01:27.872 INFO:tasks.workunit.client.0.smithi027.stdout:8/384: readlink d9/de/d22/l27 0 2024-04-26T23:01:27.873 INFO:tasks.workunit.client.0.smithi027.stdout:8/385: symlink d9/da/l91 0 2024-04-26T23:01:27.874 INFO:tasks.workunit.client.0.smithi027.stdout:8/386: mkdir d9/da/d2a/d5e/d92 0 2024-04-26T23:01:27.878 INFO:tasks.workunit.client.0.smithi027.stdout:0/326: dwrite d0/d4/f20 [4194304,4194304] 0 2024-04-26T23:01:27.892 INFO:tasks.workunit.client.0.smithi027.stdout:8/387: dread d9/da/d2a/d5e/f57 [0,4194304] 0 2024-04-26T23:01:27.894 INFO:tasks.workunit.client.0.smithi027.stdout:4/363: dread da/dd/f56 [0,4194304] 0 2024-04-26T23:01:27.896 INFO:tasks.workunit.client.0.smithi027.stdout:9/270: dwrite d4/d17/d2e/f31 [0,4194304] 0 2024-04-26T23:01:27.896 INFO:tasks.workunit.client.0.smithi027.stdout:9/271: read - d4/d10/f4a zero size 2024-04-26T23:01:27.908 INFO:tasks.workunit.client.0.smithi027.stdout:5/362: dread d4/d1a/f20 [0,4194304] 0 2024-04-26T23:01:27.908 INFO:tasks.workunit.client.0.smithi027.stdout:5/363: chown d4/d1a/d2b 1460188 1 2024-04-26T23:01:27.908 INFO:tasks.workunit.client.0.smithi027.stdout:4/364: symlink da/dd/de/d14/d17/l69 0 2024-04-26T23:01:27.908 INFO:tasks.workunit.client.0.smithi027.stdout:4/365: chown da/dd/de/d14/d17/f68 199 1 2024-04-26T23:01:27.910 INFO:tasks.workunit.client.0.smithi027.stdout:5/364: unlink d4/c6 0 2024-04-26T23:01:27.917 INFO:tasks.workunit.client.0.smithi027.stdout:7/318: dwrite d0/d2f/d4d/f64 [0,4194304] 0 2024-04-26T23:01:27.919 INFO:tasks.workunit.client.0.smithi027.stdout:7/319: creat d0/d2f/f65 x:0 0 0 2024-04-26T23:01:27.919 INFO:tasks.workunit.client.0.smithi027.stdout:7/320: stat d0/d28/d2b/l4a 0 2024-04-26T23:01:27.920 INFO:tasks.workunit.client.0.smithi027.stdout:7/321: mkdir d0/d28/d66 0 2024-04-26T23:01:27.923 INFO:tasks.workunit.client.0.smithi027.stdout:2/294: dwrite d2/d5/db/de/d21/f16 [0,4194304] 0 2024-04-26T23:01:27.923 INFO:tasks.workunit.client.0.smithi027.stdout:7/322: dread - d0/f5a zero size 2024-04-26T23:01:27.924 INFO:tasks.workunit.client.0.smithi027.stdout:7/323: write d0/d3d/f44 [446971,26144] 0 2024-04-26T23:01:27.924 INFO:tasks.workunit.client.0.smithi027.stdout:7/324: dread - d0/d28/d2b/d3a/d48/d56/f5b zero size 2024-04-26T23:01:27.925 INFO:tasks.workunit.client.0.smithi027.stdout:7/325: fdatasync d0/f30 0 2024-04-26T23:01:27.928 INFO:tasks.workunit.client.0.smithi027.stdout:7/326: unlink d0/l23 0 2024-04-26T23:01:27.930 INFO:tasks.workunit.client.0.smithi027.stdout:2/295: link d2/d5/db/d1b/d1c/d25/d19/d44/d4b/c4e d2/c4f 0 2024-04-26T23:01:27.934 INFO:tasks.workunit.client.0.smithi027.stdout:2/296: unlink d2/d5/db/f14 0 2024-04-26T23:01:27.934 INFO:tasks.workunit.client.0.smithi027.stdout:2/297: fsync d2/d5/db/d1b/d1c/d25/f4a 0 2024-04-26T23:01:27.938 INFO:tasks.workunit.client.0.smithi027.stdout:2/298: symlink d2/d5/db/d1b/d1c/d25/d19/d23/l50 0 2024-04-26T23:01:27.938 INFO:tasks.workunit.client.0.smithi027.stdout:7/327: read d0/d28/d2b/f34 [293773,29796] 0 2024-04-26T23:01:27.938 INFO:tasks.workunit.client.0.smithi027.stdout:0/327: dread d0/d4/da/f14 [0,4194304] 0 2024-04-26T23:01:27.939 INFO:tasks.workunit.client.0.smithi027.stdout:0/328: getdents d0/d4/da/d1a/d59 0 2024-04-26T23:01:27.945 INFO:tasks.workunit.client.0.smithi027.stdout:5/365: dread d4/db/f10 [4194304,4194304] 0 2024-04-26T23:01:27.945 INFO:tasks.workunit.client.0.smithi027.stdout:5/366: stat d4/d1a/d4a 0 2024-04-26T23:01:27.947 INFO:tasks.workunit.client.0.smithi027.stdout:5/367: unlink d4/d1a/d4a/d68/l53 0 2024-04-26T23:01:27.947 INFO:tasks.workunit.client.0.smithi027.stdout:9/272: dwrite d4/d10/d25/f39 [0,4194304] 0 2024-04-26T23:01:27.947 INFO:tasks.workunit.client.0.smithi027.stdout:9/273: truncate d4/f52 395159 0 2024-04-26T23:01:27.949 INFO:tasks.workunit.client.0.smithi027.stdout:9/274: symlink d4/d17/d2e/d46/l60 0 2024-04-26T23:01:27.949 INFO:tasks.workunit.client.0.smithi027.stdout:5/368: stat d4/d24/d22/d29/d2a/d3b/d50/f70 0 2024-04-26T23:01:27.950 INFO:tasks.workunit.client.0.smithi027.stdout:6/247: dwrite d1/de/f1a [0,4194304] 0 2024-04-26T23:01:27.951 INFO:tasks.workunit.client.0.smithi027.stdout:7/328: dread d0/f30 [0,4194304] 0 2024-04-26T23:01:27.958 INFO:tasks.workunit.client.0.smithi027.stdout:6/248: unlink d1/df/l24 0 2024-04-26T23:01:27.958 INFO:tasks.workunit.client.0.smithi027.stdout:6/249: dread - d1/df/f3a zero size 2024-04-26T23:01:27.959 INFO:tasks.workunit.client.0.smithi027.stdout:6/250: write d1/df/fd [1910034,58770] 0 2024-04-26T23:01:27.959 INFO:tasks.workunit.client.0.smithi027.stdout:7/329: mkdir d0/d28/d66/d67 0 2024-04-26T23:01:27.959 INFO:tasks.workunit.client.0.smithi027.stdout:7/330: readlink d0/d2f/l45 0 2024-04-26T23:01:27.959 INFO:tasks.workunit.client.0.smithi027.stdout:7/331: chown d0/d3d/c57 15412 1 2024-04-26T23:01:27.960 INFO:tasks.workunit.client.0.smithi027.stdout:6/251: symlink d1/d30/d40/l5e 0 2024-04-26T23:01:27.962 INFO:tasks.workunit.client.0.smithi027.stdout:7/332: mkdir d0/d28/d2b/d3a/d48/d68 0 2024-04-26T23:01:27.963 INFO:tasks.workunit.client.0.smithi027.stdout:3/379: sync 2024-04-26T23:01:27.964 INFO:tasks.workunit.client.0.smithi027.stdout:1/413: sync 2024-04-26T23:01:27.964 INFO:tasks.workunit.client.0.smithi027.stdout:1/414: chown d1/c28 2681 1 2024-04-26T23:01:27.964 INFO:tasks.workunit.client.0.smithi027.stdout:1/415: write d1/d2a/f62 [221171,17547] 0 2024-04-26T23:01:27.965 INFO:tasks.workunit.client.0.smithi027.stdout:4/366: dwrite da/dd/de/d14/d17/f68 [0,4194304] 0 2024-04-26T23:01:27.965 INFO:tasks.workunit.client.0.smithi027.stdout:4/367: chown da/dd/de/d14/d21/d67 148294 1 2024-04-26T23:01:27.966 INFO:tasks.workunit.client.0.smithi027.stdout:7/333: mkdir d0/d28/d2b/d55/d69 0 2024-04-26T23:01:27.966 INFO:tasks.workunit.client.0.smithi027.stdout:3/380: creat d6/dc/d2f/d6b/d74/f87 x:0 0 0 2024-04-26T23:01:27.966 INFO:tasks.workunit.client.0.smithi027.stdout:8/388: dwrite d9/de/d22/f4d [0,4194304] 0 2024-04-26T23:01:27.968 INFO:tasks.workunit.client.0.smithi027.stdout:4/368: creat da/dd/de/d14/d21/d4a/f6a x:0 0 0 2024-04-26T23:01:27.970 INFO:tasks.workunit.client.0.smithi027.stdout:7/334: creat d0/d28/f6a x:0 0 0 2024-04-26T23:01:27.970 INFO:tasks.workunit.client.0.smithi027.stdout:7/335: fsync d0/f13 0 2024-04-26T23:01:27.972 INFO:tasks.workunit.client.0.smithi027.stdout:4/369: mknod da/dd/de/d14/c6b 0 2024-04-26T23:01:27.972 INFO:tasks.workunit.client.0.smithi027.stdout:4/370: stat da/dd/de/d14/d21/d67 0 2024-04-26T23:01:27.973 INFO:tasks.workunit.client.0.smithi027.stdout:3/381: truncate d6/dc/d34/f36 3585250 0 2024-04-26T23:01:27.975 INFO:tasks.workunit.client.0.smithi027.stdout:1/416: symlink d1/l80 0 2024-04-26T23:01:27.975 INFO:tasks.workunit.client.0.smithi027.stdout:1/417: readlink d1/d5/l1a 0 2024-04-26T23:01:27.980 INFO:tasks.workunit.client.0.smithi027.stdout:3/382: rmdir d6/dc/d17/d59/d61 39 2024-04-26T23:01:27.988 INFO:tasks.workunit.client.0.smithi027.stdout:3/383: unlink d6/dc/d17/d59/d61/c68 0 2024-04-26T23:01:27.988 INFO:tasks.workunit.client.0.smithi027.stdout:3/384: fsync d6/dc/f85 0 2024-04-26T23:01:27.988 INFO:tasks.workunit.client.0.smithi027.stdout:0/329: dread d0/f12 [0,4194304] 0 2024-04-26T23:01:27.988 INFO:tasks.workunit.client.0.smithi027.stdout:0/330: rmdir d0/d4/da/d15 39 2024-04-26T23:01:27.989 INFO:tasks.workunit.client.0.smithi027.stdout:0/331: symlink d0/d4/da/d1a/l62 0 2024-04-26T23:01:27.989 INFO:tasks.workunit.client.0.smithi027.stdout:0/332: readlink d0/d4/lc 0 2024-04-26T23:01:27.994 INFO:tasks.workunit.client.0.smithi027.stdout:0/333: write d0/d4/f7 [33749,27670] 0 2024-04-26T23:01:27.994 INFO:tasks.workunit.client.0.smithi027.stdout:0/334: chown d0/d4/da/d15/d55 6 1 2024-04-26T23:01:27.996 INFO:tasks.workunit.client.0.smithi027.stdout:0/335: rename d0/d4/da/d1a/d1e/c21 to d0/df/d3f/c63 0 2024-04-26T23:01:27.996 INFO:tasks.workunit.client.0.smithi027.stdout:2/299: dwrite d2/d5/db/d1b/d1c/f37 [4194304,4194304] 0 2024-04-26T23:01:27.998 INFO:tasks.workunit.client.0.smithi027.stdout:2/300: link d2/d5/db/de/f30 d2/d5/db/de/d21/f51 0 2024-04-26T23:01:28.004 INFO:tasks.workunit.client.0.smithi027.stdout:0/336: truncate d0/d4/f24 648205 0 2024-04-26T23:01:28.004 INFO:tasks.workunit.client.0.smithi027.stdout:4/371: dread da/dd/de/d14/d17/d24/f3e [0,4194304] 0 2024-04-26T23:01:28.012 INFO:tasks.workunit.client.0.smithi027.stdout:6/252: write d1/f2b [4498492,84694] 0 2024-04-26T23:01:28.012 INFO:tasks.workunit.client.0.smithi027.stdout:6/253: write d1/df/d43/f5d [148866,21737] 0 2024-04-26T23:01:28.012 INFO:tasks.workunit.client.0.smithi027.stdout:6/254: dread - d1/de/f48 zero size 2024-04-26T23:01:28.013 INFO:tasks.workunit.client.0.smithi027.stdout:3/385: write d6/dc/d34/d47/f3c [668,74343] 0 2024-04-26T23:01:28.013 INFO:tasks.workunit.client.0.smithi027.stdout:3/386: readlink d6/dc/l6f 0 2024-04-26T23:01:28.013 INFO:tasks.workunit.client.0.smithi027.stdout:3/387: write d6/f18 [351770,38593] 0 2024-04-26T23:01:28.014 INFO:tasks.workunit.client.0.smithi027.stdout:3/388: mkdir d6/dc/d88 0 2024-04-26T23:01:28.016 INFO:tasks.workunit.client.0.smithi027.stdout:6/255: rename d1/f27 to d1/df/f5f 0 2024-04-26T23:01:28.017 INFO:tasks.workunit.client.0.smithi027.stdout:6/256: mknod d1/d30/c60 0 2024-04-26T23:01:28.018 INFO:tasks.workunit.client.0.smithi027.stdout:6/257: mknod d1/c61 0 2024-04-26T23:01:28.020 INFO:tasks.workunit.client.0.smithi027.stdout:6/258: creat d1/f62 x:0 0 0 2024-04-26T23:01:28.020 INFO:tasks.workunit.client.0.smithi027.stdout:6/259: chown d1/de/f1a 8615 1 2024-04-26T23:01:28.020 INFO:tasks.workunit.client.0.smithi027.stdout:6/260: readlink d1/df/l3b 0 2024-04-26T23:01:28.020 INFO:tasks.workunit.client.0.smithi027.stdout:6/261: write d1/df/f3a [120753,68992] 0 2024-04-26T23:01:28.021 INFO:tasks.workunit.client.0.smithi027.stdout:9/275: dwrite d4/d10/f5e [0,4194304] 0 2024-04-26T23:01:28.021 INFO:tasks.workunit.client.0.smithi027.stdout:6/262: creat d1/d30/d40/f63 x:0 0 0 2024-04-26T23:01:28.024 INFO:tasks.workunit.client.0.smithi027.stdout:9/276: mknod d4/c61 0 2024-04-26T23:01:28.028 INFO:tasks.workunit.client.0.smithi027.stdout:6/263: read d1/de/f28 [72696,66036] 0 2024-04-26T23:01:28.028 INFO:tasks.workunit.client.0.smithi027.stdout:6/264: stat d1/c61 0 2024-04-26T23:01:28.028 INFO:tasks.workunit.client.0.smithi027.stdout:6/265: fdatasync d1/f4e 0 2024-04-26T23:01:28.029 INFO:tasks.workunit.client.0.smithi027.stdout:6/266: creat d1/f64 x:0 0 0 2024-04-26T23:01:28.029 INFO:tasks.workunit.client.0.smithi027.stdout:0/337: write d0/d4/f20 [1115407,33077] 0 2024-04-26T23:01:28.036 INFO:tasks.workunit.client.0.smithi027.stdout:1/418: dread d1/d26/d36/d56/f6e [0,4194304] 0 2024-04-26T23:01:28.040 INFO:tasks.workunit.client.0.smithi027.stdout:0/338: rename d0/d4/da/d2d to d0/df/d3f/d64 0 2024-04-26T23:01:28.040 INFO:tasks.workunit.client.0.smithi027.stdout:7/336: dwrite d0/f27 [4194304,4194304] 0 2024-04-26T23:01:28.041 INFO:tasks.workunit.client.0.smithi027.stdout:6/267: mknod d1/df/d43/d56/c65 0 2024-04-26T23:01:28.041 INFO:tasks.workunit.client.0.smithi027.stdout:7/337: readlink d0/d28/d2b/l3b 0 2024-04-26T23:01:28.041 INFO:tasks.workunit.client.0.smithi027.stdout:6/268: mkdir d1/df/d3c/d66 0 2024-04-26T23:01:28.041 INFO:tasks.workunit.client.0.smithi027.stdout:6/269: dread - d1/de/f48 zero size 2024-04-26T23:01:28.045 INFO:tasks.workunit.client.0.smithi027.stdout:6/270: getdents d1/d30/d40 0 2024-04-26T23:01:28.045 INFO:tasks.workunit.client.0.smithi027.stdout:6/271: fsync d1/df/d49/f52 0 2024-04-26T23:01:28.045 INFO:tasks.workunit.client.0.smithi027.stdout:6/272: fdatasync d1/de/f20 0 2024-04-26T23:01:28.047 INFO:tasks.workunit.client.0.smithi027.stdout:6/273: mknod d1/de/c67 0 2024-04-26T23:01:28.051 INFO:tasks.workunit.client.0.smithi027.stdout:6/274: creat d1/df/f68 x:0 0 0 2024-04-26T23:01:28.055 INFO:tasks.workunit.client.0.smithi027.stdout:8/389: dwrite d9/da/d2a/f48 [0,4194304] 0 2024-04-26T23:01:28.055 INFO:tasks.workunit.client.0.smithi027.stdout:9/277: read d4/f11 [3197799,58019] 0 2024-04-26T23:01:28.057 INFO:tasks.workunit.client.0.smithi027.stdout:2/301: dwrite d2/d5/db/de/d21/f51 [4194304,4194304] 0 2024-04-26T23:01:28.060 INFO:tasks.workunit.client.0.smithi027.stdout:1/419: dread d1/d5/d21/f34 [0,4194304] 0 2024-04-26T23:01:28.061 INFO:tasks.workunit.client.0.smithi027.stdout:4/372: dwrite da/dd/de/d14/d21/d4a/f6a [0,4194304] 0 2024-04-26T23:01:28.061 INFO:tasks.workunit.client.0.smithi027.stdout:4/373: chown da/dd/de 1419 1 2024-04-26T23:01:28.064 INFO:tasks.workunit.client.0.smithi027.stdout:8/390: getdents d9/da/d2a/d5e/d86 0 2024-04-26T23:01:28.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/278: unlink d4/d10/d25/f3e 0 2024-04-26T23:01:28.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/279: getdents d4/d17 0 2024-04-26T23:01:28.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/280: write d4/d10/d40/f5d [262844,121392] 0 2024-04-26T23:01:28.067 INFO:tasks.workunit.client.0.smithi027.stdout:2/302: getdents d2/d5/db/d1b/d1c/d25/d2f 0 2024-04-26T23:01:28.071 INFO:tasks.workunit.client.0.smithi027.stdout:3/389: dwrite d6/dc/f55 [0,4194304] 0 2024-04-26T23:01:28.071 INFO:tasks.workunit.client.0.smithi027.stdout:3/390: stat d6/dc/d34/d47/l43 0 2024-04-26T23:01:28.072 INFO:tasks.workunit.client.0.smithi027.stdout:1/420: creat d1/d26/d36/d56/f81 x:0 0 0 2024-04-26T23:01:28.072 INFO:tasks.workunit.client.0.smithi027.stdout:4/374: mknod da/dd/de/c6c 0 2024-04-26T23:01:28.072 INFO:tasks.workunit.client.0.smithi027.stdout:4/375: write f2 [2242162,25958] 0 2024-04-26T23:01:28.072 INFO:tasks.workunit.client.0.smithi027.stdout:4/376: fsync da/d40/f66 0 2024-04-26T23:01:28.073 INFO:tasks.workunit.client.0.smithi027.stdout:9/281: getdents d4/d10/d40 0 2024-04-26T23:01:28.075 INFO:tasks.workunit.client.0.smithi027.stdout:3/391: creat d6/dc/d4c/f89 x:0 0 0 2024-04-26T23:01:28.076 INFO:tasks.workunit.client.0.smithi027.stdout:1/421: link d1/d5/c1e d1/d26/d66/c82 0 2024-04-26T23:01:28.077 INFO:tasks.workunit.client.0.smithi027.stdout:7/338: dwrite d0/d3d/f44 [0,4194304] 0 2024-04-26T23:01:28.089 INFO:tasks.workunit.client.0.smithi027.stdout:4/377: creat da/dd/de/d14/d21/d4a/f6d x:0 0 0 2024-04-26T23:01:28.089 INFO:tasks.workunit.client.0.smithi027.stdout:9/282: rename d4/f14 to d4/d17/d2e/d46/f62 0 2024-04-26T23:01:28.090 INFO:tasks.workunit.client.0.smithi027.stdout:3/392: rmdir d6/dc/d2f 39 2024-04-26T23:01:28.090 INFO:tasks.workunit.client.0.smithi027.stdout:9/283: truncate d4/f43 228740 0 2024-04-26T23:01:28.090 INFO:tasks.workunit.client.0.smithi027.stdout:3/393: readlink d6/dc/d34/d2e/d5f/l7a 0 2024-04-26T23:01:28.092 INFO:tasks.workunit.client.0.smithi027.stdout:8/391: symlink d9/da/d2a/d5e/d72/d84/l93 0 2024-04-26T23:01:28.092 INFO:tasks.workunit.client.0.smithi027.stdout:4/378: symlink da/dd/d5f/l6e 0 2024-04-26T23:01:28.092 INFO:tasks.workunit.client.0.smithi027.stdout:9/284: creat d4/d10/d30/f63 x:0 0 0 2024-04-26T23:01:28.092 INFO:tasks.workunit.client.0.smithi027.stdout:9/285: fdatasync d4/d17/d2e/d46/f55 0 2024-04-26T23:01:28.092 INFO:tasks.workunit.client.0.smithi027.stdout:9/286: readlink d4/l18 0 2024-04-26T23:01:28.092 INFO:tasks.workunit.client.0.smithi027.stdout:6/275: write d1/de/f1b [3990537,102019] 0 2024-04-26T23:01:28.094 INFO:tasks.workunit.client.0.smithi027.stdout:9/287: write d4/f43 [97901,55401] 0 2024-04-26T23:01:28.097 INFO:tasks.workunit.client.0.smithi027.stdout:3/394: creat d6/dc/d34/d2e/d50/f8a x:0 0 0 2024-04-26T23:01:28.097 INFO:tasks.workunit.client.0.smithi027.stdout:6/276: creat d1/f69 x:0 0 0 2024-04-26T23:01:28.097 INFO:tasks.workunit.client.0.smithi027.stdout:4/379: rename da/dd/de/d14/c2f to da/dd/de/d14/c6f 0 2024-04-26T23:01:28.097 INFO:tasks.workunit.client.0.smithi027.stdout:4/380: readlink da/dd/de/l1b 0 2024-04-26T23:01:28.097 INFO:tasks.workunit.client.0.smithi027.stdout:4/381: readlink da/dd/d5f/l6e 0 2024-04-26T23:01:28.097 INFO:tasks.workunit.client.0.smithi027.stdout:3/395: chown d6/dc/d2f/d6b/c73 402 1 2024-04-26T23:01:28.097 INFO:tasks.workunit.client.0.smithi027.stdout:3/396: chown d6/dc/d34/d2e/c1f 16 1 2024-04-26T23:01:28.098 INFO:tasks.workunit.client.0.smithi027.stdout:0/339: dwrite d0/d4/da/d15/f44 [0,4194304] 0 2024-04-26T23:01:28.098 INFO:tasks.workunit.client.0.smithi027.stdout:9/288: creat d4/d10/d40/f64 x:0 0 0 2024-04-26T23:01:28.101 INFO:tasks.workunit.client.0.smithi027.stdout:4/382: symlink da/dd/de/d14/d21/d41/d59/l70 0 2024-04-26T23:01:28.101 INFO:tasks.workunit.client.0.smithi027.stdout:0/340: unlink d0/d4/da/d1a/d1e/f25 0 2024-04-26T23:01:28.101 INFO:tasks.workunit.client.0.smithi027.stdout:4/383: chown da/dd/de/d14/d17/d24/f26 1301519975 1 2024-04-26T23:01:28.101 INFO:tasks.workunit.client.0.smithi027.stdout:4/384: chown da/dd/de/d14/d17/d24/c4e 10 1 2024-04-26T23:01:28.107 INFO:tasks.workunit.client.0.smithi027.stdout:1/422: dread d1/d5/d40/f52 [0,4194304] 0 2024-04-26T23:01:28.115 INFO:tasks.workunit.client.0.smithi027.stdout:0/341: getdents d0/d4/da/d15 0 2024-04-26T23:01:28.115 INFO:tasks.workunit.client.0.smithi027.stdout:4/385: symlink da/dd/de/d14/d21/d4a/l71 0 2024-04-26T23:01:28.115 INFO:tasks.workunit.client.0.smithi027.stdout:5/369: sync 2024-04-26T23:01:28.115 INFO:tasks.workunit.client.0.smithi027.stdout:5/370: readlink d4/d1a/l32 0 2024-04-26T23:01:28.117 INFO:tasks.workunit.client.0.smithi027.stdout:0/342: mknod d0/d4/da/d1a/c65 0 2024-04-26T23:01:28.117 INFO:tasks.workunit.client.0.smithi027.stdout:0/343: fsync d0/df/f4e 0 2024-04-26T23:01:28.119 INFO:tasks.workunit.client.0.smithi027.stdout:1/423: rename d1/d5/l15 to d1/d26/d6b/d42/d79/l83 0 2024-04-26T23:01:28.119 INFO:tasks.workunit.client.0.smithi027.stdout:1/424: chown d1/f1b 49 1 2024-04-26T23:01:28.119 INFO:tasks.workunit.client.0.smithi027.stdout:1/425: write d1/f55 [637627,79292] 0 2024-04-26T23:01:28.120 INFO:tasks.workunit.client.0.smithi027.stdout:5/371: symlink d4/d1a/d4a/d68/l78 0 2024-04-26T23:01:28.120 INFO:tasks.workunit.client.0.smithi027.stdout:5/372: fdatasync d4/d1a/f20 0 2024-04-26T23:01:28.120 INFO:tasks.workunit.client.0.smithi027.stdout:4/386: creat da/dd/de/d14/d21/d41/f72 x:0 0 0 2024-04-26T23:01:28.120 INFO:tasks.workunit.client.0.smithi027.stdout:5/373: truncate d4/d24/d22/d29/d2a/d3b/f3d 1744473 0 2024-04-26T23:01:28.120 INFO:tasks.workunit.client.0.smithi027.stdout:5/374: read d4/f15 [3237184,40008] 0 2024-04-26T23:01:28.123 INFO:tasks.workunit.client.0.smithi027.stdout:1/426: rename d1/d5/la to d1/d26/d6b/l84 0 2024-04-26T23:01:28.123 INFO:tasks.workunit.client.0.smithi027.stdout:1/427: chown d1/l45 1 1 2024-04-26T23:01:28.123 INFO:tasks.workunit.client.0.smithi027.stdout:1/428: read d1/f1b [173036,36394] 0 2024-04-26T23:01:28.123 INFO:tasks.workunit.client.0.smithi027.stdout:4/387: mknod da/dd/c73 0 2024-04-26T23:01:28.124 INFO:tasks.workunit.client.0.smithi027.stdout:4/388: write da/dd/de/f3f [1393334,84394] 0 2024-04-26T23:01:28.129 INFO:tasks.workunit.client.0.smithi027.stdout:1/429: creat d1/f85 x:0 0 0 2024-04-26T23:01:28.135 INFO:tasks.workunit.client.0.smithi027.stdout:1/430: getdents d1/d26/d6b/d42/d79 0 2024-04-26T23:01:28.143 INFO:tasks.workunit.client.0.smithi027.stdout:2/303: dwrite d2/d5/db/de/d21/f16 [4194304,4194304] 0 2024-04-26T23:01:28.150 INFO:tasks.workunit.client.0.smithi027.stdout:0/344: dread d0/d4/da/f2f [0,4194304] 0 2024-04-26T23:01:28.152 INFO:tasks.workunit.client.0.smithi027.stdout:1/431: rename d1/d5/d40/d35 to d1/d26/d86 0 2024-04-26T23:01:28.161 INFO:tasks.workunit.client.0.smithi027.stdout:8/392: dwrite d9/da/d2a/d5e/d72/d84/d35/f56 [0,4194304] 0 2024-04-26T23:01:28.164 INFO:tasks.workunit.client.0.smithi027.stdout:8/393: mkdir d9/da/d2a/d5e/d92/d94 0 2024-04-26T23:01:28.164 INFO:tasks.workunit.client.0.smithi027.stdout:8/394: readlink d9/de/d2e/l21 0 2024-04-26T23:01:28.169 INFO:tasks.workunit.client.0.smithi027.stdout:4/389: dread da/dd/de/d14/d17/f45 [0,4194304] 0 2024-04-26T23:01:28.175 INFO:tasks.workunit.client.0.smithi027.stdout:8/395: symlink d9/de/d2e/l95 0 2024-04-26T23:01:28.176 INFO:tasks.workunit.client.0.smithi027.stdout:5/375: dwrite d4/d1a/d4a/d68/d40/f6c [0,4194304] 0 2024-04-26T23:01:28.177 INFO:tasks.workunit.client.0.smithi027.stdout:6/277: dwrite d1/de/f1e [0,4194304] 0 2024-04-26T23:01:28.184 INFO:tasks.workunit.client.0.smithi027.stdout:2/304: dwrite d2/d5/db/d1b/d1c/d25/d19/d36/d39/f3f [0,4194304] 0 2024-04-26T23:01:28.188 INFO:tasks.workunit.client.0.smithi027.stdout:2/305: dread d2/d5/db/d1b/d1c/d25/f4a [0,4194304] 0 2024-04-26T23:01:28.188 INFO:tasks.workunit.client.0.smithi027.stdout:2/306: dread - d2/d5/db/d1b/d1c/d25/d19/d36/f43 zero size 2024-04-26T23:01:28.193 INFO:tasks.workunit.client.0.smithi027.stdout:2/307: dread d2/d5/db/d1b/d1c/d25/f4a [0,4194304] 0 2024-04-26T23:01:28.196 INFO:tasks.workunit.client.0.smithi027.stdout:8/396: write d9/f74 [3569425,27541] 0 2024-04-26T23:01:28.196 INFO:tasks.workunit.client.0.smithi027.stdout:8/397: write d9/da/d2a/d6b/f81 [146512,87759] 0 2024-04-26T23:01:28.198 INFO:tasks.workunit.client.0.smithi027.stdout:6/278: unlink d1/df/l53 0 2024-04-26T23:01:28.198 INFO:tasks.workunit.client.0.smithi027.stdout:5/376: link d4/d24/d22/d29/d2a/l72 d4/db/l79 0 2024-04-26T23:01:28.198 INFO:tasks.workunit.client.0.smithi027.stdout:6/279: write d1/df/f68 [84243,17218] 0 2024-04-26T23:01:28.198 INFO:tasks.workunit.client.0.smithi027.stdout:6/280: stat d1/df/d3c 0 2024-04-26T23:01:28.199 INFO:tasks.workunit.client.0.smithi027.stdout:6/281: fsync d1/f59 0 2024-04-26T23:01:28.199 INFO:tasks.workunit.client.0.smithi027.stdout:4/390: dread da/dd/de/d14/d17/d24/f3a [0,4194304] 0 2024-04-26T23:01:28.202 INFO:tasks.workunit.client.0.smithi027.stdout:8/398: truncate d9/da/d2a/f89 90284 0 2024-04-26T23:01:28.203 INFO:tasks.workunit.client.0.smithi027.stdout:4/391: mkdir da/dd/de/d14/d21/d41/d59/d74 0 2024-04-26T23:01:28.203 INFO:tasks.workunit.client.0.smithi027.stdout:4/392: truncate da/dd/de/d14/d17/f45 762928 0 2024-04-26T23:01:28.203 INFO:tasks.workunit.client.0.smithi027.stdout:4/393: write da/dd/de/d14/d21/d41/f52 [726613,68209] 0 2024-04-26T23:01:28.203 INFO:tasks.workunit.client.0.smithi027.stdout:4/394: chown da/f47 0 1 2024-04-26T23:01:28.205 INFO:tasks.workunit.client.0.smithi027.stdout:9/289: dread d4/d10/d25/f39 [0,4194304] 0 2024-04-26T23:01:28.205 INFO:tasks.workunit.client.0.smithi027.stdout:9/290: dread - d4/d10/d2c/f57 zero size 2024-04-26T23:01:28.205 INFO:tasks.workunit.client.0.smithi027.stdout:8/399: symlink d9/da/d2a/d6b/l96 0 2024-04-26T23:01:28.205 INFO:tasks.workunit.client.0.smithi027.stdout:8/400: fsync d9/de/f18 0 2024-04-26T23:01:28.205 INFO:tasks.workunit.client.0.smithi027.stdout:8/401: fsync d9/f16 0 2024-04-26T23:01:28.205 INFO:tasks.workunit.client.0.smithi027.stdout:2/308: mknod d2/d5/db/d1b/d1c/d25/c52 0 2024-04-26T23:01:28.205 INFO:tasks.workunit.client.0.smithi027.stdout:3/397: dwrite d6/dc/d34/d2e/f45 [0,4194304] 0 2024-04-26T23:01:28.206 INFO:tasks.workunit.client.0.smithi027.stdout:3/398: write d6/dc/d17/f33 [2088534,75249] 0 2024-04-26T23:01:28.206 INFO:tasks.workunit.client.0.smithi027.stdout:3/399: chown d6/c51 231225 1 2024-04-26T23:01:28.206 INFO:tasks.workunit.client.0.smithi027.stdout:1/432: dwrite d1/d26/d86/f4c [0,4194304] 0 2024-04-26T23:01:28.209 INFO:tasks.workunit.client.0.smithi027.stdout:9/291: mknod d4/d17/d2e/c65 0 2024-04-26T23:01:28.209 INFO:tasks.workunit.client.0.smithi027.stdout:8/402: creat d9/da/d2a/d5e/d72/d84/d35/d8d/f97 x:0 0 0 2024-04-26T23:01:28.209 INFO:tasks.workunit.client.0.smithi027.stdout:8/403: getdents d9/de 0 2024-04-26T23:01:28.212 INFO:tasks.workunit.client.0.smithi027.stdout:2/309: mknod d2/d5/db/d1b/d1c/d25/d19/d44/c53 0 2024-04-26T23:01:28.216 INFO:tasks.workunit.client.0.smithi027.stdout:0/345: dwrite d0/d4/f3d [0,4194304] 0 2024-04-26T23:01:28.218 INFO:tasks.workunit.client.0.smithi027.stdout:3/400: creat d6/dc/d17/d40/f8b x:0 0 0 2024-04-26T23:01:28.218 INFO:tasks.workunit.client.0.smithi027.stdout:3/401: write d6/d5e/f6d [712905,115301] 0 2024-04-26T23:01:28.222 INFO:tasks.workunit.client.0.smithi027.stdout:2/310: getdents d2/d5/db/de/d21 0 2024-04-26T23:01:28.223 INFO:tasks.workunit.client.0.smithi027.stdout:0/346: creat d0/d4/da/d15/d55/f66 x:0 0 0 2024-04-26T23:01:28.223 INFO:tasks.workunit.client.0.smithi027.stdout:5/377: dread d4/d1a/f57 [0,4194304] 0 2024-04-26T23:01:28.225 INFO:tasks.workunit.client.0.smithi027.stdout:3/402: fsync d6/dc/d17/f33 0 2024-04-26T23:01:28.225 INFO:tasks.workunit.client.0.smithi027.stdout:3/403: fdatasync d6/dc/d62/f7c 0 2024-04-26T23:01:28.227 INFO:tasks.workunit.client.0.smithi027.stdout:4/395: dread da/dd/f5d [0,4194304] 0 2024-04-26T23:01:28.228 INFO:tasks.workunit.client.0.smithi027.stdout:0/347: write d0/d4/da/f19 [407974,120999] 0 2024-04-26T23:01:28.228 INFO:tasks.workunit.client.0.smithi027.stdout:0/348: stat d0/d4/da/f1f 0 2024-04-26T23:01:28.228 INFO:tasks.workunit.client.0.smithi027.stdout:4/396: write da/f48 [277919,77354] 0 2024-04-26T23:01:28.235 INFO:tasks.workunit.client.0.smithi027.stdout:3/404: mknod d6/dc/d34/d47/c8c 0 2024-04-26T23:01:28.236 INFO:tasks.workunit.client.0.smithi027.stdout:0/349: creat d0/d4/da/d1a/d1e/d52/f67 x:0 0 0 2024-04-26T23:01:28.237 INFO:tasks.workunit.client.0.smithi027.stdout:4/397: rename da/dd/de/d14/d17/f1d to da/dd/de/d14/d21/f75 0 2024-04-26T23:01:28.237 INFO:tasks.workunit.client.0.smithi027.stdout:4/398: chown da/dd/c73 43629 1 2024-04-26T23:01:28.239 INFO:tasks.workunit.client.0.smithi027.stdout:5/378: read d4/d24/d22/d29/f58 [1037971,35568] 0 2024-04-26T23:01:28.239 INFO:tasks.workunit.client.0.smithi027.stdout:2/311: link d2/d5/db/de/l2a d2/d5/db/l54 0 2024-04-26T23:01:28.241 INFO:tasks.workunit.client.0.smithi027.stdout:3/405: creat d6/dc/d17/d59/d61/f8d x:0 0 0 2024-04-26T23:01:28.242 INFO:tasks.workunit.client.0.smithi027.stdout:2/312: symlink d2/d5/db/d1b/d1c/d25/d19/d36/l55 0 2024-04-26T23:01:28.249 INFO:tasks.workunit.client.0.smithi027.stdout:4/399: rename da/f47 to da/f76 0 2024-04-26T23:01:28.249 INFO:tasks.workunit.client.0.smithi027.stdout:2/313: link d2/d5/db/d1b/d1c/d25/d2f/f4c d2/d5/db/d1b/d1c/d25/d19/d44/d4b/f56 0 2024-04-26T23:01:28.250 INFO:tasks.workunit.client.0.smithi027.stdout:3/406: truncate d6/dc/fd 110351 0 2024-04-26T23:01:28.254 INFO:tasks.workunit.client.0.smithi027.stdout:3/407: truncate d6/dc/d17/f2b 1874217 0 2024-04-26T23:01:28.254 INFO:tasks.workunit.client.0.smithi027.stdout:3/408: chown d6/dc/f85 1865599 1 2024-04-26T23:01:28.258 INFO:tasks.workunit.client.0.smithi027.stdout:8/404: dwrite f3 [0,4194304] 0 2024-04-26T23:01:28.261 INFO:tasks.workunit.client.0.smithi027.stdout:0/350: read d0/d4/da/d15/f44 [3287990,41762] 0 2024-04-26T23:01:28.262 INFO:tasks.workunit.client.0.smithi027.stdout:8/405: symlink d9/da/d2a/d6b/l98 0 2024-04-26T23:01:28.262 INFO:tasks.workunit.client.0.smithi027.stdout:8/406: readlink d9/da/d2a/d5e/d2b/d52/l5a 0 2024-04-26T23:01:28.262 INFO:tasks.workunit.client.0.smithi027.stdout:8/407: stat d9/da/d33/d71 0 2024-04-26T23:01:28.263 INFO:tasks.workunit.client.0.smithi027.stdout:0/351: mknod d0/d4/da/d1a/d1e/c68 0 2024-04-26T23:01:28.270 INFO:tasks.workunit.client.0.smithi027.stdout:0/352: mknod d0/d4/da/d1a/d1e/c69 0 2024-04-26T23:01:28.285 INFO:tasks.workunit.client.0.smithi027.stdout:1/433: dread d1/d5/d21/f44 [0,4194304] 0 2024-04-26T23:01:28.285 INFO:tasks.workunit.client.0.smithi027.stdout:1/434: chown d1/d26/d6b/d42/l69 0 1 2024-04-26T23:01:28.285 INFO:tasks.workunit.client.0.smithi027.stdout:6/282: dwrite d1/de/f28 [0,4194304] 0 2024-04-26T23:01:28.293 INFO:tasks.workunit.client.0.smithi027.stdout:4/400: dread da/dd/de/d14/d21/d4a/f6a [0,4194304] 0 2024-04-26T23:01:28.294 INFO:tasks.workunit.client.0.smithi027.stdout:0/353: dread d0/d4/f24 [0,4194304] 0 2024-04-26T23:01:28.294 INFO:tasks.workunit.client.0.smithi027.stdout:0/354: fsync d0/d4/da/f1f 0 2024-04-26T23:01:28.294 INFO:tasks.workunit.client.0.smithi027.stdout:9/292: dwrite d4/d10/d40/f48 [4194304,4194304] 0 2024-04-26T23:01:28.294 INFO:tasks.workunit.client.0.smithi027.stdout:9/293: chown d4/d10/d30 18 1 2024-04-26T23:01:28.297 INFO:tasks.workunit.client.0.smithi027.stdout:6/283: mkdir d1/d6a 0 2024-04-26T23:01:28.298 INFO:tasks.workunit.client.0.smithi027.stdout:1/435: truncate d1/d26/d7d/f30 163630 0 2024-04-26T23:01:28.299 INFO:tasks.workunit.client.0.smithi027.stdout:9/294: creat d4/d10/d40/f66 x:0 0 0 2024-04-26T23:01:28.303 INFO:tasks.workunit.client.0.smithi027.stdout:6/284: truncate d1/df/f3a 9455 0 2024-04-26T23:01:28.303 INFO:tasks.workunit.client.0.smithi027.stdout:7/339: sync 2024-04-26T23:01:28.303 INFO:tasks.workunit.client.0.smithi027.stdout:0/355: rename d0/d4/da/d1a/d2a/f30 to d0/df/d3f/d64/f6a 0 2024-04-26T23:01:28.303 INFO:tasks.workunit.client.0.smithi027.stdout:0/356: fdatasync d0/df/d3f/d4a/f4d 0 2024-04-26T23:01:28.304 INFO:tasks.workunit.client.0.smithi027.stdout:1/436: truncate d1/f6 1035083 0 2024-04-26T23:01:28.306 INFO:tasks.workunit.client.0.smithi027.stdout:9/295: getdents d4/d10 0 2024-04-26T23:01:28.306 INFO:tasks.workunit.client.0.smithi027.stdout:9/296: chown d4/d10/d30/f63 9514917 1 2024-04-26T23:01:28.307 INFO:tasks.workunit.client.0.smithi027.stdout:6/285: mkdir d1/df/d43/d56/d6b 0 2024-04-26T23:01:28.309 INFO:tasks.workunit.client.0.smithi027.stdout:6/286: write d1/de/f48 [433471,71523] 0 2024-04-26T23:01:28.309 INFO:tasks.workunit.client.0.smithi027.stdout:6/287: chown d1/de/c3f 96 1 2024-04-26T23:01:28.313 INFO:tasks.workunit.client.0.smithi027.stdout:1/437: mknod d1/d26/d86/d5c/c87 0 2024-04-26T23:01:28.322 INFO:tasks.workunit.client.0.smithi027.stdout:2/314: dwrite d2/d5/db/d1b/d1c/d25/d19/d36/d39/f3f [0,4194304] 0 2024-04-26T23:01:28.323 INFO:tasks.workunit.client.0.smithi027.stdout:4/401: dread da/dd/de/d14/f44 [0,4194304] 0 2024-04-26T23:01:28.331 INFO:tasks.workunit.client.0.smithi027.stdout:9/297: write d4/d17/d2e/f53 [142242,17307] 0 2024-04-26T23:01:28.333 INFO:tasks.workunit.client.0.smithi027.stdout:9/298: mkdir d4/d10/d67 0 2024-04-26T23:01:28.336 INFO:tasks.workunit.client.0.smithi027.stdout:9/299: stat d4/d10/d40/f49 0 2024-04-26T23:01:28.341 INFO:tasks.workunit.client.0.smithi027.stdout:0/357: dread d0/d4/f53 [0,4194304] 0 2024-04-26T23:01:28.341 INFO:tasks.workunit.client.0.smithi027.stdout:0/358: fdatasync d0/df/d3f/d4a/f4d 0 2024-04-26T23:01:28.343 INFO:tasks.workunit.client.0.smithi027.stdout:3/409: dwrite d6/dc/f25 [0,4194304] 0 2024-04-26T23:01:28.343 INFO:tasks.workunit.client.0.smithi027.stdout:8/408: dwrite d9/f1b [0,4194304] 0 2024-04-26T23:01:28.343 INFO:tasks.workunit.client.0.smithi027.stdout:8/409: stat d9/de/d2e/c8f 0 2024-04-26T23:01:28.353 INFO:tasks.workunit.client.0.smithi027.stdout:3/410: symlink d6/dc/d88/l8e 0 2024-04-26T23:01:28.353 INFO:tasks.workunit.client.0.smithi027.stdout:3/411: chown d6/dc/d2f/l3a 419814695 1 2024-04-26T23:01:28.370 INFO:tasks.workunit.client.0.smithi027.stdout:7/340: dwrite d0/d3d/f44 [0,4194304] 0 2024-04-26T23:01:28.370 INFO:tasks.workunit.client.0.smithi027.stdout:3/412: dread d6/dc/f46 [0,4194304] 0 2024-04-26T23:01:28.371 INFO:tasks.workunit.client.0.smithi027.stdout:3/413: dread - d6/dc/d2f/d6b/d74/f87 zero size 2024-04-26T23:01:28.371 INFO:tasks.workunit.client.0.smithi027.stdout:3/414: chown d6/l30 112525144 1 2024-04-26T23:01:28.372 INFO:tasks.workunit.client.0.smithi027.stdout:7/341: creat d0/d28/d2b/d55/d59/f6b x:0 0 0 2024-04-26T23:01:28.379 INFO:tasks.workunit.client.0.smithi027.stdout:7/342: truncate d0/d3d/f44 2887822 0 2024-04-26T23:01:28.383 INFO:tasks.workunit.client.0.smithi027.stdout:7/343: mkdir d0/d6c 0 2024-04-26T23:01:28.387 INFO:tasks.workunit.client.0.smithi027.stdout:6/288: dwrite d1/df/f3a [0,4194304] 0 2024-04-26T23:01:28.388 INFO:tasks.workunit.client.0.smithi027.stdout:6/289: fdatasync d1/df/f31 0 2024-04-26T23:01:28.390 INFO:tasks.workunit.client.0.smithi027.stdout:9/300: dwrite d4/d17/d2e/f34 [0,4194304] 0 2024-04-26T23:01:28.390 INFO:tasks.workunit.client.0.smithi027.stdout:9/301: fdatasync d4/d17/f29 0 2024-04-26T23:01:28.391 INFO:tasks.workunit.client.0.smithi027.stdout:0/359: dread d0/d4/f20 [4194304,4194304] 0 2024-04-26T23:01:28.394 INFO:tasks.workunit.client.0.smithi027.stdout:0/360: dread - d0/d4/da/d15/f5a zero size 2024-04-26T23:01:28.395 INFO:tasks.workunit.client.0.smithi027.stdout:1/438: dread d1/dc/f16 [0,4194304] 0 2024-04-26T23:01:28.395 INFO:tasks.workunit.client.0.smithi027.stdout:6/290: unlink d1/df/f44 0 2024-04-26T23:01:28.395 INFO:tasks.workunit.client.0.smithi027.stdout:9/302: link d4/l1c d4/d10/d67/l68 0 2024-04-26T23:01:28.395 INFO:tasks.workunit.client.0.smithi027.stdout:4/402: read da/dd/de/d14/d21/f46 [3303531,124414] 0 2024-04-26T23:01:28.396 INFO:tasks.workunit.client.0.smithi027.stdout:9/303: mknod d4/d17/d2e/d46/c69 0 2024-04-26T23:01:28.401 INFO:tasks.workunit.client.0.smithi027.stdout:4/403: unlink da/dd/de/d14/c6f 0 2024-04-26T23:01:28.405 INFO:tasks.workunit.client.0.smithi027.stdout:4/404: creat da/dd/de/d14/d21/d41/d59/f77 x:0 0 0 2024-04-26T23:01:28.414 INFO:tasks.workunit.client.0.smithi027.stdout:7/344: write d0/f2a [2921286,68970] 0 2024-04-26T23:01:28.414 INFO:tasks.workunit.client.0.smithi027.stdout:7/345: write d0/d28/d2b/d3a/f4c [2829817,92717] 0 2024-04-26T23:01:28.415 INFO:tasks.workunit.client.0.smithi027.stdout:7/346: mkdir d0/d28/d2b/d55/d59/d6d 0 2024-04-26T23:01:28.417 INFO:tasks.workunit.client.0.smithi027.stdout:7/347: link d0/d3d/f43 d0/d28/f6e 0 2024-04-26T23:01:28.417 INFO:tasks.workunit.client.0.smithi027.stdout:7/348: chown d0/d28/d2b/d3a/d48 117748248 1 2024-04-26T23:01:28.417 INFO:tasks.workunit.client.0.smithi027.stdout:7/349: read - d0/d28/d2b/d3a/d48/d56/f5b zero size 2024-04-26T23:01:28.417 INFO:tasks.workunit.client.0.smithi027.stdout:7/350: chown d0/d28/d2b/d3a/f3f 758 1 2024-04-26T23:01:28.417 INFO:tasks.workunit.client.0.smithi027.stdout:7/351: write d0/f5a [653563,114788] 0 2024-04-26T23:01:28.417 INFO:tasks.workunit.client.0.smithi027.stdout:7/352: truncate d0/d28/d14/f39 482572 0 2024-04-26T23:01:28.418 INFO:tasks.workunit.client.0.smithi027.stdout:8/410: dwrite d9/f16 [0,4194304] 0 2024-04-26T23:01:28.422 INFO:tasks.workunit.client.0.smithi027.stdout:7/353: creat d0/d28/d2b/d55/f6f x:0 0 0 2024-04-26T23:01:28.423 INFO:tasks.workunit.client.0.smithi027.stdout:7/354: creat d0/d28/d66/f70 x:0 0 0 2024-04-26T23:01:28.423 INFO:tasks.workunit.client.0.smithi027.stdout:0/361: dread d0/d4/da/f11 [0,4194304] 0 2024-04-26T23:01:28.425 INFO:tasks.workunit.client.0.smithi027.stdout:7/355: rmdir d0/d28/d2b/d3a/d48/d68 0 2024-04-26T23:01:28.429 INFO:tasks.workunit.client.0.smithi027.stdout:0/362: mknod d0/d4/da/c6b 0 2024-04-26T23:01:28.429 INFO:tasks.workunit.client.0.smithi027.stdout:2/315: dwrite d2/d5/db/d1b/d1c/d25/d19/d36/d39/f3c [0,4194304] 0 2024-04-26T23:01:28.432 INFO:tasks.workunit.client.0.smithi027.stdout:4/405: dread da/dd/de/d14/d17/d24/f3e [0,4194304] 0 2024-04-26T23:01:28.434 INFO:tasks.workunit.client.0.smithi027.stdout:2/316: mknod d2/d5/db/c57 0 2024-04-26T23:01:28.434 INFO:tasks.workunit.client.0.smithi027.stdout:2/317: truncate d2/d5/db/d1b/d1c/d25/d19/d23/f47 745204 0 2024-04-26T23:01:28.435 INFO:tasks.workunit.client.0.smithi027.stdout:4/406: rmdir da 39 2024-04-26T23:01:28.437 INFO:tasks.workunit.client.0.smithi027.stdout:9/304: dwrite d4/d10/f4a [0,4194304] 0 2024-04-26T23:01:28.437 INFO:tasks.workunit.client.0.smithi027.stdout:9/305: dread - d4/d10/d40/f64 zero size 2024-04-26T23:01:28.437 INFO:tasks.workunit.client.0.smithi027.stdout:4/407: rmdir da/dd/de/d14/d17 39 2024-04-26T23:01:28.438 INFO:tasks.workunit.client.0.smithi027.stdout:9/306: mknod d4/d10/d30/c6a 0 2024-04-26T23:01:28.438 INFO:tasks.workunit.client.0.smithi027.stdout:9/307: chown d4/la 342887420 1 2024-04-26T23:01:28.439 INFO:tasks.workunit.client.0.smithi027.stdout:4/408: creat da/dd/de/d14/d21/d41/d59/f78 x:0 0 0 2024-04-26T23:01:28.441 INFO:tasks.workunit.client.0.smithi027.stdout:8/411: write d9/f17 [2890100,107210] 0 2024-04-26T23:01:28.441 INFO:tasks.workunit.client.0.smithi027.stdout:0/363: dread d0/d4/da/d1a/d2a/f43 [0,4194304] 0 2024-04-26T23:01:28.442 INFO:tasks.workunit.client.0.smithi027.stdout:9/308: rename d4/d10/d40/f48 to d4/d10/d25/f6b 0 2024-04-26T23:01:28.443 INFO:tasks.workunit.client.0.smithi027.stdout:4/409: mkdir da/dd/de/d79 0 2024-04-26T23:01:28.444 INFO:tasks.workunit.client.0.smithi027.stdout:0/364: creat d0/df/d3f/d4a/f6c x:0 0 0 2024-04-26T23:01:28.446 INFO:tasks.workunit.client.0.smithi027.stdout:0/365: mkdir d0/d4/da/d1a/d6d 0 2024-04-26T23:01:28.447 INFO:tasks.workunit.client.0.smithi027.stdout:0/366: readlink d0/d4/da/d1a/l62 0 2024-04-26T23:01:28.448 INFO:tasks.workunit.client.0.smithi027.stdout:0/367: mknod d0/d4/c6e 0 2024-04-26T23:01:28.448 INFO:tasks.workunit.client.0.smithi027.stdout:0/368: truncate d0/df/f4e 1000285 0 2024-04-26T23:01:28.449 INFO:tasks.workunit.client.0.smithi027.stdout:7/356: read d0/f13 [3731889,12743] 0 2024-04-26T23:01:28.449 INFO:tasks.workunit.client.0.smithi027.stdout:7/357: fdatasync d0/d28/f60 0 2024-04-26T23:01:28.449 INFO:tasks.workunit.client.0.smithi027.stdout:7/358: chown d0/f27 97838 1 2024-04-26T23:01:28.449 INFO:tasks.workunit.client.0.smithi027.stdout:7/359: truncate d0/d28/d2b/d3a/d48/f63 562874 0 2024-04-26T23:01:28.450 INFO:tasks.workunit.client.0.smithi027.stdout:0/369: creat d0/d4/da/d15/f6f x:0 0 0 2024-04-26T23:01:28.450 INFO:tasks.workunit.client.0.smithi027.stdout:3/415: dwrite d6/dc/d34/d2e/f66 [0,4194304] 0 2024-04-26T23:01:28.450 INFO:tasks.workunit.client.0.smithi027.stdout:4/410: read da/dd/f30 [530869,40931] 0 2024-04-26T23:01:28.450 INFO:tasks.workunit.client.0.smithi027.stdout:4/411: truncate da/dd/de/d14/d17/f68 5185585 0 2024-04-26T23:01:28.457 INFO:tasks.workunit.client.0.smithi027.stdout:9/309: rename d4/d10/d25/f6b to d4/d17/d2e/d46/f6c 0 2024-04-26T23:01:28.460 INFO:tasks.workunit.client.0.smithi027.stdout:9/310: write d4/f2b [3012072,86881] 0 2024-04-26T23:01:28.460 INFO:tasks.workunit.client.0.smithi027.stdout:9/311: dread - d4/d10/d30/f63 zero size 2024-04-26T23:01:28.460 INFO:tasks.workunit.client.0.smithi027.stdout:9/312: dread - d4/d10/d30/f63 zero size 2024-04-26T23:01:28.461 INFO:tasks.workunit.client.0.smithi027.stdout:0/370: rmdir d0/df 39 2024-04-26T23:01:28.464 INFO:tasks.workunit.client.0.smithi027.stdout:3/416: mknod d6/dc/d2f/d7e/c8f 0 2024-04-26T23:01:28.467 INFO:tasks.workunit.client.0.smithi027.stdout:0/371: rename d0/d4/da/f1f to d0/df/d3f/d64/f70 0 2024-04-26T23:01:28.468 INFO:tasks.workunit.client.0.smithi027.stdout:3/417: mkdir d6/dc/d34/d2e/d5f/d90 0 2024-04-26T23:01:28.468 INFO:tasks.workunit.client.0.smithi027.stdout:0/372: write d0/d4/da/d15/f33 [442657,85407] 0 2024-04-26T23:01:28.470 INFO:tasks.workunit.client.0.smithi027.stdout:3/418: unlink d6/dc/d34/d2e/f67 0 2024-04-26T23:01:28.474 INFO:tasks.workunit.client.0.smithi027.stdout:0/373: creat d0/df/d3f/d64/f71 x:0 0 0 2024-04-26T23:01:28.474 INFO:tasks.workunit.client.0.smithi027.stdout:0/374: read d0/f37 [936832,42933] 0 2024-04-26T23:01:28.474 INFO:tasks.workunit.client.0.smithi027.stdout:5/379: sync 2024-04-26T23:01:28.474 INFO:tasks.workunit.client.0.smithi027.stdout:5/380: stat d4/d24/d22/d29/d39/d61 0 2024-04-26T23:01:28.477 INFO:tasks.workunit.client.0.smithi027.stdout:0/375: unlink d0/l22 0 2024-04-26T23:01:28.490 INFO:tasks.workunit.client.0.smithi027.stdout:0/376: creat d0/df/f72 x:0 0 0 2024-04-26T23:01:28.516 INFO:tasks.workunit.client.0.smithi027.stdout:4/412: dwrite da/dd/de/d14/d21/d4a/f6a [0,4194304] 0 2024-04-26T23:01:28.519 INFO:tasks.workunit.client.0.smithi027.stdout:4/413: link da/dd/de/d14/d21/d4a/f6d da/dd/d5f/f7a 0 2024-04-26T23:01:28.519 INFO:tasks.workunit.client.0.smithi027.stdout:4/414: chown da/dd/de/f3f 43687804 1 2024-04-26T23:01:28.519 INFO:tasks.workunit.client.0.smithi027.stdout:4/415: chown da/dd/de/d14/f44 1969517759 1 2024-04-26T23:01:28.521 INFO:tasks.workunit.client.0.smithi027.stdout:3/419: dwrite d6/dc/d2f/d6b/d74/f87 [0,4194304] 0 2024-04-26T23:01:28.521 INFO:tasks.workunit.client.0.smithi027.stdout:7/360: dwrite d0/d2f/f31 [0,4194304] 0 2024-04-26T23:01:28.523 INFO:tasks.workunit.client.0.smithi027.stdout:3/420: symlink d6/dc/d34/d2e/d70/l91 0 2024-04-26T23:01:28.523 INFO:tasks.workunit.client.0.smithi027.stdout:7/361: write d0/d28/d14/f16 [3800327,27302] 0 2024-04-26T23:01:28.524 INFO:tasks.workunit.client.0.smithi027.stdout:7/362: creat d0/d6c/f71 x:0 0 0 2024-04-26T23:01:28.524 INFO:tasks.workunit.client.0.smithi027.stdout:8/412: write d9/da/d2a/f48 [3793096,59656] 0 2024-04-26T23:01:28.529 INFO:tasks.workunit.client.0.smithi027.stdout:2/318: dwrite d2/d5/db/d1b/d1c/d25/d19/d23/f47 [0,4194304] 0 2024-04-26T23:01:28.530 INFO:tasks.workunit.client.0.smithi027.stdout:7/363: creat d0/d28/d2b/d3a/d48/f72 x:0 0 0 2024-04-26T23:01:28.530 INFO:tasks.workunit.client.0.smithi027.stdout:7/364: write d0/d28/f60 [525908,56945] 0 2024-04-26T23:01:28.534 INFO:tasks.workunit.client.0.smithi027.stdout:7/365: getdents d0/d28/d2b/d3a/d48/d56 0 2024-04-26T23:01:28.547 INFO:tasks.workunit.client.0.smithi027.stdout:3/421: dread d6/dc/d17/f33 [0,4194304] 0 2024-04-26T23:01:28.547 INFO:tasks.workunit.client.0.smithi027.stdout:3/422: read - d6/f82 zero size 2024-04-26T23:01:28.548 INFO:tasks.workunit.client.0.smithi027.stdout:0/377: dwrite d0/d4/da/d1a/f3b [4194304,4194304] 0 2024-04-26T23:01:28.550 INFO:tasks.workunit.client.0.smithi027.stdout:0/378: mknod d0/df/c73 0 2024-04-26T23:01:28.550 INFO:tasks.workunit.client.0.smithi027.stdout:0/379: getdents d0/d4/da/d1a/d59 0 2024-04-26T23:01:28.550 INFO:tasks.workunit.client.0.smithi027.stdout:0/380: truncate d0/d4/da/d15/f33 1714016 0 2024-04-26T23:01:28.550 INFO:tasks.workunit.client.0.smithi027.stdout:0/381: stat d0/d4/f20 0 2024-04-26T23:01:28.551 INFO:tasks.workunit.client.0.smithi027.stdout:0/382: creat d0/d4/da/d1a/d2a/f74 x:0 0 0 2024-04-26T23:01:28.553 INFO:tasks.workunit.client.0.smithi027.stdout:0/383: link d0/d4/da/d1a/c42 d0/d4/da/d1a/d59/c75 0 2024-04-26T23:01:28.555 INFO:tasks.workunit.client.0.smithi027.stdout:5/381: dwrite d4/db/d16/d4b/f77 [0,4194304] 0 2024-04-26T23:01:28.570 INFO:tasks.workunit.client.0.smithi027.stdout:7/366: write d0/d28/f2c [1247004,15743] 0 2024-04-26T23:01:28.591 INFO:tasks.workunit.client.0.smithi027.stdout:7/367: write d0/d28/d2b/d3a/f3f [1471167,2460] 0 2024-04-26T23:01:28.592 INFO:tasks.workunit.client.0.smithi027.stdout:4/416: dwrite da/dd/de/d14/d21/d4a/f4f [0,4194304] 0 2024-04-26T23:01:28.594 INFO:tasks.workunit.client.0.smithi027.stdout:7/368: mkdir d0/d28/d14/d73 0 2024-04-26T23:01:28.594 INFO:tasks.workunit.client.0.smithi027.stdout:7/369: dread - d0/d28/d2b/d3a/d48/f72 zero size 2024-04-26T23:01:28.598 INFO:tasks.workunit.client.0.smithi027.stdout:7/370: link d0/d28/f18 d0/d28/d2b/d3a/d48/d56/f74 0 2024-04-26T23:01:28.602 INFO:tasks.workunit.client.0.smithi027.stdout:7/371: getdents d0/d28/d2b/d55/d59 0 2024-04-26T23:01:28.602 INFO:tasks.workunit.client.0.smithi027.stdout:7/372: read - d0/d28/d2b/d3a/d48/d56/f5b zero size 2024-04-26T23:01:28.607 INFO:tasks.workunit.client.0.smithi027.stdout:2/319: dwrite d2/d5/db/d1b/d1c/d25/d19/d36/f43 [0,4194304] 0 2024-04-26T23:01:28.610 INFO:tasks.workunit.client.0.smithi027.stdout:2/320: mkdir d2/d5/db/d1b/d1c/d25/d19/d36/d39/d58 0 2024-04-26T23:01:28.611 INFO:tasks.workunit.client.0.smithi027.stdout:2/321: chown d2/d5/db/d1b/d1c/d25/d19/d36 3618141 1 2024-04-26T23:01:28.616 INFO:tasks.workunit.client.0.smithi027.stdout:2/322: rename d2/d5/db/d1b/d1c/d25/d19/d23/f2c to d2/d5/db/d1b/d1c/d25/f59 0 2024-04-26T23:01:28.617 INFO:tasks.workunit.client.0.smithi027.stdout:4/417: dread da/dd/de/d14/d17/f20 [0,4194304] 0 2024-04-26T23:01:28.617 INFO:tasks.workunit.client.0.smithi027.stdout:4/418: chown da/dd/de/d14/d17/f2a 2 1 2024-04-26T23:01:28.619 INFO:tasks.workunit.client.0.smithi027.stdout:4/419: creat da/dd/de/d79/f7b x:0 0 0 2024-04-26T23:01:28.620 INFO:tasks.workunit.client.0.smithi027.stdout:5/382: dread d4/db/d16/f1f [4194304,4194304] 0 2024-04-26T23:01:28.622 INFO:tasks.workunit.client.0.smithi027.stdout:5/383: symlink d4/d1a/l7a 0 2024-04-26T23:01:28.624 INFO:tasks.workunit.client.0.smithi027.stdout:8/413: dwrite d9/da/d2a/d5e/d46/f6f [0,4194304] 0 2024-04-26T23:01:28.629 INFO:tasks.workunit.client.0.smithi027.stdout:7/373: dread d0/f30 [0,4194304] 0 2024-04-26T23:01:28.632 INFO:tasks.workunit.client.0.smithi027.stdout:3/423: dwrite d6/dc/f11 [0,4194304] 0 2024-04-26T23:01:28.658 INFO:tasks.workunit.client.0.smithi027.stdout:0/384: dwrite d0/d4/f3d [0,4194304] 0 2024-04-26T23:01:28.658 INFO:tasks.workunit.client.0.smithi027.stdout:0/385: fdatasync d0/f37 0 2024-04-26T23:01:28.666 INFO:tasks.workunit.client.0.smithi027.stdout:7/374: dread d0/d28/f60 [0,4194304] 0 2024-04-26T23:01:28.677 INFO:tasks.workunit.client.0.smithi027.stdout:0/386: unlink d0/d4/da/d15/d55/f66 0 2024-04-26T23:01:28.677 INFO:tasks.workunit.client.0.smithi027.stdout:8/414: creat d9/da/d2a/d5e/d72/d84/d35/f99 x:0 0 0 2024-04-26T23:01:28.684 INFO:tasks.workunit.client.0.smithi027.stdout:7/375: unlink d0/d28/d2b/d3a/d48/d56/f5b 0 2024-04-26T23:01:28.685 INFO:tasks.workunit.client.0.smithi027.stdout:4/420: dwrite da/dd/de/d14/f53 [0,4194304] 0 2024-04-26T23:01:28.685 INFO:tasks.workunit.client.0.smithi027.stdout:8/415: rmdir d9/da/d2a/d6b 39 2024-04-26T23:01:28.685 INFO:tasks.workunit.client.0.smithi027.stdout:0/387: creat d0/d4/da/d15/f76 x:0 0 0 2024-04-26T23:01:28.688 INFO:tasks.workunit.client.0.smithi027.stdout:8/416: write d9/da/d2a/d5e/d72/d84/f29 [338746,73412] 0 2024-04-26T23:01:28.688 INFO:tasks.workunit.client.0.smithi027.stdout:3/424: dwrite d6/dc/f21 [4194304,4194304] 0 2024-04-26T23:01:28.693 INFO:tasks.workunit.client.0.smithi027.stdout:1/439: sync 2024-04-26T23:01:28.693 INFO:tasks.workunit.client.0.smithi027.stdout:1/440: write d1/d5/d21/f34 [3298752,127876] 0 2024-04-26T23:01:28.693 INFO:tasks.workunit.client.0.smithi027.stdout:9/313: sync 2024-04-26T23:01:28.693 INFO:tasks.workunit.client.0.smithi027.stdout:6/291: sync 2024-04-26T23:01:28.693 INFO:tasks.workunit.client.0.smithi027.stdout:0/388: link d0/d4/da/d1a/d2a/f74 d0/d4/da/d15/d55/f77 0 2024-04-26T23:01:28.693 INFO:tasks.workunit.client.0.smithi027.stdout:0/389: stat d0/l10 0 2024-04-26T23:01:28.694 INFO:tasks.workunit.client.0.smithi027.stdout:3/425: creat d6/dc/d34/d47/d7b/f92 x:0 0 0 2024-04-26T23:01:28.702 INFO:tasks.workunit.client.0.smithi027.stdout:0/390: rename d0/d4/da/d1a/d1e/d52/f67 to d0/d4/da/f78 0 2024-04-26T23:01:28.703 INFO:tasks.workunit.client.0.smithi027.stdout:0/391: chown d0/d4/da/d15/d55/f77 2924042 1 2024-04-26T23:01:28.703 INFO:tasks.workunit.client.0.smithi027.stdout:0/392: stat d0/df/d3f/f61 0 2024-04-26T23:01:28.703 INFO:tasks.workunit.client.0.smithi027.stdout:1/441: unlink d1/l20 0 2024-04-26T23:01:28.714 INFO:tasks.workunit.client.0.smithi027.stdout:2/323: dwrite d2/d5/db/de/d21/f51 [0,4194304] 0 2024-04-26T23:01:28.721 INFO:tasks.workunit.client.0.smithi027.stdout:8/417: write d9/de/d2e/f53 [420286,62605] 0 2024-04-26T23:01:28.730 INFO:tasks.workunit.client.0.smithi027.stdout:4/421: dwrite da/dd/de/d14/d21/d41/d59/f78 [0,4194304] 0 2024-04-26T23:01:28.739 INFO:tasks.workunit.client.0.smithi027.stdout:3/426: dread d6/dc/f11 [0,4194304] 0 2024-04-26T23:01:28.739 INFO:tasks.workunit.client.0.smithi027.stdout:3/427: stat d6/dc/d88/l8e 0 2024-04-26T23:01:28.749 INFO:tasks.workunit.client.0.smithi027.stdout:6/292: dread d1/df/f31 [0,4194304] 0 2024-04-26T23:01:28.753 INFO:tasks.workunit.client.0.smithi027.stdout:2/324: dread d2/d5/db/d1b/d1c/d25/f3d [0,4194304] 0 2024-04-26T23:01:28.753 INFO:tasks.workunit.client.0.smithi027.stdout:4/422: mknod da/dd/d5e/c7c 0 2024-04-26T23:01:28.753 INFO:tasks.workunit.client.0.smithi027.stdout:8/418: creat d9/da/d2a/d5e/d72/d84/d5d/d8b/f9a x:0 0 0 2024-04-26T23:01:28.754 INFO:tasks.workunit.client.0.smithi027.stdout:5/384: dwrite d4/d1a/f57 [0,4194304] 0 2024-04-26T23:01:28.755 INFO:tasks.workunit.client.0.smithi027.stdout:6/293: mkdir d1/d30/d6c 0 2024-04-26T23:01:28.757 INFO:tasks.workunit.client.0.smithi027.stdout:2/325: rename d2/d5/db/d1b/l2e to d2/d5/db/d1b/d1c/d25/d19/d44/l5a 0 2024-04-26T23:01:28.758 INFO:tasks.workunit.client.0.smithi027.stdout:2/326: chown d2/d5/db/d1b/d1c/d25/c52 228952 1 2024-04-26T23:01:28.760 INFO:tasks.workunit.client.0.smithi027.stdout:0/393: dwrite d0/d4/da/f19 [0,4194304] 0 2024-04-26T23:01:28.764 INFO:tasks.workunit.client.0.smithi027.stdout:6/294: symlink d1/d30/l6d 0 2024-04-26T23:01:28.764 INFO:tasks.workunit.client.0.smithi027.stdout:2/327: stat d2/d5/db/l54 0 2024-04-26T23:01:28.765 INFO:tasks.workunit.client.0.smithi027.stdout:6/295: creat d1/df/d43/f6e x:0 0 0 2024-04-26T23:01:28.767 INFO:tasks.workunit.client.0.smithi027.stdout:8/419: creat d9/da/d2a/d5e/d2b/d3b/f9b x:0 0 0 2024-04-26T23:01:28.767 INFO:tasks.workunit.client.0.smithi027.stdout:8/420: stat d9/da/d2a/d5e/d72/d84/d5d/d8b 0 2024-04-26T23:01:28.768 INFO:tasks.workunit.client.0.smithi027.stdout:8/421: truncate d9/f74 736213 0 2024-04-26T23:01:28.770 INFO:tasks.workunit.client.0.smithi027.stdout:9/314: dwrite d4/d17/f29 [0,4194304] 0 2024-04-26T23:01:28.770 INFO:tasks.workunit.client.0.smithi027.stdout:9/315: fdatasync d4/d17/f45 0 2024-04-26T23:01:28.771 INFO:tasks.workunit.client.0.smithi027.stdout:9/316: link d4/d17/l4c d4/d17/l6d 0 2024-04-26T23:01:28.772 INFO:tasks.workunit.client.0.smithi027.stdout:8/422: creat d9/da/d2a/d6b/f9c x:0 0 0 2024-04-26T23:01:28.773 INFO:tasks.workunit.client.0.smithi027.stdout:9/317: mkdir d4/d10/d40/d58/d6e 0 2024-04-26T23:01:28.773 INFO:tasks.workunit.client.0.smithi027.stdout:9/318: dread - d4/d10/f56 zero size 2024-04-26T23:01:28.776 INFO:tasks.workunit.client.0.smithi027.stdout:8/423: rename d9/de/c4a to d9/da/d2a/d5e/d72/d84/d5d/c9d 0 2024-04-26T23:01:28.781 INFO:tasks.workunit.client.0.smithi027.stdout:8/424: mkdir d9/da/d2a/d5e/d9e 0 2024-04-26T23:01:28.785 INFO:tasks.workunit.client.0.smithi027.stdout:8/425: mknod d9/da/d2a/d5e/d72/d84/d5d/d6c/c9f 0 2024-04-26T23:01:28.795 INFO:tasks.workunit.client.0.smithi027.stdout:3/428: dwrite d6/dc/d34/d2e/f66 [0,4194304] 0 2024-04-26T23:01:28.798 INFO:tasks.workunit.client.0.smithi027.stdout:5/385: dwrite d4/d1a/d4a/d68/d40/d54/f75 [0,4194304] 0 2024-04-26T23:01:28.802 INFO:tasks.workunit.client.0.smithi027.stdout:3/429: rmdir d6/dc/d17/d52 39 2024-04-26T23:01:28.803 INFO:tasks.workunit.client.0.smithi027.stdout:3/430: truncate d6/dc/d34/d2e/d50/f8a 203226 0 2024-04-26T23:01:28.803 INFO:tasks.workunit.client.0.smithi027.stdout:5/386: unlink d4/db/d16/f38 0 2024-04-26T23:01:28.807 INFO:tasks.workunit.client.0.smithi027.stdout:5/387: dread d4/d1a/d4a/f76 [0,4194304] 0 2024-04-26T23:01:28.807 INFO:tasks.workunit.client.0.smithi027.stdout:5/388: write d4/d24/f42 [4394976,20685] 0 2024-04-26T23:01:28.808 INFO:tasks.workunit.client.0.smithi027.stdout:5/389: creat d4/d1a/d2b/f7b x:0 0 0 2024-04-26T23:01:28.810 INFO:tasks.workunit.client.0.smithi027.stdout:9/319: dread d4/d17/d2e/f3d [0,4194304] 0 2024-04-26T23:01:28.811 INFO:tasks.workunit.client.0.smithi027.stdout:9/320: chown d4/d10/d40/f64 83407841 1 2024-04-26T23:01:28.811 INFO:tasks.workunit.client.0.smithi027.stdout:9/321: fdatasync d4/f2b 0 2024-04-26T23:01:28.811 INFO:tasks.workunit.client.0.smithi027.stdout:9/322: symlink d4/d10/d2c/l6f 0 2024-04-26T23:01:28.811 INFO:tasks.workunit.client.0.smithi027.stdout:9/323: fdatasync d4/d10/d40/f5d 0 2024-04-26T23:01:28.816 INFO:tasks.workunit.client.0.smithi027.stdout:1/442: rmdir d1/d5/d21 39 2024-04-26T23:01:28.816 INFO:tasks.workunit.client.0.smithi027.stdout:6/296: rmdir d1/d30 39 2024-04-26T23:01:28.817 INFO:tasks.workunit.client.0.smithi027.stdout:1/443: mkdir d1/d26/d36/d88 0 2024-04-26T23:01:28.819 INFO:tasks.workunit.client.0.smithi027.stdout:3/431: read d6/dc/d2f/d6b/d74/f87 [1265382,121823] 0 2024-04-26T23:01:28.819 INFO:tasks.workunit.client.0.smithi027.stdout:6/297: mkdir d1/d30/d6c/d6f 0 2024-04-26T23:01:28.819 INFO:tasks.workunit.client.0.smithi027.stdout:6/298: chown d1/df/c25 1226333 1 2024-04-26T23:01:28.820 INFO:tasks.workunit.client.0.smithi027.stdout:1/444: unlink d1/d5/d21/c61 0 2024-04-26T23:01:28.821 INFO:tasks.workunit.client.0.smithi027.stdout:3/432: link d6/f82 d6/dc/d34/d47/d7b/f93 0 2024-04-26T23:01:28.821 INFO:tasks.workunit.client.0.smithi027.stdout:3/433: stat d6/dc/f11 0 2024-04-26T23:01:28.821 INFO:tasks.workunit.client.0.smithi027.stdout:3/434: read - d6/dc/d34/d2e/d70/f7d zero size 2024-04-26T23:01:28.822 INFO:tasks.workunit.client.0.smithi027.stdout:6/299: link d1/df/d43/c50 d1/de/c70 0 2024-04-26T23:01:28.824 INFO:tasks.workunit.client.0.smithi027.stdout:1/445: creat d1/d26/f89 x:0 0 0 2024-04-26T23:01:28.824 INFO:tasks.workunit.client.0.smithi027.stdout:1/446: write d1/d26/d66/d73/f7c [624862,84809] 0 2024-04-26T23:01:28.827 INFO:tasks.workunit.client.0.smithi027.stdout:3/435: getdents d6/dc/d34/d47 0 2024-04-26T23:01:28.828 INFO:tasks.workunit.client.0.smithi027.stdout:3/436: write d6/dc/fd [516414,43768] 0 2024-04-26T23:01:28.828 INFO:tasks.workunit.client.0.smithi027.stdout:3/437: fsync d6/f8 0 2024-04-26T23:01:28.828 INFO:tasks.workunit.client.0.smithi027.stdout:0/394: dwrite d0/d4/da/d15/f44 [0,4194304] 0 2024-04-26T23:01:28.828 INFO:tasks.workunit.client.0.smithi027.stdout:0/395: fdatasync d0/df/d3f/d4a/f60 0 2024-04-26T23:01:28.828 INFO:tasks.workunit.client.0.smithi027.stdout:6/300: rename d1/df/l54 to d1/df/d43/d56/l71 0 2024-04-26T23:01:28.828 INFO:tasks.workunit.client.0.smithi027.stdout:6/301: creat d1/f72 x:0 0 0 2024-04-26T23:01:28.829 INFO:tasks.workunit.client.0.smithi027.stdout:6/302: stat d1/l2d 0 2024-04-26T23:01:28.829 INFO:tasks.workunit.client.0.smithi027.stdout:6/303: chown d1/df 706 1 2024-04-26T23:01:28.829 INFO:tasks.workunit.client.0.smithi027.stdout:9/324: read d4/d10/f28 [1248988,60569] 0 2024-04-26T23:01:28.829 INFO:tasks.workunit.client.0.smithi027.stdout:6/304: mknod d1/df/d49/c73 0 2024-04-26T23:01:28.833 INFO:tasks.workunit.client.0.smithi027.stdout:9/325: mkdir d4/d17/d2e/d70 0 2024-04-26T23:01:28.833 INFO:tasks.workunit.client.0.smithi027.stdout:9/326: readlink d4/d17/l4c 0 2024-04-26T23:01:28.834 INFO:tasks.workunit.client.0.smithi027.stdout:9/327: link d4/d10/c3f d4/d10/d40/c71 0 2024-04-26T23:01:28.834 INFO:tasks.workunit.client.0.smithi027.stdout:9/328: chown d4/d17/d2e/d70 72119 1 2024-04-26T23:01:28.838 INFO:tasks.workunit.client.0.smithi027.stdout:9/329: mkdir d4/d10/d2c/d72 0 2024-04-26T23:01:28.838 INFO:tasks.workunit.client.0.smithi027.stdout:9/330: write d4/d10/f56 [705836,60771] 0 2024-04-26T23:01:28.839 INFO:tasks.workunit.client.0.smithi027.stdout:9/331: getdents d4/d10/d40/d58 0 2024-04-26T23:01:28.839 INFO:tasks.workunit.client.0.smithi027.stdout:9/332: chown d4/d10/c16 0 1 2024-04-26T23:01:28.840 INFO:tasks.workunit.client.0.smithi027.stdout:9/333: chown d4 221 1 2024-04-26T23:01:28.840 INFO:tasks.workunit.client.0.smithi027.stdout:9/334: symlink d4/d10/d40/d58/d6e/l73 0 2024-04-26T23:01:28.842 INFO:tasks.workunit.client.0.smithi027.stdout:9/335: write d4/d17/d2e/d46/f62 [3615226,92160] 0 2024-04-26T23:01:28.856 INFO:tasks.workunit.client.0.smithi027.stdout:0/396: read d0/d4/da/d1a/f3b [6893900,63298] 0 2024-04-26T23:01:28.859 INFO:tasks.workunit.client.0.smithi027.stdout:0/397: unlink d0/d4/da/d15/f44 0 2024-04-26T23:01:28.860 INFO:tasks.workunit.client.0.smithi027.stdout:6/305: dread d1/df/f2e [0,4194304] 0 2024-04-26T23:01:28.862 INFO:tasks.workunit.client.0.smithi027.stdout:0/398: creat d0/d4/da/d15/d26/f79 x:0 0 0 2024-04-26T23:01:28.865 INFO:tasks.workunit.client.0.smithi027.stdout:5/390: dwrite d4/f15 [0,4194304] 0 2024-04-26T23:01:28.867 INFO:tasks.workunit.client.0.smithi027.stdout:6/306: creat d1/df/d43/f74 x:0 0 0 2024-04-26T23:01:28.872 INFO:tasks.workunit.client.0.smithi027.stdout:6/307: creat d1/df/d3c/f75 x:0 0 0 2024-04-26T23:01:28.890 INFO:tasks.workunit.client.0.smithi027.stdout:1/447: dwrite d1/d26/d66/f77 [0,4194304] 0 2024-04-26T23:01:28.901 INFO:tasks.workunit.client.0.smithi027.stdout:3/438: dwrite d6/dc/d34/d2e/f45 [4194304,4194304] 0 2024-04-26T23:01:28.901 INFO:tasks.workunit.client.0.smithi027.stdout:3/439: truncate d6/dc/d62/f7c 1149186 0 2024-04-26T23:01:28.915 INFO:tasks.workunit.client.0.smithi027.stdout:6/308: dread d1/df/f31 [0,4194304] 0 2024-04-26T23:01:28.921 INFO:tasks.workunit.client.0.smithi027.stdout:5/391: dwrite d4/d1a/d4a/d68/d40/d54/f73 [0,4194304] 0 2024-04-26T23:01:28.921 INFO:tasks.workunit.client.0.smithi027.stdout:5/392: truncate d4/d1a/d4a/d68/d40/d54/f75 4952115 0 2024-04-26T23:01:28.927 INFO:tasks.workunit.client.0.smithi027.stdout:0/399: dwrite d0/d4/da/d15/d55/f77 [0,4194304] 0 2024-04-26T23:01:28.928 INFO:tasks.workunit.client.0.smithi027.stdout:0/400: truncate d0/d4/da/d1a/d2a/f43 773853 0 2024-04-26T23:01:28.977 INFO:tasks.workunit.client.0.smithi027.stdout:0/401: dwrite d0/d4/da/d15/f5a [0,4194304] 0 2024-04-26T23:01:28.977 INFO:tasks.workunit.client.0.smithi027.stdout:0/402: stat d0/df/c73 0 2024-04-26T23:01:29.000 INFO:tasks.workunit.client.0.smithi027.stdout:1/448: mknod d1/d26/d36/d88/c8a 0 2024-04-26T23:01:29.005 INFO:tasks.workunit.client.0.smithi027.stdout:3/440: mknod d6/dc/d34/d2e/d50/c94 0 2024-04-26T23:01:29.007 INFO:tasks.workunit.client.0.smithi027.stdout:6/309: link d1/df/fd d1/df/d43/d56/d6b/f76 0 2024-04-26T23:01:29.008 INFO:tasks.workunit.client.0.smithi027.stdout:6/310: chown d1/df/d43/f6e 32 1 2024-04-26T23:01:29.008 INFO:tasks.workunit.client.0.smithi027.stdout:6/311: truncate d1/df/d49/f4f 393169 0 2024-04-26T23:01:29.011 INFO:tasks.workunit.client.0.smithi027.stdout:5/393: mknod d4/d24/d22/d29/c7c 0 2024-04-26T23:01:29.011 INFO:tasks.workunit.client.0.smithi027.stdout:5/394: chown d4/d1a/d2b/f7b 61717 1 2024-04-26T23:01:29.019 INFO:tasks.workunit.client.0.smithi027.stdout:0/403: symlink d0/d4/da/d15/l7a 0 2024-04-26T23:01:29.019 INFO:tasks.workunit.client.0.smithi027.stdout:0/404: chown d0/d4/c47 21994761 1 2024-04-26T23:01:29.019 INFO:tasks.workunit.client.0.smithi027.stdout:0/405: write d0/df/d3f/f61 [615470,107725] 0 2024-04-26T23:01:29.021 INFO:tasks.workunit.client.0.smithi027.stdout:1/449: dread d1/d5/d21/f27 [0,4194304] 0 2024-04-26T23:01:29.023 INFO:tasks.workunit.client.0.smithi027.stdout:3/441: unlink d6/l30 0 2024-04-26T23:01:29.027 INFO:tasks.workunit.client.0.smithi027.stdout:6/312: rename d1/df/d49/f52 to d1/df/d43/d56/f77 0 2024-04-26T23:01:29.028 INFO:tasks.workunit.client.0.smithi027.stdout:1/450: creat d1/d5/f8b x:0 0 0 2024-04-26T23:01:29.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/451: symlink d1/d5/d21/l8c 0 2024-04-26T23:01:29.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/452: fsync d1/dc/f5b 0 2024-04-26T23:01:29.030 INFO:tasks.workunit.client.0.smithi027.stdout:0/406: rmdir d0/d4/da/d1a/d1e/d52 0 2024-04-26T23:01:29.046 INFO:tasks.workunit.client.0.smithi027.stdout:1/453: dread d1/f1b [0,4194304] 0 2024-04-26T23:01:29.050 INFO:tasks.workunit.client.0.smithi027.stdout:7/376: sync 2024-04-26T23:01:29.050 INFO:tasks.workunit.client.0.smithi027.stdout:7/377: fdatasync d0/d28/d2b/d3a/f52 0 2024-04-26T23:01:29.051 INFO:tasks.workunit.client.0.smithi027.stdout:7/378: mkdir d0/d28/d2b/d55/d69/d75 0 2024-04-26T23:01:29.053 INFO:tasks.workunit.client.0.smithi027.stdout:7/379: unlink d0/d28/d14/f3e 0 2024-04-26T23:01:29.058 INFO:tasks.workunit.client.0.smithi027.stdout:7/380: symlink d0/d2f/l76 0 2024-04-26T23:01:29.059 INFO:tasks.workunit.client.0.smithi027.stdout:7/381: dread - d0/d28/f6e zero size 2024-04-26T23:01:29.066 INFO:tasks.workunit.client.0.smithi027.stdout:7/382: dread d0/d28/d2b/d3a/d48/f63 [0,4194304] 0 2024-04-26T23:01:29.067 INFO:tasks.workunit.client.0.smithi027.stdout:7/383: read d0/d28/d2b/d3a/d48/f63 [221818,67285] 0 2024-04-26T23:01:29.074 INFO:tasks.workunit.client.0.smithi027.stdout:1/454: write d1/f32 [6754048,102075] 0 2024-04-26T23:01:29.080 INFO:tasks.workunit.client.0.smithi027.stdout:1/455: mkdir d1/d26/d66/d8d 0 2024-04-26T23:01:29.080 INFO:tasks.workunit.client.0.smithi027.stdout:1/456: write d1/d5/f2f [1503996,31380] 0 2024-04-26T23:01:29.080 INFO:tasks.workunit.client.0.smithi027.stdout:1/457: chown d1/d2a 2 1 2024-04-26T23:01:29.080 INFO:tasks.workunit.client.0.smithi027.stdout:1/458: write d1/d5/d21/f22 [1815583,114511] 0 2024-04-26T23:01:29.087 INFO:tasks.workunit.client.0.smithi027.stdout:3/442: write d6/dc/f85 [3037912,1107] 0 2024-04-26T23:01:29.087 INFO:tasks.workunit.client.0.smithi027.stdout:3/443: rename d6/dc/d17 to d6/dc/d17/d52/d95 22 2024-04-26T23:01:29.088 INFO:tasks.workunit.client.0.smithi027.stdout:5/395: dwrite d4/d1a/f20 [0,4194304] 0 2024-04-26T23:01:29.088 INFO:tasks.workunit.client.0.smithi027.stdout:5/396: chown d4/d1a/d4a/d68/d40/f6c 31487064 1 2024-04-26T23:01:29.088 INFO:tasks.workunit.client.0.smithi027.stdout:5/397: write d4/f4e [1580374,34383] 0 2024-04-26T23:01:29.089 INFO:tasks.workunit.client.0.smithi027.stdout:5/398: getdents d4/d1a/d2b 0 2024-04-26T23:01:29.089 INFO:tasks.workunit.client.0.smithi027.stdout:5/399: stat d4/d1a/d4a/f76 0 2024-04-26T23:01:29.091 INFO:tasks.workunit.client.0.smithi027.stdout:1/459: dread d1/d26/d7d/f30 [0,4194304] 0 2024-04-26T23:01:29.092 INFO:tasks.workunit.client.0.smithi027.stdout:1/460: chown d1/d26/d6b/d42/l69 105100 1 2024-04-26T23:01:29.094 INFO:tasks.workunit.client.0.smithi027.stdout:1/461: mkdir d1/d26/d6b/d8e 0 2024-04-26T23:01:29.096 INFO:tasks.workunit.client.0.smithi027.stdout:0/407: dwrite d0/d4/f20 [0,4194304] 0 2024-04-26T23:01:29.096 INFO:tasks.workunit.client.0.smithi027.stdout:7/384: dread d0/d28/f18 [0,4194304] 0 2024-04-26T23:01:29.096 INFO:tasks.workunit.client.0.smithi027.stdout:0/408: fsync d0/d4/da/d15/d26/f79 0 2024-04-26T23:01:29.102 INFO:tasks.workunit.client.0.smithi027.stdout:7/385: rename d0/d2f/d4d/f64 to d0/d28/f77 0 2024-04-26T23:01:29.102 INFO:tasks.workunit.client.0.smithi027.stdout:6/313: dwrite d1/df/d3c/f75 [0,4194304] 0 2024-04-26T23:01:29.114 INFO:tasks.workunit.client.0.smithi027.stdout:0/409: dread d0/d4/da/f2f [0,4194304] 0 2024-04-26T23:01:29.116 INFO:tasks.workunit.client.0.smithi027.stdout:3/444: dread d6/dc/d17/f33 [0,4194304] 0 2024-04-26T23:01:29.125 INFO:tasks.workunit.client.0.smithi027.stdout:7/386: link d0/d28/d2b/d3a/f52 d0/d28/d14/d73/f78 0 2024-04-26T23:01:29.125 INFO:tasks.workunit.client.0.smithi027.stdout:7/387: write d0/d28/d2b/f34 [707784,15437] 0 2024-04-26T23:01:29.126 INFO:tasks.workunit.client.0.smithi027.stdout:0/410: rename d0/d4/da/d1a/c51 to d0/d4/da/c7b 0 2024-04-26T23:01:29.128 INFO:tasks.workunit.client.0.smithi027.stdout:7/388: mkdir d0/d6c/d79 0 2024-04-26T23:01:29.130 INFO:tasks.workunit.client.0.smithi027.stdout:0/411: creat d0/d4/da/d15/d26/f7c x:0 0 0 2024-04-26T23:01:29.130 INFO:tasks.workunit.client.0.smithi027.stdout:0/412: chown d0/df/d3f/d64/l4b 0 1 2024-04-26T23:01:29.132 INFO:tasks.workunit.client.0.smithi027.stdout:4/423: sync 2024-04-26T23:01:29.132 INFO:tasks.workunit.client.0.smithi027.stdout:8/426: sync 2024-04-26T23:01:29.133 INFO:tasks.workunit.client.0.smithi027.stdout:2/328: sync 2024-04-26T23:01:29.133 INFO:tasks.workunit.client.0.smithi027.stdout:9/336: sync 2024-04-26T23:01:29.136 INFO:tasks.workunit.client.0.smithi027.stdout:4/424: creat da/dd/de/d14/d21/f7d x:0 0 0 2024-04-26T23:01:29.136 INFO:tasks.workunit.client.0.smithi027.stdout:4/425: stat da/dd/de/d14/d17/f28 0 2024-04-26T23:01:29.140 INFO:tasks.workunit.client.0.smithi027.stdout:4/426: creat da/dd/de/d14/d21/d41/d59/d74/f7e x:0 0 0 2024-04-26T23:01:29.140 INFO:tasks.workunit.client.0.smithi027.stdout:4/427: symlink da/dd/de/d14/d17/d24/l7f 0 2024-04-26T23:01:29.142 INFO:tasks.workunit.client.0.smithi027.stdout:4/428: getdents da 0 2024-04-26T23:01:29.147 INFO:tasks.workunit.client.0.smithi027.stdout:2/329: dread d2/d5/db/d1b/d1c/d25/f4a [0,4194304] 0 2024-04-26T23:01:29.155 INFO:tasks.workunit.client.0.smithi027.stdout:9/337: write d4/d17/d2e/f3d [570829,79088] 0 2024-04-26T23:01:29.155 INFO:tasks.workunit.client.0.smithi027.stdout:9/338: chown d4/d17/d2e/f3d 86732400 1 2024-04-26T23:01:29.158 INFO:tasks.workunit.client.0.smithi027.stdout:9/339: rmdir d4/d10/d40 39 2024-04-26T23:01:29.159 INFO:tasks.workunit.client.0.smithi027.stdout:5/400: dread d4/f4e [0,4194304] 0 2024-04-26T23:01:29.160 INFO:tasks.workunit.client.0.smithi027.stdout:9/340: symlink d4/d17/d2e/d70/l74 0 2024-04-26T23:01:29.160 INFO:tasks.workunit.client.0.smithi027.stdout:9/341: fsync d4/d10/f5e 0 2024-04-26T23:01:29.160 INFO:tasks.workunit.client.0.smithi027.stdout:9/342: chown d4/d10/d40/d58/d6e 309902 1 2024-04-26T23:01:29.161 INFO:tasks.workunit.client.0.smithi027.stdout:5/401: truncate d4/db/f10 3045358 0 2024-04-26T23:01:29.167 INFO:tasks.workunit.client.0.smithi027.stdout:5/402: unlink d4/d1a/f57 0 2024-04-26T23:01:29.168 INFO:tasks.workunit.client.0.smithi027.stdout:8/427: creat d9/da/d2a/d5e/d2b/fa0 x:0 0 0 2024-04-26T23:01:29.168 INFO:tasks.workunit.client.0.smithi027.stdout:5/403: rmdir d4/db/d16/d4b/d59 0 2024-04-26T23:01:29.171 INFO:tasks.workunit.client.0.smithi027.stdout:8/428: truncate d9/f74 135449 0 2024-04-26T23:01:29.179 INFO:tasks.workunit.client.0.smithi027.stdout:7/389: dwrite d0/f30 [0,4194304] 0 2024-04-26T23:01:29.179 INFO:tasks.workunit.client.0.smithi027.stdout:7/390: readlink d0/d2f/l76 0 2024-04-26T23:01:29.179 INFO:tasks.workunit.client.0.smithi027.stdout:7/391: truncate d0/d28/f18 5087173 0 2024-04-26T23:01:29.179 INFO:tasks.workunit.client.0.smithi027.stdout:7/392: truncate d0/d28/d2b/d3a/d48/f63 714689 0 2024-04-26T23:01:29.180 INFO:tasks.workunit.client.0.smithi027.stdout:2/330: dread d2/d5/f34 [0,4194304] 0 2024-04-26T23:01:29.180 INFO:tasks.workunit.client.0.smithi027.stdout:2/331: getdents d2/d5/db/d1b/d1c 0 2024-04-26T23:01:29.182 INFO:tasks.workunit.client.0.smithi027.stdout:5/404: dread d4/db/f23 [0,4194304] 0 2024-04-26T23:01:29.182 INFO:tasks.workunit.client.0.smithi027.stdout:7/393: mkdir d0/d28/d2b/d55/d69/d75/d7a 0 2024-04-26T23:01:29.184 INFO:tasks.workunit.client.0.smithi027.stdout:2/332: creat d2/d5/db/d1b/d1c/d25/d19/f5b x:0 0 0 2024-04-26T23:01:29.184 INFO:tasks.workunit.client.0.smithi027.stdout:7/394: getdents d0/d28/d2b/d3a/d48 0 2024-04-26T23:01:29.187 INFO:tasks.workunit.client.0.smithi027.stdout:5/405: mknod d4/d1a/d2b/c7d 0 2024-04-26T23:01:29.190 INFO:tasks.workunit.client.0.smithi027.stdout:5/406: creat d4/d24/f7e x:0 0 0 2024-04-26T23:01:29.190 INFO:tasks.workunit.client.0.smithi027.stdout:0/413: dwrite d0/d4/da/d15/f5a [0,4194304] 0 2024-04-26T23:01:29.191 INFO:tasks.workunit.client.0.smithi027.stdout:6/314: dread d1/f2b [4194304,4194304] 0 2024-04-26T23:01:29.191 INFO:tasks.workunit.client.0.smithi027.stdout:6/315: truncate d1/f64 942889 0 2024-04-26T23:01:29.192 INFO:tasks.workunit.client.0.smithi027.stdout:1/462: dwrite d1/d5/d21/f27 [0,4194304] 0 2024-04-26T23:01:29.206 INFO:tasks.workunit.client.0.smithi027.stdout:5/407: creat d4/d24/d22/d29/d39/f7f x:0 0 0 2024-04-26T23:01:29.206 INFO:tasks.workunit.client.0.smithi027.stdout:0/414: symlink d0/l7d 0 2024-04-26T23:01:29.207 INFO:tasks.workunit.client.0.smithi027.stdout:6/316: mknod d1/d30/d6c/c78 0 2024-04-26T23:01:29.207 INFO:tasks.workunit.client.0.smithi027.stdout:6/317: chown d1/de 64 1 2024-04-26T23:01:29.210 INFO:tasks.workunit.client.0.smithi027.stdout:5/408: rmdir d4/d24/d22/d29/d2a/d3b 39 2024-04-26T23:01:29.210 INFO:tasks.workunit.client.0.smithi027.stdout:7/395: dread d0/d2f/f31 [0,4194304] 0 2024-04-26T23:01:29.210 INFO:tasks.workunit.client.0.smithi027.stdout:7/396: chown d0/d2f/d4d 1921455 1 2024-04-26T23:01:29.210 INFO:tasks.workunit.client.0.smithi027.stdout:7/397: readlink d0/d2f/l45 0 2024-04-26T23:01:29.210 INFO:tasks.workunit.client.0.smithi027.stdout:7/398: dread - d0/d2f/f65 zero size 2024-04-26T23:01:29.210 INFO:tasks.workunit.client.0.smithi027.stdout:0/415: link d0/d4/da/d15/d26/f7c d0/f7e 0 2024-04-26T23:01:29.211 INFO:tasks.workunit.client.0.smithi027.stdout:6/318: creat d1/df/f79 x:0 0 0 2024-04-26T23:01:29.212 INFO:tasks.workunit.client.0.smithi027.stdout:6/319: rmdir d1/df/d43 39 2024-04-26T23:01:29.214 INFO:tasks.workunit.client.0.smithi027.stdout:6/320: creat d1/d30/d6c/f7a x:0 0 0 2024-04-26T23:01:29.214 INFO:tasks.workunit.client.0.smithi027.stdout:1/463: dread d1/dc/f1d [0,4194304] 0 2024-04-26T23:01:29.215 INFO:tasks.workunit.client.0.smithi027.stdout:3/445: rename d6/dc/f85 to d6/f96 0 2024-04-26T23:01:29.219 INFO:tasks.workunit.client.0.smithi027.stdout:1/464: unlink d1/d5/d21/l3d 0 2024-04-26T23:01:29.221 INFO:tasks.workunit.client.0.smithi027.stdout:1/465: unlink d1/d26/d6b/d42/d79/l83 0 2024-04-26T23:01:29.221 INFO:tasks.workunit.client.0.smithi027.stdout:1/466: chown d1/dc/f5b 150887420 1 2024-04-26T23:01:29.222 INFO:tasks.workunit.client.0.smithi027.stdout:3/446: chown d6/dc/f11 1755 1 2024-04-26T23:01:29.222 INFO:tasks.workunit.client.0.smithi027.stdout:3/447: stat d6/dc/d34/d47/d7b/f93 0 2024-04-26T23:01:29.222 INFO:tasks.workunit.client.0.smithi027.stdout:1/467: stat d1/d26/d36/l57 0 2024-04-26T23:01:29.222 INFO:tasks.workunit.client.0.smithi027.stdout:1/468: fsync d1/d26/d66/f77 0 2024-04-26T23:01:29.223 INFO:tasks.workunit.client.0.smithi027.stdout:3/448: creat d6/dc/d17/d59/f97 x:0 0 0 2024-04-26T23:01:29.223 INFO:tasks.workunit.client.0.smithi027.stdout:3/449: write d6/dc/d34/d2e/d50/f8a [958557,6150] 0 2024-04-26T23:01:29.223 INFO:tasks.workunit.client.0.smithi027.stdout:3/450: chown d6/f7 34428 1 2024-04-26T23:01:29.224 INFO:tasks.workunit.client.0.smithi027.stdout:1/469: rmdir d1/d5/d40 39 2024-04-26T23:01:29.225 INFO:tasks.workunit.client.0.smithi027.stdout:3/451: mkdir d6/dc/d34/d47/d7b/d98 0 2024-04-26T23:01:29.225 INFO:tasks.workunit.client.0.smithi027.stdout:3/452: readlink d6/dc/d34/d47/l43 0 2024-04-26T23:01:29.229 INFO:tasks.workunit.client.0.smithi027.stdout:9/343: rename d4/d10/d30 to d4/d10/d40/d58/d75 0 2024-04-26T23:01:29.229 INFO:tasks.workunit.client.0.smithi027.stdout:9/344: fsync d4/d17/f45 0 2024-04-26T23:01:29.229 INFO:tasks.workunit.client.0.smithi027.stdout:9/345: truncate d4/d17/f45 971965 0 2024-04-26T23:01:29.236 INFO:tasks.workunit.client.0.smithi027.stdout:5/409: dread d4/db/d16/d4b/f77 [0,4194304] 0 2024-04-26T23:01:29.237 INFO:tasks.workunit.client.0.smithi027.stdout:5/410: write d4/db/d16/f67 [32972,30748] 0 2024-04-26T23:01:29.237 INFO:tasks.workunit.client.0.smithi027.stdout:5/411: chown d4/d24/f7e 6906 1 2024-04-26T23:01:29.238 INFO:tasks.workunit.client.0.smithi027.stdout:1/470: symlink d1/d26/d86/l8f 0 2024-04-26T23:01:29.239 INFO:tasks.workunit.client.0.smithi027.stdout:1/471: unlink d1/d26/d6b/d42/d70/l7a 0 2024-04-26T23:01:29.249 INFO:tasks.workunit.client.0.smithi027.stdout:7/399: dwrite d0/d28/d14/f3c [0,4194304] 0 2024-04-26T23:01:29.251 INFO:tasks.workunit.client.0.smithi027.stdout:7/400: rmdir d0/d28/d66 39 2024-04-26T23:01:29.254 INFO:tasks.workunit.client.0.smithi027.stdout:7/401: link d0/d28/d66/f70 d0/d28/d2b/d55/d59/f7b 0 2024-04-26T23:01:29.256 INFO:tasks.workunit.client.0.smithi027.stdout:9/346: read d4/d10/f4a [2138540,60629] 0 2024-04-26T23:01:29.256 INFO:tasks.workunit.client.0.smithi027.stdout:9/347: write d4/d10/d40/f66 [1020997,83212] 0 2024-04-26T23:01:29.258 INFO:tasks.workunit.client.0.smithi027.stdout:9/348: symlink d4/d17/l76 0 2024-04-26T23:01:29.261 INFO:tasks.workunit.client.0.smithi027.stdout:2/333: dwrite d2/d5/fa [0,4194304] 0 2024-04-26T23:01:29.265 INFO:tasks.workunit.client.0.smithi027.stdout:5/412: write d4/d1a/f1e [6864317,12441] 0 2024-04-26T23:01:29.265 INFO:tasks.workunit.client.0.smithi027.stdout:0/416: write d0/d4/da/d15/d55/f58 [622359,119977] 0 2024-04-26T23:01:29.265 INFO:tasks.workunit.client.0.smithi027.stdout:5/413: truncate d4/db/d16/d4b/f77 5008925 0 2024-04-26T23:01:29.265 INFO:tasks.workunit.client.0.smithi027.stdout:1/472: dread d1/fb [4194304,4194304] 0 2024-04-26T23:01:29.269 INFO:tasks.workunit.client.0.smithi027.stdout:9/349: mkdir d4/d77 0 2024-04-26T23:01:29.269 INFO:tasks.workunit.client.0.smithi027.stdout:9/350: readlink d4/d10/d67/l68 0 2024-04-26T23:01:29.269 INFO:tasks.workunit.client.0.smithi027.stdout:9/351: fsync d4/d17/d2e/d46/f6c 0 2024-04-26T23:01:29.269 INFO:tasks.workunit.client.0.smithi027.stdout:0/417: creat d0/d4/da/d1a/d1e/f7f x:0 0 0 2024-04-26T23:01:29.270 INFO:tasks.workunit.client.0.smithi027.stdout:5/414: mknod d4/d24/d22/d29/d2a/d3b/c80 0 2024-04-26T23:01:29.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:28 smithi080 ceph-mon[32324]: pgmap v89: 129 pgs: 129 active+clean; 736 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s rd, 99 MiB/s wr, 1.07k op/s 2024-04-26T23:01:29.271 INFO:tasks.workunit.client.0.smithi027.stdout:9/352: creat d4/d10/d40/d58/d75/f78 x:0 0 0 2024-04-26T23:01:29.271 INFO:tasks.workunit.client.0.smithi027.stdout:9/353: write d4/d10/d40/f49 [910517,120439] 0 2024-04-26T23:01:29.271 INFO:tasks.workunit.client.0.smithi027.stdout:9/354: readlink d4/l1c 0 2024-04-26T23:01:29.275 INFO:tasks.workunit.client.0.smithi027.stdout:5/415: mkdir d4/db/d16/d4b/d81 0 2024-04-26T23:01:29.275 INFO:tasks.workunit.client.0.smithi027.stdout:9/355: rmdir d4/d10/d25 39 2024-04-26T23:01:29.276 INFO:tasks.workunit.client.0.smithi027.stdout:9/356: creat d4/d10/d2c/d72/f79 x:0 0 0 2024-04-26T23:01:29.282 INFO:tasks.workunit.client.0.smithi027.stdout:9/357: creat d4/d77/f7a x:0 0 0 2024-04-26T23:01:29.282 INFO:tasks.workunit.client.0.smithi027.stdout:9/358: stat d4/d10/f5e 0 2024-04-26T23:01:29.282 INFO:tasks.workunit.client.0.smithi027.stdout:9/359: chown d4/d17/d2e/l3c 3674128 1 2024-04-26T23:01:29.282 INFO:tasks.workunit.client.0.smithi027.stdout:9/360: write d4/d10/f4a [3411141,120318] 0 2024-04-26T23:01:29.282 INFO:tasks.workunit.client.0.smithi027.stdout:9/361: write d4/d10/d2c/f57 [71772,120170] 0 2024-04-26T23:01:29.283 INFO:tasks.workunit.client.0.smithi027.stdout:2/334: dread d2/d5/db/de/d21/d26/f2d [0,4194304] 0 2024-04-26T23:01:29.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:28 smithi094 ceph-mon[32889]: pgmap v89: 129 pgs: 129 active+clean; 736 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s rd, 99 MiB/s wr, 1.07k op/s 2024-04-26T23:01:29.293 INFO:tasks.workunit.client.0.smithi027.stdout:2/335: mknod d2/d5/db/d1b/d1c/d25/c5c 0 2024-04-26T23:01:29.294 INFO:tasks.workunit.client.0.smithi027.stdout:2/336: rmdir d2/d5/db/d1b/d1c/d25/d19/d36/d39/d58 0 2024-04-26T23:01:29.323 INFO:tasks.workunit.client.0.smithi027.stdout:6/321: write d1/de/f20 [1602594,72591] 0 2024-04-26T23:01:29.326 INFO:tasks.workunit.client.0.smithi027.stdout:6/322: symlink d1/df/d3c/l7b 0 2024-04-26T23:01:29.326 INFO:tasks.workunit.client.0.smithi027.stdout:6/323: chown d1/de/d57/c5c 1048180 1 2024-04-26T23:01:29.328 INFO:tasks.workunit.client.0.smithi027.stdout:6/324: symlink d1/df/d43/d56/l7c 0 2024-04-26T23:01:29.330 INFO:tasks.workunit.client.0.smithi027.stdout:1/473: dread d1/d2a/f62 [0,4194304] 0 2024-04-26T23:01:29.330 INFO:tasks.workunit.client.0.smithi027.stdout:1/474: write d1/f85 [246336,11542] 0 2024-04-26T23:01:29.330 INFO:tasks.workunit.client.0.smithi027.stdout:1/475: write d1/f3 [4205064,120319] 0 2024-04-26T23:01:29.332 INFO:tasks.workunit.client.0.smithi027.stdout:1/476: mknod d1/d26/d66/d73/c90 0 2024-04-26T23:01:29.333 INFO:tasks.workunit.client.0.smithi027.stdout:1/477: mkdir d1/d26/d66/d91 0 2024-04-26T23:01:29.333 INFO:tasks.workunit.client.0.smithi027.stdout:1/478: write d1/d26/d6b/f5a [1155708,60688] 0 2024-04-26T23:01:29.334 INFO:tasks.workunit.client.0.smithi027.stdout:1/479: creat d1/d26/d6b/d42/d79/f92 x:0 0 0 2024-04-26T23:01:29.339 INFO:tasks.workunit.client.0.smithi027.stdout:7/402: dwrite d0/d2f/f65 [0,4194304] 0 2024-04-26T23:01:29.346 INFO:tasks.workunit.client.0.smithi027.stdout:5/416: dwrite d4/f15 [4194304,4194304] 0 2024-04-26T23:01:29.347 INFO:tasks.workunit.client.0.smithi027.stdout:0/418: dwrite d0/d4/da/f11 [0,4194304] 0 2024-04-26T23:01:29.347 INFO:tasks.workunit.client.0.smithi027.stdout:0/419: chown d0/d4/da/d15/l29 572 1 2024-04-26T23:01:29.347 INFO:tasks.workunit.client.0.smithi027.stdout:2/337: dread d2/d5/db/d1b/d1c/d25/d19/d36/f43 [0,4194304] 0 2024-04-26T23:01:29.351 INFO:tasks.workunit.client.0.smithi027.stdout:7/403: symlink d0/d28/d2b/d55/d59/d6d/l7c 0 2024-04-26T23:01:29.351 INFO:tasks.workunit.client.0.smithi027.stdout:5/417: mknod d4/d1a/d4a/c82 0 2024-04-26T23:01:29.351 INFO:tasks.workunit.client.0.smithi027.stdout:5/418: readlink d4/d24/d22/d29/d39/l52 0 2024-04-26T23:01:29.351 INFO:tasks.workunit.client.0.smithi027.stdout:5/419: truncate d4/d1a/d4a/d68/f5a 5158339 0 2024-04-26T23:01:29.351 INFO:tasks.workunit.client.0.smithi027.stdout:0/420: mknod d0/df/d3f/c80 0 2024-04-26T23:01:29.351 INFO:tasks.workunit.client.0.smithi027.stdout:0/421: fdatasync d0/f12 0 2024-04-26T23:01:29.355 INFO:tasks.workunit.client.0.smithi027.stdout:5/420: mknod d4/d24/d55/d63/c83 0 2024-04-26T23:01:29.355 INFO:tasks.workunit.client.0.smithi027.stdout:0/422: symlink d0/d4/da/d1a/d6d/l81 0 2024-04-26T23:01:29.355 INFO:tasks.workunit.client.0.smithi027.stdout:0/423: chown d0/d4/c6e 296836411 1 2024-04-26T23:01:29.355 INFO:tasks.workunit.client.0.smithi027.stdout:0/424: truncate d0/df/d3f/d64/f5e 199163 0 2024-04-26T23:01:29.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:28 smithi027 ceph-mon[28034]: pgmap v89: 129 pgs: 129 active+clean; 736 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s rd, 99 MiB/s wr, 1.07k op/s 2024-04-26T23:01:29.370 INFO:tasks.workunit.client.0.smithi027.stdout:2/338: read d2/d5/db/de/d21/f16 [4446101,66597] 0 2024-04-26T23:01:29.370 INFO:tasks.workunit.client.0.smithi027.stdout:2/339: fsync d2/d5/db/d1b/d1c/d25/d19/d36/f43 0 2024-04-26T23:01:29.371 INFO:tasks.workunit.client.0.smithi027.stdout:2/340: fsync d2/d5/db/d1b/d1c/d25/d19/d23/f47 0 2024-04-26T23:01:29.373 INFO:tasks.workunit.client.0.smithi027.stdout:6/325: dwrite d1/df/fd [0,4194304] 0 2024-04-26T23:01:29.379 INFO:tasks.workunit.client.0.smithi027.stdout:5/421: read d4/f15 [4057940,47995] 0 2024-04-26T23:01:29.380 INFO:tasks.workunit.client.0.smithi027.stdout:5/422: mkdir d4/d24/d22/d29/d39/d84 0 2024-04-26T23:01:29.384 INFO:tasks.workunit.client.0.smithi027.stdout:7/404: write d0/d28/f77 [2088847,113365] 0 2024-04-26T23:01:29.384 INFO:tasks.workunit.client.0.smithi027.stdout:7/405: write d0/d6c/f71 [240581,96489] 0 2024-04-26T23:01:29.384 INFO:tasks.workunit.client.0.smithi027.stdout:7/406: read - d0/d28/d2b/f5d zero size 2024-04-26T23:01:29.384 INFO:tasks.workunit.client.0.smithi027.stdout:7/407: write d0/d28/f77 [3392668,96256] 0 2024-04-26T23:01:29.385 INFO:tasks.workunit.client.0.smithi027.stdout:9/362: dwrite d4/d10/d2c/f57 [0,4194304] 0 2024-04-26T23:01:29.385 INFO:tasks.workunit.client.0.smithi027.stdout:9/363: readlink d4/d17/d2e/d46/l60 0 2024-04-26T23:01:29.385 INFO:tasks.workunit.client.0.smithi027.stdout:9/364: write d4/d10/d40/d58/f5a [803478,131048] 0 2024-04-26T23:01:29.386 INFO:tasks.workunit.client.0.smithi027.stdout:7/408: creat d0/d28/d2b/d55/f7d x:0 0 0 2024-04-26T23:01:29.387 INFO:tasks.workunit.client.0.smithi027.stdout:9/365: creat d4/d17/d2e/d46/f7b x:0 0 0 2024-04-26T23:01:29.389 INFO:tasks.workunit.client.0.smithi027.stdout:7/409: creat d0/d28/d2b/d55/f7e x:0 0 0 2024-04-26T23:01:29.389 INFO:tasks.workunit.client.0.smithi027.stdout:7/410: write d0/d28/d2b/d3a/f3f [1028447,55408] 0 2024-04-26T23:01:29.392 INFO:tasks.workunit.client.0.smithi027.stdout:9/366: rename d4/d10/f56 to d4/d10/d40/d58/d6e/f7c 0 2024-04-26T23:01:29.394 INFO:tasks.workunit.client.0.smithi027.stdout:9/367: rmdir d4 39 2024-04-26T23:01:29.401 INFO:tasks.workunit.client.0.smithi027.stdout:0/425: dwrite d0/d4/da/d15/f6f [0,4194304] 0 2024-04-26T23:01:29.401 INFO:tasks.workunit.client.0.smithi027.stdout:0/426: chown d0/d4/da/d1a/d2a 22 1 2024-04-26T23:01:29.414 INFO:tasks.workunit.client.0.smithi027.stdout:7/411: dread d0/d28/d2b/f34 [0,4194304] 0 2024-04-26T23:01:29.414 INFO:tasks.workunit.client.0.smithi027.stdout:7/412: read - d0/d3d/f43 zero size 2024-04-26T23:01:29.415 INFO:tasks.workunit.client.0.smithi027.stdout:7/413: mknod d0/d3d/c7f 0 2024-04-26T23:01:29.416 INFO:tasks.workunit.client.0.smithi027.stdout:7/414: fsync d0/f5a 0 2024-04-26T23:01:29.416 INFO:tasks.workunit.client.0.smithi027.stdout:7/415: read - d0/d3d/f54 zero size 2024-04-26T23:01:29.417 INFO:tasks.workunit.client.0.smithi027.stdout:7/416: getdents d0 0 2024-04-26T23:01:29.419 INFO:tasks.workunit.client.0.smithi027.stdout:7/417: mkdir d0/d28/d14/d73/d80 0 2024-04-26T23:01:29.419 INFO:tasks.workunit.client.0.smithi027.stdout:7/418: readlink d0/l33 0 2024-04-26T23:01:29.426 INFO:tasks.workunit.client.0.smithi027.stdout:2/341: dwrite d2/d5/db/de/d21/f51 [0,4194304] 0 2024-04-26T23:01:29.429 INFO:tasks.workunit.client.0.smithi027.stdout:9/368: dwrite d4/d10/d40/f64 [0,4194304] 0 2024-04-26T23:01:29.430 INFO:tasks.workunit.client.0.smithi027.stdout:2/342: creat d2/d5/db/d1b/d1c/f5d x:0 0 0 2024-04-26T23:01:29.430 INFO:tasks.workunit.client.0.smithi027.stdout:5/423: dwrite d4/f64 [0,4194304] 0 2024-04-26T23:01:29.433 INFO:tasks.workunit.client.0.smithi027.stdout:9/369: rename d4/d10/d40/f49 to d4/d10/d40/d58/d75/f7d 0 2024-04-26T23:01:29.434 INFO:tasks.workunit.client.0.smithi027.stdout:2/343: truncate d2/d5/db/d1b/f46 1889870 0 2024-04-26T23:01:29.436 INFO:tasks.workunit.client.0.smithi027.stdout:9/370: getdents d4/d77 0 2024-04-26T23:01:29.436 INFO:tasks.workunit.client.0.smithi027.stdout:9/371: read - d4/d17/d2e/d46/f7b zero size 2024-04-26T23:01:29.437 INFO:tasks.workunit.client.0.smithi027.stdout:9/372: unlink d4/d10/d2c/l6f 0 2024-04-26T23:01:29.453 INFO:tasks.workunit.client.0.smithi027.stdout:2/344: read d2/d5/db/de/f30 [2603480,15029] 0 2024-04-26T23:01:29.459 INFO:tasks.workunit.client.0.smithi027.stdout:2/345: dread d2/d5/db/d1b/d1c/d25/f4a [0,4194304] 0 2024-04-26T23:01:29.468 INFO:tasks.workunit.client.0.smithi027.stdout:5/424: write d4/d1a/d4a/d68/d40/d54/f75 [707252,5975] 0 2024-04-26T23:01:29.483 INFO:tasks.workunit.client.0.smithi027.stdout:9/373: dwrite d4/d10/d2c/f41 [0,4194304] 0 2024-04-26T23:01:29.484 INFO:tasks.workunit.client.0.smithi027.stdout:9/374: creat d4/d10/f7e x:0 0 0 2024-04-26T23:01:29.491 INFO:tasks.workunit.client.0.smithi027.stdout:5/425: dread d4/f64 [0,4194304] 0 2024-04-26T23:01:29.492 INFO:tasks.workunit.client.0.smithi027.stdout:5/426: mknod d4/db/c85 0 2024-04-26T23:01:29.524 INFO:tasks.workunit.client.0.smithi027.stdout:5/427: dwrite d4/f9 [4194304,4194304] 0 2024-04-26T23:01:29.526 INFO:tasks.workunit.client.0.smithi027.stdout:5/428: write f1 [755242,9122] 0 2024-04-26T23:01:29.528 INFO:tasks.workunit.client.0.smithi027.stdout:9/375: dwrite d4/d17/d2e/f53 [0,4194304] 0 2024-04-26T23:01:29.528 INFO:tasks.workunit.client.0.smithi027.stdout:9/376: read - d4/d10/d2c/d72/f79 zero size 2024-04-26T23:01:29.528 INFO:tasks.workunit.client.0.smithi027.stdout:5/429: unlink d4/d1a/f20 0 2024-04-26T23:01:29.530 INFO:tasks.workunit.client.0.smithi027.stdout:9/377: creat d4/f7f x:0 0 0 2024-04-26T23:01:29.531 INFO:tasks.workunit.client.0.smithi027.stdout:9/378: stat d4/f24 0 2024-04-26T23:01:29.531 INFO:tasks.workunit.client.0.smithi027.stdout:9/379: write d4/d17/f29 [5173261,35375] 0 2024-04-26T23:01:29.531 INFO:tasks.workunit.client.0.smithi027.stdout:5/430: symlink d4/d24/d22/d29/d39/l86 0 2024-04-26T23:01:29.532 INFO:tasks.workunit.client.0.smithi027.stdout:9/380: unlink d4/d17/l6d 0 2024-04-26T23:01:29.533 INFO:tasks.workunit.client.0.smithi027.stdout:5/431: creat d4/d24/d22/d29/d39/d84/f87 x:0 0 0 2024-04-26T23:01:29.534 INFO:tasks.workunit.client.0.smithi027.stdout:9/381: creat d4/d10/d67/f80 x:0 0 0 2024-04-26T23:01:29.534 INFO:tasks.workunit.client.0.smithi027.stdout:9/382: chown d4/d10/f5e 1045 1 2024-04-26T23:01:29.534 INFO:tasks.workunit.client.0.smithi027.stdout:5/432: symlink d4/d24/d55/l88 0 2024-04-26T23:01:29.535 INFO:tasks.workunit.client.0.smithi027.stdout:9/383: mkdir d4/d17/d2e/d81 0 2024-04-26T23:01:29.536 INFO:tasks.workunit.client.0.smithi027.stdout:9/384: symlink d4/d10/d2c/l82 0 2024-04-26T23:01:29.536 INFO:tasks.workunit.client.0.smithi027.stdout:9/385: readlink d4/l1c 0 2024-04-26T23:01:29.561 INFO:tasks.workunit.client.0.smithi027.stdout:5/433: write d4/f9 [2408863,124023] 0 2024-04-26T23:01:29.563 INFO:tasks.workunit.client.0.smithi027.stdout:5/434: mknod d4/d1a/d4a/d68/d40/c89 0 2024-04-26T23:01:29.571 INFO:tasks.workunit.client.0.smithi027.stdout:5/435: dread d4/db/d16/d4b/f77 [4194304,4194304] 0 2024-04-26T23:01:29.572 INFO:tasks.workunit.client.0.smithi027.stdout:5/436: truncate d4/d24/d22/d29/d2a/d3b/f3d 1168417 0 2024-04-26T23:01:29.584 INFO:tasks.workunit.client.0.smithi027.stdout:9/386: dwrite d4/f2b [0,4194304] 0 2024-04-26T23:01:29.584 INFO:tasks.workunit.client.0.smithi027.stdout:9/387: read d4/d10/f1a [1403702,16682] 0 2024-04-26T23:01:29.585 INFO:tasks.workunit.client.0.smithi027.stdout:9/388: mknod d4/d10/d2c/c83 0 2024-04-26T23:01:29.587 INFO:tasks.workunit.client.0.smithi027.stdout:9/389: mkdir d4/d10/d40/d58/d75/d84 0 2024-04-26T23:01:29.587 INFO:tasks.workunit.client.0.smithi027.stdout:9/390: fsync d4/f11 0 2024-04-26T23:01:29.587 INFO:tasks.workunit.client.0.smithi027.stdout:9/391: chown d4/d10/d40/d58/d75/d84 7629 1 2024-04-26T23:01:29.588 INFO:tasks.workunit.client.0.smithi027.stdout:9/392: mknod d4/d10/d40/d58/d75/d84/c85 0 2024-04-26T23:01:29.589 INFO:tasks.workunit.client.0.smithi027.stdout:9/393: mkdir d4/d10/d40/d58/d75/d84/d86 0 2024-04-26T23:01:29.593 INFO:tasks.workunit.client.0.smithi027.stdout:5/437: write d4/db/d16/d4b/f77 [4054370,129992] 0 2024-04-26T23:01:29.595 INFO:tasks.workunit.client.0.smithi027.stdout:9/394: read d4/d17/f29 [4869319,41779] 0 2024-04-26T23:01:29.618 INFO:tasks.workunit.client.0.smithi027.stdout:9/395: write d4/d17/d2e/d46/f62 [2138588,14229] 0 2024-04-26T23:01:29.620 INFO:tasks.workunit.client.0.smithi027.stdout:8/429: sync 2024-04-26T23:01:29.620 INFO:tasks.workunit.client.0.smithi027.stdout:3/453: sync 2024-04-26T23:01:29.620 INFO:tasks.workunit.client.0.smithi027.stdout:4/429: sync 2024-04-26T23:01:29.620 INFO:tasks.workunit.client.0.smithi027.stdout:1/480: sync 2024-04-26T23:01:29.620 INFO:tasks.workunit.client.0.smithi027.stdout:1/481: fdatasync d1/d26/d66/d73/f7c 0 2024-04-26T23:01:29.620 INFO:tasks.workunit.client.0.smithi027.stdout:4/430: dread - da/dd/de/d14/d17/d24/f26 zero size 2024-04-26T23:01:29.620 INFO:tasks.workunit.client.0.smithi027.stdout:4/431: chown da/dd/de/d79 7 1 2024-04-26T23:01:29.622 INFO:tasks.workunit.client.0.smithi027.stdout:9/396: symlink d4/l87 0 2024-04-26T23:01:29.623 INFO:tasks.workunit.client.0.smithi027.stdout:1/482: symlink d1/d26/d6b/d42/d70/l93 0 2024-04-26T23:01:29.623 INFO:tasks.workunit.client.0.smithi027.stdout:1/483: chown d1/d26/d66/d73/c90 3156 1 2024-04-26T23:01:29.623 INFO:tasks.workunit.client.0.smithi027.stdout:1/484: readlink d1/d5/d21/l6f 0 2024-04-26T23:01:29.623 INFO:tasks.workunit.client.0.smithi027.stdout:4/432: unlink da/dd/de/d14/d17/f68 0 2024-04-26T23:01:29.624 INFO:tasks.workunit.client.0.smithi027.stdout:3/454: creat d6/dc/d17/d52/d5a/f99 x:0 0 0 2024-04-26T23:01:29.625 INFO:tasks.workunit.client.0.smithi027.stdout:9/397: truncate d4/d17/f1d 3329875 0 2024-04-26T23:01:29.626 INFO:tasks.workunit.client.0.smithi027.stdout:3/455: creat d6/dc/d88/f9a x:0 0 0 2024-04-26T23:01:29.626 INFO:tasks.workunit.client.0.smithi027.stdout:3/456: stat d6/dc/d34/d47/f65 0 2024-04-26T23:01:29.626 INFO:tasks.workunit.client.0.smithi027.stdout:9/398: mknod d4/d17/d2e/d81/c88 0 2024-04-26T23:01:29.627 INFO:tasks.workunit.client.0.smithi027.stdout:5/438: dread f1 [0,4194304] 0 2024-04-26T23:01:29.629 INFO:tasks.workunit.client.0.smithi027.stdout:9/399: mknod d4/d10/d40/d58/d75/d84/c89 0 2024-04-26T23:01:29.630 INFO:tasks.workunit.client.0.smithi027.stdout:5/439: mknod d4/d24/d22/d29/d39/d84/c8a 0 2024-04-26T23:01:29.630 INFO:tasks.workunit.client.0.smithi027.stdout:4/433: write da/dd/de/f3c [773804,64292] 0 2024-04-26T23:01:29.630 INFO:tasks.workunit.client.0.smithi027.stdout:5/440: write d4/db/d16/f67 [1020303,126931] 0 2024-04-26T23:01:29.631 INFO:tasks.workunit.client.0.smithi027.stdout:9/400: symlink d4/d10/d67/l8a 0 2024-04-26T23:01:29.633 INFO:tasks.workunit.client.0.smithi027.stdout:5/441: mknod d4/db/c8b 0 2024-04-26T23:01:29.633 INFO:tasks.workunit.client.0.smithi027.stdout:5/442: write d4/d24/f4c [143606,22843] 0 2024-04-26T23:01:29.640 INFO:tasks.workunit.client.0.smithi027.stdout:3/457: rmdir d6/dc/d34/d2e/d5f 39 2024-04-26T23:01:29.643 INFO:tasks.workunit.client.0.smithi027.stdout:3/458: mkdir d6/dc/d34/d9b 0 2024-04-26T23:01:29.656 INFO:tasks.workunit.client.0.smithi027.stdout:4/434: write da/dd/de/d14/f44 [1989584,115044] 0 2024-04-26T23:01:29.699 INFO:tasks.workunit.client.0.smithi027.stdout:3/459: dread d6/dc/f21 [0,4194304] 0 2024-04-26T23:01:29.700 INFO:tasks.workunit.client.0.smithi027.stdout:9/401: dwrite d4/d17/f1d [0,4194304] 0 2024-04-26T23:01:29.700 INFO:tasks.workunit.client.0.smithi027.stdout:3/460: write d6/dc/d34/d47/d7b/f92 [101891,82032] 0 2024-04-26T23:01:29.701 INFO:tasks.workunit.client.0.smithi027.stdout:3/461: link d6/dc/d34/d47/l4b d6/dc/d34/d9b/l9c 0 2024-04-26T23:01:29.703 INFO:tasks.workunit.client.0.smithi027.stdout:3/462: mkdir d6/dc/d34/d2e/d9d 0 2024-04-26T23:01:29.712 INFO:tasks.workunit.client.0.smithi027.stdout:3/463: dread d6/dc/d34/d2e/d50/f8a [0,4194304] 0 2024-04-26T23:01:29.712 INFO:tasks.workunit.client.0.smithi027.stdout:3/464: write d6/dc/d17/d59/f97 [690247,98836] 0 2024-04-26T23:01:29.715 INFO:tasks.workunit.client.0.smithi027.stdout:3/465: mkdir d6/dc/d2f/d6b/d74/d9e 0 2024-04-26T23:01:29.718 INFO:tasks.workunit.client.0.smithi027.stdout:3/466: rename d6/dc/d34/d2e/f49 to d6/dc/d34/d2e/d70/d71/f9f 0 2024-04-26T23:01:29.721 INFO:tasks.workunit.client.0.smithi027.stdout:5/443: dwrite d4/d1a/d4a/f76 [0,4194304] 0 2024-04-26T23:01:29.722 INFO:tasks.workunit.client.0.smithi027.stdout:5/444: write d4/d24/d22/d29/f36 [2803,3235] 0 2024-04-26T23:01:29.722 INFO:tasks.workunit.client.0.smithi027.stdout:8/430: dwrite d9/da/d2a/d6b/f9c [0,4194304] 0 2024-04-26T23:01:29.722 INFO:tasks.workunit.client.0.smithi027.stdout:8/431: chown d9/da/d2a/d5e/d72/d84/d4f 2782327 1 2024-04-26T23:01:29.724 INFO:tasks.workunit.client.0.smithi027.stdout:3/467: read d6/dc/d34/d2e/d50/f8a [858175,71293] 0 2024-04-26T23:01:29.725 INFO:tasks.workunit.client.0.smithi027.stdout:5/445: symlink d4/d1a/d4a/d68/d40/d54/l8c 0 2024-04-26T23:01:29.730 INFO:tasks.workunit.client.0.smithi027.stdout:8/432: rmdir d9/da/d2a/d5e/d92/d94 0 2024-04-26T23:01:29.730 INFO:tasks.workunit.client.0.smithi027.stdout:3/468: creat d6/d5e/fa0 x:0 0 0 2024-04-26T23:01:29.735 INFO:tasks.workunit.client.0.smithi027.stdout:3/469: rename d6/dc/d62 to d6/dc/d34/d78/da1 0 2024-04-26T23:01:29.735 INFO:tasks.workunit.client.0.smithi027.stdout:3/470: dread - d6/dc/d34/d2e/d70/d71/f9f zero size 2024-04-26T23:01:29.736 INFO:tasks.workunit.client.0.smithi027.stdout:3/471: creat d6/dc/d34/fa2 x:0 0 0 2024-04-26T23:01:29.737 INFO:tasks.workunit.client.0.smithi027.stdout:3/472: read - d6/dc/d34/d2e/d70/f7d zero size 2024-04-26T23:01:29.737 INFO:tasks.workunit.client.0.smithi027.stdout:3/473: write d6/dc/d17/d59/f97 [1197669,21583] 0 2024-04-26T23:01:29.739 INFO:tasks.workunit.client.0.smithi027.stdout:5/446: dread d4/db/f23 [0,4194304] 0 2024-04-26T23:01:29.739 INFO:tasks.workunit.client.0.smithi027.stdout:4/435: dwrite da/dd/de/d14/f53 [0,4194304] 0 2024-04-26T23:01:29.743 INFO:tasks.workunit.client.0.smithi027.stdout:5/447: rmdir d4/d24/d22 39 2024-04-26T23:01:29.743 INFO:tasks.workunit.client.0.smithi027.stdout:3/474: rename d6/dc/d4c to d6/dc/d34/d2e/d9d/da3 0 2024-04-26T23:01:29.743 INFO:tasks.workunit.client.0.smithi027.stdout:0/427: sync 2024-04-26T23:01:29.744 INFO:tasks.workunit.client.0.smithi027.stdout:6/326: sync 2024-04-26T23:01:29.744 INFO:tasks.workunit.client.0.smithi027.stdout:0/428: chown d0/df/c54 13059 1 2024-04-26T23:01:29.744 INFO:tasks.workunit.client.0.smithi027.stdout:0/429: write d0/df/d3f/d64/f6a [885488,11329] 0 2024-04-26T23:01:29.744 INFO:tasks.workunit.client.0.smithi027.stdout:6/327: write d1/f59 [853603,89172] 0 2024-04-26T23:01:29.744 INFO:tasks.workunit.client.0.smithi027.stdout:6/328: fdatasync d1/df/f3a 0 2024-04-26T23:01:29.745 INFO:tasks.workunit.client.0.smithi027.stdout:6/329: mkdir d1/d30/d6c/d6f/d7d 0 2024-04-26T23:01:29.745 INFO:tasks.workunit.client.0.smithi027.stdout:8/433: write d9/de/f14 [5766552,39428] 0 2024-04-26T23:01:29.745 INFO:tasks.workunit.client.0.smithi027.stdout:8/434: stat d9/da/d2a/d5e/d72 0 2024-04-26T23:01:29.746 INFO:tasks.workunit.client.0.smithi027.stdout:6/330: mknod d1/df/d49/c7e 0 2024-04-26T23:01:29.746 INFO:tasks.workunit.client.0.smithi027.stdout:6/331: dread - d1/f4e zero size 2024-04-26T23:01:29.749 INFO:tasks.workunit.client.0.smithi027.stdout:4/436: dread da/f48 [0,4194304] 0 2024-04-26T23:01:29.750 INFO:tasks.workunit.client.0.smithi027.stdout:3/475: creat d6/dc/d34/d47/d7b/fa4 x:0 0 0 2024-04-26T23:01:29.750 INFO:tasks.workunit.client.0.smithi027.stdout:3/476: chown d6/dc/d34/d2e/f45 434 1 2024-04-26T23:01:29.750 INFO:tasks.workunit.client.0.smithi027.stdout:3/477: readlink d6/dc/d34/d2e/d5f/l7a 0 2024-04-26T23:01:29.758 INFO:tasks.workunit.client.0.smithi027.stdout:3/478: mknod d6/dc/d34/d78/ca5 0 2024-04-26T23:01:29.759 INFO:tasks.workunit.client.0.smithi027.stdout:3/479: fdatasync d6/d5e/f6d 0 2024-04-26T23:01:29.765 INFO:tasks.workunit.client.0.smithi027.stdout:3/480: rename d6/dc/d17 to d6/dc/d2f/d6b/d74/da6 0 2024-04-26T23:01:29.767 INFO:tasks.workunit.client.0.smithi027.stdout:3/481: mkdir d6/dc/d34/d2e/d9d/da7 0 2024-04-26T23:01:29.767 INFO:tasks.workunit.client.0.smithi027.stdout:3/482: chown d6/dc/d34/d2e/d50/f8a 33 1 2024-04-26T23:01:29.773 INFO:tasks.workunit.client.0.smithi027.stdout:5/448: dread d4/d1a/d4a/d68/d40/f6c [0,4194304] 0 2024-04-26T23:01:29.776 INFO:tasks.workunit.client.0.smithi027.stdout:5/449: rename d4/db/l35 to d4/d1a/d4a/l8d 0 2024-04-26T23:01:29.776 INFO:tasks.workunit.client.0.smithi027.stdout:5/450: write d4/d1a/d4a/d68/d40/f6c [4467602,70574] 0 2024-04-26T23:01:29.777 INFO:tasks.workunit.client.0.smithi027.stdout:5/451: write d4/d24/d22/d29/d2a/f45 [900881,2552] 0 2024-04-26T23:01:29.778 INFO:tasks.workunit.client.0.smithi027.stdout:5/452: mknod d4/d1a/d4a/d68/d40/d54/c8e 0 2024-04-26T23:01:29.779 INFO:tasks.workunit.client.0.smithi027.stdout:5/453: chown d4/d24/c11 4287 1 2024-04-26T23:01:29.780 INFO:tasks.workunit.client.0.smithi027.stdout:5/454: mknod d4/d1a/d4a/d68/d40/c8f 0 2024-04-26T23:01:29.780 INFO:tasks.workunit.client.0.smithi027.stdout:5/455: write d4/d24/f42 [520747,18380] 0 2024-04-26T23:01:29.789 INFO:tasks.workunit.client.0.smithi027.stdout:3/483: dread d6/f2d [0,4194304] 0 2024-04-26T23:01:29.789 INFO:tasks.workunit.client.0.smithi027.stdout:8/435: dwrite d9/da/d2a/d5e/d2b/d3b/f9b [0,4194304] 0 2024-04-26T23:01:29.796 INFO:tasks.workunit.client.0.smithi027.stdout:8/436: getdents d9/da/d33 0 2024-04-26T23:01:29.803 INFO:tasks.workunit.client.0.smithi027.stdout:6/332: dwrite d1/d30/d6c/f7a [0,4194304] 0 2024-04-26T23:01:29.804 INFO:tasks.workunit.client.0.smithi027.stdout:5/456: dread d4/d1a/d4a/d68/f5a [0,4194304] 0 2024-04-26T23:01:29.806 INFO:tasks.workunit.client.0.smithi027.stdout:5/457: symlink d4/d24/d22/l90 0 2024-04-26T23:01:29.809 INFO:tasks.workunit.client.0.smithi027.stdout:4/437: write da/dd/de/d14/d17/d24/f19 [2677829,102972] 0 2024-04-26T23:01:29.809 INFO:tasks.workunit.client.0.smithi027.stdout:4/438: dread - da/dd/de/d14/d17/d24/d37/f5a zero size 2024-04-26T23:01:29.809 INFO:tasks.workunit.client.0.smithi027.stdout:5/458: rename d4/d1a/d2b/c7d to d4/d1a/d4a/d68/d40/d54/c91 0 2024-04-26T23:01:29.810 INFO:tasks.workunit.client.0.smithi027.stdout:5/459: read d4/db/d16/f1f [4425999,120003] 0 2024-04-26T23:01:29.812 INFO:tasks.workunit.client.0.smithi027.stdout:4/439: mkdir da/dd/de/d14/d17/d80 0 2024-04-26T23:01:29.812 INFO:tasks.workunit.client.0.smithi027.stdout:3/484: dread d6/dc/f10 [4194304,4194304] 0 2024-04-26T23:01:29.812 INFO:tasks.workunit.client.0.smithi027.stdout:3/485: readlink d6/l23 0 2024-04-26T23:01:29.813 INFO:tasks.workunit.client.0.smithi027.stdout:3/486: chown d6/dc/d2f/d7e 48075 1 2024-04-26T23:01:29.816 INFO:tasks.workunit.client.0.smithi027.stdout:0/430: dwrite d0/d4/da/d15/d26/f7c [0,4194304] 0 2024-04-26T23:01:29.825 INFO:tasks.workunit.client.0.smithi027.stdout:2/346: sync 2024-04-26T23:01:29.825 INFO:tasks.workunit.client.0.smithi027.stdout:7/419: sync 2024-04-26T23:01:29.825 INFO:tasks.workunit.client.0.smithi027.stdout:2/347: chown d2/d5/db/d1b/d1c/d25/d19/d44/d4b/f56 24 1 2024-04-26T23:01:29.826 INFO:tasks.workunit.client.0.smithi027.stdout:3/487: dread d6/dc/f46 [4194304,4194304] 0 2024-04-26T23:01:29.827 INFO:tasks.workunit.client.0.smithi027.stdout:4/440: dread da/f1f [4194304,4194304] 0 2024-04-26T23:01:29.828 INFO:tasks.workunit.client.0.smithi027.stdout:7/420: read d0/d28/f60 [260173,62755] 0 2024-04-26T23:01:29.830 INFO:tasks.workunit.client.0.smithi027.stdout:7/421: creat d0/d28/d66/d67/f81 x:0 0 0 2024-04-26T23:01:29.834 INFO:tasks.workunit.client.0.smithi027.stdout:7/422: rename d0/d6c/d79 to d0/d28/d2b/d55/d59/d6d/d82 0 2024-04-26T23:01:29.834 INFO:tasks.workunit.client.0.smithi027.stdout:7/423: truncate d0/d6c/f71 698402 0 2024-04-26T23:01:29.837 INFO:tasks.workunit.client.0.smithi027.stdout:7/424: mknod d0/d28/d2b/d55/d59/c83 0 2024-04-26T23:01:29.837 INFO:tasks.workunit.client.0.smithi027.stdout:5/460: write d4/f15 [8104256,108961] 0 2024-04-26T23:01:29.838 INFO:tasks.workunit.client.0.smithi027.stdout:5/461: truncate d4/d24/d22/d29/d2a/d3b/d50/f70 4619873 0 2024-04-26T23:01:29.839 INFO:tasks.workunit.client.0.smithi027.stdout:6/333: dwrite d1/df/d49/f4d [0,4194304] 0 2024-04-26T23:01:29.842 INFO:tasks.workunit.client.0.smithi027.stdout:6/334: truncate d1/df/fd 3358212 0 2024-04-26T23:01:29.844 INFO:tasks.workunit.client.0.smithi027.stdout:6/335: truncate d1/f19 57856 0 2024-04-26T23:01:29.851 INFO:tasks.workunit.client.0.smithi027.stdout:2/348: dread d2/d5/db/de/d21/f16 [4194304,4194304] 0 2024-04-26T23:01:29.852 INFO:tasks.workunit.client.0.smithi027.stdout:4/441: dread da/dd/f5d [0,4194304] 0 2024-04-26T23:01:29.852 INFO:tasks.workunit.client.0.smithi027.stdout:4/442: fdatasync da/dd/de/d14/d21/d41/d59/d74/f7e 0 2024-04-26T23:01:29.855 INFO:tasks.workunit.client.0.smithi027.stdout:1/485: sync 2024-04-26T23:01:29.855 INFO:tasks.workunit.client.0.smithi027.stdout:1/486: chown d1/d26/d36/d88/c8a 849980 1 2024-04-26T23:01:29.857 INFO:tasks.workunit.client.0.smithi027.stdout:2/349: mknod d2/d5/db/d1b/d1c/d25/d19/d36/d39/c5e 0 2024-04-26T23:01:29.859 INFO:tasks.workunit.client.0.smithi027.stdout:4/443: creat da/dd/de/d14/d17/f81 x:0 0 0 2024-04-26T23:01:29.859 INFO:tasks.workunit.client.0.smithi027.stdout:2/350: creat d2/d5/db/de/d21/f5f x:0 0 0 2024-04-26T23:01:29.861 INFO:tasks.workunit.client.0.smithi027.stdout:2/351: rename d2/d5/db/d1b/d1c/d25/d19/d36/d39/f3c to d2/d5/db/d1b/d1c/d25/d19/d44/f60 0 2024-04-26T23:01:29.861 INFO:tasks.workunit.client.0.smithi027.stdout:2/352: chown d2/d5/db/d1b/d1c/d25/f4a 232396 1 2024-04-26T23:01:29.861 INFO:tasks.workunit.client.0.smithi027.stdout:2/353: write d2/d5/db/d1b/d1c/f5d [464192,111515] 0 2024-04-26T23:01:29.861 INFO:tasks.workunit.client.0.smithi027.stdout:4/444: mknod da/dd/de/d14/d17/d80/c82 0 2024-04-26T23:01:29.867 INFO:tasks.workunit.client.0.smithi027.stdout:6/336: dread d1/de/f20 [0,4194304] 0 2024-04-26T23:01:29.867 INFO:tasks.workunit.client.0.smithi027.stdout:6/337: read - d1/f72 zero size 2024-04-26T23:01:29.880 INFO:tasks.workunit.client.0.smithi027.stdout:7/425: dwrite d0/d28/d14/f39 [0,4194304] 0 2024-04-26T23:01:29.882 INFO:tasks.workunit.client.0.smithi027.stdout:7/426: link d0/d28/f60 d0/d2f/f84 0 2024-04-26T23:01:29.882 INFO:tasks.workunit.client.0.smithi027.stdout:7/427: fdatasync d0/d3d/f44 0 2024-04-26T23:01:29.886 INFO:tasks.workunit.client.0.smithi027.stdout:7/428: mkdir d0/d28/d14/d73/d80/d85 0 2024-04-26T23:01:29.886 INFO:tasks.workunit.client.0.smithi027.stdout:7/429: rename d0/d28/d2b/l3b to d0/d6c/l86 0 2024-04-26T23:01:29.890 INFO:tasks.workunit.client.0.smithi027.stdout:6/338: write d1/de/f1e [1095122,6420] 0 2024-04-26T23:01:29.892 INFO:tasks.workunit.client.0.smithi027.stdout:6/339: mkdir d1/df/d49/d7f 0 2024-04-26T23:01:29.893 INFO:tasks.workunit.client.0.smithi027.stdout:6/340: write d1/f59 [1820020,12122] 0 2024-04-26T23:01:29.897 INFO:tasks.workunit.client.0.smithi027.stdout:7/430: dread d0/d6c/f71 [0,4194304] 0 2024-04-26T23:01:29.899 INFO:tasks.workunit.client.0.smithi027.stdout:1/487: dwrite d1/d26/d66/f74 [0,4194304] 0 2024-04-26T23:01:29.901 INFO:tasks.workunit.client.0.smithi027.stdout:2/354: dwrite d2/d5/db/d1b/f38 [0,4194304] 0 2024-04-26T23:01:29.903 INFO:tasks.workunit.client.0.smithi027.stdout:1/488: truncate d1/dc/f16 863076 0 2024-04-26T23:01:29.903 INFO:tasks.workunit.client.0.smithi027.stdout:4/445: dwrite da/dd/de/d14/d21/f7d [0,4194304] 0 2024-04-26T23:01:29.903 INFO:tasks.workunit.client.0.smithi027.stdout:2/355: creat d2/d5/db/d1b/d1c/f61 x:0 0 0 2024-04-26T23:01:29.906 INFO:tasks.workunit.client.0.smithi027.stdout:1/489: creat d1/f94 x:0 0 0 2024-04-26T23:01:29.906 INFO:tasks.workunit.client.0.smithi027.stdout:1/490: chown d1/d26/d66/f77 11545 1 2024-04-26T23:01:29.906 INFO:tasks.workunit.client.0.smithi027.stdout:1/491: truncate d1/f94 487190 0 2024-04-26T23:01:29.907 INFO:tasks.workunit.client.0.smithi027.stdout:1/492: mknod d1/d26/d66/d91/c95 0 2024-04-26T23:01:29.939 INFO:tasks.workunit.client.0.smithi027.stdout:1/493: dread d1/d5/d21/f34 [0,4194304] 0 2024-04-26T23:01:29.939 INFO:tasks.workunit.client.0.smithi027.stdout:1/494: chown d1/f85 14 1 2024-04-26T23:01:29.940 INFO:tasks.workunit.client.0.smithi027.stdout:1/495: readlink d1/l12 0 2024-04-26T23:01:29.940 INFO:tasks.workunit.client.0.smithi027.stdout:4/446: write da/dd/de/d14/d21/d41/d59/f78 [3912848,94041] 0 2024-04-26T23:01:29.940 INFO:tasks.workunit.client.0.smithi027.stdout:4/447: chown da/dd/de/d14/d21/d41/f49 0 1 2024-04-26T23:01:29.940 INFO:tasks.workunit.client.0.smithi027.stdout:4/448: chown da/dd/de/d14/d17/f45 904811 1 2024-04-26T23:01:29.940 INFO:tasks.workunit.client.0.smithi027.stdout:4/449: read - da/dd/de/d14/d21/d41/f72 zero size 2024-04-26T23:01:29.941 INFO:tasks.workunit.client.0.smithi027.stdout:6/341: dwrite d1/f2 [0,4194304] 0 2024-04-26T23:01:29.944 INFO:tasks.workunit.client.0.smithi027.stdout:7/431: dwrite d0/d28/d2b/d3a/d48/f72 [0,4194304] 0 2024-04-26T23:01:29.947 INFO:tasks.workunit.client.0.smithi027.stdout:6/342: dread d1/df/d49/f4f [0,4194304] 0 2024-04-26T23:01:29.954 INFO:tasks.workunit.client.0.smithi027.stdout:7/432: dread d0/d2f/f84 [0,4194304] 0 2024-04-26T23:01:29.955 INFO:tasks.workunit.client.0.smithi027.stdout:7/433: symlink d0/d28/d2b/l87 0 2024-04-26T23:01:29.956 INFO:tasks.workunit.client.0.smithi027.stdout:2/356: dwrite d2/d5/db/de/f30 [0,4194304] 0 2024-04-26T23:01:29.956 INFO:tasks.workunit.client.0.smithi027.stdout:2/357: truncate d2/d5/fa 5532801 0 2024-04-26T23:01:29.958 INFO:tasks.workunit.client.0.smithi027.stdout:2/358: mknod d2/d5/db/d1b/d1c/c62 0 2024-04-26T23:01:29.958 INFO:tasks.workunit.client.0.smithi027.stdout:2/359: chown d2/d5/db/d1b/d1c/c62 0 1 2024-04-26T23:01:29.958 INFO:tasks.workunit.client.0.smithi027.stdout:2/360: write d2/d5/db/d1b/d1c/d25/d2f/f4c [276931,127130] 0 2024-04-26T23:01:29.963 INFO:tasks.workunit.client.0.smithi027.stdout:7/434: rename d0/d28/d14 to d0/d28/d2b/d3a/d48/d56/d88 0 2024-04-26T23:01:29.963 INFO:tasks.workunit.client.0.smithi027.stdout:2/361: rename d2/d5/db/d1b to d2/d5/db/d1b/d1c/d25/d63 22 2024-04-26T23:01:29.963 INFO:tasks.workunit.client.0.smithi027.stdout:6/343: dread d1/df/f39 [0,4194304] 0 2024-04-26T23:01:29.964 INFO:tasks.workunit.client.0.smithi027.stdout:7/435: creat d0/d28/d2b/d55/d69/d75/f89 x:0 0 0 2024-04-26T23:01:29.965 INFO:tasks.workunit.client.0.smithi027.stdout:7/436: stat d0/f30 0 2024-04-26T23:01:29.965 INFO:tasks.workunit.client.0.smithi027.stdout:7/437: fdatasync d0/d28/d66/d67/f81 0 2024-04-26T23:01:29.965 INFO:tasks.workunit.client.0.smithi027.stdout:7/438: write d0/d28/d2b/f5d [104364,22409] 0 2024-04-26T23:01:29.966 INFO:tasks.workunit.client.0.smithi027.stdout:6/344: truncate d1/de/f1b 936104 0 2024-04-26T23:01:29.967 INFO:tasks.workunit.client.0.smithi027.stdout:6/345: creat d1/d30/d6c/d6f/d7d/f80 x:0 0 0 2024-04-26T23:01:29.971 INFO:tasks.workunit.client.0.smithi027.stdout:6/346: rmdir d1/d30 39 2024-04-26T23:01:29.973 INFO:tasks.workunit.client.0.smithi027.stdout:6/347: mknod d1/d30/c81 0 2024-04-26T23:01:29.974 INFO:tasks.workunit.client.0.smithi027.stdout:6/348: stat d1/d30/l37 0 2024-04-26T23:01:29.976 INFO:tasks.workunit.client.0.smithi027.stdout:6/349: creat d1/de/f82 x:0 0 0 2024-04-26T23:01:29.985 INFO:tasks.workunit.client.0.smithi027.stdout:2/362: read d2/d5/db/d1b/d1c/d25/d19/d44/f60 [344209,56135] 0 2024-04-26T23:01:30.007 INFO:tasks.workunit.client.0.smithi027.stdout:2/363: dread d2/d5/db/d1b/f38 [0,4194304] 0 2024-04-26T23:01:30.009 INFO:tasks.workunit.client.0.smithi027.stdout:1/496: dwrite d1/dc/f5b [0,4194304] 0 2024-04-26T23:01:30.012 INFO:tasks.workunit.client.0.smithi027.stdout:1/497: truncate d1/d26/d66/f74 565880 0 2024-04-26T23:01:30.014 INFO:tasks.workunit.client.0.smithi027.stdout:9/402: sync 2024-04-26T23:01:30.014 INFO:tasks.workunit.client.0.smithi027.stdout:9/403: fsync d4/d77/f7a 0 2024-04-26T23:01:30.014 INFO:tasks.workunit.client.0.smithi027.stdout:4/450: dwrite da/f1f [0,4194304] 0 2024-04-26T23:01:30.015 INFO:tasks.workunit.client.0.smithi027.stdout:9/404: symlink d4/d10/d25/l8b 0 2024-04-26T23:01:30.016 INFO:tasks.workunit.client.0.smithi027.stdout:4/451: creat da/dd/de/d79/f83 x:0 0 0 2024-04-26T23:01:30.017 INFO:tasks.workunit.client.0.smithi027.stdout:9/405: getdents d4/d10 0 2024-04-26T23:01:30.018 INFO:tasks.workunit.client.0.smithi027.stdout:2/364: dread d2/d5/db/d1b/d1c/d25/d19/d44/d4b/f56 [0,4194304] 0 2024-04-26T23:01:30.019 INFO:tasks.workunit.client.0.smithi027.stdout:4/452: rename da/dd/de/d14/d17/l69 to da/dd/de/d14/d21/d41/l84 0 2024-04-26T23:01:30.019 INFO:tasks.workunit.client.0.smithi027.stdout:9/406: symlink d4/d10/d2c/d72/l8c 0 2024-04-26T23:01:30.022 INFO:tasks.workunit.client.0.smithi027.stdout:2/365: creat d2/d5/db/d1b/d1c/d25/d19/d23/f64 x:0 0 0 2024-04-26T23:01:30.022 INFO:tasks.workunit.client.0.smithi027.stdout:2/366: write d2/d5/db/de/f42 [3325680,99620] 0 2024-04-26T23:01:30.023 INFO:tasks.workunit.client.0.smithi027.stdout:4/453: creat da/dd/de/d79/f85 x:0 0 0 2024-04-26T23:01:30.023 INFO:tasks.workunit.client.0.smithi027.stdout:9/407: mkdir d4/d10/d2c/d8d 0 2024-04-26T23:01:30.025 INFO:tasks.workunit.client.0.smithi027.stdout:9/408: unlink d4/c7 0 2024-04-26T23:01:30.025 INFO:tasks.workunit.client.0.smithi027.stdout:9/409: getdents d4/d77 0 2024-04-26T23:01:30.026 INFO:tasks.workunit.client.0.smithi027.stdout:9/410: creat d4/f8e x:0 0 0 2024-04-26T23:01:30.057 INFO:tasks.workunit.client.0.smithi027.stdout:4/454: dread da/dd/de/d14/d17/f33 [0,4194304] 0 2024-04-26T23:01:30.058 INFO:tasks.workunit.client.0.smithi027.stdout:4/455: mknod da/dd/de/c86 0 2024-04-26T23:01:30.065 INFO:tasks.workunit.client.0.smithi027.stdout:4/456: getdents da/dd/de/d14/d21/d41 0 2024-04-26T23:01:30.066 INFO:tasks.workunit.client.0.smithi027.stdout:9/411: dwrite d4/f11 [0,4194304] 0 2024-04-26T23:01:30.071 INFO:tasks.workunit.client.0.smithi027.stdout:2/367: dwrite d2/d5/db/de/f42 [0,4194304] 0 2024-04-26T23:01:30.071 INFO:tasks.workunit.client.0.smithi027.stdout:2/368: chown d2/d5/db/d1b/d1c/d25/d19/d44 2 1 2024-04-26T23:01:30.071 INFO:tasks.workunit.client.0.smithi027.stdout:2/369: write d2/d5/db/d1b/d1c/d25/d19/d44/d4b/f56 [742377,6431] 0 2024-04-26T23:01:30.071 INFO:tasks.workunit.client.0.smithi027.stdout:4/457: unlink da/dd/de/c6c 0 2024-04-26T23:01:30.073 INFO:tasks.workunit.client.0.smithi027.stdout:4/458: mkdir da/d40/d87 0 2024-04-26T23:01:30.077 INFO:tasks.workunit.client.0.smithi027.stdout:9/412: dread d4/d10/d40/d58/d75/f7d [0,4194304] 0 2024-04-26T23:01:30.077 INFO:tasks.workunit.client.0.smithi027.stdout:9/413: fsync d4/d17/d2e/f31 0 2024-04-26T23:01:30.078 INFO:tasks.workunit.client.0.smithi027.stdout:9/414: creat d4/d10/d40/d58/d75/f8f x:0 0 0 2024-04-26T23:01:30.078 INFO:tasks.workunit.client.0.smithi027.stdout:2/370: dread d2/d5/db/d1b/d1c/f5d [0,4194304] 0 2024-04-26T23:01:30.093 INFO:tasks.workunit.client.0.smithi027.stdout:9/415: write d4/f11 [1657129,70672] 0 2024-04-26T23:01:30.093 INFO:tasks.workunit.client.0.smithi027.stdout:9/416: chown d4/d10/d2c 57339 1 2024-04-26T23:01:30.101 INFO:tasks.workunit.client.0.smithi027.stdout:9/417: dread d4/d10/d40/d58/d75/f7d [0,4194304] 0 2024-04-26T23:01:30.132 INFO:tasks.workunit.client.0.smithi027.stdout:9/418: dwrite d4/d10/d2c/d72/f79 [0,4194304] 0 2024-04-26T23:01:30.134 INFO:tasks.workunit.client.0.smithi027.stdout:9/419: truncate d4/d17/d2e/d46/f6c 3809612 0 2024-04-26T23:01:30.137 INFO:tasks.workunit.client.0.smithi027.stdout:9/420: creat d4/d10/d2c/d72/f90 x:0 0 0 2024-04-26T23:01:30.137 INFO:tasks.workunit.client.0.smithi027.stdout:9/421: write d4/d17/d2e/d46/f55 [576645,81577] 0 2024-04-26T23:01:30.137 INFO:tasks.workunit.client.0.smithi027.stdout:9/422: truncate d4/d10/d40/d58/f5a 5228170 0 2024-04-26T23:01:30.139 INFO:tasks.workunit.client.0.smithi027.stdout:9/423: symlink d4/d17/l91 0 2024-04-26T23:01:30.140 INFO:tasks.workunit.client.0.smithi027.stdout:9/424: getdents d4/d10/d40/d58/d75/d84 0 2024-04-26T23:01:30.141 INFO:tasks.workunit.client.0.smithi027.stdout:9/425: mknod d4/d17/d2e/d81/c92 0 2024-04-26T23:01:30.142 INFO:tasks.workunit.client.0.smithi027.stdout:9/426: write d4/f43 [1275948,122081] 0 2024-04-26T23:01:30.146 INFO:tasks.workunit.client.0.smithi027.stdout:2/371: write d2/d5/f34 [1684693,28013] 0 2024-04-26T23:01:30.146 INFO:tasks.workunit.client.0.smithi027.stdout:9/427: creat d4/d17/f93 x:0 0 0 2024-04-26T23:01:30.146 INFO:tasks.workunit.client.0.smithi027.stdout:4/459: dwrite da/dd/de/d14/d17/f2a [4194304,4194304] 0 2024-04-26T23:01:30.147 INFO:tasks.workunit.client.0.smithi027.stdout:4/460: chown da/dd/de/d14/d17/f20 67 1 2024-04-26T23:01:30.147 INFO:tasks.workunit.client.0.smithi027.stdout:4/461: fsync da/f76 0 2024-04-26T23:01:30.148 INFO:tasks.workunit.client.0.smithi027.stdout:2/372: unlink d2/d5/db/de/f41 0 2024-04-26T23:01:30.149 INFO:tasks.workunit.client.0.smithi027.stdout:4/462: unlink da/f76 0 2024-04-26T23:01:30.150 INFO:tasks.workunit.client.0.smithi027.stdout:4/463: fsync da/dd/de/d14/d21/d41/f72 0 2024-04-26T23:01:30.151 INFO:tasks.workunit.client.0.smithi027.stdout:4/464: mknod da/dd/de/d14/d21/d67/c88 0 2024-04-26T23:01:30.151 INFO:tasks.workunit.client.0.smithi027.stdout:4/465: read da/dd/de/d14/d21/f46 [2279133,127820] 0 2024-04-26T23:01:30.153 INFO:tasks.workunit.client.0.smithi027.stdout:4/466: unlink da/dd/de/d14/d17/f65 0 2024-04-26T23:01:30.153 INFO:tasks.workunit.client.0.smithi027.stdout:4/467: chown da/dd/de/d14/d17/d24/c62 96092 1 2024-04-26T23:01:30.154 INFO:tasks.workunit.client.0.smithi027.stdout:4/468: mknod da/dd/de/d14/d21/c89 0 2024-04-26T23:01:30.156 INFO:tasks.workunit.client.0.smithi027.stdout:4/469: mkdir da/dd/de/d14/d21/d67/d8a 0 2024-04-26T23:01:30.157 INFO:tasks.workunit.client.0.smithi027.stdout:4/470: creat da/dd/d5f/f8b x:0 0 0 2024-04-26T23:01:30.158 INFO:tasks.workunit.client.0.smithi027.stdout:1/498: sync 2024-04-26T23:01:30.158 INFO:tasks.workunit.client.0.smithi027.stdout:7/439: sync 2024-04-26T23:01:30.158 INFO:tasks.workunit.client.0.smithi027.stdout:1/499: chown d1/d26/d66/c82 86 1 2024-04-26T23:01:30.162 INFO:tasks.workunit.client.0.smithi027.stdout:0/431: sync 2024-04-26T23:01:30.162 INFO:tasks.workunit.client.0.smithi027.stdout:3/488: sync 2024-04-26T23:01:30.162 INFO:tasks.workunit.client.0.smithi027.stdout:6/350: sync 2024-04-26T23:01:30.162 INFO:tasks.workunit.client.0.smithi027.stdout:5/462: sync 2024-04-26T23:01:30.162 INFO:tasks.workunit.client.0.smithi027.stdout:8/437: sync 2024-04-26T23:01:30.163 INFO:tasks.workunit.client.0.smithi027.stdout:6/351: truncate d1/d30/d40/f63 483587 0 2024-04-26T23:01:30.163 INFO:tasks.workunit.client.0.smithi027.stdout:3/489: creat d6/dc/d34/d78/da1/fa8 x:0 0 0 2024-04-26T23:01:30.163 INFO:tasks.workunit.client.0.smithi027.stdout:3/490: readlink d6/dc/d34/d2e/d5f/l7a 0 2024-04-26T23:01:30.163 INFO:tasks.workunit.client.0.smithi027.stdout:5/463: creat d4/d24/d22/d29/d39/d84/f92 x:0 0 0 2024-04-26T23:01:30.163 INFO:tasks.workunit.client.0.smithi027.stdout:5/464: chown d4/d1a/d2b 1988535 1 2024-04-26T23:01:30.163 INFO:tasks.workunit.client.0.smithi027.stdout:3/491: fsync d6/dc/d2f/d6b/d74/da6/d40/f57 0 2024-04-26T23:01:30.163 INFO:tasks.workunit.client.0.smithi027.stdout:5/465: write d4/db/f10 [2977693,86370] 0 2024-04-26T23:01:30.165 INFO:tasks.workunit.client.0.smithi027.stdout:6/352: write d1/df/f8 [4205796,9133] 0 2024-04-26T23:01:30.165 INFO:tasks.workunit.client.0.smithi027.stdout:3/492: rmdir d6/dc/d2f/d6b/d74/da6 39 2024-04-26T23:01:30.169 INFO:tasks.workunit.client.0.smithi027.stdout:5/466: mknod d4/d24/d22/d29/d2a/d3b/c93 0 2024-04-26T23:01:30.170 INFO:tasks.workunit.client.0.smithi027.stdout:3/493: creat d6/dc/d34/d2e/d5f/d90/fa9 x:0 0 0 2024-04-26T23:01:30.175 INFO:tasks.workunit.client.0.smithi027.stdout:2/373: dread d2/d5/db/d1b/d1c/d25/d19/d36/d39/f3f [0,4194304] 0 2024-04-26T23:01:30.177 INFO:tasks.workunit.client.0.smithi027.stdout:5/467: creat d4/d1a/f94 x:0 0 0 2024-04-26T23:01:30.177 INFO:tasks.workunit.client.0.smithi027.stdout:5/468: readlink d4/l21 0 2024-04-26T23:01:30.181 INFO:tasks.workunit.client.0.smithi027.stdout:2/374: creat d2/d5/db/de/d21/f65 x:0 0 0 2024-04-26T23:01:30.181 INFO:tasks.workunit.client.0.smithi027.stdout:5/469: rename d4/db/d16/c47 to d4/d24/d55/c95 0 2024-04-26T23:01:30.181 INFO:tasks.workunit.client.0.smithi027.stdout:5/470: fdatasync d4/f15 0 2024-04-26T23:01:30.182 INFO:tasks.workunit.client.0.smithi027.stdout:2/375: write d2/d5/db/d1b/d1c/d25/f4a [495864,27785] 0 2024-04-26T23:01:30.182 INFO:tasks.workunit.client.0.smithi027.stdout:2/376: write d2/d5/db/de/d21/f5f [273957,23537] 0 2024-04-26T23:01:30.182 INFO:tasks.workunit.client.0.smithi027.stdout:2/377: fdatasync d2/d5/db/d1b/d1c/d25/d19/f5b 0 2024-04-26T23:01:30.185 INFO:tasks.workunit.client.0.smithi027.stdout:2/378: mknod d2/d5/db/de/c66 0 2024-04-26T23:01:30.185 INFO:tasks.workunit.client.0.smithi027.stdout:2/379: write d2/d5/f34 [2071665,33320] 0 2024-04-26T23:01:30.196 INFO:tasks.workunit.client.0.smithi027.stdout:5/471: write d4/f4e [1356711,9296] 0 2024-04-26T23:01:30.197 INFO:tasks.workunit.client.0.smithi027.stdout:5/472: chown d4/d24/d22/d29/f33 10 1 2024-04-26T23:01:30.197 INFO:tasks.workunit.client.0.smithi027.stdout:5/473: write d4/d1a/d2b/f7b [656995,22993] 0 2024-04-26T23:01:30.197 INFO:tasks.workunit.client.0.smithi027.stdout:5/474: fdatasync d4/d24/f5d 0 2024-04-26T23:01:30.198 INFO:tasks.workunit.client.0.smithi027.stdout:9/428: dwrite d4/d17/d2e/d46/f6c [0,4194304] 0 2024-04-26T23:01:30.207 INFO:tasks.workunit.client.0.smithi027.stdout:7/440: write d0/f30 [2462716,11625] 0 2024-04-26T23:01:30.207 INFO:tasks.workunit.client.0.smithi027.stdout:7/441: chown d0/d28/d2b/d3a/d48/d56/d88/f29 1 1 2024-04-26T23:01:30.210 INFO:tasks.workunit.client.0.smithi027.stdout:7/442: truncate d0/d28/d2b/d3a/d48/f63 226229 0 2024-04-26T23:01:30.215 INFO:tasks.workunit.client.0.smithi027.stdout:7/443: mknod d0/d28/d2b/d55/c8a 0 2024-04-26T23:01:30.222 INFO:tasks.workunit.client.0.smithi027.stdout:2/380: dread d2/d5/f22 [0,4194304] 0 2024-04-26T23:01:30.223 INFO:tasks.workunit.client.0.smithi027.stdout:2/381: write d2/d5/db/d1b/d1c/d25/d19/d44/d4b/f56 [1028263,113690] 0 2024-04-26T23:01:30.225 INFO:tasks.workunit.client.0.smithi027.stdout:2/382: creat d2/f67 x:0 0 0 2024-04-26T23:01:30.230 INFO:tasks.workunit.client.0.smithi027.stdout:2/383: readlink d2/d5/db/de/l2a 0 2024-04-26T23:01:30.230 INFO:tasks.workunit.client.0.smithi027.stdout:2/384: write d2/d5/db/d1b/d1c/d25/f4a [320736,117373] 0 2024-04-26T23:01:30.237 INFO:tasks.workunit.client.0.smithi027.stdout:7/444: write d0/d2f/f31 [2307901,55073] 0 2024-04-26T23:01:30.245 INFO:tasks.workunit.client.0.smithi027.stdout:9/429: dread d4/d17/d2e/f53 [0,4194304] 0 2024-04-26T23:01:30.251 INFO:tasks.workunit.client.0.smithi027.stdout:0/432: dwrite d0/d4/da/d15/d26/f7c [4194304,4194304] 0 2024-04-26T23:01:30.251 INFO:tasks.workunit.client.0.smithi027.stdout:0/433: fsync d0/d4/f7 0 2024-04-26T23:01:30.252 INFO:tasks.workunit.client.0.smithi027.stdout:4/471: dwrite da/dd/f2b [0,4194304] 0 2024-04-26T23:01:30.258 INFO:tasks.workunit.client.0.smithi027.stdout:7/445: read d0/d28/d2b/d3a/d48/d56/d88/f3c [636580,24358] 0 2024-04-26T23:01:30.259 INFO:tasks.workunit.client.0.smithi027.stdout:7/446: creat d0/d28/d2b/d3a/f8b x:0 0 0 2024-04-26T23:01:30.262 INFO:tasks.workunit.client.0.smithi027.stdout:6/353: write d1/df/d43/d56/d6b/f76 [2715976,93210] 0 2024-04-26T23:01:30.262 INFO:tasks.workunit.client.0.smithi027.stdout:1/500: dwrite d1/d5/d21/f22 [0,4194304] 0 2024-04-26T23:01:30.267 INFO:tasks.workunit.client.0.smithi027.stdout:6/354: creat d1/d30/f83 x:0 0 0 2024-04-26T23:01:30.267 INFO:tasks.workunit.client.0.smithi027.stdout:1/501: truncate d1/d26/d36/d56/f81 678005 0 2024-04-26T23:01:30.268 INFO:tasks.workunit.client.0.smithi027.stdout:0/434: dread d0/d4/da/d15/f48 [0,4194304] 0 2024-04-26T23:01:30.270 INFO:tasks.workunit.client.0.smithi027.stdout:0/435: mknod d0/df/d3f/d64/c82 0 2024-04-26T23:01:30.270 INFO:tasks.workunit.client.0.smithi027.stdout:0/436: chown d0/d4/da/d1a/d1e/l46 0 1 2024-04-26T23:01:30.270 INFO:tasks.workunit.client.0.smithi027.stdout:0/437: stat d0/d4/da/d1a/d59 0 2024-04-26T23:01:30.275 INFO:tasks.workunit.client.0.smithi027.stdout:0/438: rename d0/d4/c5b to d0/df/d3f/d4a/c83 0 2024-04-26T23:01:30.275 INFO:tasks.workunit.client.0.smithi027.stdout:8/438: dwrite d9/da/d2a/d5e/d2b/d3b/f9b [0,4194304] 0 2024-04-26T23:01:30.275 INFO:tasks.workunit.client.0.smithi027.stdout:3/494: dwrite d6/dc/d34/d78/da1/fa8 [0,4194304] 0 2024-04-26T23:01:30.278 INFO:tasks.workunit.client.0.smithi027.stdout:3/495: mknod d6/dc/d2f/d6b/d74/da6/d40/caa 0 2024-04-26T23:01:30.280 INFO:tasks.workunit.client.0.smithi027.stdout:8/439: mkdir d9/da/da1 0 2024-04-26T23:01:30.285 INFO:tasks.workunit.client.0.smithi027.stdout:4/472: write da/dd/de/d14/d21/d41/f49 [2301235,90337] 0 2024-04-26T23:01:30.286 INFO:tasks.workunit.client.0.smithi027.stdout:3/496: symlink d6/dc/d2f/d6b/d74/da6/d59/d61/lab 0 2024-04-26T23:01:30.290 INFO:tasks.workunit.client.0.smithi027.stdout:4/473: link da/dd/de/d14/f43 da/dd/de/d14/d21/d67/d8a/f8c 0 2024-04-26T23:01:30.291 INFO:tasks.workunit.client.0.smithi027.stdout:4/474: rmdir da/dd/d5f 39 2024-04-26T23:01:30.291 INFO:tasks.workunit.client.0.smithi027.stdout:4/475: dread - da/dd/de/d14/d21/d41/d59/f77 zero size 2024-04-26T23:01:30.293 INFO:tasks.workunit.client.0.smithi027.stdout:2/385: dwrite d2/d5/db/d1b/f38 [4194304,4194304] 0 2024-04-26T23:01:30.293 INFO:tasks.workunit.client.0.smithi027.stdout:2/386: readlink d2/d5/l33 0 2024-04-26T23:01:30.296 INFO:tasks.workunit.client.0.smithi027.stdout:0/439: dread d0/d4/da/f11 [0,4194304] 0 2024-04-26T23:01:30.296 INFO:tasks.workunit.client.0.smithi027.stdout:4/476: link da/dd/de/d14/d21/f7d da/dd/de/d14/d21/d41/d59/f8d 0 2024-04-26T23:01:30.298 INFO:tasks.workunit.client.0.smithi027.stdout:0/440: getdents d0/df/d3f 0 2024-04-26T23:01:30.300 INFO:tasks.workunit.client.0.smithi027.stdout:4/477: creat da/dd/de/d79/f8e x:0 0 0 2024-04-26T23:01:30.300 INFO:tasks.workunit.client.0.smithi027.stdout:7/447: dread d0/f27 [0,4194304] 0 2024-04-26T23:01:30.318 INFO:tasks.workunit.client.0.smithi027.stdout:2/387: read d2/d5/db/d1b/d1c/f37 [827834,13158] 0 2024-04-26T23:01:30.320 INFO:tasks.workunit.client.0.smithi027.stdout:2/388: creat d2/d5/db/de/f68 x:0 0 0 2024-04-26T23:01:30.321 INFO:tasks.workunit.client.0.smithi027.stdout:2/389: symlink d2/d5/db/d1b/d1c/d25/d19/d36/l69 0 2024-04-26T23:01:30.322 INFO:tasks.workunit.client.0.smithi027.stdout:6/355: dwrite d1/df/f39 [4194304,4194304] 0 2024-04-26T23:01:30.326 INFO:tasks.workunit.client.0.smithi027.stdout:2/390: rename d2/d5/db/d1b/d1c/d25/d2f to d2/d5/db/d1b/d1c/d25/d6a 0 2024-04-26T23:01:30.326 INFO:tasks.workunit.client.0.smithi027.stdout:2/391: chown d2/d5/db/d1b/d1c/d25/d19/d44/c53 121657117 1 2024-04-26T23:01:30.336 INFO:tasks.workunit.client.0.smithi027.stdout:1/502: dwrite d1/f1f [0,4194304] 0 2024-04-26T23:01:30.338 INFO:tasks.workunit.client.0.smithi027.stdout:1/503: mknod d1/c96 0 2024-04-26T23:01:30.339 INFO:tasks.workunit.client.0.smithi027.stdout:1/504: symlink d1/d5/l97 0 2024-04-26T23:01:30.341 INFO:tasks.workunit.client.0.smithi027.stdout:1/505: truncate d1/f32 5842648 0 2024-04-26T23:01:30.341 INFO:tasks.workunit.client.0.smithi027.stdout:1/506: fdatasync d1/d26/d6b/f5a 0 2024-04-26T23:01:30.343 INFO:tasks.workunit.client.0.smithi027.stdout:2/392: dread d2/d5/db/d1b/d1c/d25/f3d [0,4194304] 0 2024-04-26T23:01:30.343 INFO:tasks.workunit.client.0.smithi027.stdout:2/393: write d2/d5/db/d1b/d1c/d25/d19/d44/f60 [4171228,79369] 0 2024-04-26T23:01:30.343 INFO:tasks.workunit.client.0.smithi027.stdout:2/394: fsync d2/d5/db/d1b/d1c/f61 0 2024-04-26T23:01:30.353 INFO:tasks.workunit.client.0.smithi027.stdout:6/356: write d1/f59 [1070666,100002] 0 2024-04-26T23:01:30.353 INFO:tasks.workunit.client.0.smithi027.stdout:6/357: chown d1/df/f8 7 1 2024-04-26T23:01:30.353 INFO:tasks.workunit.client.0.smithi027.stdout:6/358: stat d1/d30/c3e 0 2024-04-26T23:01:30.353 INFO:tasks.workunit.client.0.smithi027.stdout:6/359: truncate d1/de/f1b 1222033 0 2024-04-26T23:01:30.353 INFO:tasks.workunit.client.0.smithi027.stdout:2/395: unlink d2/d5/db/d1b/d1c/d25/d19/d44/f60 0 2024-04-26T23:01:30.353 INFO:tasks.workunit.client.0.smithi027.stdout:2/396: symlink d2/d5/db/d1b/d1c/l6b 0 2024-04-26T23:01:30.353 INFO:tasks.workunit.client.0.smithi027.stdout:2/397: readlink d2/d5/l33 0 2024-04-26T23:01:30.353 INFO:tasks.workunit.client.0.smithi027.stdout:1/507: dread d1/f94 [0,4194304] 0 2024-04-26T23:01:30.353 INFO:tasks.workunit.client.0.smithi027.stdout:8/440: write d9/da/d2a/d5e/d72/d84/d35/f56 [2256933,126071] 0 2024-04-26T23:01:30.353 INFO:tasks.workunit.client.0.smithi027.stdout:1/508: link d1/d26/d86/l4f d1/d5/d21/l98 0 2024-04-26T23:01:30.353 INFO:tasks.workunit.client.0.smithi027.stdout:1/509: creat d1/d26/d36/d88/f99 x:0 0 0 2024-04-26T23:01:30.356 INFO:tasks.workunit.client.0.smithi027.stdout:4/478: write da/dd/de/d14/d21/d4a/f6a [319412,53737] 0 2024-04-26T23:01:30.357 INFO:tasks.workunit.client.0.smithi027.stdout:4/479: chown da/dd/de/d14/d21/d41/f72 17941 1 2024-04-26T23:01:30.357 INFO:tasks.workunit.client.0.smithi027.stdout:0/441: dwrite d0/d4/da/f14 [0,4194304] 0 2024-04-26T23:01:30.357 INFO:tasks.workunit.client.0.smithi027.stdout:1/510: rename d1/c9 to d1/d26/d36/d56/c9a 0 2024-04-26T23:01:30.358 INFO:tasks.workunit.client.0.smithi027.stdout:6/360: dread d1/df/f68 [0,4194304] 0 2024-04-26T23:01:30.358 INFO:tasks.workunit.client.0.smithi027.stdout:6/361: truncate d1/f72 657248 0 2024-04-26T23:01:30.364 INFO:tasks.workunit.client.0.smithi027.stdout:0/442: link d0/df/d3f/d64/f38 d0/d4/da/d1a/d1e/f84 0 2024-04-26T23:01:30.364 INFO:tasks.workunit.client.0.smithi027.stdout:2/398: write d2/d5/db/d1b/f38 [2210849,51487] 0 2024-04-26T23:01:30.364 INFO:tasks.workunit.client.0.smithi027.stdout:1/511: mknod d1/dc/c9b 0 2024-04-26T23:01:30.366 INFO:tasks.workunit.client.0.smithi027.stdout:0/443: getdents d0/df 0 2024-04-26T23:01:30.366 INFO:tasks.workunit.client.0.smithi027.stdout:0/444: fdatasync d0/f12 0 2024-04-26T23:01:30.369 INFO:tasks.workunit.client.0.smithi027.stdout:1/512: creat d1/d26/d86/f9c x:0 0 0 2024-04-26T23:01:30.369 INFO:tasks.workunit.client.0.smithi027.stdout:0/445: mknod d0/d4/da/d1a/d6d/c85 0 2024-04-26T23:01:30.369 INFO:tasks.workunit.client.0.smithi027.stdout:0/446: chown d0/df/d3f/d64/l4b 56480341 1 2024-04-26T23:01:30.369 INFO:tasks.workunit.client.0.smithi027.stdout:0/447: fsync d0/d4/da/d15/d26/f7c 0 2024-04-26T23:01:30.369 INFO:tasks.workunit.client.0.smithi027.stdout:1/513: write d1/d26/d6b/d42/d79/f92 [1026826,128701] 0 2024-04-26T23:01:30.369 INFO:tasks.workunit.client.0.smithi027.stdout:1/514: chown d1/d26/d36 2100515732 1 2024-04-26T23:01:30.369 INFO:tasks.workunit.client.0.smithi027.stdout:1/515: chown d1/d26/d86/d5c 119 1 2024-04-26T23:01:30.369 INFO:tasks.workunit.client.0.smithi027.stdout:2/399: read d2/d5/db/d1b/d1c/d25/d19/d44/d4b/f56 [810431,112625] 0 2024-04-26T23:01:30.378 INFO:tasks.workunit.client.0.smithi027.stdout:0/448: mknod d0/d4/da/d1a/d2a/c86 0 2024-04-26T23:01:30.379 INFO:tasks.workunit.client.0.smithi027.stdout:2/400: creat d2/d5/db/f6c x:0 0 0 2024-04-26T23:01:30.379 INFO:tasks.workunit.client.0.smithi027.stdout:2/401: chown d2/d5/db/d1b/d1c/d25/d19/d23/f47 1559835 1 2024-04-26T23:01:30.380 INFO:tasks.workunit.client.0.smithi027.stdout:0/449: rename d0/d4/da/d15/c2b to d0/d4/da/d1a/d2a/c87 0 2024-04-26T23:01:30.381 INFO:tasks.workunit.client.0.smithi027.stdout:8/441: dread d9/da/d2a/d5e/f57 [4194304,4194304] 0 2024-04-26T23:01:30.381 INFO:tasks.workunit.client.0.smithi027.stdout:2/402: creat d2/d5/db/de/d21/d26/f6d x:0 0 0 2024-04-26T23:01:30.382 INFO:tasks.workunit.client.0.smithi027.stdout:0/450: rename d0/d4/da/d1a/d2a/c32 to d0/df/c88 0 2024-04-26T23:01:30.386 INFO:tasks.workunit.client.0.smithi027.stdout:2/403: getdents d2/d5 0 2024-04-26T23:01:30.387 INFO:tasks.workunit.client.0.smithi027.stdout:0/451: creat d0/d4/da/d1a/f89 x:0 0 0 2024-04-26T23:01:30.387 INFO:tasks.workunit.client.0.smithi027.stdout:8/442: creat d9/da/d2a/fa2 x:0 0 0 2024-04-26T23:01:30.387 INFO:tasks.workunit.client.0.smithi027.stdout:0/452: read d0/d4/f24 [361137,101590] 0 2024-04-26T23:01:30.389 INFO:tasks.workunit.client.0.smithi027.stdout:8/443: getdents d9/da/d2a/d5e/d72/d84 0 2024-04-26T23:01:30.390 INFO:tasks.workunit.client.0.smithi027.stdout:8/444: rmdir d9/da/d2a/d5e/d72/d84/d5d 39 2024-04-26T23:01:30.391 INFO:tasks.workunit.client.0.smithi027.stdout:8/445: truncate d9/da/d33/f60 3616963 0 2024-04-26T23:01:30.394 INFO:tasks.workunit.client.0.smithi027.stdout:8/446: mkdir d9/da/d2a/d5e/d72/d84/d35/d8d/da3 0 2024-04-26T23:01:30.410 INFO:tasks.workunit.client.0.smithi027.stdout:2/404: dread d2/d5/db/de/f30 [0,4194304] 0 2024-04-26T23:01:30.414 INFO:tasks.workunit.client.0.smithi027.stdout:1/516: dread d1/dc/f5b [0,4194304] 0 2024-04-26T23:01:30.415 INFO:tasks.workunit.client.0.smithi027.stdout:4/480: dwrite da/dd/de/d14/d21/d4a/f4f [4194304,4194304] 0 2024-04-26T23:01:30.415 INFO:tasks.workunit.client.0.smithi027.stdout:4/481: write da/dd/de/d14/d21/d4a/f4f [8974566,69648] 0 2024-04-26T23:01:30.416 INFO:tasks.workunit.client.0.smithi027.stdout:1/517: mknod d1/d26/d6b/d8e/c9d 0 2024-04-26T23:01:30.416 INFO:tasks.workunit.client.0.smithi027.stdout:4/482: creat da/f8f x:0 0 0 2024-04-26T23:01:30.418 INFO:tasks.workunit.client.0.smithi027.stdout:1/518: mkdir d1/d26/d66/d73/d9e 0 2024-04-26T23:01:30.418 INFO:tasks.workunit.client.0.smithi027.stdout:1/519: chown d1/d5/f46 10884 1 2024-04-26T23:01:30.419 INFO:tasks.workunit.client.0.smithi027.stdout:1/520: fsync d1/d5/d40/f52 0 2024-04-26T23:01:30.426 INFO:tasks.workunit.client.0.smithi027.stdout:2/405: write d2/d5/db/d1b/d1c/f37 [5671743,87040] 0 2024-04-26T23:01:30.426 INFO:tasks.workunit.client.0.smithi027.stdout:2/406: write d2/d5/db/d1b/d1c/f5d [1299657,71990] 0 2024-04-26T23:01:30.437 INFO:tasks.workunit.client.0.smithi027.stdout:6/362: dwrite d1/df/f31 [0,4194304] 0 2024-04-26T23:01:30.496 INFO:tasks.workunit.client.0.smithi027.stdout:4/483: rename da/l61 to da/dd/de/d14/d21/d67/l90 0 2024-04-26T23:01:30.502 INFO:tasks.workunit.client.0.smithi027.stdout:1/521: link d1/d5/d21/f27 d1/d26/d66/d73/d9e/f9f 0 2024-04-26T23:01:30.504 INFO:tasks.workunit.client.0.smithi027.stdout:8/447: dwrite d9/da/d2a/d5e/f6e [0,4194304] 0 2024-04-26T23:01:30.505 INFO:tasks.workunit.client.0.smithi027.stdout:0/453: dwrite d0/d4/da/f2f [4194304,4194304] 0 2024-04-26T23:01:30.506 INFO:tasks.workunit.client.0.smithi027.stdout:8/448: mkdir d9/da/d2a/d5e/d9e/da4 0 2024-04-26T23:01:30.506 INFO:tasks.workunit.client.0.smithi027.stdout:8/449: truncate d9/da/d2a/fa2 96684 0 2024-04-26T23:01:30.508 INFO:tasks.workunit.client.0.smithi027.stdout:2/407: rename d2/d5/db/d1b/d1c/f5d to d2/d5/f6e 0 2024-04-26T23:01:30.509 INFO:tasks.workunit.client.0.smithi027.stdout:4/484: mknod da/dd/de/d14/d17/c91 0 2024-04-26T23:01:30.509 INFO:tasks.workunit.client.0.smithi027.stdout:4/485: dread - da/dd/f5c zero size 2024-04-26T23:01:30.509 INFO:tasks.workunit.client.0.smithi027.stdout:4/486: fdatasync da/dd/f2b 0 2024-04-26T23:01:30.511 INFO:tasks.workunit.client.0.smithi027.stdout:4/487: read da/dd/de/d14/d17/f20 [1386848,39150] 0 2024-04-26T23:01:30.512 INFO:tasks.workunit.client.0.smithi027.stdout:6/363: dwrite d1/df/d43/d56/f77 [0,4194304] 0 2024-04-26T23:01:30.513 INFO:tasks.workunit.client.0.smithi027.stdout:1/522: creat d1/d26/d6b/fa0 x:0 0 0 2024-04-26T23:01:30.522 INFO:tasks.workunit.client.0.smithi027.stdout:0/454: creat d0/d4/da/d15/d26/f8a x:0 0 0 2024-04-26T23:01:30.522 INFO:tasks.workunit.client.0.smithi027.stdout:0/455: chown d0/d4/da/d15/d26/f8a 49565852 1 2024-04-26T23:01:30.524 INFO:tasks.workunit.client.0.smithi027.stdout:4/488: mkdir da/dd/d92 0 2024-04-26T23:01:30.527 INFO:tasks.workunit.client.0.smithi027.stdout:6/364: mkdir d1/df/d49/d7f/d84 0 2024-04-26T23:01:30.528 INFO:tasks.workunit.client.0.smithi027.stdout:1/523: creat d1/d26/d6b/fa1 x:0 0 0 2024-04-26T23:01:30.529 INFO:tasks.workunit.client.0.smithi027.stdout:0/456: symlink d0/d4/da/d15/l8b 0 2024-04-26T23:01:30.529 INFO:tasks.workunit.client.0.smithi027.stdout:0/457: write d0/d4/da/f78 [705209,64894] 0 2024-04-26T23:01:30.530 INFO:tasks.workunit.client.0.smithi027.stdout:4/489: mkdir da/dd/de/d14/d93 0 2024-04-26T23:01:30.532 INFO:tasks.workunit.client.0.smithi027.stdout:6/365: rmdir d1/de/d57 39 2024-04-26T23:01:30.534 INFO:tasks.workunit.client.0.smithi027.stdout:4/490: creat da/dd/de/f94 x:0 0 0 2024-04-26T23:01:30.535 INFO:tasks.workunit.client.0.smithi027.stdout:4/491: write da/dd/de/d14/d21/f7d [4343491,72714] 0 2024-04-26T23:01:30.535 INFO:tasks.workunit.client.0.smithi027.stdout:9/430: sync 2024-04-26T23:01:30.535 INFO:tasks.workunit.client.0.smithi027.stdout:6/366: mkdir d1/d6a/d85 0 2024-04-26T23:01:30.537 INFO:tasks.workunit.client.0.smithi027.stdout:9/431: write d4/d10/f7e [816441,120252] 0 2024-04-26T23:01:30.537 INFO:tasks.workunit.client.0.smithi027.stdout:6/367: write d1/de/f1b [1367001,125316] 0 2024-04-26T23:01:30.538 INFO:tasks.workunit.client.0.smithi027.stdout:5/475: sync 2024-04-26T23:01:30.539 INFO:tasks.workunit.client.0.smithi027.stdout:9/432: rename d4/d17/d2e/f31 to d4/d10/d40/d58/d75/d84/d86/f94 0 2024-04-26T23:01:30.540 INFO:tasks.workunit.client.0.smithi027.stdout:9/433: creat d4/d17/d2e/f95 x:0 0 0 2024-04-26T23:01:30.541 INFO:tasks.workunit.client.0.smithi027.stdout:5/476: rmdir d4/d24/d22/d29/d2a 39 2024-04-26T23:01:30.542 INFO:tasks.workunit.client.0.smithi027.stdout:9/434: creat d4/d10/d40/d58/f96 x:0 0 0 2024-04-26T23:01:30.542 INFO:tasks.workunit.client.0.smithi027.stdout:9/435: chown d4/d17/d2e/d46/c69 13552897 1 2024-04-26T23:01:30.545 INFO:tasks.workunit.client.0.smithi027.stdout:2/408: dwrite d2/d5/db/de/d21/f5f [0,4194304] 0 2024-04-26T23:01:30.555 INFO:tasks.workunit.client.0.smithi027.stdout:1/524: write d1/d26/d66/d73/d9e/f9f [3168928,48268] 0 2024-04-26T23:01:30.557 INFO:tasks.workunit.client.0.smithi027.stdout:1/525: creat d1/d26/d86/fa2 x:0 0 0 2024-04-26T23:01:30.558 INFO:tasks.workunit.client.0.smithi027.stdout:4/492: dread da/dd/de/d14/d17/d24/f35 [0,4194304] 0 2024-04-26T23:01:30.558 INFO:tasks.workunit.client.0.smithi027.stdout:4/493: truncate da/dd/de/d14/d21/d41/d59/d74/f7e 413749 0 2024-04-26T23:01:30.569 INFO:tasks.workunit.client.0.smithi027.stdout:1/526: symlink d1/d26/d6b/d8e/la3 0 2024-04-26T23:01:30.570 INFO:tasks.workunit.client.0.smithi027.stdout:4/494: creat da/dd/f95 x:0 0 0 2024-04-26T23:01:30.570 INFO:tasks.workunit.client.0.smithi027.stdout:4/495: readlink da/dd/de/d14/d21/d67/l90 0 2024-04-26T23:01:30.573 INFO:tasks.workunit.client.0.smithi027.stdout:4/496: link da/d40/f66 da/dd/de/d14/d93/f96 0 2024-04-26T23:01:30.574 INFO:tasks.workunit.client.0.smithi027.stdout:4/497: symlink da/dd/d5e/l97 0 2024-04-26T23:01:30.598 INFO:tasks.workunit.client.0.smithi027.stdout:4/498: write da/dd/de/d14/d21/f23 [889485,47441] 0 2024-04-26T23:01:30.600 INFO:tasks.workunit.client.0.smithi027.stdout:4/499: rename da/dd/d5e/l97 to da/dd/d92/l98 0 2024-04-26T23:01:30.602 INFO:tasks.workunit.client.0.smithi027.stdout:5/477: dwrite d4/d24/d22/d29/d39/f7f [0,4194304] 0 2024-04-26T23:01:30.602 INFO:tasks.workunit.client.0.smithi027.stdout:4/500: symlink da/dd/d5f/l99 0 2024-04-26T23:01:30.605 INFO:tasks.workunit.client.0.smithi027.stdout:5/478: rename d4/d1a/d2b/c46 to d4/d1a/d2b/c96 0 2024-04-26T23:01:30.607 INFO:tasks.workunit.client.0.smithi027.stdout:4/501: unlink da/dd/f30 0 2024-04-26T23:01:30.607 INFO:tasks.workunit.client.0.smithi027.stdout:4/502: write da/dd/f95 [233849,78888] 0 2024-04-26T23:01:30.608 INFO:tasks.workunit.client.0.smithi027.stdout:5/479: creat d4/d1a/d4a/d68/d40/f97 x:0 0 0 2024-04-26T23:01:30.610 INFO:tasks.workunit.client.0.smithi027.stdout:4/503: mknod da/dd/d5f/c9a 0 2024-04-26T23:01:30.612 INFO:tasks.workunit.client.0.smithi027.stdout:5/480: creat d4/d24/f98 x:0 0 0 2024-04-26T23:01:30.612 INFO:tasks.workunit.client.0.smithi027.stdout:5/481: write d4/f4e [954248,36940] 0 2024-04-26T23:01:30.613 INFO:tasks.workunit.client.0.smithi027.stdout:4/504: rmdir da/dd/de/d14/d21/d41/d59/d74 39 2024-04-26T23:01:30.616 INFO:tasks.workunit.client.0.smithi027.stdout:5/482: mknod d4/d1a/d4a/c99 0 2024-04-26T23:01:30.617 INFO:tasks.workunit.client.0.smithi027.stdout:4/505: mknod da/dd/d92/c9b 0 2024-04-26T23:01:30.620 INFO:tasks.workunit.client.0.smithi027.stdout:5/483: fsync d4/f4e 0 2024-04-26T23:01:30.627 INFO:tasks.workunit.client.0.smithi027.stdout:1/527: dwrite d1/f6 [0,4194304] 0 2024-04-26T23:01:30.629 INFO:tasks.workunit.client.0.smithi027.stdout:1/528: rename d1/d26/d36/l4b to d1/d26/d36/la4 0 2024-04-26T23:01:30.631 INFO:tasks.workunit.client.0.smithi027.stdout:6/368: write d1/df/f31 [2380386,74914] 0 2024-04-26T23:01:30.631 INFO:tasks.workunit.client.0.smithi027.stdout:1/529: getdents d1/d26/d86/d5c 0 2024-04-26T23:01:30.632 INFO:tasks.workunit.client.0.smithi027.stdout:0/458: dwrite d0/d4/f53 [0,4194304] 0 2024-04-26T23:01:30.632 INFO:tasks.workunit.client.0.smithi027.stdout:1/530: write d1/f94 [194153,20835] 0 2024-04-26T23:01:30.633 INFO:tasks.workunit.client.0.smithi027.stdout:1/531: write d1/d26/d6b/fa0 [198500,100813] 0 2024-04-26T23:01:30.633 INFO:tasks.workunit.client.0.smithi027.stdout:1/532: truncate d1/d2a/f62 873035 0 2024-04-26T23:01:30.637 INFO:tasks.workunit.client.0.smithi027.stdout:5/484: dread d4/db/f23 [0,4194304] 0 2024-04-26T23:01:30.639 INFO:tasks.workunit.client.0.smithi027.stdout:6/369: creat d1/df/d49/f86 x:0 0 0 2024-04-26T23:01:30.640 INFO:tasks.workunit.client.0.smithi027.stdout:0/459: getdents d0/d4/da/d1a/d1e 0 2024-04-26T23:01:30.640 INFO:tasks.workunit.client.0.smithi027.stdout:0/460: chown d0/d4/da/d1a/d6d 62181549 1 2024-04-26T23:01:30.640 INFO:tasks.workunit.client.0.smithi027.stdout:2/409: dwrite d2/d5/db/de/f30 [4194304,4194304] 0 2024-04-26T23:01:30.640 INFO:tasks.workunit.client.0.smithi027.stdout:2/410: truncate d2/f67 464591 0 2024-04-26T23:01:30.641 INFO:tasks.workunit.client.0.smithi027.stdout:1/533: symlink d1/d26/d66/d91/la5 0 2024-04-26T23:01:30.641 INFO:tasks.workunit.client.0.smithi027.stdout:4/506: dread da/dd/de/d14/d17/f33 [0,4194304] 0 2024-04-26T23:01:30.644 INFO:tasks.workunit.client.0.smithi027.stdout:5/485: rmdir d4/d24/d22/d29/d2a 39 2024-04-26T23:01:30.645 INFO:tasks.workunit.client.0.smithi027.stdout:0/461: rename d0/f1b to d0/d4/da/d1a/d1e/f8c 0 2024-04-26T23:01:30.646 INFO:tasks.workunit.client.0.smithi027.stdout:6/370: getdents d1/df/d49 0 2024-04-26T23:01:30.649 INFO:tasks.workunit.client.0.smithi027.stdout:2/411: creat d2/f6f x:0 0 0 2024-04-26T23:01:30.650 INFO:tasks.workunit.client.0.smithi027.stdout:1/534: mknod d1/d26/d6b/d42/ca6 0 2024-04-26T23:01:30.651 INFO:tasks.workunit.client.0.smithi027.stdout:5/486: symlink d4/d1a/d2b/l9a 0 2024-04-26T23:01:30.653 INFO:tasks.workunit.client.0.smithi027.stdout:4/507: rename da/dd/de/d14/d21/d4a/f4f to da/dd/de/d14/d21/d41/d59/f9c 0 2024-04-26T23:01:30.653 INFO:tasks.workunit.client.0.smithi027.stdout:4/508: chown da/dd/de/d14/d17/d24/c62 1759 1 2024-04-26T23:01:30.653 INFO:tasks.workunit.client.0.smithi027.stdout:4/509: write f2 [17738,7426] 0 2024-04-26T23:01:30.653 INFO:tasks.workunit.client.0.smithi027.stdout:0/462: symlink d0/df/d3f/d64/l8d 0 2024-04-26T23:01:30.653 INFO:tasks.workunit.client.0.smithi027.stdout:0/463: readlink d0/d4/l36 0 2024-04-26T23:01:30.654 INFO:tasks.workunit.client.0.smithi027.stdout:0/464: write d0/d4/da/d15/d26/f7c [8550919,121725] 0 2024-04-26T23:01:30.654 INFO:tasks.workunit.client.0.smithi027.stdout:6/371: getdents d1 0 2024-04-26T23:01:30.655 INFO:tasks.workunit.client.0.smithi027.stdout:2/412: mknod d2/d5/db/d1b/d1c/d25/d19/d23/c70 0 2024-04-26T23:01:30.655 INFO:tasks.workunit.client.0.smithi027.stdout:2/413: write d2/d5/f34 [1665483,19677] 0 2024-04-26T23:01:30.656 INFO:tasks.workunit.client.0.smithi027.stdout:5/487: creat d4/db/d16/f9b x:0 0 0 2024-04-26T23:01:30.657 INFO:tasks.workunit.client.0.smithi027.stdout:4/510: getdents da/dd/de/d14/d17 0 2024-04-26T23:01:30.657 INFO:tasks.workunit.client.0.smithi027.stdout:4/511: chown da/dd/de/d14/d93 732514410 1 2024-04-26T23:01:30.657 INFO:tasks.workunit.client.0.smithi027.stdout:6/372: mknod d1/de/d57/c87 0 2024-04-26T23:01:30.659 INFO:tasks.workunit.client.0.smithi027.stdout:5/488: rename d4/f4e to d4/d24/d22/d29/d39/f9c 0 2024-04-26T23:01:30.661 INFO:tasks.workunit.client.0.smithi027.stdout:4/512: rmdir da/dd/de/d14/d17 39 2024-04-26T23:01:30.661 INFO:tasks.workunit.client.0.smithi027.stdout:4/513: chown da/dd/de/d14/d21/d4a/c5b 238 1 2024-04-26T23:01:30.662 INFO:tasks.workunit.client.0.smithi027.stdout:4/514: symlink da/d40/d87/l9d 0 2024-04-26T23:01:30.663 INFO:tasks.workunit.client.0.smithi027.stdout:4/515: mknod da/c9e 0 2024-04-26T23:01:30.663 INFO:tasks.workunit.client.0.smithi027.stdout:4/516: readlink da/l63 0 2024-04-26T23:01:30.664 INFO:tasks.workunit.client.0.smithi027.stdout:4/517: symlink da/dd/de/d14/d21/d67/d8a/l9f 0 2024-04-26T23:01:30.669 INFO:tasks.workunit.client.0.smithi027.stdout:7/448: sync 2024-04-26T23:01:30.670 INFO:tasks.workunit.client.0.smithi027.stdout:7/449: link d0/d28/d2b/d55/d59/f7b d0/d6c/f8c 0 2024-04-26T23:01:30.672 INFO:tasks.workunit.client.0.smithi027.stdout:7/450: mkdir d0/d28/d2b/d3a/d48/d56/d8d 0 2024-04-26T23:01:30.672 INFO:tasks.workunit.client.0.smithi027.stdout:7/451: fdatasync d0/d28/d66/f70 0 2024-04-26T23:01:30.672 INFO:tasks.workunit.client.0.smithi027.stdout:7/452: dread - d0/d28/d2b/d3a/f52 zero size 2024-04-26T23:01:30.679 INFO:tasks.workunit.client.0.smithi027.stdout:0/465: dread d0/f37 [0,4194304] 0 2024-04-26T23:01:30.690 INFO:tasks.workunit.client.0.smithi027.stdout:7/453: rename d0/l7 to d0/d28/d2b/d3a/d48/d56/d88/d73/l8e 0 2024-04-26T23:01:30.690 INFO:tasks.workunit.client.0.smithi027.stdout:2/414: dwrite d2/f6f [0,4194304] 0 2024-04-26T23:01:30.693 INFO:tasks.workunit.client.0.smithi027.stdout:2/415: symlink d2/d5/db/de/d21/d26/l71 0 2024-04-26T23:01:30.693 INFO:tasks.workunit.client.0.smithi027.stdout:0/466: creat d0/d4/da/d15/f8e x:0 0 0 2024-04-26T23:01:30.693 INFO:tasks.workunit.client.0.smithi027.stdout:0/467: readlink d0/d4/da/d15/l29 0 2024-04-26T23:01:30.693 INFO:tasks.workunit.client.0.smithi027.stdout:0/468: fsync d0/df/d3f/d64/f71 0 2024-04-26T23:01:30.693 INFO:tasks.workunit.client.0.smithi027.stdout:0/469: write d0/df/f72 [248800,12826] 0 2024-04-26T23:01:30.693 INFO:tasks.workunit.client.0.smithi027.stdout:0/470: fdatasync d0/d4/da/d1a/d1e/f84 0 2024-04-26T23:01:30.694 INFO:tasks.workunit.client.0.smithi027.stdout:0/471: mknod d0/d4/da/d1a/d59/c8f 0 2024-04-26T23:01:30.696 INFO:tasks.workunit.client.0.smithi027.stdout:1/535: dwrite d1/d5/d21/f44 [0,4194304] 0 2024-04-26T23:01:30.718 INFO:tasks.workunit.client.0.smithi027.stdout:6/373: dwrite d1/df/d43/f55 [0,4194304] 0 2024-04-26T23:01:30.718 INFO:tasks.workunit.client.0.smithi027.stdout:6/374: chown d1/de/c2c 1 1 2024-04-26T23:01:30.718 INFO:tasks.workunit.client.0.smithi027.stdout:6/375: write d1/df/d49/f86 [972752,78613] 0 2024-04-26T23:01:30.721 INFO:tasks.workunit.client.0.smithi027.stdout:5/489: dwrite d4/db/f10 [0,4194304] 0 2024-04-26T23:01:30.736 INFO:tasks.workunit.client.0.smithi027.stdout:7/454: dwrite d0/d28/d2b/d55/d59/f7b [0,4194304] 0 2024-04-26T23:01:30.739 INFO:tasks.workunit.client.0.smithi027.stdout:0/472: dwrite d0/d4/da/d15/f5a [0,4194304] 0 2024-04-26T23:01:30.739 INFO:tasks.workunit.client.0.smithi027.stdout:0/473: write d0/d4/da/d1a/d1e/f8c [2777924,121423] 0 2024-04-26T23:01:30.759 INFO:tasks.workunit.client.0.smithi027.stdout:7/455: dread d0/d28/d2b/d3a/d48/d56/d88/f39 [0,4194304] 0 2024-04-26T23:01:30.759 INFO:tasks.workunit.client.0.smithi027.stdout:6/376: dread d1/df/d43/d56/f77 [0,4194304] 0 2024-04-26T23:01:30.759 INFO:tasks.workunit.client.0.smithi027.stdout:7/456: write d0/d28/f77 [2754631,107344] 0 2024-04-26T23:01:30.760 INFO:tasks.workunit.client.0.smithi027.stdout:6/377: mknod d1/df/d49/d7f/c88 0 2024-04-26T23:01:30.762 INFO:tasks.workunit.client.0.smithi027.stdout:7/457: creat d0/d28/d66/d67/f8f x:0 0 0 2024-04-26T23:01:30.763 INFO:tasks.workunit.client.0.smithi027.stdout:1/536: dwrite d1/d26/d6b/fa0 [0,4194304] 0 2024-04-26T23:01:30.765 INFO:tasks.workunit.client.0.smithi027.stdout:7/458: mkdir d0/d28/d2b/d3a/d90 0 2024-04-26T23:01:30.779 INFO:tasks.workunit.client.0.smithi027.stdout:5/490: dwrite d4/d24/f4c [0,4194304] 0 2024-04-26T23:01:30.780 INFO:tasks.workunit.client.0.smithi027.stdout:5/491: symlink d4/d1a/d4a/d68/d40/l9d 0 2024-04-26T23:01:30.781 INFO:tasks.workunit.client.0.smithi027.stdout:6/378: dread d1/df/d43/d56/d6b/f76 [0,4194304] 0 2024-04-26T23:01:30.782 INFO:tasks.workunit.client.0.smithi027.stdout:5/492: symlink d4/d24/d22/d29/d39/d84/l9e 0 2024-04-26T23:01:30.782 INFO:tasks.workunit.client.0.smithi027.stdout:5/493: stat d4/d24/d22/d29 0 2024-04-26T23:01:30.782 INFO:tasks.workunit.client.0.smithi027.stdout:5/494: chown d4/db/l44 6996568 1 2024-04-26T23:01:30.782 INFO:tasks.workunit.client.0.smithi027.stdout:5/495: dread - d4/d24/f5d zero size 2024-04-26T23:01:30.783 INFO:tasks.workunit.client.0.smithi027.stdout:5/496: unlink d4/c3a 0 2024-04-26T23:01:30.795 INFO:tasks.workunit.client.0.smithi027.stdout:3/497: sync 2024-04-26T23:01:30.796 INFO:tasks.workunit.client.0.smithi027.stdout:0/474: dwrite d0/d4/da/d1a/d1e/f7f [0,4194304] 0 2024-04-26T23:01:30.812 INFO:tasks.workunit.client.0.smithi027.stdout:0/475: dread d0/d4/da/d1a/d1e/f7f [0,4194304] 0 2024-04-26T23:01:30.813 INFO:tasks.workunit.client.0.smithi027.stdout:0/476: readlink d0/d4/da/d1a/l5f 0 2024-04-26T23:01:30.814 INFO:tasks.workunit.client.0.smithi027.stdout:0/477: creat d0/d4/da/d1a/f90 x:0 0 0 2024-04-26T23:01:30.817 INFO:tasks.workunit.client.0.smithi027.stdout:3/498: rename d6/dc/d2f/d6b to d6/dc/d34/d2e/d9d/da7/dac 0 2024-04-26T23:01:30.818 INFO:tasks.workunit.client.0.smithi027.stdout:3/499: getdents d6/d5e 0 2024-04-26T23:01:30.818 INFO:tasks.workunit.client.0.smithi027.stdout:3/500: getdents d6/dc/d34/d47/d7b/d98 0 2024-04-26T23:01:30.829 INFO:tasks.workunit.client.0.smithi027.stdout:6/379: dwrite d1/df/f8 [0,4194304] 0 2024-04-26T23:01:30.831 INFO:tasks.workunit.client.0.smithi027.stdout:6/380: creat d1/d30/d6c/d6f/f89 x:0 0 0 2024-04-26T23:01:30.833 INFO:tasks.workunit.client.0.smithi027.stdout:6/381: creat d1/df/d3c/d66/f8a x:0 0 0 2024-04-26T23:01:30.834 INFO:tasks.workunit.client.0.smithi027.stdout:6/382: creat d1/df/d43/d56/d6b/f8b x:0 0 0 2024-04-26T23:01:30.834 INFO:tasks.workunit.client.0.smithi027.stdout:6/383: read - d1/de/f42 zero size 2024-04-26T23:01:30.843 INFO:tasks.workunit.client.0.smithi027.stdout:0/478: dwrite d0/d4/da/d1a/f3b [8388608,4194304] 0 2024-04-26T23:01:30.845 INFO:tasks.workunit.client.0.smithi027.stdout:0/479: rmdir d0/d4/da/d1a/d6d 39 2024-04-26T23:01:30.846 INFO:tasks.workunit.client.0.smithi027.stdout:0/480: unlink d0/d4/da/d15/d55/f58 0 2024-04-26T23:01:30.848 INFO:tasks.workunit.client.0.smithi027.stdout:0/481: creat d0/d4/da/d1a/d1e/f91 x:0 0 0 2024-04-26T23:01:30.849 INFO:tasks.workunit.client.0.smithi027.stdout:0/482: mkdir d0/d4/da/d1a/d59/d92 0 2024-04-26T23:01:30.849 INFO:tasks.workunit.client.0.smithi027.stdout:0/483: fsync d0/df/d3f/f61 0 2024-04-26T23:01:30.851 INFO:tasks.workunit.client.0.smithi027.stdout:0/484: creat d0/d4/da/d15/f93 x:0 0 0 2024-04-26T23:01:30.868 INFO:tasks.workunit.client.0.smithi027.stdout:0/485: dread d0/d4/da/d15/f48 [0,4194304] 0 2024-04-26T23:01:30.868 INFO:tasks.workunit.client.0.smithi027.stdout:0/486: chown d0/d4/da/d1a/d59/d92 24 1 2024-04-26T23:01:30.868 INFO:tasks.workunit.client.0.smithi027.stdout:0/487: chown d0/d4/da/d15/d26/f8a 20775007 1 2024-04-26T23:01:30.870 INFO:tasks.workunit.client.0.smithi027.stdout:0/488: creat d0/d4/da/d1a/d59/d92/f94 x:0 0 0 2024-04-26T23:01:30.872 INFO:tasks.workunit.client.0.smithi027.stdout:0/489: link d0/d4/c6e d0/d4/da/c95 0 2024-04-26T23:01:30.874 INFO:tasks.workunit.client.0.smithi027.stdout:3/501: dwrite d6/d5e/fa0 [0,4194304] 0 2024-04-26T23:01:30.874 INFO:tasks.workunit.client.0.smithi027.stdout:3/502: dread - d6/dc/d34/f5d zero size 2024-04-26T23:01:30.878 INFO:tasks.workunit.client.0.smithi027.stdout:3/503: creat d6/dc/d34/d2e/d9d/fad x:0 0 0 2024-04-26T23:01:30.883 INFO:tasks.workunit.client.0.smithi027.stdout:3/504: rename d6/dc/d34/d2e/d9d/da7/dac/d74/d9e to d6/dc/d34/d2e/d70/d71/dae 0 2024-04-26T23:01:30.887 INFO:tasks.workunit.client.0.smithi027.stdout:3/505: rename d6/dc/d34/d2e/d9d/da7/dac/d74/da6/l4e to d6/dc/d34/d2e/d9d/da7/dac/laf 0 2024-04-26T23:01:30.889 INFO:tasks.workunit.client.0.smithi027.stdout:3/506: unlink d6/dc/d34/d78/l81 0 2024-04-26T23:01:30.891 INFO:tasks.workunit.client.0.smithi027.stdout:3/507: creat d6/dc/d34/d2e/d5f/d90/fb0 x:0 0 0 2024-04-26T23:01:30.893 INFO:tasks.workunit.client.0.smithi027.stdout:3/508: link d6/dc/d34/d47/l43 d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d59/lb1 0 2024-04-26T23:01:30.895 INFO:tasks.workunit.client.0.smithi027.stdout:6/384: dwrite d1/de/f48 [0,4194304] 0 2024-04-26T23:01:30.896 INFO:tasks.workunit.client.0.smithi027.stdout:6/385: chown d1/d30/l37 42295545 1 2024-04-26T23:01:30.898 INFO:tasks.workunit.client.0.smithi027.stdout:6/386: getdents d1/d30 0 2024-04-26T23:01:30.902 INFO:tasks.workunit.client.0.smithi027.stdout:6/387: unlink d1/df/f58 0 2024-04-26T23:01:30.905 INFO:tasks.workunit.client.0.smithi027.stdout:6/388: unlink d1/f19 0 2024-04-26T23:01:30.905 INFO:tasks.workunit.client.0.smithi027.stdout:6/389: truncate d1/df/f79 971561 0 2024-04-26T23:01:30.905 INFO:tasks.workunit.client.0.smithi027.stdout:6/390: dread - d1/d30/d6c/d6f/f89 zero size 2024-04-26T23:01:30.906 INFO:tasks.workunit.client.0.smithi027.stdout:6/391: unlink d1/df/f68 0 2024-04-26T23:01:30.908 INFO:tasks.workunit.client.0.smithi027.stdout:6/392: mknod d1/de/c8c 0 2024-04-26T23:01:30.908 INFO:tasks.workunit.client.0.smithi027.stdout:6/393: fdatasync d1/df/f5f 0 2024-04-26T23:01:30.908 INFO:tasks.workunit.client.0.smithi027.stdout:0/490: dwrite d0/d4/da/d15/d26/f7c [0,4194304] 0 2024-04-26T23:01:30.908 INFO:tasks.workunit.client.0.smithi027.stdout:0/491: fdatasync d0/f12 0 2024-04-26T23:01:30.908 INFO:tasks.workunit.client.0.smithi027.stdout:0/492: write d0/d4/f7 [401654,34796] 0 2024-04-26T23:01:30.909 INFO:tasks.workunit.client.0.smithi027.stdout:6/394: creat d1/df/d3c/d66/f8d x:0 0 0 2024-04-26T23:01:30.910 INFO:tasks.workunit.client.0.smithi027.stdout:0/493: creat d0/d4/da/d15/d55/f96 x:0 0 0 2024-04-26T23:01:30.910 INFO:tasks.workunit.client.0.smithi027.stdout:0/494: stat d0/d4/c4f 0 2024-04-26T23:01:30.914 INFO:tasks.workunit.client.0.smithi027.stdout:6/395: mkdir d1/de/d57/d8e 0 2024-04-26T23:01:30.914 INFO:tasks.workunit.client.0.smithi027.stdout:6/396: mknod d1/df/d43/d56/d6b/c8f 0 2024-04-26T23:01:30.914 INFO:tasks.workunit.client.0.smithi027.stdout:3/509: write d6/dc/d2f/f39 [670564,34896] 0 2024-04-26T23:01:30.915 INFO:tasks.workunit.client.0.smithi027.stdout:8/450: sync 2024-04-26T23:01:30.915 INFO:tasks.workunit.client.0.smithi027.stdout:9/436: sync 2024-04-26T23:01:30.916 INFO:tasks.workunit.client.0.smithi027.stdout:0/495: dread d0/df/d3f/d64/f5e [0,4194304] 0 2024-04-26T23:01:30.918 INFO:tasks.workunit.client.0.smithi027.stdout:8/451: getdents d9/da/d2a/d5e/d72/d84/d5d/d6c 0 2024-04-26T23:01:30.919 INFO:tasks.workunit.client.0.smithi027.stdout:9/437: rmdir d4 39 2024-04-26T23:01:30.921 INFO:tasks.workunit.client.0.smithi027.stdout:9/438: mknod d4/d77/c97 0 2024-04-26T23:01:30.922 INFO:tasks.workunit.client.0.smithi027.stdout:9/439: readlink d4/l1c 0 2024-04-26T23:01:30.924 INFO:tasks.workunit.client.0.smithi027.stdout:9/440: creat d4/d17/d2e/d70/f98 x:0 0 0 2024-04-26T23:01:30.924 INFO:tasks.workunit.client.0.smithi027.stdout:9/441: stat d4/d10/d40/d58/d75/d84/d86/f94 0 2024-04-26T23:01:30.927 INFO:tasks.workunit.client.0.smithi027.stdout:3/510: creat d6/dc/d34/d2e/d50/fb2 x:0 0 0 2024-04-26T23:01:30.927 INFO:tasks.workunit.client.0.smithi027.stdout:6/397: dread d1/de/f1b [0,4194304] 0 2024-04-26T23:01:30.935 INFO:tasks.workunit.client.0.smithi027.stdout:3/511: dread d6/f18 [0,4194304] 0 2024-04-26T23:01:30.935 INFO:tasks.workunit.client.0.smithi027.stdout:3/512: chown d6/dc 6617 1 2024-04-26T23:01:30.937 INFO:tasks.workunit.client.0.smithi027.stdout:2/416: sync 2024-04-26T23:01:30.937 INFO:tasks.workunit.client.0.smithi027.stdout:4/518: sync 2024-04-26T23:01:30.937 INFO:tasks.workunit.client.0.smithi027.stdout:3/513: mkdir d6/dc/d34/db3 0 2024-04-26T23:01:30.938 INFO:tasks.workunit.client.0.smithi027.stdout:2/417: dread - d2/d5/db/d1b/d1c/d25/d19/d23/f64 zero size 2024-04-26T23:01:30.938 INFO:tasks.workunit.client.0.smithi027.stdout:2/418: dread - d2/d5/db/f6c zero size 2024-04-26T23:01:30.938 INFO:tasks.workunit.client.0.smithi027.stdout:4/519: read da/dd/de/d14/d21/d67/d8a/f8c [32692,37824] 0 2024-04-26T23:01:30.939 INFO:tasks.workunit.client.0.smithi027.stdout:3/514: mkdir d6/dc/d34/db3/db4 0 2024-04-26T23:01:30.939 INFO:tasks.workunit.client.0.smithi027.stdout:3/515: truncate d6/dc/d34/d2e/d5f/d90/fa9 455534 0 2024-04-26T23:01:30.939 INFO:tasks.workunit.client.0.smithi027.stdout:2/419: unlink d2/d5/db/f1a 0 2024-04-26T23:01:30.939 INFO:tasks.workunit.client.0.smithi027.stdout:4/520: rmdir da/dd/d92 39 2024-04-26T23:01:30.939 INFO:tasks.workunit.client.0.smithi027.stdout:4/521: stat da/dd/de/d14/d21/d4a/f6d 0 2024-04-26T23:01:30.963 INFO:tasks.workunit.client.0.smithi027.stdout:4/522: write da/dd/de/d14/d17/f33 [370895,110636] 0 2024-04-26T23:01:30.964 INFO:tasks.workunit.client.0.smithi027.stdout:8/452: dwrite d9/da/d33/f88 [0,4194304] 0 2024-04-26T23:01:30.969 INFO:tasks.workunit.client.0.smithi027.stdout:9/442: dwrite d4/d17/d2e/f95 [0,4194304] 0 2024-04-26T23:01:30.969 INFO:tasks.workunit.client.0.smithi027.stdout:9/443: truncate d4/d17/d2e/f53 4357760 0 2024-04-26T23:01:30.969 INFO:tasks.workunit.client.0.smithi027.stdout:0/496: dwrite d0/df/d3f/d64/f31 [0,4194304] 0 2024-04-26T23:01:30.973 INFO:tasks.workunit.client.0.smithi027.stdout:8/453: creat d9/de/fa5 x:0 0 0 2024-04-26T23:01:30.976 INFO:tasks.workunit.client.0.smithi027.stdout:8/454: unlink d9/da/d2a/c73 0 2024-04-26T23:01:30.978 INFO:tasks.workunit.client.0.smithi027.stdout:6/398: dread d1/de/f20 [0,4194304] 0 2024-04-26T23:01:30.983 INFO:tasks.workunit.client.0.smithi027.stdout:9/444: write d4/d17/d2e/f53 [2943266,122191] 0 2024-04-26T23:01:30.989 INFO:tasks.workunit.client.0.smithi027.stdout:6/399: unlink d1/d30/d40/f63 0 2024-04-26T23:01:30.989 INFO:tasks.workunit.client.0.smithi027.stdout:8/455: mkdir d9/da6 0 2024-04-26T23:01:30.990 INFO:tasks.workunit.client.0.smithi027.stdout:9/445: mknod d4/d17/d2e/c99 0 2024-04-26T23:01:30.991 INFO:tasks.workunit.client.0.smithi027.stdout:6/400: mknod d1/c90 0 2024-04-26T23:01:30.991 INFO:tasks.workunit.client.0.smithi027.stdout:6/401: fdatasync d1/de/f1b 0 2024-04-26T23:01:30.991 INFO:tasks.workunit.client.0.smithi027.stdout:6/402: fdatasync d1/d30/d6c/d6f/f89 0 2024-04-26T23:01:30.991 INFO:tasks.workunit.client.0.smithi027.stdout:6/403: stat d1/df/d43/d56 0 2024-04-26T23:01:30.991 INFO:tasks.workunit.client.0.smithi027.stdout:9/446: creat d4/d10/d40/f9a x:0 0 0 2024-04-26T23:01:30.992 INFO:tasks.workunit.client.0.smithi027.stdout:8/456: rename d9/da/d2a/d5e/d9e/da4 to d9/da/d2a/d5e/d72/d84/d35/da7 0 2024-04-26T23:01:30.995 INFO:tasks.workunit.client.0.smithi027.stdout:8/457: truncate d9/da/d2a/d5e/f64 3577511 0 2024-04-26T23:01:30.996 INFO:tasks.workunit.client.0.smithi027.stdout:8/458: creat d9/de/d2e/fa8 x:0 0 0 2024-04-26T23:01:30.999 INFO:tasks.workunit.client.0.smithi027.stdout:8/459: creat d9/da/d2a/d5e/d72/d84/d5d/d6c/fa9 x:0 0 0 2024-04-26T23:01:30.999 INFO:tasks.workunit.client.0.smithi027.stdout:0/497: dread d0/d4/da/d1a/f40 [0,4194304] 0 2024-04-26T23:01:31.003 INFO:tasks.workunit.client.0.smithi027.stdout:8/460: symlink d9/da/d2a/d5e/d72/d84/d4f/laa 0 2024-04-26T23:01:31.004 INFO:tasks.workunit.client.0.smithi027.stdout:4/523: dwrite da/dd/f56 [0,4194304] 0 2024-04-26T23:01:31.005 INFO:tasks.workunit.client.0.smithi027.stdout:4/524: write da/dd/d5f/f8b [189372,106659] 0 2024-04-26T23:01:31.005 INFO:tasks.workunit.client.0.smithi027.stdout:0/498: rmdir d0/d4/da/d15 39 2024-04-26T23:01:31.005 INFO:tasks.workunit.client.0.smithi027.stdout:8/461: creat d9/da/d2a/d6b/fab x:0 0 0 2024-04-26T23:01:31.010 INFO:tasks.workunit.client.0.smithi027.stdout:0/499: link d0/d4/da/d1a/f89 d0/d4/da/d15/d55/f97 0 2024-04-26T23:01:31.010 INFO:tasks.workunit.client.0.smithi027.stdout:0/500: read d0/d4/da/f2f [64145,73085] 0 2024-04-26T23:01:31.010 INFO:tasks.workunit.client.0.smithi027.stdout:0/501: dread - d0/df/d3f/d64/f71 zero size 2024-04-26T23:01:31.010 INFO:tasks.workunit.client.0.smithi027.stdout:0/502: stat d0/d4/f3d 0 2024-04-26T23:01:31.016 INFO:tasks.workunit.client.0.smithi027.stdout:7/459: sync 2024-04-26T23:01:31.017 INFO:tasks.workunit.client.0.smithi027.stdout:1/537: sync 2024-04-26T23:01:31.017 INFO:tasks.workunit.client.0.smithi027.stdout:5/497: sync 2024-04-26T23:01:31.017 INFO:tasks.workunit.client.0.smithi027.stdout:7/460: creat d0/d28/d2b/d3a/d48/d56/d88/f91 x:0 0 0 2024-04-26T23:01:31.017 INFO:tasks.workunit.client.0.smithi027.stdout:7/461: dread - d0/d28/d2b/d55/f6f zero size 2024-04-26T23:01:31.017 INFO:tasks.workunit.client.0.smithi027.stdout:7/462: fdatasync d0/f62 0 2024-04-26T23:01:31.017 INFO:tasks.workunit.client.0.smithi027.stdout:5/498: mkdir d4/d9f 0 2024-04-26T23:01:31.017 INFO:tasks.workunit.client.0.smithi027.stdout:7/463: mknod d0/d28/d2b/d3a/d90/c92 0 2024-04-26T23:01:31.019 INFO:tasks.workunit.client.0.smithi027.stdout:6/404: write d1/de/f28 [2418261,2553] 0 2024-04-26T23:01:31.020 INFO:tasks.workunit.client.0.smithi027.stdout:6/405: chown d1/f2b 82937070 1 2024-04-26T23:01:31.020 INFO:tasks.workunit.client.0.smithi027.stdout:6/406: truncate d1/f69 931032 0 2024-04-26T23:01:31.020 INFO:tasks.workunit.client.0.smithi027.stdout:1/538: dread d1/d26/d7d/f30 [0,4194304] 0 2024-04-26T23:01:31.026 INFO:tasks.workunit.client.0.smithi027.stdout:7/464: rmdir d0/d28/d2b/d55/d69/d75/d7a 0 2024-04-26T23:01:31.026 INFO:tasks.workunit.client.0.smithi027.stdout:5/499: symlink d4/d24/d22/d29/d2a/la0 0 2024-04-26T23:01:31.027 INFO:tasks.workunit.client.0.smithi027.stdout:0/503: dread d0/d4/da/f11 [0,4194304] 0 2024-04-26T23:01:31.028 INFO:tasks.workunit.client.0.smithi027.stdout:6/407: unlink d1/f33 0 2024-04-26T23:01:31.029 INFO:tasks.workunit.client.0.smithi027.stdout:7/465: creat d0/d6c/f93 x:0 0 0 2024-04-26T23:01:31.030 INFO:tasks.workunit.client.0.smithi027.stdout:8/462: dread d9/de/d22/f3e [0,4194304] 0 2024-04-26T23:01:31.030 INFO:tasks.workunit.client.0.smithi027.stdout:8/463: write d9/de/d2e/f1f [122854,29675] 0 2024-04-26T23:01:31.030 INFO:tasks.workunit.client.0.smithi027.stdout:6/408: symlink d1/de/d57/d8e/l91 0 2024-04-26T23:01:31.030 INFO:tasks.workunit.client.0.smithi027.stdout:6/409: write d1/d30/d6c/d6f/f89 [27923,24856] 0 2024-04-26T23:01:31.034 INFO:tasks.workunit.client.0.smithi027.stdout:6/410: creat d1/df/d49/d7f/f92 x:0 0 0 2024-04-26T23:01:31.049 INFO:tasks.workunit.client.0.smithi027.stdout:5/500: dread d4/d24/f42 [0,4194304] 0 2024-04-26T23:01:31.057 INFO:tasks.workunit.client.0.smithi027.stdout:6/411: write d1/df/f3a [1954380,23814] 0 2024-04-26T23:01:31.057 INFO:tasks.workunit.client.0.smithi027.stdout:6/412: chown d1/df/d3c/d66 30 1 2024-04-26T23:01:31.059 INFO:tasks.workunit.client.0.smithi027.stdout:7/466: write d0/d28/d66/f70 [4033268,111012] 0 2024-04-26T23:01:31.060 INFO:tasks.workunit.client.0.smithi027.stdout:7/467: truncate d0/d28/f40 862521 0 2024-04-26T23:01:31.062 INFO:tasks.workunit.client.0.smithi027.stdout:7/468: creat d0/d28/d2b/d3a/d90/f94 x:0 0 0 2024-04-26T23:01:31.062 INFO:tasks.workunit.client.0.smithi027.stdout:7/469: write d0/d28/d2b/d3a/d48/f72 [4545701,88032] 0 2024-04-26T23:01:31.062 INFO:tasks.workunit.client.0.smithi027.stdout:7/470: write d0/d2f/f84 [257797,36109] 0 2024-04-26T23:01:31.066 INFO:tasks.workunit.client.0.smithi027.stdout:7/471: rename d0/d28/f6e to d0/d28/d2b/d55/f95 0 2024-04-26T23:01:31.067 INFO:tasks.workunit.client.0.smithi027.stdout:7/472: truncate d0/d28/d2b/d55/f6f 199709 0 2024-04-26T23:01:31.084 INFO:tasks.workunit.client.0.smithi027.stdout:5/501: dwrite d4/d24/d22/d29/d39/f7f [0,4194304] 0 2024-04-26T23:01:31.084 INFO:tasks.workunit.client.0.smithi027.stdout:5/502: truncate d4/f64 4569608 0 2024-04-26T23:01:31.088 INFO:tasks.workunit.client.0.smithi027.stdout:8/464: dwrite d9/de/d2e/fa8 [0,4194304] 0 2024-04-26T23:01:31.090 INFO:tasks.workunit.client.0.smithi027.stdout:8/465: mknod d9/da/d2a/d5e/d72/d84/d4f/cac 0 2024-04-26T23:01:31.096 INFO:tasks.workunit.client.0.smithi027.stdout:8/466: unlink d9/da/d2a/d5e/d2b/fa0 0 2024-04-26T23:01:31.098 INFO:tasks.workunit.client.0.smithi027.stdout:8/467: mknod d9/da/d2a/d5e/d92/cad 0 2024-04-26T23:01:31.102 INFO:tasks.workunit.client.0.smithi027.stdout:8/468: unlink d9/da/fd 0 2024-04-26T23:01:31.102 INFO:tasks.workunit.client.0.smithi027.stdout:6/413: dwrite d1/d30/d6c/d6f/d7d/f80 [0,4194304] 0 2024-04-26T23:01:31.105 INFO:tasks.workunit.client.0.smithi027.stdout:5/503: dread d4/db/f10 [0,4194304] 0 2024-04-26T23:01:31.105 INFO:tasks.workunit.client.0.smithi027.stdout:5/504: stat d4/l21 0 2024-04-26T23:01:31.107 INFO:tasks.workunit.client.0.smithi027.stdout:5/505: creat d4/d24/d22/d29/d2a/fa1 x:0 0 0 2024-04-26T23:01:31.107 INFO:tasks.workunit.client.0.smithi027.stdout:5/506: stat d4/d24/d22/d29/d2a/fa1 0 2024-04-26T23:01:31.107 INFO:tasks.workunit.client.0.smithi027.stdout:5/507: chown d4/db/d16 15 1 2024-04-26T23:01:31.107 INFO:tasks.workunit.client.0.smithi027.stdout:5/508: stat d4/d24/c41 0 2024-04-26T23:01:31.108 INFO:tasks.workunit.client.0.smithi027.stdout:5/509: link d4/f64 d4/d24/d22/d29/d2a/d3b/fa2 0 2024-04-26T23:01:31.109 INFO:tasks.workunit.client.0.smithi027.stdout:5/510: stat d4/d24/d22/d29/d2a/l72 0 2024-04-26T23:01:31.124 INFO:tasks.workunit.client.0.smithi027.stdout:5/511: getdents d4/d24/d22/d29/d2a/d3b 0 2024-04-26T23:01:31.124 INFO:tasks.workunit.client.0.smithi027.stdout:7/473: dwrite d0/d28/d2b/d3a/d48/d56/d88/f3c [0,4194304] 0 2024-04-26T23:01:31.127 INFO:tasks.workunit.client.0.smithi027.stdout:8/469: dread d9/de/d2e/f1f [0,4194304] 0 2024-04-26T23:01:31.129 INFO:tasks.workunit.client.0.smithi027.stdout:8/470: creat d9/da/d2a/d5e/d92/fae x:0 0 0 2024-04-26T23:01:31.141 INFO:tasks.workunit.client.0.smithi027.stdout:7/474: write d0/d28/d2b/f4e [2743333,107982] 0 2024-04-26T23:01:31.141 INFO:tasks.workunit.client.0.smithi027.stdout:7/475: dread - d0/d28/d2b/d55/f95 zero size 2024-04-26T23:01:31.141 INFO:tasks.workunit.client.0.smithi027.stdout:7/476: truncate d0/d28/d2b/d3a/d48/f72 2141506 0 2024-04-26T23:01:31.143 INFO:tasks.workunit.client.0.smithi027.stdout:7/477: creat d0/d28/d2b/d55/d59/d6d/d82/f96 x:0 0 0 2024-04-26T23:01:31.144 INFO:tasks.workunit.client.0.smithi027.stdout:7/478: mknod d0/d28/d2b/d55/d69/c97 0 2024-04-26T23:01:31.144 INFO:tasks.workunit.client.0.smithi027.stdout:7/479: truncate d0/d3d/f43 808046 0 2024-04-26T23:01:31.160 INFO:tasks.workunit.client.0.smithi027.stdout:5/512: dread d4/d1a/d4a/d68/d40/d54/f75 [0,4194304] 0 2024-04-26T23:01:31.162 INFO:tasks.workunit.client.0.smithi027.stdout:5/513: creat d4/d24/d22/fa3 x:0 0 0 2024-04-26T23:01:31.164 INFO:tasks.workunit.client.0.smithi027.stdout:6/414: dwrite d1/de/f1b [0,4194304] 0 2024-04-26T23:01:31.164 INFO:tasks.workunit.client.0.smithi027.stdout:6/415: chown d1/df/d49 32271999 1 2024-04-26T23:01:31.168 INFO:tasks.workunit.client.0.smithi027.stdout:8/471: write d9/f16 [1322912,34155] 0 2024-04-26T23:01:31.168 INFO:tasks.workunit.client.0.smithi027.stdout:8/472: dread - d9/da/d2a/d5e/d72/f7e zero size 2024-04-26T23:01:31.171 INFO:tasks.workunit.client.0.smithi027.stdout:8/473: rmdir d9/da/d2a/d5e/d2b 39 2024-04-26T23:01:31.171 INFO:tasks.workunit.client.0.smithi027.stdout:8/474: stat d9/de/f14 0 2024-04-26T23:01:31.172 INFO:tasks.workunit.client.0.smithi027.stdout:8/475: creat d9/da/da1/faf x:0 0 0 2024-04-26T23:01:31.186 INFO:tasks.workunit.client.0.smithi027.stdout:5/514: write d4/f9 [4630254,56296] 0 2024-04-26T23:01:31.188 INFO:tasks.workunit.client.0.smithi027.stdout:5/515: symlink d4/d9f/la4 0 2024-04-26T23:01:31.189 INFO:tasks.workunit.client.0.smithi027.stdout:7/480: dwrite d0/d28/d2b/d3a/d48/d56/d88/f39 [4194304,4194304] 0 2024-04-26T23:01:31.189 INFO:tasks.workunit.client.0.smithi027.stdout:8/476: dread d9/da/d2a/d5e/d2b/f76 [0,4194304] 0 2024-04-26T23:01:31.190 INFO:tasks.workunit.client.0.smithi027.stdout:8/477: dread - d9/da/d2a/d5e/d72/d84/d5d/d6c/fa9 zero size 2024-04-26T23:01:31.190 INFO:tasks.workunit.client.0.smithi027.stdout:7/481: read d0/f11 [3659015,42641] 0 2024-04-26T23:01:31.190 INFO:tasks.workunit.client.0.smithi027.stdout:5/516: creat d4/fa5 x:0 0 0 2024-04-26T23:01:31.192 INFO:tasks.workunit.client.0.smithi027.stdout:7/482: mknod d0/d28/d2b/d55/d59/d6d/c98 0 2024-04-26T23:01:31.192 INFO:tasks.workunit.client.0.smithi027.stdout:7/483: chown d0/d28/d66/d67 2 1 2024-04-26T23:01:31.192 INFO:tasks.workunit.client.0.smithi027.stdout:7/484: dread - d0/f62 zero size 2024-04-26T23:01:31.193 INFO:tasks.workunit.client.0.smithi027.stdout:5/517: rename d4/d24/d22/d29/d39/d84/f92 to d4/d1a/d4a/d68/d40/d54/fa6 0 2024-04-26T23:01:31.194 INFO:tasks.workunit.client.0.smithi027.stdout:7/485: mknod d0/d28/d2b/d55/d69/d75/c99 0 2024-04-26T23:01:31.198 INFO:tasks.workunit.client.0.smithi027.stdout:5/518: mknod d4/d24/ca7 0 2024-04-26T23:01:31.198 INFO:tasks.workunit.client.0.smithi027.stdout:5/519: chown d4/d1a/d4a/d68/d40/d54/l8c 804089 1 2024-04-26T23:01:31.200 INFO:tasks.workunit.client.0.smithi027.stdout:6/416: dwrite d1/df/d43/d56/f77 [0,4194304] 0 2024-04-26T23:01:31.202 INFO:tasks.workunit.client.0.smithi027.stdout:7/486: dread d0/d28/f18 [4194304,4194304] 0 2024-04-26T23:01:31.202 INFO:tasks.workunit.client.0.smithi027.stdout:7/487: truncate d0/d6c/f93 517787 0 2024-04-26T23:01:31.203 INFO:tasks.workunit.client.0.smithi027.stdout:5/520: link d4/d1a/d4a/d68/d40/d54/fa6 d4/d24/d22/d29/d39/fa8 0 2024-04-26T23:01:31.203 INFO:tasks.workunit.client.0.smithi027.stdout:5/521: write d4/d24/d22/d29/f58 [173866,16338] 0 2024-04-26T23:01:31.204 INFO:tasks.workunit.client.0.smithi027.stdout:7/488: creat d0/d28/d2b/d3a/d48/d56/f9a x:0 0 0 2024-04-26T23:01:31.205 INFO:tasks.workunit.client.0.smithi027.stdout:5/522: symlink d4/d24/d22/d29/d2a/la9 0 2024-04-26T23:01:31.206 INFO:tasks.workunit.client.0.smithi027.stdout:5/523: mkdir d4/d1a/d2b/daa 0 2024-04-26T23:01:31.214 INFO:tasks.workunit.client.0.smithi027.stdout:6/417: dread d1/df/fd [0,4194304] 0 2024-04-26T23:01:31.237 INFO:tasks.workunit.client.0.smithi027.stdout:7/489: write d0/d28/d2b/d55/f95 [346049,66921] 0 2024-04-26T23:01:31.242 INFO:tasks.workunit.client.0.smithi027.stdout:7/490: fsync d0/f42 0 2024-04-26T23:01:31.242 INFO:tasks.workunit.client.0.smithi027.stdout:6/418: read d1/df/f2e [2346276,106417] 0 2024-04-26T23:01:31.250 INFO:tasks.workunit.client.0.smithi027.stdout:8/478: dwrite d9/da/d2a/d5e/d72/d84/d35/f56 [0,4194304] 0 2024-04-26T23:01:31.251 INFO:tasks.workunit.client.0.smithi027.stdout:8/479: mknod d9/da/d2a/d5e/d72/d84/d5d/d8b/cb0 0 2024-04-26T23:01:31.251 INFO:tasks.workunit.client.0.smithi027.stdout:8/480: fsync d9/da/d33/f3a 0 2024-04-26T23:01:31.256 INFO:tasks.workunit.client.0.smithi027.stdout:5/524: dwrite d4/d24/d22/d29/d2a/d3b/fa2 [0,4194304] 0 2024-04-26T23:01:31.263 INFO:tasks.workunit.client.0.smithi027.stdout:6/419: dread d1/f64 [0,4194304] 0 2024-04-26T23:01:31.266 INFO:tasks.workunit.client.0.smithi027.stdout:5/525: rename d4/db to d4/d24/d22/dab 0 2024-04-26T23:01:31.266 INFO:tasks.workunit.client.0.smithi027.stdout:6/420: creat d1/df/d43/d56/d6b/f93 x:0 0 0 2024-04-26T23:01:31.266 INFO:tasks.workunit.client.0.smithi027.stdout:6/421: write d1/df/d43/f5d [445817,28556] 0 2024-04-26T23:01:31.267 INFO:tasks.workunit.client.0.smithi027.stdout:6/422: unlink d1/df/d43/d56/f77 0 2024-04-26T23:01:31.268 INFO:tasks.workunit.client.0.smithi027.stdout:6/423: mknod d1/df/d49/d7f/d84/c94 0 2024-04-26T23:01:31.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:30 smithi080 ceph-mon[32324]: pgmap v90: 129 pgs: 129 active+clean; 736 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s rd, 98 MiB/s wr, 1.04k op/s 2024-04-26T23:01:31.280 INFO:tasks.workunit.client.0.smithi027.stdout:7/491: dwrite d0/d28/d66/d67/f81 [0,4194304] 0 2024-04-26T23:01:31.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:30 smithi094 ceph-mon[32889]: pgmap v90: 129 pgs: 129 active+clean; 736 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s rd, 98 MiB/s wr, 1.04k op/s 2024-04-26T23:01:31.311 INFO:tasks.workunit.client.0.smithi027.stdout:6/424: dwrite d1/f64 [0,4194304] 0 2024-04-26T23:01:31.321 INFO:tasks.workunit.client.0.smithi027.stdout:5/526: dwrite d4/d24/d22/d29/f58 [0,4194304] 0 2024-04-26T23:01:31.321 INFO:tasks.workunit.client.0.smithi027.stdout:5/527: chown d4/d24/d22/d29/d39 149537 1 2024-04-26T23:01:31.322 INFO:tasks.workunit.client.0.smithi027.stdout:5/528: fdatasync d4/d24/f5d 0 2024-04-26T23:01:31.322 INFO:tasks.workunit.client.0.smithi027.stdout:5/529: fdatasync d4/d24/f1b 0 2024-04-26T23:01:31.324 INFO:tasks.workunit.client.0.smithi027.stdout:5/530: rename d4/d1a/d4a/d68 to d4/d24/d5f/dac 0 2024-04-26T23:01:31.324 INFO:tasks.workunit.client.0.smithi027.stdout:8/481: dwrite d9/da/d2a/d5e/d72/d84/d4f/f79 [0,4194304] 0 2024-04-26T23:01:31.327 INFO:tasks.workunit.client.0.smithi027.stdout:8/482: truncate d9/da/d2a/d5e/d2b/f3d 692379 0 2024-04-26T23:01:31.329 INFO:tasks.workunit.client.0.smithi027.stdout:8/483: symlink d9/da/d2a/lb1 0 2024-04-26T23:01:31.330 INFO:tasks.workunit.client.0.smithi027.stdout:8/484: creat d9/da/d2a/d5e/d72/d84/d35/d8d/fb2 x:0 0 0 2024-04-26T23:01:31.331 INFO:tasks.workunit.client.0.smithi027.stdout:6/425: dread d1/d30/d6c/f7a [0,4194304] 0 2024-04-26T23:01:31.332 INFO:tasks.workunit.client.0.smithi027.stdout:7/492: dwrite d0/d28/d2b/d55/d59/d6d/d82/f96 [0,4194304] 0 2024-04-26T23:01:31.332 INFO:tasks.workunit.client.0.smithi027.stdout:7/493: readlink d0/d28/l12 0 2024-04-26T23:01:31.332 INFO:tasks.workunit.client.0.smithi027.stdout:7/494: dread - d0/d28/d2b/d55/d59/f6b zero size 2024-04-26T23:01:31.332 INFO:tasks.workunit.client.0.smithi027.stdout:8/485: mknod d9/da/d2a/d5e/d46/cb3 0 2024-04-26T23:01:31.333 INFO:tasks.workunit.client.0.smithi027.stdout:6/426: rename d1/de/l2f to d1/l95 0 2024-04-26T23:01:31.333 INFO:tasks.workunit.client.0.smithi027.stdout:6/427: chown d1/l21 105138970 1 2024-04-26T23:01:31.335 INFO:tasks.workunit.client.0.smithi027.stdout:7/495: rename d0/d28/d2b/d3a/d90/f94 to d0/d28/d2b/f9b 0 2024-04-26T23:01:31.336 INFO:tasks.workunit.client.0.smithi027.stdout:7/496: write d0/f42 [1951072,106151] 0 2024-04-26T23:01:31.336 INFO:tasks.workunit.client.0.smithi027.stdout:7/497: write d0/d28/f77 [4462808,83792] 0 2024-04-26T23:01:31.336 INFO:tasks.workunit.client.0.smithi027.stdout:7/498: fsync d0/f13 0 2024-04-26T23:01:31.339 INFO:tasks.workunit.client.0.smithi027.stdout:7/499: symlink d0/d28/d2b/d3a/d48/d56/d88/d73/l9c 0 2024-04-26T23:01:31.341 INFO:tasks.workunit.client.0.smithi027.stdout:7/500: mknod d0/d28/c9d 0 2024-04-26T23:01:31.341 INFO:tasks.workunit.client.0.smithi027.stdout:7/501: chown d0/d3d/f44 0 1 2024-04-26T23:01:31.341 INFO:tasks.workunit.client.0.smithi027.stdout:7/502: write d0/d28/d2b/d55/f95 [305822,103385] 0 2024-04-26T23:01:31.341 INFO:tasks.workunit.client.0.smithi027.stdout:3/516: sync 2024-04-26T23:01:31.341 INFO:tasks.workunit.client.0.smithi027.stdout:2/420: sync 2024-04-26T23:01:31.341 INFO:tasks.workunit.client.0.smithi027.stdout:2/421: chown d2/d5/db/f6c 3 1 2024-04-26T23:01:31.342 INFO:tasks.workunit.client.0.smithi027.stdout:4/525: sync 2024-04-26T23:01:31.343 INFO:tasks.workunit.client.0.smithi027.stdout:9/447: sync 2024-04-26T23:01:31.343 INFO:tasks.workunit.client.0.smithi027.stdout:9/448: chown d4/f2b 2 1 2024-04-26T23:01:31.343 INFO:tasks.workunit.client.0.smithi027.stdout:9/449: fdatasync d4/d10/d2c/d72/f79 0 2024-04-26T23:01:31.345 INFO:tasks.workunit.client.0.smithi027.stdout:7/503: mknod d0/c9e 0 2024-04-26T23:01:31.345 INFO:tasks.workunit.client.0.smithi027.stdout:7/504: write d0/f30 [612774,43836] 0 2024-04-26T23:01:31.346 INFO:tasks.workunit.client.0.smithi027.stdout:4/526: symlink da/dd/d5f/la0 0 2024-04-26T23:01:31.349 INFO:tasks.workunit.client.0.smithi027.stdout:9/450: mknod d4/d10/d67/c9b 0 2024-04-26T23:01:31.350 INFO:tasks.workunit.client.0.smithi027.stdout:8/486: read d9/da/d2a/d5e/f6e [3783090,50866] 0 2024-04-26T23:01:31.350 INFO:tasks.workunit.client.0.smithi027.stdout:9/451: creat d4/d10/d40/d58/f9c x:0 0 0 2024-04-26T23:01:31.350 INFO:tasks.workunit.client.0.smithi027.stdout:3/517: dread d6/dc/d34/d2e/d5f/d90/fa9 [0,4194304] 0 2024-04-26T23:01:31.361 INFO:tasks.workunit.client.0.smithi027.stdout:8/487: dread d9/f17 [4194304,4194304] 0 2024-04-26T23:01:31.361 INFO:tasks.workunit.client.0.smithi027.stdout:8/488: truncate d9/da/d2a/d5e/d72/d84/d5d/d8b/f9a 395280 0 2024-04-26T23:01:31.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:30 smithi027 ceph-mon[28034]: pgmap v90: 129 pgs: 129 active+clean; 736 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s rd, 98 MiB/s wr, 1.04k op/s 2024-04-26T23:01:31.386 INFO:tasks.workunit.client.0.smithi027.stdout:2/422: dwrite d2/d5/db/de/d21/d26/f2d [0,4194304] 0 2024-04-26T23:01:31.390 INFO:tasks.workunit.client.0.smithi027.stdout:2/423: rename d2/d5/db/d1b/d1c/d25/c52 to d2/d5/db/d1b/d1c/d25/d19/d44/c72 0 2024-04-26T23:01:31.390 INFO:tasks.workunit.client.0.smithi027.stdout:2/424: fsync d2/d5/db/d1b/d1c/d25/d19/d36/f43 0 2024-04-26T23:01:31.391 INFO:tasks.workunit.client.0.smithi027.stdout:2/425: unlink d2/d5/f22 0 2024-04-26T23:01:31.393 INFO:tasks.workunit.client.0.smithi027.stdout:2/426: creat d2/d5/db/d1b/d1c/d25/d19/d36/d39/f73 x:0 0 0 2024-04-26T23:01:31.394 INFO:tasks.workunit.client.0.smithi027.stdout:2/427: dread - d2/d5/db/de/d21/d26/f6d zero size 2024-04-26T23:01:31.394 INFO:tasks.workunit.client.0.smithi027.stdout:7/505: dwrite d0/d28/d2b/d55/d59/f6b [0,4194304] 0 2024-04-26T23:01:31.394 INFO:tasks.workunit.client.0.smithi027.stdout:3/518: write d6/f8 [2692417,87328] 0 2024-04-26T23:01:31.395 INFO:tasks.workunit.client.0.smithi027.stdout:2/428: symlink d2/d5/db/d1b/d1c/d25/d6a/l74 0 2024-04-26T23:01:31.399 INFO:tasks.workunit.client.0.smithi027.stdout:3/519: mknod d6/cb5 0 2024-04-26T23:01:31.399 INFO:tasks.workunit.client.0.smithi027.stdout:2/429: rmdir d2/d5/db/d1b/d1c/d25/d19/d36 39 2024-04-26T23:01:31.399 INFO:tasks.workunit.client.0.smithi027.stdout:2/430: stat d2/d5/db/d1b/d1c/d25/d19/d44/d4b/f56 0 2024-04-26T23:01:31.400 INFO:tasks.workunit.client.0.smithi027.stdout:6/428: dwrite d1/f69 [0,4194304] 0 2024-04-26T23:01:31.401 INFO:tasks.workunit.client.0.smithi027.stdout:8/489: dread d9/de/d2e/f53 [0,4194304] 0 2024-04-26T23:01:31.401 INFO:tasks.workunit.client.0.smithi027.stdout:2/431: mknod d2/d5/db/d1b/d1c/d25/d19/d44/c75 0 2024-04-26T23:01:31.403 INFO:tasks.workunit.client.0.smithi027.stdout:3/520: rename d6/dc/d34/d47/d7b/f93 to d6/dc/d34/d78/fb6 0 2024-04-26T23:01:31.404 INFO:tasks.workunit.client.0.smithi027.stdout:9/452: dwrite d4/d17/d2e/f95 [0,4194304] 0 2024-04-26T23:01:31.404 INFO:tasks.workunit.client.0.smithi027.stdout:8/490: truncate d9/f16 172229 0 2024-04-26T23:01:31.405 INFO:tasks.workunit.client.0.smithi027.stdout:3/521: creat d6/dc/d34/d2e/d9d/da3/fb7 x:0 0 0 2024-04-26T23:01:31.406 INFO:tasks.workunit.client.0.smithi027.stdout:6/429: symlink d1/df/d49/l96 0 2024-04-26T23:01:31.406 INFO:tasks.workunit.client.0.smithi027.stdout:6/430: write d1/df/d49/f4f [1172504,35201] 0 2024-04-26T23:01:31.406 INFO:tasks.workunit.client.0.smithi027.stdout:8/491: truncate d9/de/f18 631231 0 2024-04-26T23:01:31.408 INFO:tasks.workunit.client.0.smithi027.stdout:3/522: creat d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/fb8 x:0 0 0 2024-04-26T23:01:31.408 INFO:tasks.workunit.client.0.smithi027.stdout:3/523: write d6/dc/d34/d47/d7b/fa4 [971612,75564] 0 2024-04-26T23:01:31.408 INFO:tasks.workunit.client.0.smithi027.stdout:3/524: fsync d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d52/d5a/f99 0 2024-04-26T23:01:31.408 INFO:tasks.workunit.client.0.smithi027.stdout:3/525: fdatasync d6/dc/f11 0 2024-04-26T23:01:31.409 INFO:tasks.workunit.client.0.smithi027.stdout:3/526: readlink d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d59/lb1 0 2024-04-26T23:01:31.409 INFO:tasks.workunit.client.0.smithi027.stdout:9/453: mknod d4/d10/d40/d58/c9d 0 2024-04-26T23:01:31.409 INFO:tasks.workunit.client.0.smithi027.stdout:8/492: creat d9/de/fb4 x:0 0 0 2024-04-26T23:01:31.412 INFO:tasks.workunit.client.0.smithi027.stdout:4/527: dwrite f2 [0,4194304] 0 2024-04-26T23:01:31.412 INFO:tasks.workunit.client.0.smithi027.stdout:6/431: getdents d1/d30/d40 0 2024-04-26T23:01:31.412 INFO:tasks.workunit.client.0.smithi027.stdout:2/432: symlink d2/d5/db/d1b/d1c/l76 0 2024-04-26T23:01:31.413 INFO:tasks.workunit.client.0.smithi027.stdout:7/506: dread d0/d28/d66/f70 [0,4194304] 0 2024-04-26T23:01:31.418 INFO:tasks.workunit.client.0.smithi027.stdout:9/454: rename d4/d17/d2e/d70/f98 to d4/d17/f9e 0 2024-04-26T23:01:31.418 INFO:tasks.workunit.client.0.smithi027.stdout:4/528: link da/dd/de/f3f da/dd/de/d14/d21/d41/d59/d74/fa1 0 2024-04-26T23:01:31.418 INFO:tasks.workunit.client.0.smithi027.stdout:4/529: fsync da/dd/de/d14/d21/f75 0 2024-04-26T23:01:31.418 INFO:tasks.workunit.client.0.smithi027.stdout:4/530: fdatasync da/d40/f66 0 2024-04-26T23:01:31.418 INFO:tasks.workunit.client.0.smithi027.stdout:4/531: write da/dd/de/d14/d21/d67/d8a/f8c [1177570,113171] 0 2024-04-26T23:01:31.418 INFO:tasks.workunit.client.0.smithi027.stdout:6/432: creat d1/d30/d6c/f97 x:0 0 0 2024-04-26T23:01:31.419 INFO:tasks.workunit.client.0.smithi027.stdout:8/493: creat d9/da/d2a/d5e/d9e/fb5 x:0 0 0 2024-04-26T23:01:31.423 INFO:tasks.workunit.client.0.smithi027.stdout:7/507: rmdir d0/d28/d2b/d3a/d48/d56/d88/d73/d80 39 2024-04-26T23:01:31.434 INFO:tasks.workunit.client.0.smithi027.stdout:4/532: creat da/dd/de/d14/d21/d67/d8a/fa2 x:0 0 0 2024-04-26T23:01:31.434 INFO:tasks.workunit.client.0.smithi027.stdout:7/508: mkdir d0/d28/d9f 0 2024-04-26T23:01:31.435 INFO:tasks.workunit.client.0.smithi027.stdout:7/509: stat d0/d28/d2b/d55/c8a 0 2024-04-26T23:01:31.436 INFO:tasks.workunit.client.0.smithi027.stdout:0/504: sync 2024-04-26T23:01:31.438 INFO:tasks.workunit.client.0.smithi027.stdout:2/433: rename d2/d5/db/d1b/f46 to d2/d5/db/d1b/d1c/f77 0 2024-04-26T23:01:31.442 INFO:tasks.workunit.client.0.smithi027.stdout:7/510: truncate d0/d28/d66/f70 1483671 0 2024-04-26T23:01:31.443 INFO:tasks.workunit.client.0.smithi027.stdout:7/511: write d0/d28/d2b/d3a/d48/d56/d88/f29 [2015680,17963] 0 2024-04-26T23:01:31.443 INFO:tasks.workunit.client.0.smithi027.stdout:7/512: chown d0/c4f 90 1 2024-04-26T23:01:31.443 INFO:tasks.workunit.client.0.smithi027.stdout:1/539: sync 2024-04-26T23:01:31.443 INFO:tasks.workunit.client.0.smithi027.stdout:7/513: fdatasync d0/d28/d2b/d3a/d48/d56/d88/f3c 0 2024-04-26T23:01:31.443 INFO:tasks.workunit.client.0.smithi027.stdout:7/514: readlink d0/d28/l12 0 2024-04-26T23:01:31.443 INFO:tasks.workunit.client.0.smithi027.stdout:2/434: rmdir d2/d5/db/d1b/d1c/d25/d19/d44/d4b 39 2024-04-26T23:01:31.443 INFO:tasks.workunit.client.0.smithi027.stdout:4/533: mknod da/dd/d5e/ca3 0 2024-04-26T23:01:31.443 INFO:tasks.workunit.client.0.smithi027.stdout:9/455: dread d4/d17/d2e/f95 [0,4194304] 0 2024-04-26T23:01:31.445 INFO:tasks.workunit.client.0.smithi027.stdout:4/534: creat da/dd/de/d14/d21/d41/fa4 x:0 0 0 2024-04-26T23:01:31.445 INFO:tasks.workunit.client.0.smithi027.stdout:4/535: chown da/dd/c73 5 1 2024-04-26T23:01:31.448 INFO:tasks.workunit.client.0.smithi027.stdout:1/540: link d1/d2a/f49 d1/dc/fa7 0 2024-04-26T23:01:31.449 INFO:tasks.workunit.client.0.smithi027.stdout:9/456: truncate d4/d17/f29 3124574 0 2024-04-26T23:01:31.451 INFO:tasks.workunit.client.0.smithi027.stdout:2/435: symlink d2/d5/db/d1b/d1c/l78 0 2024-04-26T23:01:31.451 INFO:tasks.workunit.client.0.smithi027.stdout:4/536: mkdir da/dd/de/d79/da5 0 2024-04-26T23:01:31.451 INFO:tasks.workunit.client.0.smithi027.stdout:2/436: fsync d2/d5/db/f6c 0 2024-04-26T23:01:31.451 INFO:tasks.workunit.client.0.smithi027.stdout:4/537: fsync da/dd/de/d14/d21/d4a/f50 0 2024-04-26T23:01:31.452 INFO:tasks.workunit.client.0.smithi027.stdout:4/538: mkdir da/dd/de/d14/d21/d67/da6 0 2024-04-26T23:01:31.452 INFO:tasks.workunit.client.0.smithi027.stdout:4/539: readlink da/dd/de/d14/l34 0 2024-04-26T23:01:31.452 INFO:tasks.workunit.client.0.smithi027.stdout:4/540: readlink da/dd/d5f/la0 0 2024-04-26T23:01:31.452 INFO:tasks.workunit.client.0.smithi027.stdout:4/541: chown da/d40/l54 32 1 2024-04-26T23:01:31.452 INFO:tasks.workunit.client.0.smithi027.stdout:4/542: fsync da/dd/de/d14/d21/d41/fa4 0 2024-04-26T23:01:31.453 INFO:tasks.workunit.client.0.smithi027.stdout:2/437: creat d2/d5/db/d1b/d1c/d25/d19/f79 x:0 0 0 2024-04-26T23:01:31.454 INFO:tasks.workunit.client.0.smithi027.stdout:4/543: symlink da/dd/de/d14/la7 0 2024-04-26T23:01:31.458 INFO:tasks.workunit.client.0.smithi027.stdout:2/438: mkdir d2/d5/db/d1b/d1c/d25/d7a 0 2024-04-26T23:01:31.468 INFO:tasks.workunit.client.0.smithi027.stdout:4/544: read da/dd/de/d14/d21/d41/d59/d74/f7e [262300,61505] 0 2024-04-26T23:01:31.469 INFO:tasks.workunit.client.0.smithi027.stdout:4/545: chown da/dd/de/d14/d21/d41/f49 659 1 2024-04-26T23:01:31.482 INFO:tasks.workunit.client.0.smithi027.stdout:8/494: dwrite d9/de/d22/f4d [4194304,4194304] 0 2024-04-26T23:01:31.486 INFO:tasks.workunit.client.0.smithi027.stdout:8/495: rmdir d9/da/d2a/d5e/d72/d84/d35/d8d 39 2024-04-26T23:01:31.502 INFO:tasks.workunit.client.0.smithi027.stdout:3/527: dwrite d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d59/f97 [0,4194304] 0 2024-04-26T23:01:31.505 INFO:tasks.workunit.client.0.smithi027.stdout:4/546: read da/f1f [4831427,93705] 0 2024-04-26T23:01:31.505 INFO:tasks.workunit.client.0.smithi027.stdout:4/547: stat da/dd/de/d14/f43 0 2024-04-26T23:01:31.506 INFO:tasks.workunit.client.0.smithi027.stdout:2/439: dwrite d2/d5/db/d1b/d1c/d25/d6a/f4c [0,4194304] 0 2024-04-26T23:01:31.511 INFO:tasks.workunit.client.0.smithi027.stdout:4/548: mkdir da/dd/de/d79/da8 0 2024-04-26T23:01:31.518 INFO:tasks.workunit.client.0.smithi027.stdout:3/528: unlink d6/dc/d34/d2e/d9d/da7/dac/laf 0 2024-04-26T23:01:31.519 INFO:tasks.workunit.client.0.smithi027.stdout:3/529: rmdir d6/dc/d34/db3 39 2024-04-26T23:01:31.526 INFO:tasks.workunit.client.0.smithi027.stdout:3/530: mkdir d6/dc/d34/d2e/d5f/d90/db9 0 2024-04-26T23:01:31.526 INFO:tasks.workunit.client.0.smithi027.stdout:6/433: dwrite d1/df/d49/f4d [0,4194304] 0 2024-04-26T23:01:31.526 INFO:tasks.workunit.client.0.smithi027.stdout:6/434: write d1/d30/d6c/f7a [5052779,23411] 0 2024-04-26T23:01:31.532 INFO:tasks.workunit.client.0.smithi027.stdout:7/515: dwrite d0/d28/d2b/f5d [0,4194304] 0 2024-04-26T23:01:31.537 INFO:tasks.workunit.client.0.smithi027.stdout:2/440: read d2/d5/db/d1b/d1c/d25/f3d [1062994,50305] 0 2024-04-26T23:01:31.537 INFO:tasks.workunit.client.0.smithi027.stdout:2/441: chown d2/d5/db/de/d21/d26/c4d 15 1 2024-04-26T23:01:31.537 INFO:tasks.workunit.client.0.smithi027.stdout:2/442: chown d2/d5/db/de/f42 278 1 2024-04-26T23:01:31.539 INFO:tasks.workunit.client.0.smithi027.stdout:0/505: dwrite d0/df/d3f/d64/f38 [0,4194304] 0 2024-04-26T23:01:31.539 INFO:tasks.workunit.client.0.smithi027.stdout:0/506: truncate d0/d4/da/d15/f6f 4616783 0 2024-04-26T23:01:31.539 INFO:tasks.workunit.client.0.smithi027.stdout:0/507: stat d0/d4/da/c6b 0 2024-04-26T23:01:31.539 INFO:tasks.workunit.client.0.smithi027.stdout:7/516: mknod d0/d28/d2b/d55/d59/d6d/d82/ca0 0 2024-04-26T23:01:31.539 INFO:tasks.workunit.client.0.smithi027.stdout:7/517: read d0/d28/d2b/d3a/d48/d56/d88/f16 [1626899,123586] 0 2024-04-26T23:01:31.539 INFO:tasks.workunit.client.0.smithi027.stdout:7/518: stat d0/d28/d2b/d3a/d48/f63 0 2024-04-26T23:01:31.539 INFO:tasks.workunit.client.0.smithi027.stdout:2/443: truncate d2/d5/db/d1b/d1c/d25/d19/d36/d39/f3f 4152174 0 2024-04-26T23:01:31.541 INFO:tasks.workunit.client.0.smithi027.stdout:1/541: dwrite d1/dc/f5b [0,4194304] 0 2024-04-26T23:01:31.542 INFO:tasks.workunit.client.0.smithi027.stdout:1/542: write d1/f55 [1341953,33956] 0 2024-04-26T23:01:31.542 INFO:tasks.workunit.client.0.smithi027.stdout:0/508: unlink d0/d4/da/d15/l7a 0 2024-04-26T23:01:31.543 INFO:tasks.workunit.client.0.smithi027.stdout:7/519: creat d0/d28/d9f/fa1 x:0 0 0 2024-04-26T23:01:31.544 INFO:tasks.workunit.client.0.smithi027.stdout:7/520: stat d0/d28/d2b/d55/d59 0 2024-04-26T23:01:31.544 INFO:tasks.workunit.client.0.smithi027.stdout:2/444: creat d2/d5/f7b x:0 0 0 2024-04-26T23:01:31.547 INFO:tasks.workunit.client.0.smithi027.stdout:7/521: unlink d0/d28/d2b/d3a/f4c 0 2024-04-26T23:01:31.548 INFO:tasks.workunit.client.0.smithi027.stdout:2/445: mkdir d2/d5/db/d1b/d1c/d25/d19/d23/d7c 0 2024-04-26T23:01:31.549 INFO:tasks.workunit.client.0.smithi027.stdout:7/522: symlink d0/d28/d2b/d55/d59/la2 0 2024-04-26T23:01:31.550 INFO:tasks.workunit.client.0.smithi027.stdout:2/446: unlink d2/d5/l33 0 2024-04-26T23:01:31.552 INFO:tasks.workunit.client.0.smithi027.stdout:7/523: mkdir d0/d28/d66/da3 0 2024-04-26T23:01:31.552 INFO:tasks.workunit.client.0.smithi027.stdout:7/524: chown d0/d2f 3 1 2024-04-26T23:01:31.554 INFO:tasks.workunit.client.0.smithi027.stdout:0/509: dread d0/d4/da/f78 [0,4194304] 0 2024-04-26T23:01:31.554 INFO:tasks.workunit.client.0.smithi027.stdout:0/510: chown d0/d4/da/d1a/d59/c75 1 1 2024-04-26T23:01:31.555 INFO:tasks.workunit.client.0.smithi027.stdout:9/457: dwrite d4/d17/d2e/f3d [0,4194304] 0 2024-04-26T23:01:31.555 INFO:tasks.workunit.client.0.smithi027.stdout:9/458: fdatasync d4/f52 0 2024-04-26T23:01:31.556 INFO:tasks.workunit.client.0.smithi027.stdout:0/511: rename d0/d4/da/c7b to d0/df/d3f/d4a/c98 0 2024-04-26T23:01:31.556 INFO:tasks.workunit.client.0.smithi027.stdout:0/512: chown d0/d4/da/f14 46021 1 2024-04-26T23:01:31.559 INFO:tasks.workunit.client.0.smithi027.stdout:9/459: rmdir d4 39 2024-04-26T23:01:31.562 INFO:tasks.workunit.client.0.smithi027.stdout:9/460: creat d4/d10/d40/d58/d6e/f9f x:0 0 0 2024-04-26T23:01:31.562 INFO:tasks.workunit.client.0.smithi027.stdout:9/461: write d4/d10/d40/d58/d6e/f9f [20003,86135] 0 2024-04-26T23:01:31.564 INFO:tasks.workunit.client.0.smithi027.stdout:9/462: rename d4/d10/d2c to d4/d17/d2e/d46/da0 0 2024-04-26T23:01:31.565 INFO:tasks.workunit.client.0.smithi027.stdout:9/463: chown d4/d17/d2e/f34 0 1 2024-04-26T23:01:31.565 INFO:tasks.workunit.client.0.smithi027.stdout:9/464: readlink d4/l5 0 2024-04-26T23:01:31.568 INFO:tasks.workunit.client.0.smithi027.stdout:9/465: mkdir d4/d10/d40/d58/d75/d84/d86/da1 0 2024-04-26T23:01:31.573 INFO:tasks.workunit.client.0.smithi027.stdout:6/435: dwrite d1/df/d3c/d66/f8a [0,4194304] 0 2024-04-26T23:01:31.574 INFO:tasks.workunit.client.0.smithi027.stdout:4/549: dwrite da/dd/de/f94 [0,4194304] 0 2024-04-26T23:01:31.574 INFO:tasks.workunit.client.0.smithi027.stdout:1/543: dread d1/d26/d36/d56/f6e [0,4194304] 0 2024-04-26T23:01:31.575 INFO:tasks.workunit.client.0.smithi027.stdout:1/544: stat d1/d26/d7d/l33 0 2024-04-26T23:01:31.579 INFO:tasks.workunit.client.0.smithi027.stdout:4/550: link da/dd/de/d14/d21/f75 da/dd/de/d14/d17/d80/fa9 0 2024-04-26T23:01:31.582 INFO:tasks.workunit.client.0.smithi027.stdout:4/551: unlink da/dd/f5c 0 2024-04-26T23:01:31.584 INFO:tasks.workunit.client.0.smithi027.stdout:4/552: creat da/dd/de/d14/d21/d41/d59/d74/faa x:0 0 0 2024-04-26T23:01:31.584 INFO:tasks.workunit.client.0.smithi027.stdout:4/553: read - da/dd/de/d79/f8e zero size 2024-04-26T23:01:31.591 INFO:tasks.workunit.client.0.smithi027.stdout:4/554: rename da/dd/de/d14/d17/d24 to da/dd/de/d14/d17/d80/dab 0 2024-04-26T23:01:31.591 INFO:tasks.workunit.client.0.smithi027.stdout:4/555: write da/dd/de/d14/d21/d67/d8a/fa2 [1003574,64667] 0 2024-04-26T23:01:31.595 INFO:tasks.workunit.client.0.smithi027.stdout:4/556: rename da/dd/de/d14/d93 to da/dd/d5f/dac 0 2024-04-26T23:01:31.597 INFO:tasks.workunit.client.0.smithi027.stdout:4/557: mkdir da/dd/de/d14/d21/dad 0 2024-04-26T23:01:31.597 INFO:tasks.workunit.client.0.smithi027.stdout:4/558: fdatasync da/dd/de/d14/d17/d80/dab/f16 0 2024-04-26T23:01:31.600 INFO:tasks.workunit.client.0.smithi027.stdout:7/525: dwrite d0/d28/d2b/d3a/d48/d56/d88/f16 [0,4194304] 0 2024-04-26T23:01:31.600 INFO:tasks.workunit.client.0.smithi027.stdout:7/526: fsync d0/d3d/f54 0 2024-04-26T23:01:31.600 INFO:tasks.workunit.client.0.smithi027.stdout:7/527: chown d0/d28/d2b/d3a/d48/d56 2692949 1 2024-04-26T23:01:31.600 INFO:tasks.workunit.client.0.smithi027.stdout:1/545: write d1/f1f [2885208,123796] 0 2024-04-26T23:01:31.600 INFO:tasks.workunit.client.0.smithi027.stdout:1/546: fsync d1/d26/d66/d73/d9e/f9f 0 2024-04-26T23:01:31.602 INFO:tasks.workunit.client.0.smithi027.stdout:1/547: mknod d1/d26/d36/d56/ca8 0 2024-04-26T23:01:31.624 INFO:tasks.workunit.client.0.smithi027.stdout:1/548: dread d1/dc/f37 [0,4194304] 0 2024-04-26T23:01:31.625 INFO:tasks.workunit.client.0.smithi027.stdout:1/549: readlink d1/l45 0 2024-04-26T23:01:31.625 INFO:tasks.workunit.client.0.smithi027.stdout:1/550: write d1/d26/d86/f9c [16410,127829] 0 2024-04-26T23:01:31.628 INFO:tasks.workunit.client.0.smithi027.stdout:1/551: link d1/d26/d66/d73/f7c d1/dc/fa9 0 2024-04-26T23:01:31.629 INFO:tasks.workunit.client.0.smithi027.stdout:1/552: rename d1/f6 to d1/d26/d7d/faa 0 2024-04-26T23:01:31.631 INFO:tasks.workunit.client.0.smithi027.stdout:1/553: mkdir d1/d26/d66/dab 0 2024-04-26T23:01:31.632 INFO:tasks.workunit.client.0.smithi027.stdout:1/554: creat d1/d2a/fac x:0 0 0 2024-04-26T23:01:31.633 INFO:tasks.workunit.client.0.smithi027.stdout:1/555: rmdir d1/d26/d36/d88 39 2024-04-26T23:01:31.637 INFO:tasks.workunit.client.0.smithi027.stdout:7/528: dwrite d0/d28/d2b/d55/d59/f6b [4194304,4194304] 0 2024-04-26T23:01:31.644 INFO:tasks.workunit.client.0.smithi027.stdout:1/556: dread d1/d5/f2f [0,4194304] 0 2024-04-26T23:01:31.663 INFO:tasks.workunit.client.0.smithi027.stdout:4/559: dwrite da/dd/de/d14/f53 [0,4194304] 0 2024-04-26T23:01:31.666 INFO:tasks.workunit.client.0.smithi027.stdout:4/560: getdents da/d40/d87 0 2024-04-26T23:01:31.669 INFO:tasks.workunit.client.0.smithi027.stdout:4/561: creat da/dd/de/d14/d21/d4a/fae x:0 0 0 2024-04-26T23:01:31.674 INFO:tasks.workunit.client.0.smithi027.stdout:7/529: dwrite d0/d28/f40 [0,4194304] 0 2024-04-26T23:01:31.675 INFO:tasks.workunit.client.0.smithi027.stdout:7/530: mknod d0/d28/d2b/d3a/d48/d56/d88/d73/d80/ca4 0 2024-04-26T23:01:31.676 INFO:tasks.workunit.client.0.smithi027.stdout:7/531: truncate d0/f42 968286 0 2024-04-26T23:01:31.687 INFO:tasks.workunit.client.0.smithi027.stdout:4/562: read da/dd/f2b [900777,93184] 0 2024-04-26T23:01:31.688 INFO:tasks.workunit.client.0.smithi027.stdout:5/531: sync 2024-04-26T23:01:31.689 INFO:tasks.workunit.client.0.smithi027.stdout:5/532: dread - d4/d24/d5f/dac/d40/f97 zero size 2024-04-26T23:01:31.693 INFO:tasks.workunit.client.0.smithi027.stdout:7/532: dread d0/d28/d2b/d55/d59/f7b [0,4194304] 0 2024-04-26T23:01:31.694 INFO:tasks.workunit.client.0.smithi027.stdout:7/533: fsync d0/d28/d2b/d3a/d48/d56/f9a 0 2024-04-26T23:01:31.694 INFO:tasks.workunit.client.0.smithi027.stdout:7/534: symlink d0/d28/d2b/d3a/d48/d56/d88/la5 0 2024-04-26T23:01:31.695 INFO:tasks.workunit.client.0.smithi027.stdout:7/535: chown d0/c4f 262425 1 2024-04-26T23:01:31.695 INFO:tasks.workunit.client.0.smithi027.stdout:7/536: write d0/d3d/f43 [1119377,89561] 0 2024-04-26T23:01:31.697 INFO:tasks.workunit.client.0.smithi027.stdout:7/537: getdents d0/d28/d66/d67 0 2024-04-26T23:01:31.736 INFO:tasks.workunit.client.0.smithi027.stdout:7/538: dread d0/d28/d2b/d55/d59/f7b [0,4194304] 0 2024-04-26T23:01:31.736 INFO:tasks.workunit.client.0.smithi027.stdout:7/539: dread - d0/d28/d2b/d3a/d48/d56/f9a zero size 2024-04-26T23:01:31.736 INFO:tasks.workunit.client.0.smithi027.stdout:7/540: stat d0/d28/d66 0 2024-04-26T23:01:31.736 INFO:tasks.workunit.client.0.smithi027.stdout:7/541: write d0/d28/f6a [874966,121651] 0 2024-04-26T23:01:31.737 INFO:tasks.workunit.client.0.smithi027.stdout:7/542: mkdir d0/d28/d66/d67/da6 0 2024-04-26T23:01:31.738 INFO:tasks.workunit.client.0.smithi027.stdout:7/543: write d0/d28/d2b/f34 [1272705,9654] 0 2024-04-26T23:01:31.738 INFO:tasks.workunit.client.0.smithi027.stdout:7/544: readlink d0/d28/d2b/d3a/l5f 0 2024-04-26T23:01:31.738 INFO:tasks.workunit.client.0.smithi027.stdout:7/545: chown d0/l33 20 1 2024-04-26T23:01:31.747 INFO:tasks.workunit.client.0.smithi027.stdout:5/533: dwrite d4/d24/d22/dab/d16/f1f [4194304,4194304] 0 2024-04-26T23:01:31.747 INFO:tasks.workunit.client.0.smithi027.stdout:5/534: read - d4/fa5 zero size 2024-04-26T23:01:31.748 INFO:tasks.workunit.client.0.smithi027.stdout:5/535: mkdir d4/d24/d22/dab/d16/dad 0 2024-04-26T23:01:31.749 INFO:tasks.workunit.client.0.smithi027.stdout:5/536: write d4/d24/d5f/dac/d40/f97 [633474,588] 0 2024-04-26T23:01:31.750 INFO:tasks.workunit.client.0.smithi027.stdout:5/537: truncate d4/d24/d22/d29/f58 1290111 0 2024-04-26T23:01:31.750 INFO:tasks.workunit.client.0.smithi027.stdout:5/538: truncate d4/d24/f7e 11289 0 2024-04-26T23:01:31.755 INFO:tasks.workunit.client.0.smithi027.stdout:5/539: rename d4/d24/d22/dab/d16/f1f to d4/d24/d5f/dac/d40/fae 0 2024-04-26T23:01:31.757 INFO:tasks.workunit.client.0.smithi027.stdout:5/540: mknod d4/d24/d22/d29/caf 0 2024-04-26T23:01:31.778 INFO:tasks.workunit.client.0.smithi027.stdout:5/541: truncate d4/d24/d22/d29/f58 797944 0 2024-04-26T23:01:31.778 INFO:tasks.workunit.client.0.smithi027.stdout:5/542: chown d4/f15 3 1 2024-04-26T23:01:31.792 INFO:tasks.workunit.client.0.smithi027.stdout:7/546: dwrite d0/d6c/f93 [0,4194304] 0 2024-04-26T23:01:31.792 INFO:tasks.workunit.client.0.smithi027.stdout:7/547: chown d0/d28/d2b/d55/d69/c97 27848722 1 2024-04-26T23:01:31.828 INFO:tasks.workunit.client.0.smithi027.stdout:5/543: dwrite d4/f15 [4194304,4194304] 0 2024-04-26T23:01:31.830 INFO:tasks.workunit.client.0.smithi027.stdout:5/544: rmdir d4/d24/d22/dab/d16/dad 0 2024-04-26T23:01:31.831 INFO:tasks.workunit.client.0.smithi027.stdout:5/545: unlink d4/ca 0 2024-04-26T23:01:31.832 INFO:tasks.workunit.client.0.smithi027.stdout:7/548: dwrite d0/d2f/f31 [0,4194304] 0 2024-04-26T23:01:31.833 INFO:tasks.workunit.client.0.smithi027.stdout:7/549: getdents d0/d2f 0 2024-04-26T23:01:31.835 INFO:tasks.workunit.client.0.smithi027.stdout:7/550: mkdir d0/d28/d2b/d55/d69/da7 0 2024-04-26T23:01:31.835 INFO:tasks.workunit.client.0.smithi027.stdout:7/551: truncate d0/d28/d9f/fa1 33631 0 2024-04-26T23:01:31.870 INFO:tasks.workunit.client.0.smithi027.stdout:7/552: dread d0/d28/d2b/d3a/d48/d56/d88/f29 [0,4194304] 0 2024-04-26T23:01:31.882 INFO:tasks.workunit.client.0.smithi027.stdout:5/546: dwrite d4/d24/f42 [0,4194304] 0 2024-04-26T23:01:31.882 INFO:tasks.workunit.client.0.smithi027.stdout:5/547: readlink d4/d24/d22/d29/d2a/d3b/d50/l71 0 2024-04-26T23:01:31.882 INFO:tasks.workunit.client.0.smithi027.stdout:5/548: readlink d4/d24/d22/d29/d2a/d3b/d50/l71 0 2024-04-26T23:01:31.882 INFO:tasks.workunit.client.0.smithi027.stdout:5/549: chown d4/d24/d5f/dac 78938462 1 2024-04-26T23:01:31.884 INFO:tasks.workunit.client.0.smithi027.stdout:5/550: mknod d4/d24/d22/d29/cb0 0 2024-04-26T23:01:31.884 INFO:tasks.workunit.client.0.smithi027.stdout:5/551: write d4/d24/d22/d29/d39/f7f [4531273,5727] 0 2024-04-26T23:01:31.884 INFO:tasks.workunit.client.0.smithi027.stdout:5/552: fsync d4/d1a/d2b/f7b 0 2024-04-26T23:01:31.924 INFO:tasks.workunit.client.0.smithi027.stdout:7/553: dwrite d0/d28/f40 [0,4194304] 0 2024-04-26T23:01:31.927 INFO:tasks.workunit.client.0.smithi027.stdout:7/554: rename d0/f5a to d0/d28/d2b/d55/fa8 0 2024-04-26T23:01:31.928 INFO:tasks.workunit.client.0.smithi027.stdout:7/555: truncate d0/d28/d2b/d3a/d48/d56/d88/f3c 1764693 0 2024-04-26T23:01:31.928 INFO:tasks.workunit.client.0.smithi027.stdout:7/556: readlink d0/d28/d2b/l4a 0 2024-04-26T23:01:31.929 INFO:tasks.workunit.client.0.smithi027.stdout:5/553: dwrite f1 [4194304,4194304] 0 2024-04-26T23:01:31.934 INFO:tasks.workunit.client.0.smithi027.stdout:7/557: mknod d0/d28/d2b/d3a/d48/d56/ca9 0 2024-04-26T23:01:31.935 INFO:tasks.workunit.client.0.smithi027.stdout:7/558: symlink d0/d28/d2b/d3a/laa 0 2024-04-26T23:01:31.935 INFO:tasks.workunit.client.0.smithi027.stdout:8/496: sync 2024-04-26T23:01:31.935 INFO:tasks.workunit.client.0.smithi027.stdout:2/447: sync 2024-04-26T23:01:31.935 INFO:tasks.workunit.client.0.smithi027.stdout:3/531: sync 2024-04-26T23:01:31.936 INFO:tasks.workunit.client.0.smithi027.stdout:7/559: getdents d0/d28/d2b/d3a/d90 0 2024-04-26T23:01:31.937 INFO:tasks.workunit.client.0.smithi027.stdout:1/557: sync 2024-04-26T23:01:31.937 INFO:tasks.workunit.client.0.smithi027.stdout:6/436: sync 2024-04-26T23:01:31.937 INFO:tasks.workunit.client.0.smithi027.stdout:9/466: sync 2024-04-26T23:01:31.938 INFO:tasks.workunit.client.0.smithi027.stdout:2/448: write d2/f67 [40216,112765] 0 2024-04-26T23:01:31.938 INFO:tasks.workunit.client.0.smithi027.stdout:0/513: sync 2024-04-26T23:01:31.938 INFO:tasks.workunit.client.0.smithi027.stdout:6/437: rmdir d1/df/d3c 39 2024-04-26T23:01:31.938 INFO:tasks.workunit.client.0.smithi027.stdout:0/514: fdatasync d0/d4/da/f2f 0 2024-04-26T23:01:31.939 INFO:tasks.workunit.client.0.smithi027.stdout:9/467: mkdir d4/d77/da2 0 2024-04-26T23:01:31.940 INFO:tasks.workunit.client.0.smithi027.stdout:2/449: mkdir d2/d5/db/d1b/d1c/d25/d19/d23/d7c/d7d 0 2024-04-26T23:01:31.940 INFO:tasks.workunit.client.0.smithi027.stdout:6/438: mkdir d1/de/d57/d98 0 2024-04-26T23:01:31.941 INFO:tasks.workunit.client.0.smithi027.stdout:4/563: sync 2024-04-26T23:01:31.941 INFO:tasks.workunit.client.0.smithi027.stdout:8/497: creat d9/da/d2a/fb6 x:0 0 0 2024-04-26T23:01:31.942 INFO:tasks.workunit.client.0.smithi027.stdout:0/515: unlink d0/l10 0 2024-04-26T23:01:31.942 INFO:tasks.workunit.client.0.smithi027.stdout:0/516: fdatasync d0/df/f72 0 2024-04-26T23:01:31.944 INFO:tasks.workunit.client.0.smithi027.stdout:9/468: symlink d4/d10/d40/d58/d75/la3 0 2024-04-26T23:01:31.944 INFO:tasks.workunit.client.0.smithi027.stdout:2/450: mkdir d2/d5/db/d1b/d1c/d25/d19/d44/d7e 0 2024-04-26T23:01:31.950 INFO:tasks.workunit.client.0.smithi027.stdout:6/439: unlink d1/de/f20 0 2024-04-26T23:01:31.950 INFO:tasks.workunit.client.0.smithi027.stdout:6/440: chown d1/de/c67 109807958 1 2024-04-26T23:01:31.951 INFO:tasks.workunit.client.0.smithi027.stdout:6/441: fdatasync d1/de/f48 0 2024-04-26T23:01:31.951 INFO:tasks.workunit.client.0.smithi027.stdout:6/442: chown d1/de/l46 5645 1 2024-04-26T23:01:31.951 INFO:tasks.workunit.client.0.smithi027.stdout:6/443: chown d1/d30/d6c 34870 1 2024-04-26T23:01:31.951 INFO:tasks.workunit.client.0.smithi027.stdout:3/532: symlink d6/dc/lba 0 2024-04-26T23:01:31.951 INFO:tasks.workunit.client.0.smithi027.stdout:6/444: creat d1/d30/d6c/f99 x:0 0 0 2024-04-26T23:01:31.952 INFO:tasks.workunit.client.0.smithi027.stdout:6/445: mkdir d1/df/d3c/d9a 0 2024-04-26T23:01:31.953 INFO:tasks.workunit.client.0.smithi027.stdout:5/554: write d4/f15 [6010727,13241] 0 2024-04-26T23:01:31.958 INFO:tasks.workunit.client.0.smithi027.stdout:6/446: creat d1/df/f9b x:0 0 0 2024-04-26T23:01:31.958 INFO:tasks.workunit.client.0.smithi027.stdout:6/447: write d1/de/f42 [983532,49940] 0 2024-04-26T23:01:31.958 INFO:tasks.workunit.client.0.smithi027.stdout:3/533: unlink d6/cb 0 2024-04-26T23:01:31.959 INFO:tasks.workunit.client.0.smithi027.stdout:5/555: creat d4/d24/d22/d29/fb1 x:0 0 0 2024-04-26T23:01:31.961 INFO:tasks.workunit.client.0.smithi027.stdout:5/556: link d4/d24/d22/dab/l44 d4/d24/d22/dab/d16/d4b/d81/lb2 0 2024-04-26T23:01:31.992 INFO:tasks.workunit.client.0.smithi027.stdout:3/534: write d6/dc/f25 [1756458,114125] 0 2024-04-26T23:01:31.992 INFO:tasks.workunit.client.0.smithi027.stdout:3/535: write d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d52/d5a/f99 [896679,97435] 0 2024-04-26T23:01:31.993 INFO:tasks.workunit.client.0.smithi027.stdout:3/536: mknod d6/dc/d34/d78/da1/cbb 0 2024-04-26T23:01:31.995 INFO:tasks.workunit.client.0.smithi027.stdout:3/537: creat d6/fbc x:0 0 0 2024-04-26T23:01:31.997 INFO:tasks.workunit.client.0.smithi027.stdout:3/538: rmdir d6/dc/d34/d2e/d9d/da7 39 2024-04-26T23:01:31.997 INFO:tasks.workunit.client.0.smithi027.stdout:3/539: write d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d40/f8b [888210,28877] 0 2024-04-26T23:01:31.997 INFO:tasks.workunit.client.0.smithi027.stdout:3/540: dread - d6/dc/d34/d2e/d70/d71/f9f zero size 2024-04-26T23:01:31.997 INFO:tasks.workunit.client.0.smithi027.stdout:3/541: dread - d6/dc/d34/f58 zero size 2024-04-26T23:01:32.001 INFO:tasks.workunit.client.0.smithi027.stdout:4/564: dwrite da/dd/de/d14/d21/d41/f72 [0,4194304] 0 2024-04-26T23:01:32.001 INFO:tasks.workunit.client.0.smithi027.stdout:3/542: mknod d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d52/cbd 0 2024-04-26T23:01:32.001 INFO:tasks.workunit.client.0.smithi027.stdout:3/543: truncate d6/dc/d34/d2e/d70/d71/f9f 381835 0 2024-04-26T23:01:32.006 INFO:tasks.workunit.client.0.smithi027.stdout:3/544: creat d6/dc/d34/d2e/d9d/da7/fbe x:0 0 0 2024-04-26T23:01:32.006 INFO:tasks.workunit.client.0.smithi027.stdout:3/545: dread - d6/dc/d34/d2e/d70/f7d zero size 2024-04-26T23:01:32.006 INFO:tasks.workunit.client.0.smithi027.stdout:3/546: write d6/dc/f72 [58761,92682] 0 2024-04-26T23:01:32.007 INFO:tasks.workunit.client.0.smithi027.stdout:3/547: readlink d6/dc/d2f/l3a 0 2024-04-26T23:01:32.015 INFO:tasks.workunit.client.0.smithi027.stdout:2/451: write d2/d5/db/de/d21/d26/f2d [2679209,118657] 0 2024-04-26T23:01:32.015 INFO:tasks.workunit.client.0.smithi027.stdout:2/452: stat d2/d5/db/de/d21/d26/f6d 0 2024-04-26T23:01:32.016 INFO:tasks.workunit.client.0.smithi027.stdout:8/498: dwrite d9/da/d2a/d5e/d72/d84/d5d/d6c/fa9 [0,4194304] 0 2024-04-26T23:01:32.020 INFO:tasks.workunit.client.0.smithi027.stdout:4/565: rename da/d40/d87 to da/d40/daf 0 2024-04-26T23:01:32.020 INFO:tasks.workunit.client.0.smithi027.stdout:9/469: dread d4/d10/d40/d58/d6e/f7c [0,4194304] 0 2024-04-26T23:01:32.020 INFO:tasks.workunit.client.0.smithi027.stdout:9/470: chown d4/l6 144442030 1 2024-04-26T23:01:32.020 INFO:tasks.workunit.client.0.smithi027.stdout:9/471: write d4/f24 [302070,13117] 0 2024-04-26T23:01:32.021 INFO:tasks.workunit.client.0.smithi027.stdout:1/558: dread d1/d26/d6b/d42/d79/f92 [0,4194304] 0 2024-04-26T23:01:32.023 INFO:tasks.workunit.client.0.smithi027.stdout:2/453: creat d2/d5/db/d1b/d1c/d25/d19/d44/f7f x:0 0 0 2024-04-26T23:01:32.027 INFO:tasks.workunit.client.0.smithi027.stdout:0/517: dwrite d0/d4/da/f19 [0,4194304] 0 2024-04-26T23:01:32.027 INFO:tasks.workunit.client.0.smithi027.stdout:0/518: chown d0/d4/c47 40253456 1 2024-04-26T23:01:32.027 INFO:tasks.workunit.client.0.smithi027.stdout:0/519: dread - d0/d4/da/d1a/f90 zero size 2024-04-26T23:01:32.027 INFO:tasks.workunit.client.0.smithi027.stdout:8/499: creat d9/da/d2a/d5e/d72/d84/d35/da7/fb7 x:0 0 0 2024-04-26T23:01:32.028 INFO:tasks.workunit.client.0.smithi027.stdout:0/520: symlink d0/d4/da/d1a/d59/d92/l99 0 2024-04-26T23:01:32.028 INFO:tasks.workunit.client.0.smithi027.stdout:0/521: write d0/d4/da/d15/d55/f96 [291160,25708] 0 2024-04-26T23:01:32.028 INFO:tasks.workunit.client.0.smithi027.stdout:8/500: truncate d9/de/d2e/f53 325493 0 2024-04-26T23:01:32.028 INFO:tasks.workunit.client.0.smithi027.stdout:0/522: symlink d0/d4/da/d15/d26/l9a 0 2024-04-26T23:01:32.035 INFO:tasks.workunit.client.0.smithi027.stdout:7/560: dwrite d0/d28/f77 [0,4194304] 0 2024-04-26T23:01:32.039 INFO:tasks.workunit.client.0.smithi027.stdout:6/448: dwrite d1/df/d43/d56/d6b/f8b [0,4194304] 0 2024-04-26T23:01:32.039 INFO:tasks.workunit.client.0.smithi027.stdout:6/449: write d1/de/f13 [219322,79268] 0 2024-04-26T23:01:32.042 INFO:tasks.workunit.client.0.smithi027.stdout:6/450: symlink d1/df/d43/d56/d6b/l9c 0 2024-04-26T23:01:32.070 INFO:tasks.workunit.client.0.smithi027.stdout:8/501: rename d9/da/d2a/d5e/d2b/d3b to d9/da/d2a/d5e/d72/d84/db8 0 2024-04-26T23:01:32.071 INFO:tasks.workunit.client.0.smithi027.stdout:9/472: dread d4/f24 [0,4194304] 0 2024-04-26T23:01:32.074 INFO:tasks.workunit.client.0.smithi027.stdout:7/561: rename d0/d28/d2b/d3a/d48/d56/d88/d73/d80/ca4 to d0/d28/d2b/d55/d59/d6d/cab 0 2024-04-26T23:01:32.074 INFO:tasks.workunit.client.0.smithi027.stdout:7/562: dread - d0/d28/d2b/d55/d69/d75/f89 zero size 2024-04-26T23:01:32.079 INFO:tasks.workunit.client.0.smithi027.stdout:3/548: dwrite d6/dc/d34/d2e/d9d/da7/dac/d74/da6/f2b [0,4194304] 0 2024-04-26T23:01:32.079 INFO:tasks.workunit.client.0.smithi027.stdout:9/473: rename d4/d10/f1a to d4/d77/da2/fa4 0 2024-04-26T23:01:32.079 INFO:tasks.workunit.client.0.smithi027.stdout:1/559: dwrite d1/f85 [0,4194304] 0 2024-04-26T23:01:32.079 INFO:tasks.workunit.client.0.smithi027.stdout:1/560: truncate d1/dc/fa7 437269 0 2024-04-26T23:01:32.079 INFO:tasks.workunit.client.0.smithi027.stdout:1/561: fdatasync d1/d5/f50 0 2024-04-26T23:01:32.081 INFO:tasks.workunit.client.0.smithi027.stdout:9/474: creat d4/d10/d40/d58/d75/d84/d86/fa5 x:0 0 0 2024-04-26T23:01:32.081 INFO:tasks.workunit.client.0.smithi027.stdout:9/475: write d4/d10/d40/d58/f96 [930522,29987] 0 2024-04-26T23:01:32.083 INFO:tasks.workunit.client.0.smithi027.stdout:9/476: unlink d4/d10/d67/f80 0 2024-04-26T23:01:32.090 INFO:tasks.workunit.client.0.smithi027.stdout:8/502: link d9/de/d2e/f53 d9/da/d2a/d5e/d92/fb9 0 2024-04-26T23:01:32.096 INFO:tasks.workunit.client.0.smithi027.stdout:3/549: rmdir d6/dc/d34/d2e 39 2024-04-26T23:01:32.096 INFO:tasks.workunit.client.0.smithi027.stdout:3/550: chown d6/dc/d34/d2e/c1f 11 1 2024-04-26T23:01:32.099 INFO:tasks.workunit.client.0.smithi027.stdout:7/563: dread d0/d28/d2b/d3a/f3f [0,4194304] 0 2024-04-26T23:01:32.101 INFO:tasks.workunit.client.0.smithi027.stdout:3/551: rmdir d6/dc/d34/d2e/d9d/da7/dac/d74/da6 39 2024-04-26T23:01:32.102 INFO:tasks.workunit.client.0.smithi027.stdout:3/552: write d6/dc/d34/d2e/d5f/d90/fa9 [336455,88050] 0 2024-04-26T23:01:32.116 INFO:tasks.workunit.client.0.smithi027.stdout:2/454: dwrite d2/d5/db/d1b/d1c/d25/d19/f45 [0,4194304] 0 2024-04-26T23:01:32.124 INFO:tasks.workunit.client.0.smithi027.stdout:4/566: dwrite da/dd/de/d14/d17/d80/dab/d37/f5a [0,4194304] 0 2024-04-26T23:01:32.126 INFO:tasks.workunit.client.0.smithi027.stdout:3/553: dread d6/dc/d34/d78/da1/fa8 [0,4194304] 0 2024-04-26T23:01:32.129 INFO:tasks.workunit.client.0.smithi027.stdout:4/567: mknod da/dd/de/d14/d17/d80/dab/cb0 0 2024-04-26T23:01:32.129 INFO:tasks.workunit.client.0.smithi027.stdout:4/568: readlink da/dd/d5f/la0 0 2024-04-26T23:01:32.130 INFO:tasks.workunit.client.0.smithi027.stdout:3/554: creat d6/dc/d34/d78/fbf x:0 0 0 2024-04-26T23:01:32.133 INFO:tasks.workunit.client.0.smithi027.stdout:3/555: getdents d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d52/d5a 0 2024-04-26T23:01:32.134 INFO:tasks.workunit.client.0.smithi027.stdout:4/569: truncate da/dd/de/d14/d17/d80/dab/d37/f5a 2147517 0 2024-04-26T23:01:32.134 INFO:tasks.workunit.client.0.smithi027.stdout:4/570: stat da/dd/de/d14/d21/d4a/f50 0 2024-04-26T23:01:32.137 INFO:tasks.workunit.client.0.smithi027.stdout:4/571: mknod da/dd/d5f/cb1 0 2024-04-26T23:01:32.138 INFO:tasks.workunit.client.0.smithi027.stdout:4/572: truncate da/dd/de/d14/d21/f75 4328287 0 2024-04-26T23:01:32.138 INFO:tasks.workunit.client.0.smithi027.stdout:3/556: mkdir d6/dc/d34/d2e/d9d/da7/dac/dc0 0 2024-04-26T23:01:32.142 INFO:tasks.workunit.client.0.smithi027.stdout:3/557: unlink d6/l23 0 2024-04-26T23:01:32.144 INFO:tasks.workunit.client.0.smithi027.stdout:3/558: creat d6/dc/d34/db3/fc1 x:0 0 0 2024-04-26T23:01:32.149 INFO:tasks.workunit.client.0.smithi027.stdout:6/451: dwrite d1/de/f28 [0,4194304] 0 2024-04-26T23:01:32.149 INFO:tasks.workunit.client.0.smithi027.stdout:7/564: dwrite d0/d28/d2b/d3a/f52 [0,4194304] 0 2024-04-26T23:01:32.164 INFO:tasks.workunit.client.0.smithi027.stdout:2/455: dwrite d2/d5/db/d1b/d1c/f61 [0,4194304] 0 2024-04-26T23:01:32.166 INFO:tasks.workunit.client.0.smithi027.stdout:4/573: dread da/dd/de/d14/d21/d4a/f6a [0,4194304] 0 2024-04-26T23:01:32.172 INFO:tasks.workunit.client.0.smithi027.stdout:1/562: dwrite d1/d26/d66/d73/d9e/f9f [0,4194304] 0 2024-04-26T23:01:32.179 INFO:tasks.workunit.client.0.smithi027.stdout:3/559: dread d6/dc/d34/d2e/f22 [0,4194304] 0 2024-04-26T23:01:32.181 INFO:tasks.workunit.client.0.smithi027.stdout:4/574: mknod da/dd/de/d14/d21/d67/d8a/cb2 0 2024-04-26T23:01:32.181 INFO:tasks.workunit.client.0.smithi027.stdout:1/563: rmdir d1/d26/d6b/d42/d70 39 2024-04-26T23:01:32.182 INFO:tasks.workunit.client.0.smithi027.stdout:3/560: creat d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/fc2 x:0 0 0 2024-04-26T23:01:32.183 INFO:tasks.workunit.client.0.smithi027.stdout:4/575: mkdir da/dd/de/d14/db3 0 2024-04-26T23:01:32.183 INFO:tasks.workunit.client.0.smithi027.stdout:4/576: chown da/dd/de/d14/d21/d41/l84 9854977 1 2024-04-26T23:01:32.185 INFO:tasks.workunit.client.0.smithi027.stdout:4/577: link da/dd/de/d14/d21/d67/l90 da/dd/lb4 0 2024-04-26T23:01:32.185 INFO:tasks.workunit.client.0.smithi027.stdout:4/578: chown da/dd/de/f94 5248 1 2024-04-26T23:01:32.193 INFO:tasks.workunit.client.0.smithi027.stdout:8/503: dread d9/da/d2a/d6b/f9c [0,4194304] 0 2024-04-26T23:01:32.198 INFO:tasks.workunit.client.0.smithi027.stdout:6/452: dwrite d1/d30/f36 [0,4194304] 0 2024-04-26T23:01:32.198 INFO:tasks.workunit.client.0.smithi027.stdout:6/453: write d1/df/d43/d56/d6b/f76 [3446245,129331] 0 2024-04-26T23:01:32.209 INFO:tasks.workunit.client.0.smithi027.stdout:6/454: rename d1/d30/c81 to d1/df/d3c/d9a/c9d 0 2024-04-26T23:01:32.212 INFO:tasks.workunit.client.0.smithi027.stdout:9/477: dwrite d4/d10/d40/d58/d75/d84/d86/fa5 [0,4194304] 0 2024-04-26T23:01:32.212 INFO:tasks.workunit.client.0.smithi027.stdout:9/478: fdatasync d4/d10/d40/d58/d75/f7d 0 2024-04-26T23:01:32.219 INFO:tasks.workunit.client.0.smithi027.stdout:9/479: symlink d4/d10/d25/la6 0 2024-04-26T23:01:32.219 INFO:tasks.workunit.client.0.smithi027.stdout:9/480: chown d4/l15 31 1 2024-04-26T23:01:32.220 INFO:tasks.workunit.client.0.smithi027.stdout:9/481: rmdir d4/d10/d40/d58/d75/d84/d86 39 2024-04-26T23:01:32.220 INFO:tasks.workunit.client.0.smithi027.stdout:9/482: chown d4/d10/d40/f66 3 1 2024-04-26T23:01:32.222 INFO:tasks.workunit.client.0.smithi027.stdout:9/483: mkdir d4/d17/d2e/d81/da7 0 2024-04-26T23:01:32.228 INFO:tasks.workunit.client.0.smithi027.stdout:0/523: sync 2024-04-26T23:01:32.228 INFO:tasks.workunit.client.0.smithi027.stdout:0/524: write d0/d4/da/d1a/f89 [807068,8760] 0 2024-04-26T23:01:32.228 INFO:tasks.workunit.client.0.smithi027.stdout:5/557: sync 2024-04-26T23:01:32.232 INFO:tasks.workunit.client.0.smithi027.stdout:0/525: mkdir d0/d4/d9b 0 2024-04-26T23:01:32.238 INFO:tasks.workunit.client.0.smithi027.stdout:2/456: dwrite d2/d5/db/d1b/d1c/d25/d19/d23/f64 [0,4194304] 0 2024-04-26T23:01:32.238 INFO:tasks.workunit.client.0.smithi027.stdout:6/455: write d1/f69 [3704274,45535] 0 2024-04-26T23:01:32.238 INFO:tasks.workunit.client.0.smithi027.stdout:6/456: read d1/f69 [2505582,80094] 0 2024-04-26T23:01:32.238 INFO:tasks.workunit.client.0.smithi027.stdout:2/457: write d2/d5/db/d1b/d1c/d25/f4a [1206085,20080] 0 2024-04-26T23:01:32.238 INFO:tasks.workunit.client.0.smithi027.stdout:2/458: write d2/d5/db/d1b/d1c/d25/d19/f5b [691987,102770] 0 2024-04-26T23:01:32.240 INFO:tasks.workunit.client.0.smithi027.stdout:0/526: creat d0/d4/da/d1a/d59/f9c x:0 0 0 2024-04-26T23:01:32.240 INFO:tasks.workunit.client.0.smithi027.stdout:0/527: dread - d0/d4/da/d15/f76 zero size 2024-04-26T23:01:32.240 INFO:tasks.workunit.client.0.smithi027.stdout:1/564: dread d1/dc/f5b [0,4194304] 0 2024-04-26T23:01:32.240 INFO:tasks.workunit.client.0.smithi027.stdout:1/565: chown d1/d26/d86/f9c 312495 1 2024-04-26T23:01:32.240 INFO:tasks.workunit.client.0.smithi027.stdout:1/566: chown d1/f85 382 1 2024-04-26T23:01:32.244 INFO:tasks.workunit.client.0.smithi027.stdout:2/459: creat d2/d5/db/d1b/f80 x:0 0 0 2024-04-26T23:01:32.248 INFO:tasks.workunit.client.0.smithi027.stdout:1/567: creat d1/d26/d6b/fad x:0 0 0 2024-04-26T23:01:32.248 INFO:tasks.workunit.client.0.smithi027.stdout:1/568: chown d1/d26/d6b/d8e/c9d 134769 1 2024-04-26T23:01:32.249 INFO:tasks.workunit.client.0.smithi027.stdout:1/569: mkdir d1/d26/d66/d73/dae 0 2024-04-26T23:01:32.251 INFO:tasks.workunit.client.0.smithi027.stdout:7/565: dwrite d0/d6c/f71 [0,4194304] 0 2024-04-26T23:01:32.251 INFO:tasks.workunit.client.0.smithi027.stdout:1/570: rename d1/dc/c6a to d1/d26/d66/d73/dae/caf 0 2024-04-26T23:01:32.252 INFO:tasks.workunit.client.0.smithi027.stdout:7/566: creat d0/fac x:0 0 0 2024-04-26T23:01:32.252 INFO:tasks.workunit.client.0.smithi027.stdout:7/567: read - d0/d28/d2b/d3a/d48/d56/d88/f91 zero size 2024-04-26T23:01:32.254 INFO:tasks.workunit.client.0.smithi027.stdout:1/571: rmdir d1/d26/d66/d91 39 2024-04-26T23:01:32.254 INFO:tasks.workunit.client.0.smithi027.stdout:1/572: readlink d1/d26/l5f 0 2024-04-26T23:01:32.254 INFO:tasks.workunit.client.0.smithi027.stdout:7/568: creat d0/d28/d2b/d55/d59/d6d/fad x:0 0 0 2024-04-26T23:01:32.256 INFO:tasks.workunit.client.0.smithi027.stdout:6/457: fdatasync d1/df/fd 0 2024-04-26T23:01:32.256 INFO:tasks.workunit.client.0.smithi027.stdout:4/579: dwrite da/dd/de/f94 [0,4194304] 0 2024-04-26T23:01:32.257 INFO:tasks.workunit.client.0.smithi027.stdout:1/573: symlink d1/d26/d6b/d8e/lb0 0 2024-04-26T23:01:32.257 INFO:tasks.workunit.client.0.smithi027.stdout:1/574: chown d1/d26/d7d/l38 91758 1 2024-04-26T23:01:32.258 INFO:tasks.workunit.client.0.smithi027.stdout:7/569: symlink d0/d28/d2b/d3a/d48/d56/lae 0 2024-04-26T23:01:32.258 INFO:tasks.workunit.client.0.smithi027.stdout:7/570: fdatasync d0/d28/d2b/d3a/f52 0 2024-04-26T23:01:32.260 INFO:tasks.workunit.client.0.smithi027.stdout:4/580: creat da/dd/de/d79/fb5 x:0 0 0 2024-04-26T23:01:32.260 INFO:tasks.workunit.client.0.smithi027.stdout:4/581: fdatasync da/dd/f5d 0 2024-04-26T23:01:32.264 INFO:tasks.workunit.client.0.smithi027.stdout:1/575: truncate d1/f1f 2063696 0 2024-04-26T23:01:32.264 INFO:tasks.workunit.client.0.smithi027.stdout:7/571: link d0/c9e d0/d28/d2b/d55/d59/d6d/d82/caf 0 2024-04-26T23:01:32.264 INFO:tasks.workunit.client.0.smithi027.stdout:3/561: dwrite d6/dc/d2f/d7e/f6e [0,4194304] 0 2024-04-26T23:01:32.264 INFO:tasks.workunit.client.0.smithi027.stdout:4/582: rename da/dd/f95 to da/dd/d5f/fb6 0 2024-04-26T23:01:32.266 INFO:tasks.workunit.client.0.smithi027.stdout:1/576: creat d1/dc/fb1 x:0 0 0 2024-04-26T23:01:32.267 INFO:tasks.workunit.client.0.smithi027.stdout:7/572: getdents d0/d28/d2b/d3a/d48/d56/d88/d73/d80 0 2024-04-26T23:01:32.269 INFO:tasks.workunit.client.0.smithi027.stdout:4/583: getdents da/dd/d5f/dac 0 2024-04-26T23:01:32.271 INFO:tasks.workunit.client.0.smithi027.stdout:1/577: rename d1/d5/l53 to d1/d26/d7d/lb2 0 2024-04-26T23:01:32.271 INFO:tasks.workunit.client.0.smithi027.stdout:8/504: dwrite d9/de/d2e/f49 [0,4194304] 0 2024-04-26T23:01:32.271 INFO:tasks.workunit.client.0.smithi027.stdout:8/505: fsync d9/da/d2a/d5e/d92/fb9 0 2024-04-26T23:01:32.272 INFO:tasks.workunit.client.0.smithi027.stdout:8/506: write d9/da/d2a/d5e/d46/f6f [4567646,84567] 0 2024-04-26T23:01:32.272 INFO:tasks.workunit.client.0.smithi027.stdout:8/507: read - d9/da/da1/faf zero size 2024-04-26T23:01:32.276 INFO:tasks.workunit.client.0.smithi027.stdout:4/584: creat da/dd/de/d14/d17/d80/fb7 x:0 0 0 2024-04-26T23:01:32.276 INFO:tasks.workunit.client.0.smithi027.stdout:4/585: stat da/dd/de/d14/d17/d80/dab/l27 0 2024-04-26T23:01:32.276 INFO:tasks.workunit.client.0.smithi027.stdout:1/578: symlink d1/d5/d21/lb3 0 2024-04-26T23:01:32.276 INFO:tasks.workunit.client.0.smithi027.stdout:7/573: symlink d0/d28/d2b/d3a/d48/d56/d88/d73/d80/d85/lb0 0 2024-04-26T23:01:32.279 INFO:tasks.workunit.client.0.smithi027.stdout:8/508: getdents d9/da/d2a/d5e/d46 0 2024-04-26T23:01:32.279 INFO:tasks.workunit.client.0.smithi027.stdout:4/586: mkdir da/dd/de/db8 0 2024-04-26T23:01:32.279 INFO:tasks.workunit.client.0.smithi027.stdout:8/509: chown d9/da/d33/c5b 392535567 1 2024-04-26T23:01:32.279 INFO:tasks.workunit.client.0.smithi027.stdout:1/579: link d1/d26/d6b/d42/d70/c72 d1/d26/d6b/d42/d70/cb4 0 2024-04-26T23:01:32.280 INFO:tasks.workunit.client.0.smithi027.stdout:1/580: rmdir d1/d26/d66/d91 39 2024-04-26T23:01:32.288 INFO:tasks.workunit.client.0.smithi027.stdout:8/510: mkdir d9/da/d2a/d5e/d9e/dba 0 2024-04-26T23:01:32.288 INFO:tasks.workunit.client.0.smithi027.stdout:8/511: chown d9/de/d2e/c1a 97 1 2024-04-26T23:01:32.288 INFO:tasks.workunit.client.0.smithi027.stdout:1/581: mkdir d1/d26/d66/d73/db5 0 2024-04-26T23:01:32.288 INFO:tasks.workunit.client.0.smithi027.stdout:8/512: creat d9/da/d2a/d5e/d72/d84/d5d/d8b/fbb x:0 0 0 2024-04-26T23:01:32.288 INFO:tasks.workunit.client.0.smithi027.stdout:8/513: stat d9/da/d2a/d5e/d72/d84/d5d/d8b/f9a 0 2024-04-26T23:01:32.293 INFO:tasks.workunit.client.0.smithi027.stdout:9/484: dwrite d4/d17/d2e/d46/da0/f57 [0,4194304] 0 2024-04-26T23:01:32.295 INFO:tasks.workunit.client.0.smithi027.stdout:9/485: creat d4/d17/d2e/d46/da0/d8d/fa8 x:0 0 0 2024-04-26T23:01:32.297 INFO:tasks.workunit.client.0.smithi027.stdout:9/486: symlink d4/d77/la9 0 2024-04-26T23:01:32.298 INFO:tasks.workunit.client.0.smithi027.stdout:9/487: symlink d4/d10/laa 0 2024-04-26T23:01:32.300 INFO:tasks.workunit.client.0.smithi027.stdout:9/488: creat d4/d10/d40/fab x:0 0 0 2024-04-26T23:01:32.300 INFO:tasks.workunit.client.0.smithi027.stdout:9/489: write d4/d77/f7a [408431,32205] 0 2024-04-26T23:01:32.322 INFO:tasks.workunit.client.0.smithi027.stdout:2/460: dwrite d2/d5/db/de/f68 [0,4194304] 0 2024-04-26T23:01:32.341 INFO:tasks.workunit.client.0.smithi027.stdout:5/558: dwrite d4/d1a/d4a/f76 [0,4194304] 0 2024-04-26T23:01:32.341 INFO:tasks.workunit.client.0.smithi027.stdout:5/559: chown d4/d24/d22/d29/d2a/d3b/d50/f70 7803512 1 2024-04-26T23:01:32.342 INFO:tasks.workunit.client.0.smithi027.stdout:1/582: dread d1/d26/d86/f9c [0,4194304] 0 2024-04-26T23:01:32.344 INFO:tasks.workunit.client.0.smithi027.stdout:1/583: mkdir d1/d26/d66/d73/d9e/db6 0 2024-04-26T23:01:32.347 INFO:tasks.workunit.client.0.smithi027.stdout:6/458: dwrite d1/df/d3c/d66/f8a [0,4194304] 0 2024-04-26T23:01:32.347 INFO:tasks.workunit.client.0.smithi027.stdout:1/584: rename d1/d26/d6b/d8e/lb0 to d1/d26/d6b/d8e/lb7 0 2024-04-26T23:01:32.349 INFO:tasks.workunit.client.0.smithi027.stdout:1/585: stat d1/d26/d66/c82 0 2024-04-26T23:01:32.351 INFO:tasks.workunit.client.0.smithi027.stdout:1/586: symlink d1/dc/lb8 0 2024-04-26T23:01:32.351 INFO:tasks.workunit.client.0.smithi027.stdout:3/562: dwrite d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/f8d [0,4194304] 0 2024-04-26T23:01:32.352 INFO:tasks.workunit.client.0.smithi027.stdout:3/563: write d6/dc/d88/f9a [527907,115144] 0 2024-04-26T23:01:32.352 INFO:tasks.workunit.client.0.smithi027.stdout:6/459: unlink d1/f64 0 2024-04-26T23:01:32.356 INFO:tasks.workunit.client.0.smithi027.stdout:1/587: rename d1/d26/d6b/fa0 to d1/d26/d66/dab/fb9 0 2024-04-26T23:01:32.356 INFO:tasks.workunit.client.0.smithi027.stdout:1/588: chown d1/d26/d36/c48 30878 1 2024-04-26T23:01:32.356 INFO:tasks.workunit.client.0.smithi027.stdout:9/490: dwrite d4/d17/d2e/d46/f7b [0,4194304] 0 2024-04-26T23:01:32.358 INFO:tasks.workunit.client.0.smithi027.stdout:3/564: rename d6/dc/d34/d2e/d9d/da3/fb7 to d6/dc/d34/d2e/d70/fc3 0 2024-04-26T23:01:32.358 INFO:tasks.workunit.client.0.smithi027.stdout:9/491: symlink d4/d17/d2e/d81/da7/lac 0 2024-04-26T23:01:32.360 INFO:tasks.workunit.client.0.smithi027.stdout:9/492: creat d4/d10/d40/fad x:0 0 0 2024-04-26T23:01:32.362 INFO:tasks.workunit.client.0.smithi027.stdout:9/493: link d4/l18 d4/d10/d25/lae 0 2024-04-26T23:01:32.364 INFO:tasks.workunit.client.0.smithi027.stdout:9/494: mkdir d4/d77/da2/daf 0 2024-04-26T23:01:32.367 INFO:tasks.workunit.client.0.smithi027.stdout:4/587: dwrite da/dd/de/d14/d21/d41/f52 [0,4194304] 0 2024-04-26T23:01:32.372 INFO:tasks.workunit.client.0.smithi027.stdout:3/565: dread d6/dc/d34/d2e/d5f/d90/fa9 [0,4194304] 0 2024-04-26T23:01:32.383 INFO:tasks.workunit.client.0.smithi027.stdout:7/574: dwrite d0/d28/d2b/d3a/d48/d56/d88/f16 [4194304,4194304] 0 2024-04-26T23:01:32.383 INFO:tasks.workunit.client.0.smithi027.stdout:7/575: truncate d0/d28/d2b/d3a/d48/d56/f9a 1002322 0 2024-04-26T23:01:32.383 INFO:tasks.workunit.client.0.smithi027.stdout:7/576: getdents d0/d28/d2b/d3a/d48/d56/d88/d73/d80 0 2024-04-26T23:01:32.397 INFO:tasks.workunit.client.0.smithi027.stdout:3/566: dread d6/f2d [0,4194304] 0 2024-04-26T23:01:32.401 INFO:tasks.workunit.client.0.smithi027.stdout:9/495: dwrite d4/d77/f7a [0,4194304] 0 2024-04-26T23:01:32.413 INFO:tasks.workunit.client.0.smithi027.stdout:5/560: dread d4/d24/d22/d29/d39/f9c [0,4194304] 0 2024-04-26T23:01:32.413 INFO:tasks.workunit.client.0.smithi027.stdout:5/561: fsync d4/d24/d22/d29/d39/f7f 0 2024-04-26T23:01:32.414 INFO:tasks.workunit.client.0.smithi027.stdout:2/461: dwrite d2/d5/db/d1b/f80 [0,4194304] 0 2024-04-26T23:01:32.414 INFO:tasks.workunit.client.0.smithi027.stdout:2/462: chown d2/d5/db/de/d21/d26/c3a 8904 1 2024-04-26T23:01:32.414 INFO:tasks.workunit.client.0.smithi027.stdout:2/463: chown d2/d5/db/d1b/d1c/d25 863053572 1 2024-04-26T23:01:32.417 INFO:tasks.workunit.client.0.smithi027.stdout:5/562: creat d4/d24/d22/d29/d39/d84/fb3 x:0 0 0 2024-04-26T23:01:32.417 INFO:tasks.workunit.client.0.smithi027.stdout:5/563: truncate d4/d1a/d4a/f76 5433331 0 2024-04-26T23:01:32.418 INFO:tasks.workunit.client.0.smithi027.stdout:5/564: truncate d4/d1a/f94 923071 0 2024-04-26T23:01:32.434 INFO:tasks.workunit.client.0.smithi027.stdout:1/589: dwrite d1/d26/f89 [0,4194304] 0 2024-04-26T23:01:32.434 INFO:tasks.workunit.client.0.smithi027.stdout:6/460: dwrite d1/df/d43/f45 [0,4194304] 0 2024-04-26T23:01:32.434 INFO:tasks.workunit.client.0.smithi027.stdout:6/461: truncate d1/f62 336765 0 2024-04-26T23:01:32.443 INFO:tasks.workunit.client.0.smithi027.stdout:4/588: dread da/dd/de/d14/f53 [0,4194304] 0 2024-04-26T23:01:32.443 INFO:tasks.workunit.client.0.smithi027.stdout:4/589: read - da/dd/de/d14/d21/d4a/f6d zero size 2024-04-26T23:01:32.444 INFO:tasks.workunit.client.0.smithi027.stdout:5/565: write d4/d24/d5f/dac/d40/d54/f73 [931175,19808] 0 2024-04-26T23:01:32.448 INFO:tasks.workunit.client.0.smithi027.stdout:7/577: dwrite d0/d28/f77 [0,4194304] 0 2024-04-26T23:01:32.448 INFO:tasks.workunit.client.0.smithi027.stdout:7/578: fdatasync d0/d28/d2b/d3a/d48/d56/d88/d73/f78 0 2024-04-26T23:01:32.448 INFO:tasks.workunit.client.0.smithi027.stdout:4/590: mknod da/dd/d5f/dac/cb9 0 2024-04-26T23:01:32.449 INFO:tasks.workunit.client.0.smithi027.stdout:7/579: creat d0/d28/d2b/d3a/fb1 x:0 0 0 2024-04-26T23:01:32.449 INFO:tasks.workunit.client.0.smithi027.stdout:9/496: dwrite d4/d10/d40/f5d [0,4194304] 0 2024-04-26T23:01:32.450 INFO:tasks.workunit.client.0.smithi027.stdout:3/567: dwrite d6/dc/d34/d2e/d9d/da7/dac/d74/da6/f2b [0,4194304] 0 2024-04-26T23:01:32.450 INFO:tasks.workunit.client.0.smithi027.stdout:9/497: write d4/f52 [1253897,103063] 0 2024-04-26T23:01:32.455 INFO:tasks.workunit.client.0.smithi027.stdout:9/498: write d4/d10/d40/f66 [1681310,59428] 0 2024-04-26T23:01:32.458 INFO:tasks.workunit.client.0.smithi027.stdout:7/580: creat d0/d28/d2b/d3a/d48/fb2 x:0 0 0 2024-04-26T23:01:32.458 INFO:tasks.workunit.client.0.smithi027.stdout:4/591: creat da/dd/de/d79/da8/fba x:0 0 0 2024-04-26T23:01:32.458 INFO:tasks.workunit.client.0.smithi027.stdout:7/581: write d0/d28/d2b/d3a/f3f [4930706,92136] 0 2024-04-26T23:01:32.458 INFO:tasks.workunit.client.0.smithi027.stdout:4/592: dread - da/dd/de/d14/d17/d80/fb7 zero size 2024-04-26T23:01:32.458 INFO:tasks.workunit.client.0.smithi027.stdout:4/593: dread - da/dd/de/d79/fb5 zero size 2024-04-26T23:01:32.461 INFO:tasks.workunit.client.0.smithi027.stdout:9/499: symlink d4/d10/d40/d58/d75/lb0 0 2024-04-26T23:01:32.462 INFO:tasks.workunit.client.0.smithi027.stdout:7/582: unlink d0/d28/d2b/d55/d59/la2 0 2024-04-26T23:01:32.463 INFO:tasks.workunit.client.0.smithi027.stdout:4/594: mknod da/dd/d5f/dac/cbb 0 2024-04-26T23:01:32.463 INFO:tasks.workunit.client.0.smithi027.stdout:4/595: readlink da/dd/de/d14/la7 0 2024-04-26T23:01:32.463 INFO:tasks.workunit.client.0.smithi027.stdout:4/596: stat da/dd/de/d14/d21/d4a/l4c 0 2024-04-26T23:01:32.463 INFO:tasks.workunit.client.0.smithi027.stdout:4/597: stat da/dd/de/d14/d17/d80/c82 0 2024-04-26T23:01:32.463 INFO:tasks.workunit.client.0.smithi027.stdout:4/598: stat da/dd/de/d14/d21/d67/da6 0 2024-04-26T23:01:32.464 INFO:tasks.workunit.client.0.smithi027.stdout:9/500: creat d4/d17/d2e/d46/da0/fb1 x:0 0 0 2024-04-26T23:01:32.464 INFO:tasks.workunit.client.0.smithi027.stdout:7/583: symlink d0/d28/d66/d67/lb3 0 2024-04-26T23:01:32.465 INFO:tasks.workunit.client.0.smithi027.stdout:7/584: stat d0/d28/d2b/d3a/d48/c51 0 2024-04-26T23:01:32.466 INFO:tasks.workunit.client.0.smithi027.stdout:4/599: rename da/dd/de/d14/d21/c89 to da/dd/de/d14/d17/d80/dab/cbc 0 2024-04-26T23:01:32.470 INFO:tasks.workunit.client.0.smithi027.stdout:9/501: symlink d4/d17/lb2 0 2024-04-26T23:01:32.471 INFO:tasks.workunit.client.0.smithi027.stdout:7/585: mkdir d0/d28/d2b/d55/d69/db4 0 2024-04-26T23:01:32.471 INFO:tasks.workunit.client.0.smithi027.stdout:9/502: write d4/d17/d2e/d46/da0/d72/f90 [429419,66884] 0 2024-04-26T23:01:32.471 INFO:tasks.workunit.client.0.smithi027.stdout:9/503: chown d4/d10/c16 3576755 1 2024-04-26T23:01:32.471 INFO:tasks.workunit.client.0.smithi027.stdout:4/600: getdents da/dd/de/d14/d17/d80 0 2024-04-26T23:01:32.472 INFO:tasks.workunit.client.0.smithi027.stdout:6/462: dwrite d1/df/f3a [0,4194304] 0 2024-04-26T23:01:32.473 INFO:tasks.workunit.client.0.smithi027.stdout:7/586: link d0/d28/d2b/d3a/d48/d56/d88/f29 d0/d28/d2b/d55/d59/d6d/fb5 0 2024-04-26T23:01:32.474 INFO:tasks.workunit.client.0.smithi027.stdout:6/463: mknod d1/de/d57/d8e/c9e 0 2024-04-26T23:01:32.476 INFO:tasks.workunit.client.0.smithi027.stdout:7/587: mkdir d0/d28/d2b/d3a/d48/d56/db6 0 2024-04-26T23:01:32.477 INFO:tasks.workunit.client.0.smithi027.stdout:0/528: sync 2024-04-26T23:01:32.477 INFO:tasks.workunit.client.0.smithi027.stdout:0/529: write d0/d4/da/d15/f5a [4544769,47622] 0 2024-04-26T23:01:32.478 INFO:tasks.workunit.client.0.smithi027.stdout:0/530: write d0/d4/da/d1a/f3b [6248955,74945] 0 2024-04-26T23:01:32.478 INFO:tasks.workunit.client.0.smithi027.stdout:7/588: rename d0/d28/f40 to d0/d28/d66/d67/fb7 0 2024-04-26T23:01:32.479 INFO:tasks.workunit.client.0.smithi027.stdout:7/589: write d0/d2f/f84 [1470952,93252] 0 2024-04-26T23:01:32.483 INFO:tasks.workunit.client.0.smithi027.stdout:0/531: rmdir d0/d4/da/d1a/d1e 39 2024-04-26T23:01:32.483 INFO:tasks.workunit.client.0.smithi027.stdout:7/590: unlink d0/d28/d2b/d3a/d90/c92 0 2024-04-26T23:01:32.488 INFO:tasks.workunit.client.0.smithi027.stdout:0/532: read d0/d4/da/d1a/d2a/f43 [351339,57564] 0 2024-04-26T23:01:32.492 INFO:tasks.workunit.client.0.smithi027.stdout:0/533: mkdir d0/df/d9d 0 2024-04-26T23:01:32.494 INFO:tasks.workunit.client.0.smithi027.stdout:0/534: unlink d0/d4/da/d1a/f89 0 2024-04-26T23:01:32.494 INFO:tasks.workunit.client.0.smithi027.stdout:5/566: dread d4/d24/f4c [0,4194304] 0 2024-04-26T23:01:32.495 INFO:tasks.workunit.client.0.smithi027.stdout:0/535: unlink d0/d4/da/d15/l49 0 2024-04-26T23:01:32.496 INFO:tasks.workunit.client.0.smithi027.stdout:3/568: dread d6/dc/d34/d2e/f31 [0,4194304] 0 2024-04-26T23:01:32.496 INFO:tasks.workunit.client.0.smithi027.stdout:7/591: mkdir d0/d28/d9f/db8 0 2024-04-26T23:01:32.498 INFO:tasks.workunit.client.0.smithi027.stdout:0/536: mknod d0/d4/da/d15/d26/c9e 0 2024-04-26T23:01:32.498 INFO:tasks.workunit.client.0.smithi027.stdout:5/567: unlink d4/d24/d22/d29/fb1 0 2024-04-26T23:01:32.499 INFO:tasks.workunit.client.0.smithi027.stdout:0/537: mkdir d0/df/d9d/d9f 0 2024-04-26T23:01:32.501 INFO:tasks.workunit.client.0.smithi027.stdout:5/568: getdents d4/d1a/d2b 0 2024-04-26T23:01:32.502 INFO:tasks.workunit.client.0.smithi027.stdout:2/464: dwrite d2/d5/db/d1b/d1c/f20 [4194304,4194304] 0 2024-04-26T23:01:32.509 INFO:tasks.workunit.client.0.smithi027.stdout:1/590: dwrite d1/d26/d7d/f30 [0,4194304] 0 2024-04-26T23:01:32.511 INFO:tasks.workunit.client.0.smithi027.stdout:2/465: symlink d2/d5/db/d1b/d1c/d25/d19/d44/l81 0 2024-04-26T23:01:32.511 INFO:tasks.workunit.client.0.smithi027.stdout:2/466: readlink d2/d5/db/d1b/d1c/l78 0 2024-04-26T23:01:32.521 INFO:tasks.workunit.client.0.smithi027.stdout:7/592: dread d0/d28/d2b/f34 [0,4194304] 0 2024-04-26T23:01:32.521 INFO:tasks.workunit.client.0.smithi027.stdout:1/591: creat d1/d26/d66/d73/d9e/fba x:0 0 0 2024-04-26T23:01:32.521 INFO:tasks.workunit.client.0.smithi027.stdout:1/592: fsync d1/fb 0 2024-04-26T23:01:32.528 INFO:tasks.workunit.client.0.smithi027.stdout:4/601: dwrite da/dd/de/d14/d21/d4a/f6a [0,4194304] 0 2024-04-26T23:01:32.536 INFO:tasks.workunit.client.0.smithi027.stdout:9/504: dwrite d4/d77/f7a [0,4194304] 0 2024-04-26T23:01:32.538 INFO:tasks.workunit.client.0.smithi027.stdout:4/602: creat da/dd/de/d14/db3/fbd x:0 0 0 2024-04-26T23:01:32.541 INFO:tasks.workunit.client.0.smithi027.stdout:9/505: creat d4/d10/d67/fb3 x:0 0 0 2024-04-26T23:01:32.542 INFO:tasks.workunit.client.0.smithi027.stdout:4/603: symlink da/dd/de/d14/d21/d67/da6/lbe 0 2024-04-26T23:01:32.542 INFO:tasks.workunit.client.0.smithi027.stdout:4/604: readlink da/dd/de/d14/d21/d41/l84 0 2024-04-26T23:01:32.545 INFO:tasks.workunit.client.0.smithi027.stdout:7/593: dread d0/d28/d2b/f4e [0,4194304] 0 2024-04-26T23:01:32.547 INFO:tasks.workunit.client.0.smithi027.stdout:9/506: mknod d4/d10/cb4 0 2024-04-26T23:01:32.549 INFO:tasks.workunit.client.0.smithi027.stdout:7/594: unlink d0/d28/f60 0 2024-04-26T23:01:32.549 INFO:tasks.workunit.client.0.smithi027.stdout:7/595: stat d0/d28/d2b/f5c 0 2024-04-26T23:01:32.549 INFO:tasks.workunit.client.0.smithi027.stdout:8/514: sync 2024-04-26T23:01:32.549 INFO:tasks.workunit.client.0.smithi027.stdout:7/596: read - d0/d3d/f54 zero size 2024-04-26T23:01:32.551 INFO:tasks.workunit.client.0.smithi027.stdout:9/507: rename d4/f8e to d4/d17/d2e/d46/da0/d8d/fb5 0 2024-04-26T23:01:32.563 INFO:tasks.workunit.client.0.smithi027.stdout:5/569: write d4/f64 [4098066,23025] 0 2024-04-26T23:01:32.564 INFO:tasks.workunit.client.0.smithi027.stdout:6/464: dwrite d1/f69 [0,4194304] 0 2024-04-26T23:01:32.565 INFO:tasks.workunit.client.0.smithi027.stdout:3/569: dwrite d6/dc/f72 [0,4194304] 0 2024-04-26T23:01:32.567 INFO:tasks.workunit.client.0.smithi027.stdout:5/570: symlink d4/d9f/lb4 0 2024-04-26T23:01:32.569 INFO:tasks.workunit.client.0.smithi027.stdout:6/465: creat d1/df/d49/d7f/f9f x:0 0 0 2024-04-26T23:01:32.570 INFO:tasks.workunit.client.0.smithi027.stdout:5/571: mknod d4/d24/cb5 0 2024-04-26T23:01:32.570 INFO:tasks.workunit.client.0.smithi027.stdout:5/572: write d4/d1a/d4a/f76 [904569,106266] 0 2024-04-26T23:01:32.571 INFO:tasks.workunit.client.0.smithi027.stdout:2/467: dwrite d2/d5/db/d1b/d1c/f20 [0,4194304] 0 2024-04-26T23:01:32.575 INFO:tasks.workunit.client.0.smithi027.stdout:2/468: rename d2/d5/db/d1b/d1c/d25/d7a to d2/d5/db/d1b/d1c/d25/d19/d23/d82 0 2024-04-26T23:01:32.575 INFO:tasks.workunit.client.0.smithi027.stdout:2/469: rename d2/d5 to d2/d5/db/d1b/d1c/d25/d19/d36/d39/d83 22 2024-04-26T23:01:32.580 INFO:tasks.workunit.client.0.smithi027.stdout:5/573: rmdir d4/d24/d55 39 2024-04-26T23:01:32.580 INFO:tasks.workunit.client.0.smithi027.stdout:3/570: link d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d59/f97 d6/dc/d34/d2e/d5f/fc4 0 2024-04-26T23:01:32.580 INFO:tasks.workunit.client.0.smithi027.stdout:3/571: truncate d6/dc/d34/d2e/d50/f8a 1509584 0 2024-04-26T23:01:32.580 INFO:tasks.workunit.client.0.smithi027.stdout:3/572: fsync d6/dc/fd 0 2024-04-26T23:01:32.580 INFO:tasks.workunit.client.0.smithi027.stdout:3/573: fdatasync d6/dc/d34/d2e/f45 0 2024-04-26T23:01:32.583 INFO:tasks.workunit.client.0.smithi027.stdout:5/574: symlink d4/d1a/d2b/daa/lb6 0 2024-04-26T23:01:32.587 INFO:tasks.workunit.client.0.smithi027.stdout:1/593: dwrite d1/d26/d66/d73/f7c [0,4194304] 0 2024-04-26T23:01:32.592 INFO:tasks.workunit.client.0.smithi027.stdout:1/594: rename d1/dc/f11 to d1/d26/d66/dab/fbb 0 2024-04-26T23:01:32.603 INFO:tasks.workunit.client.0.smithi027.stdout:1/595: dread d1/f94 [0,4194304] 0 2024-04-26T23:01:32.605 INFO:tasks.workunit.client.0.smithi027.stdout:1/596: creat d1/d2a/fbc x:0 0 0 2024-04-26T23:01:32.605 INFO:tasks.workunit.client.0.smithi027.stdout:1/597: chown d1/d26/d86/c59 0 1 2024-04-26T23:01:32.605 INFO:tasks.workunit.client.0.smithi027.stdout:1/598: dread - d1/d2a/fac zero size 2024-04-26T23:01:32.617 INFO:tasks.workunit.client.0.smithi027.stdout:5/575: dread f1 [4194304,4194304] 0 2024-04-26T23:01:32.617 INFO:tasks.workunit.client.0.smithi027.stdout:5/576: chown d4/d24/d22/dab/d16/d4b 1363672 1 2024-04-26T23:01:32.623 INFO:tasks.workunit.client.0.smithi027.stdout:2/470: dwrite d2/d5/db/de/f42 [0,4194304] 0 2024-04-26T23:01:32.623 INFO:tasks.workunit.client.0.smithi027.stdout:2/471: write d2/d5/fa [6007281,128918] 0 2024-04-26T23:01:32.625 INFO:tasks.workunit.client.0.smithi027.stdout:9/508: dwrite d4/d17/f93 [0,4194304] 0 2024-04-26T23:01:32.625 INFO:tasks.workunit.client.0.smithi027.stdout:2/472: link d2/d5/db/d1b/d1c/d25/d19/d44/l81 d2/d5/db/de/l84 0 2024-04-26T23:01:32.628 INFO:tasks.workunit.client.0.smithi027.stdout:4/605: dwrite da/dd/de/f1c [0,4194304] 0 2024-04-26T23:01:32.628 INFO:tasks.workunit.client.0.smithi027.stdout:8/515: write d9/da/d2a/d5e/d72/d84/f3f [4148858,104053] 0 2024-04-26T23:01:32.628 INFO:tasks.workunit.client.0.smithi027.stdout:8/516: readlink d9/da/d2a/d5e/d72/d84/d4f/laa 0 2024-04-26T23:01:32.628 INFO:tasks.workunit.client.0.smithi027.stdout:8/517: readlink d9/da/d2a/lb1 0 2024-04-26T23:01:32.630 INFO:tasks.workunit.client.0.smithi027.stdout:9/509: rmdir d4/d10/d40/d58/d75/d84/d86/da1 0 2024-04-26T23:01:32.630 INFO:tasks.workunit.client.0.smithi027.stdout:4/606: creat da/dd/de/d79/da8/fbf x:0 0 0 2024-04-26T23:01:32.632 INFO:tasks.workunit.client.0.smithi027.stdout:4/607: mknod da/cc0 0 2024-04-26T23:01:32.635 INFO:tasks.workunit.client.0.smithi027.stdout:4/608: rename da/dd/de/d14/d21/d41/l42 to da/dd/de/d14/d21/d67/d8a/lc1 0 2024-04-26T23:01:32.635 INFO:tasks.workunit.client.0.smithi027.stdout:4/609: fdatasync da/dd/de/d14/d17/f81 0 2024-04-26T23:01:32.641 INFO:tasks.workunit.client.0.smithi027.stdout:6/466: dwrite d1/de/f13 [0,4194304] 0 2024-04-26T23:01:32.645 INFO:tasks.workunit.client.0.smithi027.stdout:6/467: rmdir d1/df/d49 39 2024-04-26T23:01:32.647 INFO:tasks.workunit.client.0.smithi027.stdout:6/468: rename d1/d30/l37 to d1/df/d3c/la0 0 2024-04-26T23:01:32.647 INFO:tasks.workunit.client.0.smithi027.stdout:6/469: read d1/de/f1e [1655936,119180] 0 2024-04-26T23:01:32.647 INFO:tasks.workunit.client.0.smithi027.stdout:6/470: write d1/de/f42 [1892511,18370] 0 2024-04-26T23:01:32.647 INFO:tasks.workunit.client.0.smithi027.stdout:6/471: truncate d1/df/d43/f55 2761050 0 2024-04-26T23:01:32.652 INFO:tasks.workunit.client.0.smithi027.stdout:2/473: write d2/d5/db/d1b/d1c/d25/d19/f45 [702025,89111] 0 2024-04-26T23:01:32.654 INFO:tasks.workunit.client.0.smithi027.stdout:2/474: symlink d2/d5/db/de/l85 0 2024-04-26T23:01:32.669 INFO:tasks.workunit.client.0.smithi027.stdout:8/518: dwrite d9/da/d2a/d5e/d86/f8c [0,4194304] 0 2024-04-26T23:01:32.669 INFO:tasks.workunit.client.0.smithi027.stdout:8/519: dread - d9/da/d2a/d5e/d72/d84/d5d/d8b/fbb zero size 2024-04-26T23:01:32.669 INFO:tasks.workunit.client.0.smithi027.stdout:8/520: write d9/da/d2a/d5e/d72/d84/d5d/f7d [1598206,43201] 0 2024-04-26T23:01:32.670 INFO:tasks.workunit.client.0.smithi027.stdout:8/521: write d9/da/d2a/d5e/d92/fae [358190,19167] 0 2024-04-26T23:01:32.672 INFO:tasks.workunit.client.0.smithi027.stdout:8/522: rmdir d9/da/d2a/d5e/d72/d84/d4f 39 2024-04-26T23:01:32.676 INFO:tasks.workunit.client.0.smithi027.stdout:6/472: write d1/df/f8 [1701938,74190] 0 2024-04-26T23:01:32.678 INFO:tasks.workunit.client.0.smithi027.stdout:6/473: mkdir d1/df/d49/d7f/da1 0 2024-04-26T23:01:32.693 INFO:tasks.workunit.client.0.smithi027.stdout:5/577: dwrite d4/d24/d22/d29/d39/f6d [0,4194304] 0 2024-04-26T23:01:32.693 INFO:tasks.workunit.client.0.smithi027.stdout:5/578: stat d4/d24/d22/d29/cb0 0 2024-04-26T23:01:32.693 INFO:tasks.workunit.client.0.smithi027.stdout:9/510: dread d4/d17/d2e/d46/f55 [0,4194304] 0 2024-04-26T23:01:32.693 INFO:tasks.workunit.client.0.smithi027.stdout:1/599: dwrite d1/d2a/f62 [0,4194304] 0 2024-04-26T23:01:32.693 INFO:tasks.workunit.client.0.smithi027.stdout:1/600: fdatasync d1/d5/d40/f52 0 2024-04-26T23:01:32.696 INFO:tasks.workunit.client.0.smithi027.stdout:5/579: getdents d4/d24/d22/d29/d39/d84 0 2024-04-26T23:01:32.700 INFO:tasks.workunit.client.0.smithi027.stdout:9/511: rename d4/d17/d2e/d46/f6c to d4/d10/d67/fb6 0 2024-04-26T23:01:32.701 INFO:tasks.workunit.client.0.smithi027.stdout:1/601: creat d1/d26/d66/d73/fbd x:0 0 0 2024-04-26T23:01:32.704 INFO:tasks.workunit.client.0.smithi027.stdout:5/580: creat d4/d24/d22/d29/d2a/d3b/fb7 x:0 0 0 2024-04-26T23:01:32.704 INFO:tasks.workunit.client.0.smithi027.stdout:6/474: dwrite d1/df/f3a [0,4194304] 0 2024-04-26T23:01:32.707 INFO:tasks.workunit.client.0.smithi027.stdout:1/602: getdents d1/d26/d6b/d8e 0 2024-04-26T23:01:32.707 INFO:tasks.workunit.client.0.smithi027.stdout:9/512: rename d4/f7f to d4/d17/d2e/fb7 0 2024-04-26T23:01:32.707 INFO:tasks.workunit.client.0.smithi027.stdout:9/513: chown d4/d10/d40/d58/d6e/l73 5 1 2024-04-26T23:01:32.708 INFO:tasks.workunit.client.0.smithi027.stdout:1/603: symlink d1/d26/d66/d73/d9e/db6/lbe 0 2024-04-26T23:01:32.708 INFO:tasks.workunit.client.0.smithi027.stdout:9/514: truncate d4/d10/d40/f64 3344553 0 2024-04-26T23:01:32.708 INFO:tasks.workunit.client.0.smithi027.stdout:9/515: chown d4/d77/da2 51741587 1 2024-04-26T23:01:32.708 INFO:tasks.workunit.client.0.smithi027.stdout:9/516: fdatasync d4/d10/d40/d58/d75/f7d 0 2024-04-26T23:01:32.710 INFO:tasks.workunit.client.0.smithi027.stdout:1/604: rmdir d1/d26/d6b/d42 39 2024-04-26T23:01:32.711 INFO:tasks.workunit.client.0.smithi027.stdout:1/605: mkdir d1/d26/d66/d73/dae/dbf 0 2024-04-26T23:01:32.725 INFO:tasks.workunit.client.0.smithi027.stdout:2/475: dread d2/d5/db/d1b/d1c/d25/d6a/f4c [0,4194304] 0 2024-04-26T23:01:32.725 INFO:tasks.workunit.client.0.smithi027.stdout:2/476: chown d2/d5/db/d1b/d1c/d25/d6a/c3e 72272 1 2024-04-26T23:01:32.725 INFO:tasks.workunit.client.0.smithi027.stdout:6/475: dread d1/df/f31 [0,4194304] 0 2024-04-26T23:01:32.725 INFO:tasks.workunit.client.0.smithi027.stdout:6/476: write d1/de/f1e [1786702,126241] 0 2024-04-26T23:01:32.725 INFO:tasks.workunit.client.0.smithi027.stdout:4/610: dwrite da/dd/d5f/fb6 [0,4194304] 0 2024-04-26T23:01:32.728 INFO:tasks.workunit.client.0.smithi027.stdout:6/477: symlink d1/df/d49/la2 0 2024-04-26T23:01:32.746 INFO:tasks.workunit.client.0.smithi027.stdout:2/477: dread d2/d5/db/de/d21/f51 [0,4194304] 0 2024-04-26T23:01:32.761 INFO:tasks.workunit.client.0.smithi027.stdout:6/478: dread d1/f69 [0,4194304] 0 2024-04-26T23:01:32.761 INFO:tasks.workunit.client.0.smithi027.stdout:8/523: dwrite d9/de/d2e/f1f [0,4194304] 0 2024-04-26T23:01:32.761 INFO:tasks.workunit.client.0.smithi027.stdout:8/524: write d9/de/f18 [730479,38554] 0 2024-04-26T23:01:32.761 INFO:tasks.workunit.client.0.smithi027.stdout:8/525: truncate d9/da/d2a/d5e/d72/d84/db8/f5f 542891 0 2024-04-26T23:01:32.764 INFO:tasks.workunit.client.0.smithi027.stdout:6/479: truncate d1/f72 599717 0 2024-04-26T23:01:32.764 INFO:tasks.workunit.client.0.smithi027.stdout:6/480: dread - d1/d30/d6c/f97 zero size 2024-04-26T23:01:32.766 INFO:tasks.workunit.client.0.smithi027.stdout:8/526: link d9/da/d2a/d6b/l98 d9/da/d2a/d5e/d72/d84/d4f/lbc 0 2024-04-26T23:01:32.766 INFO:tasks.workunit.client.0.smithi027.stdout:6/481: symlink d1/d30/d6c/la3 0 2024-04-26T23:01:32.767 INFO:tasks.workunit.client.0.smithi027.stdout:8/527: rmdir d9/da/d33/d71 39 2024-04-26T23:01:32.769 INFO:tasks.workunit.client.0.smithi027.stdout:4/611: dwrite da/dd/de/d14/d21/d41/f52 [0,4194304] 0 2024-04-26T23:01:32.770 INFO:tasks.workunit.client.0.smithi027.stdout:6/482: link d1/df/d43/d56/d6b/c8f d1/d30/d40/ca4 0 2024-04-26T23:01:32.770 INFO:tasks.workunit.client.0.smithi027.stdout:6/483: stat d1/df/d49/l96 0 2024-04-26T23:01:32.773 INFO:tasks.workunit.client.0.smithi027.stdout:6/484: truncate d1/de/f1b 951502 0 2024-04-26T23:01:32.777 INFO:tasks.workunit.client.0.smithi027.stdout:6/485: creat d1/df/fa5 x:0 0 0 2024-04-26T23:01:32.779 INFO:tasks.workunit.client.0.smithi027.stdout:6/486: link d1/l95 d1/df/d3c/d66/la6 0 2024-04-26T23:01:32.788 INFO:tasks.workunit.client.0.smithi027.stdout:2/478: write d2/d5/db/d1b/d1c/d25/d6a/f4c [2935341,52915] 0 2024-04-26T23:01:32.788 INFO:tasks.workunit.client.0.smithi027.stdout:2/479: stat d2/d5/db/de/d21/d26/c3a 0 2024-04-26T23:01:32.804 INFO:tasks.workunit.client.0.smithi027.stdout:5/581: dwrite d4/d1a/f94 [0,4194304] 0 2024-04-26T23:01:32.805 INFO:tasks.workunit.client.0.smithi027.stdout:6/487: dread d1/df/d49/f4d [0,4194304] 0 2024-04-26T23:01:32.811 INFO:tasks.workunit.client.0.smithi027.stdout:6/488: creat d1/d30/fa7 x:0 0 0 2024-04-26T23:01:32.812 INFO:tasks.workunit.client.0.smithi027.stdout:6/489: truncate d1/f2 3568084 0 2024-04-26T23:01:32.814 INFO:tasks.workunit.client.0.smithi027.stdout:6/490: unlink d1/d30/d6c/f7a 0 2024-04-26T23:01:32.814 INFO:tasks.workunit.client.0.smithi027.stdout:6/491: write d1/df/f8 [4633621,98919] 0 2024-04-26T23:01:32.816 INFO:tasks.workunit.client.0.smithi027.stdout:2/480: dread d2/d5/db/d1b/d1c/d25/d6a/f4c [0,4194304] 0 2024-04-26T23:01:32.816 INFO:tasks.workunit.client.0.smithi027.stdout:6/492: rename d1/df/d49/d7f to d1/df/d43/da8 0 2024-04-26T23:01:32.817 INFO:tasks.workunit.client.0.smithi027.stdout:6/493: fdatasync d1/de/f28 0 2024-04-26T23:01:32.817 INFO:tasks.workunit.client.0.smithi027.stdout:6/494: fdatasync d1/df/f2e 0 2024-04-26T23:01:32.817 INFO:tasks.workunit.client.0.smithi027.stdout:6/495: chown d1/df/f31 1977088209 1 2024-04-26T23:01:32.817 INFO:tasks.workunit.client.0.smithi027.stdout:1/606: dwrite d1/d26/d66/d73/d9e/fba [0,4194304] 0 2024-04-26T23:01:32.818 INFO:tasks.workunit.client.0.smithi027.stdout:4/612: dwrite da/dd/de/d14/d17/d80/dab/f26 [0,4194304] 0 2024-04-26T23:01:32.818 INFO:tasks.workunit.client.0.smithi027.stdout:4/613: chown da/l63 3901667 1 2024-04-26T23:01:32.818 INFO:tasks.workunit.client.0.smithi027.stdout:4/614: chown da/cc0 0 1 2024-04-26T23:01:32.819 INFO:tasks.workunit.client.0.smithi027.stdout:5/582: dread d4/d24/d22/d29/f58 [0,4194304] 0 2024-04-26T23:01:32.819 INFO:tasks.workunit.client.0.smithi027.stdout:8/528: dwrite d9/da/f13 [0,4194304] 0 2024-04-26T23:01:32.819 INFO:tasks.workunit.client.0.smithi027.stdout:8/529: truncate d9/da/d2a/f89 156333 0 2024-04-26T23:01:32.823 INFO:tasks.workunit.client.0.smithi027.stdout:2/481: symlink d2/d5/db/d1b/d1c/d25/d19/d44/l86 0 2024-04-26T23:01:32.823 INFO:tasks.workunit.client.0.smithi027.stdout:2/482: readlink d2/d5/db/de/l84 0 2024-04-26T23:01:32.824 INFO:tasks.workunit.client.0.smithi027.stdout:6/496: getdents d1/d30 0 2024-04-26T23:01:32.825 INFO:tasks.workunit.client.0.smithi027.stdout:1/607: mknod d1/d26/d66/d73/d9e/cc0 0 2024-04-26T23:01:32.825 INFO:tasks.workunit.client.0.smithi027.stdout:1/608: chown d1/d2a 24 1 2024-04-26T23:01:32.825 INFO:tasks.workunit.client.0.smithi027.stdout:5/583: write d4/d24/d5f/dac/d40/f97 [327315,83792] 0 2024-04-26T23:01:32.829 INFO:tasks.workunit.client.0.smithi027.stdout:6/497: rmdir d1/d6a/d85 0 2024-04-26T23:01:32.829 INFO:tasks.workunit.client.0.smithi027.stdout:5/584: mkdir d4/d1a/d2b/db8 0 2024-04-26T23:01:32.831 INFO:tasks.workunit.client.0.smithi027.stdout:1/609: creat d1/d26/d86/fc1 x:0 0 0 2024-04-26T23:01:32.831 INFO:tasks.workunit.client.0.smithi027.stdout:1/610: write d1/d26/d66/d73/fbd [296892,69571] 0 2024-04-26T23:01:32.831 INFO:tasks.workunit.client.0.smithi027.stdout:1/611: chown d1/d26/d66/d73/d9e/fba 175 1 2024-04-26T23:01:32.832 INFO:tasks.workunit.client.0.smithi027.stdout:2/483: mknod d2/d5/db/c87 0 2024-04-26T23:01:32.832 INFO:tasks.workunit.client.0.smithi027.stdout:7/597: sync 2024-04-26T23:01:32.832 INFO:tasks.workunit.client.0.smithi027.stdout:0/538: sync 2024-04-26T23:01:32.832 INFO:tasks.workunit.client.0.smithi027.stdout:0/539: truncate d0/d4/da/d15/d26/f7c 9418700 0 2024-04-26T23:01:32.833 INFO:tasks.workunit.client.0.smithi027.stdout:6/498: rename d1/df/l3d to d1/df/d43/d56/la9 0 2024-04-26T23:01:32.834 INFO:tasks.workunit.client.0.smithi027.stdout:1/612: mknod d1/d26/d6b/d8e/cc2 0 2024-04-26T23:01:32.835 INFO:tasks.workunit.client.0.smithi027.stdout:3/574: sync 2024-04-26T23:01:32.836 INFO:tasks.workunit.client.0.smithi027.stdout:2/484: mkdir d2/d5/db/de/d21/d26/d88 0 2024-04-26T23:01:32.836 INFO:tasks.workunit.client.0.smithi027.stdout:3/575: creat d6/dc/d34/d2e/d70/d71/fc5 x:0 0 0 2024-04-26T23:01:32.837 INFO:tasks.workunit.client.0.smithi027.stdout:3/576: write d6/fbc [245921,75873] 0 2024-04-26T23:01:32.839 INFO:tasks.workunit.client.0.smithi027.stdout:6/499: link d1/f2b d1/df/d43/faa 0 2024-04-26T23:01:32.839 INFO:tasks.workunit.client.0.smithi027.stdout:6/500: fdatasync d1/df/d43/f45 0 2024-04-26T23:01:32.839 INFO:tasks.workunit.client.0.smithi027.stdout:8/530: dread d9/da/f13 [4194304,4194304] 0 2024-04-26T23:01:32.841 INFO:tasks.workunit.client.0.smithi027.stdout:8/531: creat d9/da/d2a/fbd x:0 0 0 2024-04-26T23:01:32.841 INFO:tasks.workunit.client.0.smithi027.stdout:8/532: chown d9/de/d22/c36 9 1 2024-04-26T23:01:32.841 INFO:tasks.workunit.client.0.smithi027.stdout:8/533: chown d9/da/d2a/d6b/f9c 8608104 1 2024-04-26T23:01:32.841 INFO:tasks.workunit.client.0.smithi027.stdout:8/534: chown d9/da/d2a/d5e/d2b/f76 43 1 2024-04-26T23:01:32.841 INFO:tasks.workunit.client.0.smithi027.stdout:3/577: creat d6/dc/d34/db3/db4/fc6 x:0 0 0 2024-04-26T23:01:32.842 INFO:tasks.workunit.client.0.smithi027.stdout:7/598: rename d0/d28/d2b/d55/f6f to d0/d28/d2b/d55/d59/d6d/d82/fb9 0 2024-04-26T23:01:32.846 INFO:tasks.workunit.client.0.smithi027.stdout:7/599: write d0/d28/d2b/d3a/fb1 [528700,128391] 0 2024-04-26T23:01:32.846 INFO:tasks.workunit.client.0.smithi027.stdout:2/485: creat d2/f89 x:0 0 0 2024-04-26T23:01:32.857 INFO:tasks.workunit.client.0.smithi027.stdout:8/535: rename d9/de/fb4 to d9/da/d2a/d5e/d72/d84/db8/fbe 0 2024-04-26T23:01:32.857 INFO:tasks.workunit.client.0.smithi027.stdout:8/536: write d9/da/d2a/d5e/d72/d84/d5d/d8b/f9a [1433660,1068] 0 2024-04-26T23:01:32.864 INFO:tasks.workunit.client.0.smithi027.stdout:8/537: unlink d9/l23 0 2024-04-26T23:01:32.875 INFO:tasks.workunit.client.0.smithi027.stdout:8/538: read d9/de/f18 [630021,37497] 0 2024-04-26T23:01:32.882 INFO:tasks.workunit.client.0.smithi027.stdout:1/613: dwrite d1/d26/d66/f77 [0,4194304] 0 2024-04-26T23:01:32.884 INFO:tasks.workunit.client.0.smithi027.stdout:1/614: read d1/d26/d36/d56/f5e [3085626,49562] 0 2024-04-26T23:01:32.888 INFO:tasks.workunit.client.0.smithi027.stdout:1/615: unlink d1/f85 0 2024-04-26T23:01:32.892 INFO:tasks.workunit.client.0.smithi027.stdout:1/616: unlink d1/d5/f50 0 2024-04-26T23:01:32.893 INFO:tasks.workunit.client.0.smithi027.stdout:1/617: creat d1/d26/d66/dab/fc3 x:0 0 0 2024-04-26T23:01:32.894 INFO:tasks.workunit.client.0.smithi027.stdout:7/600: dwrite d0/d28/f2c [0,4194304] 0 2024-04-26T23:01:32.894 INFO:tasks.workunit.client.0.smithi027.stdout:7/601: chown d0/d28/d2b/d3a/d48 4148 1 2024-04-26T23:01:32.895 INFO:tasks.workunit.client.0.smithi027.stdout:7/602: chown d0/d28/d2b/d55/d59/d6d/d82/fb9 74 1 2024-04-26T23:01:32.895 INFO:tasks.workunit.client.0.smithi027.stdout:1/618: creat d1/dc/fc4 x:0 0 0 2024-04-26T23:01:32.895 INFO:tasks.workunit.client.0.smithi027.stdout:7/603: write d0/f2a [4196720,83049] 0 2024-04-26T23:01:32.896 INFO:tasks.workunit.client.0.smithi027.stdout:1/619: unlink d1/dc/f5b 0 2024-04-26T23:01:32.897 INFO:tasks.workunit.client.0.smithi027.stdout:7/604: unlink d0/d28/d2b/d3a/d48/d56/d88/f16 0 2024-04-26T23:01:32.898 INFO:tasks.workunit.client.0.smithi027.stdout:1/620: getdents d1/d2a 0 2024-04-26T23:01:32.898 INFO:tasks.workunit.client.0.smithi027.stdout:1/621: readlink d1/l45 0 2024-04-26T23:01:32.899 INFO:tasks.workunit.client.0.smithi027.stdout:7/605: mkdir d0/d3d/dba 0 2024-04-26T23:01:32.900 INFO:tasks.workunit.client.0.smithi027.stdout:1/622: unlink d1/d26/d36/c48 0 2024-04-26T23:01:32.900 INFO:tasks.workunit.client.0.smithi027.stdout:1/623: write d1/d5/f2f [1593997,35688] 0 2024-04-26T23:01:32.904 INFO:tasks.workunit.client.0.smithi027.stdout:7/606: symlink d0/d28/d2b/d55/d59/d6d/lbb 0 2024-04-26T23:01:32.906 INFO:tasks.workunit.client.0.smithi027.stdout:6/501: write d1/df/d43/f45 [4063743,125094] 0 2024-04-26T23:01:32.906 INFO:tasks.workunit.client.0.smithi027.stdout:0/540: dread d0/d4/f24 [0,4194304] 0 2024-04-26T23:01:32.907 INFO:tasks.workunit.client.0.smithi027.stdout:4/615: dwrite da/dd/de/d79/f7b [0,4194304] 0 2024-04-26T23:01:32.910 INFO:tasks.workunit.client.0.smithi027.stdout:6/502: link d1/de/l46 d1/df/d49/lab 0 2024-04-26T23:01:32.912 INFO:tasks.workunit.client.0.smithi027.stdout:4/616: creat da/fc2 x:0 0 0 2024-04-26T23:01:32.912 INFO:tasks.workunit.client.0.smithi027.stdout:6/503: creat d1/fac x:0 0 0 2024-04-26T23:01:32.912 INFO:tasks.workunit.client.0.smithi027.stdout:4/617: write da/fc2 [924945,81309] 0 2024-04-26T23:01:32.915 INFO:tasks.workunit.client.0.smithi027.stdout:6/504: getdents d1/d30/d6c/d6f/d7d 0 2024-04-26T23:01:32.915 INFO:tasks.workunit.client.0.smithi027.stdout:5/585: dwrite d4/d24/f5d [0,4194304] 0 2024-04-26T23:01:32.924 INFO:tasks.workunit.client.0.smithi027.stdout:4/618: rename da/c9e to da/dd/d5f/dac/cc3 0 2024-04-26T23:01:32.924 INFO:tasks.workunit.client.0.smithi027.stdout:4/619: write da/dd/de/d14/d21/d4a/f50 [595175,111611] 0 2024-04-26T23:01:32.926 INFO:tasks.workunit.client.0.smithi027.stdout:8/539: dwrite d9/da/d2a/f78 [4194304,4194304] 0 2024-04-26T23:01:32.932 INFO:tasks.workunit.client.0.smithi027.stdout:4/620: link da/c2e da/dd/d5f/dac/cc4 0 2024-04-26T23:01:32.932 INFO:tasks.workunit.client.0.smithi027.stdout:2/486: dwrite d2/d5/db/d1b/d1c/f20 [4194304,4194304] 0 2024-04-26T23:01:32.933 INFO:tasks.workunit.client.0.smithi027.stdout:2/487: truncate d2/f49 690055 0 2024-04-26T23:01:32.934 INFO:tasks.workunit.client.0.smithi027.stdout:4/621: mkdir da/dd/de/d79/da8/dc5 0 2024-04-26T23:01:32.939 INFO:tasks.workunit.client.0.smithi027.stdout:2/488: mkdir d2/d5/db/d1b/d1c/d25/d6a/d8a 0 2024-04-26T23:01:32.943 INFO:tasks.workunit.client.0.smithi027.stdout:2/489: rename d2/d5/db/d1b/d1c/d25/d19/d44/d4b to d2/d5/db/d1b/d1c/d8b 0 2024-04-26T23:01:32.950 INFO:tasks.workunit.client.0.smithi027.stdout:7/607: dwrite d0/d28/d2b/d3a/d48/f72 [0,4194304] 0 2024-04-26T23:01:32.953 INFO:tasks.workunit.client.0.smithi027.stdout:7/608: mkdir d0/d28/d9f/db8/dbc 0 2024-04-26T23:01:32.958 INFO:tasks.workunit.client.0.smithi027.stdout:8/540: write d9/de/d2e/f1f [193554,103845] 0 2024-04-26T23:01:32.961 INFO:tasks.workunit.client.0.smithi027.stdout:8/541: rename d9/da/d2a/d5e/d72/d84/d4f/lbc to d9/da/d2a/d5e/d72/lbf 0 2024-04-26T23:01:32.961 INFO:tasks.workunit.client.0.smithi027.stdout:8/542: write d9/da/d2a/d5e/d72/d84/db8/fbe [518793,27390] 0 2024-04-26T23:01:32.962 INFO:tasks.workunit.client.0.smithi027.stdout:8/543: mkdir d9/da/d2a/d5e/d72/d84/d5d/d6c/dc0 0 2024-04-26T23:01:32.963 INFO:tasks.workunit.client.0.smithi027.stdout:6/505: write d1/df/f39 [2267626,74084] 0 2024-04-26T23:01:32.964 INFO:tasks.workunit.client.0.smithi027.stdout:8/544: rename d9/da/d2a/f78 to d9/da/d33/d71/fc1 0 2024-04-26T23:01:32.967 INFO:tasks.workunit.client.0.smithi027.stdout:8/545: mkdir d9/da/d2a/d5e/d72/d84/d35/dc2 0 2024-04-26T23:01:32.967 INFO:tasks.workunit.client.0.smithi027.stdout:8/546: read - d9/da/d2a/d5e/d72/d84/d5d/d8b/fbb zero size 2024-04-26T23:01:32.967 INFO:tasks.workunit.client.0.smithi027.stdout:2/490: dread d2/d5/db/d1b/d1c/d25/d19/d36/f43 [0,4194304] 0 2024-04-26T23:01:32.967 INFO:tasks.workunit.client.0.smithi027.stdout:2/491: fsync d2/d5/db/d1b/d1c/d25/d19/f45 0 2024-04-26T23:01:32.974 INFO:tasks.workunit.client.0.smithi027.stdout:1/624: write d1/d26/d7d/f30 [27159,101152] 0 2024-04-26T23:01:32.975 INFO:tasks.workunit.client.0.smithi027.stdout:1/625: truncate d1/d26/d6b/d42/d79/f92 427556 0 2024-04-26T23:01:32.983 INFO:tasks.workunit.client.0.smithi027.stdout:1/626: write d1/d26/d6b/f5a [253389,26313] 0 2024-04-26T23:01:32.988 INFO:tasks.workunit.client.0.smithi027.stdout:1/627: mkdir d1/d26/d6b/d8e/dc5 0 2024-04-26T23:01:32.988 INFO:tasks.workunit.client.0.smithi027.stdout:1/628: dread - d1/d26/d86/fa2 zero size 2024-04-26T23:01:32.988 INFO:tasks.workunit.client.0.smithi027.stdout:1/629: getdents d1/d26/d66/d8d 0 2024-04-26T23:01:32.989 INFO:tasks.workunit.client.0.smithi027.stdout:1/630: getdents d1/d5/d21 0 2024-04-26T23:01:32.995 INFO:tasks.workunit.client.0.smithi027.stdout:1/631: symlink d1/d26/d36/d88/lc6 0 2024-04-26T23:01:32.996 INFO:tasks.workunit.client.0.smithi027.stdout:9/517: sync 2024-04-26T23:01:32.996 INFO:tasks.workunit.client.0.smithi027.stdout:9/518: stat d4/l87 0 2024-04-26T23:01:32.996 INFO:tasks.workunit.client.0.smithi027.stdout:9/519: readlink d4/d10/d40/d58/d6e/l73 0 2024-04-26T23:01:32.997 INFO:tasks.workunit.client.0.smithi027.stdout:0/541: dwrite d0/df/d3f/d64/f5e [0,4194304] 0 2024-04-26T23:01:32.998 INFO:tasks.workunit.client.0.smithi027.stdout:1/632: unlink d1/d5/l6c 0 2024-04-26T23:01:33.000 INFO:tasks.workunit.client.0.smithi027.stdout:9/520: truncate d4/f24 314946 0 2024-04-26T23:01:33.002 INFO:tasks.workunit.client.0.smithi027.stdout:0/542: unlink d0/c28 0 2024-04-26T23:01:33.006 INFO:tasks.workunit.client.0.smithi027.stdout:0/543: truncate d0/d4/f20 5803861 0 2024-04-26T23:01:33.014 INFO:tasks.workunit.client.0.smithi027.stdout:7/609: dwrite d0/d28/d2b/d55/d59/d6d/fad [0,4194304] 0 2024-04-26T23:01:33.016 INFO:tasks.workunit.client.0.smithi027.stdout:7/610: creat d0/d28/d2b/d3a/d90/fbd x:0 0 0 2024-04-26T23:01:33.017 INFO:tasks.workunit.client.0.smithi027.stdout:5/586: dwrite d4/d24/d22/d29/d2a/d3b/d50/f70 [4194304,4194304] 0 2024-04-26T23:01:33.017 INFO:tasks.workunit.client.0.smithi027.stdout:6/506: dread d1/f35 [4194304,4194304] 0 2024-04-26T23:01:33.017 INFO:tasks.workunit.client.0.smithi027.stdout:7/611: unlink d0/d28/f77 0 2024-04-26T23:01:33.021 INFO:tasks.workunit.client.0.smithi027.stdout:6/507: mknod d1/df/d43/da8/cad 0 2024-04-26T23:01:33.023 INFO:tasks.workunit.client.0.smithi027.stdout:5/587: creat d4/d24/fb9 x:0 0 0 2024-04-26T23:01:33.023 INFO:tasks.workunit.client.0.smithi027.stdout:5/588: write d4/d24/d22/d29/d39/f9c [810427,18575] 0 2024-04-26T23:01:33.027 INFO:tasks.workunit.client.0.smithi027.stdout:2/492: dwrite d2/d5/db/de/d21/f16 [0,4194304] 0 2024-04-26T23:01:33.031 INFO:tasks.workunit.client.0.smithi027.stdout:6/508: creat d1/de/d57/d98/fae x:0 0 0 2024-04-26T23:01:33.033 INFO:tasks.workunit.client.0.smithi027.stdout:8/547: dwrite f3 [0,4194304] 0 2024-04-26T23:01:33.033 INFO:tasks.workunit.client.0.smithi027.stdout:8/548: fsync d9/da/d2a/d5e/d72/d84/d5d/f7d 0 2024-04-26T23:01:33.035 INFO:tasks.workunit.client.0.smithi027.stdout:8/549: creat d9/da/d2a/d5e/d72/d84/d35/d8d/da3/fc3 x:0 0 0 2024-04-26T23:01:33.035 INFO:tasks.workunit.client.0.smithi027.stdout:0/544: dread d0/d4/da/d1a/f3b [8388608,4194304] 0 2024-04-26T23:01:33.036 INFO:tasks.workunit.client.0.smithi027.stdout:0/545: write d0/d4/da/d15/f76 [845133,39326] 0 2024-04-26T23:01:33.044 INFO:tasks.workunit.client.0.smithi027.stdout:5/589: dread d4/d24/f5d [0,4194304] 0 2024-04-26T23:01:33.050 INFO:tasks.workunit.client.0.smithi027.stdout:1/633: write d1/d26/d66/f77 [210438,55849] 0 2024-04-26T23:01:33.050 INFO:tasks.workunit.client.0.smithi027.stdout:6/509: rename d1/df/f8 to d1/d30/faf 0 2024-04-26T23:01:33.050 INFO:tasks.workunit.client.0.smithi027.stdout:2/493: creat d2/d5/db/d1b/d1c/d25/d19/d36/d39/f8c x:0 0 0 2024-04-26T23:01:33.051 INFO:tasks.workunit.client.0.smithi027.stdout:0/546: symlink d0/df/d3f/d64/la0 0 2024-04-26T23:01:33.051 INFO:tasks.workunit.client.0.smithi027.stdout:0/547: dread - d0/d4/da/d15/f93 zero size 2024-04-26T23:01:33.051 INFO:tasks.workunit.client.0.smithi027.stdout:0/548: write d0/d4/da/d15/d26/f79 [946913,63178] 0 2024-04-26T23:01:33.052 INFO:tasks.workunit.client.0.smithi027.stdout:5/590: symlink d4/d1a/d2b/lba 0 2024-04-26T23:01:33.054 INFO:tasks.workunit.client.0.smithi027.stdout:2/494: rename d2/d5/db/d1b/d1c/d25/d19/d23/d7c/d7d to d2/d5/db/d1b/d1c/d25/d6a/d8a/d8d 0 2024-04-26T23:01:33.055 INFO:tasks.workunit.client.0.smithi027.stdout:6/510: symlink d1/d30/lb0 0 2024-04-26T23:01:33.055 INFO:tasks.workunit.client.0.smithi027.stdout:5/591: rmdir d4/d24/d5f/dac/d40/d54 39 2024-04-26T23:01:33.058 INFO:tasks.workunit.client.0.smithi027.stdout:2/495: rename d2/d5/db/d1b/d1c/d25/d19/d36/l55 to d2/d5/db/d1b/d1c/d25/d6a/l8e 0 2024-04-26T23:01:33.059 INFO:tasks.workunit.client.0.smithi027.stdout:6/511: symlink d1/lb1 0 2024-04-26T23:01:33.059 INFO:tasks.workunit.client.0.smithi027.stdout:6/512: fsync d1/de/f82 0 2024-04-26T23:01:33.059 INFO:tasks.workunit.client.0.smithi027.stdout:5/592: getdents d4/d24/d55 0 2024-04-26T23:01:33.060 INFO:tasks.workunit.client.0.smithi027.stdout:5/593: readlink d4/d24/d5f/dac/d40/l6e 0 2024-04-26T23:01:33.076 INFO:tasks.workunit.client.0.smithi027.stdout:1/634: read d1/d26/d86/f4c [8026076,72277] 0 2024-04-26T23:01:33.076 INFO:tasks.workunit.client.0.smithi027.stdout:6/513: mknod d1/df/d43/da8/cb2 0 2024-04-26T23:01:33.076 INFO:tasks.workunit.client.0.smithi027.stdout:6/514: readlink d1/l2d 0 2024-04-26T23:01:33.076 INFO:tasks.workunit.client.0.smithi027.stdout:6/515: chown d1/d6a 51967 1 2024-04-26T23:01:33.076 INFO:tasks.workunit.client.0.smithi027.stdout:2/496: rename d2/d5/db/de/d21/d26 to d2/d5/db/de/d21/d8f 0 2024-04-26T23:01:33.076 INFO:tasks.workunit.client.0.smithi027.stdout:5/594: creat d4/d24/d22/d29/d2a/d3b/d50/fbb x:0 0 0 2024-04-26T23:01:33.076 INFO:tasks.workunit.client.0.smithi027.stdout:5/595: stat d4/d24/d5f/dac/d40/c8f 0 2024-04-26T23:01:33.078 INFO:tasks.workunit.client.0.smithi027.stdout:6/516: truncate d1/df/d3c/f75 459783 0 2024-04-26T23:01:33.080 INFO:tasks.workunit.client.0.smithi027.stdout:5/596: getdents d4 0 2024-04-26T23:01:33.084 INFO:tasks.workunit.client.0.smithi027.stdout:0/549: dwrite d0/d4/da/d1a/d1e/f84 [0,4194304] 0 2024-04-26T23:01:33.089 INFO:tasks.workunit.client.0.smithi027.stdout:7/612: dwrite d0/d3d/f54 [0,4194304] 0 2024-04-26T23:01:33.091 INFO:tasks.workunit.client.0.smithi027.stdout:0/550: symlink d0/d4/da/d15/d55/la1 0 2024-04-26T23:01:33.097 INFO:tasks.workunit.client.0.smithi027.stdout:7/613: symlink d0/d28/d66/da3/lbe 0 2024-04-26T23:01:33.098 INFO:tasks.workunit.client.0.smithi027.stdout:6/517: dread d1/de/f1a [0,4194304] 0 2024-04-26T23:01:33.099 INFO:tasks.workunit.client.0.smithi027.stdout:7/614: rename d0/d28/d2b/f5c to d0/d28/d9f/db8/fbf 0 2024-04-26T23:01:33.103 INFO:tasks.workunit.client.0.smithi027.stdout:6/518: mkdir d1/d30/d6c/db3 0 2024-04-26T23:01:33.103 INFO:tasks.workunit.client.0.smithi027.stdout:0/551: rmdir d0/d4 39 2024-04-26T23:01:33.103 INFO:tasks.workunit.client.0.smithi027.stdout:6/519: chown d1/de/f1a 46480739 1 2024-04-26T23:01:33.103 INFO:tasks.workunit.client.0.smithi027.stdout:2/497: dread d2/d5/db/d1b/f38 [0,4194304] 0 2024-04-26T23:01:33.103 INFO:tasks.workunit.client.0.smithi027.stdout:6/520: readlink d1/d30/lb0 0 2024-04-26T23:01:33.103 INFO:tasks.workunit.client.0.smithi027.stdout:2/498: readlink d2/d5/db/d1b/d1c/d25/d19/d23/l50 0 2024-04-26T23:01:33.104 INFO:tasks.workunit.client.0.smithi027.stdout:5/597: dread d4/d24/f5d [0,4194304] 0 2024-04-26T23:01:33.104 INFO:tasks.workunit.client.0.smithi027.stdout:5/598: dread - d4/d24/d22/d29/d2a/d3b/fb7 zero size 2024-04-26T23:01:33.104 INFO:tasks.workunit.client.0.smithi027.stdout:6/521: creat d1/de/d57/fb4 x:0 0 0 2024-04-26T23:01:33.105 INFO:tasks.workunit.client.0.smithi027.stdout:2/499: symlink d2/d5/db/d1b/d1c/d25/l90 0 2024-04-26T23:01:33.106 INFO:tasks.workunit.client.0.smithi027.stdout:6/522: read d1/df/f79 [676152,110355] 0 2024-04-26T23:01:33.106 INFO:tasks.workunit.client.0.smithi027.stdout:0/552: link d0/df/d3f/d64/l4b d0/d4/da/d15/d55/la2 0 2024-04-26T23:01:33.108 INFO:tasks.workunit.client.0.smithi027.stdout:6/523: truncate d1/df/d49/f86 740471 0 2024-04-26T23:01:33.108 INFO:tasks.workunit.client.0.smithi027.stdout:5/599: symlink d4/d24/d22/lbc 0 2024-04-26T23:01:33.109 INFO:tasks.workunit.client.0.smithi027.stdout:6/524: mknod d1/df/d3c/cb5 0 2024-04-26T23:01:33.110 INFO:tasks.workunit.client.0.smithi027.stdout:6/525: write d1/df/d43/d56/d6b/f93 [125643,77826] 0 2024-04-26T23:01:33.122 INFO:tasks.workunit.client.0.smithi027.stdout:3/578: sync 2024-04-26T23:01:33.123 INFO:tasks.workunit.client.0.smithi027.stdout:5/600: rename d4/d24/d22/dab/d16/c30 to d4/d1a/d2b/db8/cbd 0 2024-04-26T23:01:33.126 INFO:tasks.workunit.client.0.smithi027.stdout:5/601: symlink d4/d24/d5f/lbe 0 2024-04-26T23:01:33.127 INFO:tasks.workunit.client.0.smithi027.stdout:5/602: mknod d4/d1a/cbf 0 2024-04-26T23:01:33.128 INFO:tasks.workunit.client.0.smithi027.stdout:5/603: mkdir d4/d9f/dc0 0 2024-04-26T23:01:33.129 INFO:tasks.workunit.client.0.smithi027.stdout:5/604: write d4/d24/d22/d29/d2a/d3b/f3d [1840286,63691] 0 2024-04-26T23:01:33.129 INFO:tasks.workunit.client.0.smithi027.stdout:5/605: dread - d4/d24/fb9 zero size 2024-04-26T23:01:33.129 INFO:tasks.workunit.client.0.smithi027.stdout:5/606: truncate d4/d24/d22/d29/d39/d84/fb3 871333 0 2024-04-26T23:01:33.134 INFO:tasks.workunit.client.0.smithi027.stdout:0/553: read d0/df/d3f/d64/f31 [280093,53015] 0 2024-04-26T23:01:33.134 INFO:tasks.workunit.client.0.smithi027.stdout:0/554: dread - d0/d4/da/d15/d26/f8a zero size 2024-04-26T23:01:33.144 INFO:tasks.workunit.client.0.smithi027.stdout:0/555: dread d0/df/d3f/f61 [0,4194304] 0 2024-04-26T23:01:33.144 INFO:tasks.workunit.client.0.smithi027.stdout:2/500: read d2/f6f [3379634,40829] 0 2024-04-26T23:01:33.146 INFO:tasks.workunit.client.0.smithi027.stdout:1/635: dwrite d1/dc/fa7 [0,4194304] 0 2024-04-26T23:01:33.146 INFO:tasks.workunit.client.0.smithi027.stdout:2/501: creat d2/d5/db/f91 x:0 0 0 2024-04-26T23:01:33.165 INFO:tasks.workunit.client.0.smithi027.stdout:3/579: dwrite d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d59/f97 [0,4194304] 0 2024-04-26T23:01:33.166 INFO:tasks.workunit.client.0.smithi027.stdout:0/556: dread d0/d4/da/d15/d55/f96 [0,4194304] 0 2024-04-26T23:01:33.166 INFO:tasks.workunit.client.0.smithi027.stdout:3/580: fdatasync d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/fb8 0 2024-04-26T23:01:33.167 INFO:tasks.workunit.client.0.smithi027.stdout:7/615: dwrite d0/d28/d2b/f4e [0,4194304] 0 2024-04-26T23:01:33.167 INFO:tasks.workunit.client.0.smithi027.stdout:7/616: fdatasync d0/d28/f18 0 2024-04-26T23:01:33.168 INFO:tasks.workunit.client.0.smithi027.stdout:7/617: mkdir d0/d28/d2b/d3a/d48/dc0 0 2024-04-26T23:01:33.169 INFO:tasks.workunit.client.0.smithi027.stdout:7/618: symlink d0/d28/d9f/db8/lc1 0 2024-04-26T23:01:33.171 INFO:tasks.workunit.client.0.smithi027.stdout:6/526: dwrite d1/df/d49/f4d [0,4194304] 0 2024-04-26T23:01:33.174 INFO:tasks.workunit.client.0.smithi027.stdout:7/619: creat d0/fc2 x:0 0 0 2024-04-26T23:01:33.176 INFO:tasks.workunit.client.0.smithi027.stdout:7/620: symlink d0/d28/d2b/d55/d69/lc3 0 2024-04-26T23:01:33.178 INFO:tasks.workunit.client.0.smithi027.stdout:7/621: rmdir d0/d28/d2b/d55/d69/d75 39 2024-04-26T23:01:33.179 INFO:tasks.workunit.client.0.smithi027.stdout:7/622: symlink d0/d28/d2b/d55/d69/d75/lc4 0 2024-04-26T23:01:33.180 INFO:tasks.workunit.client.0.smithi027.stdout:2/502: dread d2/d5/db/de/d21/f51 [4194304,4194304] 0 2024-04-26T23:01:33.181 INFO:tasks.workunit.client.0.smithi027.stdout:7/623: mkdir d0/d28/d2b/d3a/d48/d56/d8d/dc5 0 2024-04-26T23:01:33.184 INFO:tasks.workunit.client.0.smithi027.stdout:2/503: truncate d2/d5/db/d1b/d1c/d25/d19/d23/f64 3536265 0 2024-04-26T23:01:33.184 INFO:tasks.workunit.client.0.smithi027.stdout:2/504: fdatasync d2/f89 0 2024-04-26T23:01:33.189 INFO:tasks.workunit.client.0.smithi027.stdout:2/505: unlink d2/d5/db/d1b/d1c/d25/d19/d36/d39/c5e 0 2024-04-26T23:01:33.190 INFO:tasks.workunit.client.0.smithi027.stdout:2/506: write d2/d5/db/d1b/d1c/d8b/f56 [213737,24712] 0 2024-04-26T23:01:33.191 INFO:tasks.workunit.client.0.smithi027.stdout:0/557: write d0/f7e [5619447,74233] 0 2024-04-26T23:01:33.192 INFO:tasks.workunit.client.0.smithi027.stdout:2/507: getdents d2/d5/db/d1b/d1c/d25/d19/d44 0 2024-04-26T23:01:33.193 INFO:tasks.workunit.client.0.smithi027.stdout:0/558: mknod d0/df/d3f/d4a/ca3 0 2024-04-26T23:01:33.193 INFO:tasks.workunit.client.0.smithi027.stdout:3/581: dread d6/dc/d34/d78/da1/f7c [0,4194304] 0 2024-04-26T23:01:33.193 INFO:tasks.workunit.client.0.smithi027.stdout:2/508: rename d2/d5/db/d1b/d1c/d25/d19/d44/c75 to d2/d5/c92 0 2024-04-26T23:01:33.195 INFO:tasks.workunit.client.0.smithi027.stdout:0/559: link d0/d4/da/d1a/l3e d0/df/d3f/la4 0 2024-04-26T23:01:33.195 INFO:tasks.workunit.client.0.smithi027.stdout:0/560: chown d0/d4/da/d1a/d2a/f43 226069523 1 2024-04-26T23:01:33.196 INFO:tasks.workunit.client.0.smithi027.stdout:0/561: rmdir d0/d4/da/d1a/d6d 39 2024-04-26T23:01:33.198 INFO:tasks.workunit.client.0.smithi027.stdout:1/636: dwrite d1/d26/d36/d56/f5e [0,4194304] 0 2024-04-26T23:01:33.198 INFO:tasks.workunit.client.0.smithi027.stdout:0/562: unlink d0/d4/da/d1a/l62 0 2024-04-26T23:01:33.198 INFO:tasks.workunit.client.0.smithi027.stdout:0/563: readlink d0/df/d3f/la4 0 2024-04-26T23:01:33.198 INFO:tasks.workunit.client.0.smithi027.stdout:3/582: write d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d52/d5a/f99 [353489,53105] 0 2024-04-26T23:01:33.200 INFO:tasks.workunit.client.0.smithi027.stdout:0/564: rename d0/d4/da/d1a/f40 to d0/d4/da/d1a/d1e/fa5 0 2024-04-26T23:01:33.202 INFO:tasks.workunit.client.0.smithi027.stdout:1/637: rename d1/d26/d36/d56/f5e to d1/d26/d6b/d42/d70/fc7 0 2024-04-26T23:01:33.206 INFO:tasks.workunit.client.0.smithi027.stdout:0/565: rmdir d0/d4/d9b 0 2024-04-26T23:01:33.206 INFO:tasks.workunit.client.0.smithi027.stdout:0/566: chown d0/d4/f3d 28396002 1 2024-04-26T23:01:33.208 INFO:tasks.workunit.client.0.smithi027.stdout:0/567: unlink d0/df/d3f/d4a/ca3 0 2024-04-26T23:01:33.212 INFO:tasks.workunit.client.0.smithi027.stdout:0/568: write d0/d4/da/d1a/d59/f9c [29987,61815] 0 2024-04-26T23:01:33.214 INFO:tasks.workunit.client.0.smithi027.stdout:2/509: dread d2/d5/db/de/d21/f16 [0,4194304] 0 2024-04-26T23:01:33.214 INFO:tasks.workunit.client.0.smithi027.stdout:7/624: dread d0/d28/d2b/d3a/d48/d56/f74 [0,4194304] 0 2024-04-26T23:01:33.216 INFO:tasks.workunit.client.0.smithi027.stdout:2/510: link d2/d5/db/d1b/d1c/d25/d19/d36/l69 d2/d5/db/de/l93 0 2024-04-26T23:01:33.216 INFO:tasks.workunit.client.0.smithi027.stdout:2/511: fsync d2/d5/db/d1b/d1c/d25/d19/d36/d39/f8c 0 2024-04-26T23:01:33.217 INFO:tasks.workunit.client.0.smithi027.stdout:7/625: creat d0/d28/d2b/d55/d69/da7/fc6 x:0 0 0 2024-04-26T23:01:33.220 INFO:tasks.workunit.client.0.smithi027.stdout:6/527: dwrite d1/d30/d6c/d6f/f89 [0,4194304] 0 2024-04-26T23:01:33.220 INFO:tasks.workunit.client.0.smithi027.stdout:6/528: write d1/f69 [5183865,73692] 0 2024-04-26T23:01:33.222 INFO:tasks.workunit.client.0.smithi027.stdout:6/529: rmdir d1/df/d43/d56/d6b 39 2024-04-26T23:01:33.228 INFO:tasks.workunit.client.0.smithi027.stdout:2/512: link d2/d5/db/de/l85 d2/d5/db/d1b/d1c/l94 0 2024-04-26T23:01:33.231 INFO:tasks.workunit.client.0.smithi027.stdout:4/622: sync 2024-04-26T23:01:33.232 INFO:tasks.workunit.client.0.smithi027.stdout:4/623: creat da/dd/de/d14/db3/fc6 x:0 0 0 2024-04-26T23:01:33.239 INFO:tasks.workunit.client.0.smithi027.stdout:7/626: write d0/d28/f37 [1474374,87866] 0 2024-04-26T23:01:33.242 INFO:tasks.workunit.client.0.smithi027.stdout:7/627: unlink d0/d28/d2b/l87 0 2024-04-26T23:01:33.251 INFO:tasks.workunit.client.0.smithi027.stdout:3/583: dwrite d6/dc/d34/f5d [0,4194304] 0 2024-04-26T23:01:33.251 INFO:tasks.workunit.client.0.smithi027.stdout:6/530: write d1/de/f28 [2999090,33116] 0 2024-04-26T23:01:33.251 INFO:tasks.workunit.client.0.smithi027.stdout:6/531: stat d1/d30/l38 0 2024-04-26T23:01:33.258 INFO:tasks.workunit.client.0.smithi027.stdout:4/624: write da/dd/f5d [3764625,22252] 0 2024-04-26T23:01:33.266 INFO:tasks.workunit.client.0.smithi027.stdout:3/584: unlink d6/dc/d2f/d7e/c8f 0 2024-04-26T23:01:33.267 INFO:tasks.workunit.client.0.smithi027.stdout:0/569: dwrite d0/df/d3f/d64/f6a [0,4194304] 0 2024-04-26T23:01:33.269 INFO:tasks.workunit.client.0.smithi027.stdout:3/585: rmdir d6/dc/d34/d2e/d70/d71/dae 0 2024-04-26T23:01:33.269 INFO:tasks.workunit.client.0.smithi027.stdout:0/570: truncate d0/d4/da/d15/d55/f77 3276562 0 2024-04-26T23:01:33.270 INFO:tasks.workunit.client.0.smithi027.stdout:0/571: link d0/d4/da/f78 d0/d4/da/d1a/d6d/fa6 0 2024-04-26T23:01:33.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:32 smithi080 ceph-mon[32324]: pgmap v91: 129 pgs: 129 active+clean; 1.2 GiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 147 MiB/s rd, 181 MiB/s wr, 1.16k op/s 2024-04-26T23:01:33.272 INFO:tasks.workunit.client.0.smithi027.stdout:0/572: creat d0/d4/da/d1a/d6d/fa7 x:0 0 0 2024-04-26T23:01:33.272 INFO:tasks.workunit.client.0.smithi027.stdout:0/573: write d0/d4/da/d1a/d59/d92/f94 [327269,80374] 0 2024-04-26T23:01:33.272 INFO:tasks.workunit.client.0.smithi027.stdout:0/574: fsync d0/d4/da/f2f 0 2024-04-26T23:01:33.272 INFO:tasks.workunit.client.0.smithi027.stdout:0/575: readlink d0/l7d 0 2024-04-26T23:01:33.272 INFO:tasks.workunit.client.0.smithi027.stdout:3/586: creat d6/dc/d34/d47/d7b/d98/fc7 x:0 0 0 2024-04-26T23:01:33.272 INFO:tasks.workunit.client.0.smithi027.stdout:3/587: truncate d6/dc/d34/d2e/d70/d71/fc5 479490 0 2024-04-26T23:01:33.273 INFO:tasks.workunit.client.0.smithi027.stdout:0/576: mkdir d0/df/d3f/d64/da8 0 2024-04-26T23:01:33.275 INFO:tasks.workunit.client.0.smithi027.stdout:7/628: write d0/d2f/f65 [2808858,25162] 0 2024-04-26T23:01:33.275 INFO:tasks.workunit.client.0.smithi027.stdout:7/629: fdatasync d0/d28/d2b/d3a/fb1 0 2024-04-26T23:01:33.275 INFO:tasks.workunit.client.0.smithi027.stdout:7/630: read - d0/d28/d9f/db8/fbf zero size 2024-04-26T23:01:33.275 INFO:tasks.workunit.client.0.smithi027.stdout:3/588: symlink d6/dc/d34/db3/db4/lc8 0 2024-04-26T23:01:33.276 INFO:tasks.workunit.client.0.smithi027.stdout:3/589: mknod d6/dc/d34/d78/cc9 0 2024-04-26T23:01:33.276 INFO:tasks.workunit.client.0.smithi027.stdout:3/590: stat d6/dc/d34/d78/da1/cbb 0 2024-04-26T23:01:33.276 INFO:tasks.workunit.client.0.smithi027.stdout:3/591: truncate d6/dc/d34/db3/fc1 876405 0 2024-04-26T23:01:33.278 INFO:tasks.workunit.client.0.smithi027.stdout:0/577: mknod d0/d4/da/d1a/d1e/ca9 0 2024-04-26T23:01:33.279 INFO:tasks.workunit.client.0.smithi027.stdout:7/631: creat d0/d28/d2b/d3a/d48/d56/fc7 x:0 0 0 2024-04-26T23:01:33.280 INFO:tasks.workunit.client.0.smithi027.stdout:2/513: dwrite d2/d5/db/de/f30 [0,4194304] 0 2024-04-26T23:01:33.282 INFO:tasks.workunit.client.0.smithi027.stdout:7/632: rename d0/d28/d2b/d55/c8a to d0/d3d/cc8 0 2024-04-26T23:01:33.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:32 smithi094 ceph-mon[32889]: pgmap v91: 129 pgs: 129 active+clean; 1.2 GiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 147 MiB/s rd, 181 MiB/s wr, 1.16k op/s 2024-04-26T23:01:33.285 INFO:tasks.workunit.client.0.smithi027.stdout:2/514: rename d2/d5/db/de/f68 to d2/d5/db/de/d21/d8f/f95 0 2024-04-26T23:01:33.289 INFO:tasks.workunit.client.0.smithi027.stdout:4/625: write da/dd/de/d14/d17/d80/dab/f26 [1235591,35313] 0 2024-04-26T23:01:33.289 INFO:tasks.workunit.client.0.smithi027.stdout:4/626: fdatasync da/dd/de/f3f 0 2024-04-26T23:01:33.289 INFO:tasks.workunit.client.0.smithi027.stdout:4/627: fsync da/dd/de/d14/d21/d41/fa4 0 2024-04-26T23:01:33.297 INFO:tasks.workunit.client.0.smithi027.stdout:3/592: dread d6/dc/f55 [0,4194304] 0 2024-04-26T23:01:33.297 INFO:tasks.workunit.client.0.smithi027.stdout:3/593: dread - d6/dc/d34/d78/fbf zero size 2024-04-26T23:01:33.298 INFO:tasks.workunit.client.0.smithi027.stdout:3/594: stat d6/dc/d34/d2e/d70/f7d 0 2024-04-26T23:01:33.298 INFO:tasks.workunit.client.0.smithi027.stdout:3/595: write d6/dc/d34/d2e/d5f/d90/fb0 [833481,99708] 0 2024-04-26T23:01:33.303 INFO:tasks.workunit.client.0.smithi027.stdout:0/578: read d0/d4/da/d15/d26/f7c [2897422,8945] 0 2024-04-26T23:01:33.304 INFO:tasks.workunit.client.0.smithi027.stdout:3/596: unlink d6/c51 0 2024-04-26T23:01:33.306 INFO:tasks.workunit.client.0.smithi027.stdout:3/597: mkdir d6/dc/d34/d2e/d9d/da7/dac/dca 0 2024-04-26T23:01:33.308 INFO:tasks.workunit.client.0.smithi027.stdout:7/633: dread d0/d28/d66/d67/fb7 [0,4194304] 0 2024-04-26T23:01:33.310 INFO:tasks.workunit.client.0.smithi027.stdout:7/634: rmdir d0/d28/d2b/d3a 39 2024-04-26T23:01:33.311 INFO:tasks.workunit.client.0.smithi027.stdout:7/635: mknod d0/d28/d2b/cc9 0 2024-04-26T23:01:33.311 INFO:tasks.workunit.client.0.smithi027.stdout:7/636: chown d0/d28/d2b/d3a/f52 870 1 2024-04-26T23:01:33.312 INFO:tasks.workunit.client.0.smithi027.stdout:7/637: symlink d0/d28/d2b/d3a/d48/d56/d88/lca 0 2024-04-26T23:01:33.312 INFO:tasks.workunit.client.0.smithi027.stdout:7/638: truncate d0/d28/d2b/d3a/f8b 943536 0 2024-04-26T23:01:33.312 INFO:tasks.workunit.client.0.smithi027.stdout:7/639: write d0/f13 [2208481,12148] 0 2024-04-26T23:01:33.313 INFO:tasks.workunit.client.0.smithi027.stdout:3/598: link d6/dc/d88/l8e d6/dc/d34/d47/lcb 0 2024-04-26T23:01:33.313 INFO:tasks.workunit.client.0.smithi027.stdout:3/599: write d6/dc/d34/d78/fb6 [18018,69048] 0 2024-04-26T23:01:33.313 INFO:tasks.workunit.client.0.smithi027.stdout:3/600: chown d6/dc/d34/d2e/d5f 63 1 2024-04-26T23:01:33.313 INFO:tasks.workunit.client.0.smithi027.stdout:3/601: stat d6/dc/f21 0 2024-04-26T23:01:33.314 INFO:tasks.workunit.client.0.smithi027.stdout:3/602: getdents d6/dc/d34/d2e/d5f 0 2024-04-26T23:01:33.334 INFO:tasks.workunit.client.0.smithi027.stdout:0/579: dread d0/d4/da/d1a/d1e/f35 [0,4194304] 0 2024-04-26T23:01:33.335 INFO:tasks.workunit.client.0.smithi027.stdout:3/603: dread d6/dc/d2f/f39 [0,4194304] 0 2024-04-26T23:01:33.335 INFO:tasks.workunit.client.0.smithi027.stdout:3/604: write d6/dc/d34/d2e/d70/d71/fc5 [1256407,83449] 0 2024-04-26T23:01:33.335 INFO:tasks.workunit.client.0.smithi027.stdout:3/605: write d6/dc/d34/d78/fb6 [687106,97907] 0 2024-04-26T23:01:33.336 INFO:tasks.workunit.client.0.smithi027.stdout:2/515: dwrite d2/d5/db/d1b/d1c/d25/d19/d36/d39/f3f [0,4194304] 0 2024-04-26T23:01:33.336 INFO:tasks.workunit.client.0.smithi027.stdout:2/516: dread - d2/d5/db/d1b/d1c/d25/d19/d36/d39/f8c zero size 2024-04-26T23:01:33.336 INFO:tasks.workunit.client.0.smithi027.stdout:4/628: dread da/dd/de/d14/d17/f45 [0,4194304] 0 2024-04-26T23:01:33.336 INFO:tasks.workunit.client.0.smithi027.stdout:7/640: write d0/d28/d66/d67/fb7 [1917796,4442] 0 2024-04-26T23:01:33.337 INFO:tasks.workunit.client.0.smithi027.stdout:0/580: rename d0/d4/da/d1a/c65 to d0/d4/caa 0 2024-04-26T23:01:33.339 INFO:tasks.workunit.client.0.smithi027.stdout:0/581: truncate d0/d4/da/d15/d26/f7c 7843500 0 2024-04-26T23:01:33.339 INFO:tasks.workunit.client.0.smithi027.stdout:0/582: chown d0/d4/da/f11 205648 1 2024-04-26T23:01:33.340 INFO:tasks.workunit.client.0.smithi027.stdout:3/606: creat d6/dc/d34/d2e/d5f/d90/db9/fcc x:0 0 0 2024-04-26T23:01:33.341 INFO:tasks.workunit.client.0.smithi027.stdout:0/583: symlink d0/d4/da/d1a/d59/lab 0 2024-04-26T23:01:33.342 INFO:tasks.workunit.client.0.smithi027.stdout:2/517: write d2/d5/db/d1b/d1c/d25/d19/f5b [505611,23301] 0 2024-04-26T23:01:33.344 INFO:tasks.workunit.client.0.smithi027.stdout:2/518: creat d2/d5/db/d1b/d1c/d25/d19/d36/d39/f96 x:0 0 0 2024-04-26T23:01:33.344 INFO:tasks.workunit.client.0.smithi027.stdout:0/584: write d0/d4/da/d15/f33 [1959389,72568] 0 2024-04-26T23:01:33.346 INFO:tasks.workunit.client.0.smithi027.stdout:2/519: link d2/f67 d2/d5/db/d1b/d1c/d25/f97 0 2024-04-26T23:01:33.346 INFO:tasks.workunit.client.0.smithi027.stdout:4/629: dread da/dd/de/d14/d21/d41/d59/d74/f7e [0,4194304] 0 2024-04-26T23:01:33.350 INFO:tasks.workunit.client.0.smithi027.stdout:2/520: getdents d2/d5/db/de/d21 0 2024-04-26T23:01:33.350 INFO:tasks.workunit.client.0.smithi027.stdout:4/630: creat da/dd/de/fc7 x:0 0 0 2024-04-26T23:01:33.350 INFO:tasks.workunit.client.0.smithi027.stdout:0/585: creat d0/fac x:0 0 0 2024-04-26T23:01:33.350 INFO:tasks.workunit.client.0.smithi027.stdout:0/586: fsync d0/df/d3f/d64/f31 0 2024-04-26T23:01:33.355 INFO:tasks.workunit.client.0.smithi027.stdout:4/631: symlink da/dd/de/d14/d21/dad/lc8 0 2024-04-26T23:01:33.355 INFO:tasks.workunit.client.0.smithi027.stdout:4/632: truncate da/dd/de/d79/da8/fba 826494 0 2024-04-26T23:01:33.355 INFO:tasks.workunit.client.0.smithi027.stdout:4/633: stat da/dd/de/d14/d17/d80/fb7 0 2024-04-26T23:01:33.358 INFO:tasks.workunit.client.0.smithi027.stdout:9/521: sync 2024-04-26T23:01:33.358 INFO:tasks.workunit.client.0.smithi027.stdout:8/550: sync 2024-04-26T23:01:33.363 INFO:tasks.workunit.client.0.smithi027.stdout:8/551: creat d9/da/d2a/d5e/d72/d84/d5d/d6c/fc4 x:0 0 0 2024-04-26T23:01:33.363 INFO:tasks.workunit.client.0.smithi027.stdout:9/522: mkdir d4/d17/d2e/d81/da7/db8 0 2024-04-26T23:01:33.363 INFO:tasks.workunit.client.0.smithi027.stdout:3/607: dread d6/f82 [0,4194304] 0 2024-04-26T23:01:33.370 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:32 smithi027 ceph-mon[28034]: pgmap v91: 129 pgs: 129 active+clean; 1.2 GiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 147 MiB/s rd, 181 MiB/s wr, 1.16k op/s 2024-04-26T23:01:33.375 INFO:tasks.workunit.client.0.smithi027.stdout:9/523: rename d4/d10/d40/d58/d75/la3 to d4/d17/lb9 0 2024-04-26T23:01:33.376 INFO:tasks.workunit.client.0.smithi027.stdout:9/524: fdatasync d4/d10/f28 0 2024-04-26T23:01:33.376 INFO:tasks.workunit.client.0.smithi027.stdout:2/521: dread d2/d5/db/de/f42 [0,4194304] 0 2024-04-26T23:01:33.376 INFO:tasks.workunit.client.0.smithi027.stdout:2/522: chown d2/d5/db/de/l84 0 1 2024-04-26T23:01:33.378 INFO:tasks.workunit.client.0.smithi027.stdout:2/523: unlink d2/d5/db/d1b/d1c/c62 0 2024-04-26T23:01:33.379 INFO:tasks.workunit.client.0.smithi027.stdout:2/524: getdents d2/d5/db/d1b 0 2024-04-26T23:01:33.379 INFO:tasks.workunit.client.0.smithi027.stdout:2/525: chown d2/d5/db/d1b/d1c/l76 1 1 2024-04-26T23:01:33.382 INFO:tasks.workunit.client.0.smithi027.stdout:2/526: unlink d2/l35 0 2024-04-26T23:01:33.382 INFO:tasks.workunit.client.0.smithi027.stdout:2/527: chown d2/d5/db/d1b/d1c/f61 60138 1 2024-04-26T23:01:33.382 INFO:tasks.workunit.client.0.smithi027.stdout:0/587: read d0/d4/f53 [1838440,33224] 0 2024-04-26T23:01:33.382 INFO:tasks.workunit.client.0.smithi027.stdout:0/588: fdatasync d0/d4/da/f14 0 2024-04-26T23:01:33.384 INFO:tasks.workunit.client.0.smithi027.stdout:0/589: mknod d0/d4/da/d1a/d59/d92/cad 0 2024-04-26T23:01:33.386 INFO:tasks.workunit.client.0.smithi027.stdout:2/528: rmdir d2/d5 39 2024-04-26T23:01:33.387 INFO:tasks.workunit.client.0.smithi027.stdout:8/552: dread d9/da/d2a/d5e/d72/d84/db8/f9b [0,4194304] 0 2024-04-26T23:01:33.387 INFO:tasks.workunit.client.0.smithi027.stdout:8/553: fdatasync d9/da/d2a/d5e/d72/d84/d4f/f79 0 2024-04-26T23:01:33.389 INFO:tasks.workunit.client.0.smithi027.stdout:2/529: fsync d2/d5/db/d1b/d1c/d25/d19/d36/f43 0 2024-04-26T23:01:33.389 INFO:tasks.workunit.client.0.smithi027.stdout:2/530: rename d2/d5 to d2/d5/db/d1b/d1c/d25/d19/d36/d39/d98 22 2024-04-26T23:01:33.389 INFO:tasks.workunit.client.0.smithi027.stdout:3/608: write d6/dc/fd [273660,59458] 0 2024-04-26T23:01:33.392 INFO:tasks.workunit.client.0.smithi027.stdout:3/609: truncate d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d40/f4d 528184 0 2024-04-26T23:01:33.392 INFO:tasks.workunit.client.0.smithi027.stdout:3/610: stat d6/dc/d34/d2e/d70 0 2024-04-26T23:01:33.392 INFO:tasks.workunit.client.0.smithi027.stdout:2/531: creat d2/d5/db/d1b/d1c/f99 x:0 0 0 2024-04-26T23:01:33.392 INFO:tasks.workunit.client.0.smithi027.stdout:2/532: write d2/d5/db/d1b/d1c/d25/d19/d36/d39/f96 [65962,42876] 0 2024-04-26T23:01:33.392 INFO:tasks.workunit.client.0.smithi027.stdout:2/533: readlink d2/d5/db/d1b/d1c/d25/l90 0 2024-04-26T23:01:33.392 INFO:tasks.workunit.client.0.smithi027.stdout:3/611: write d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d40/f57 [755622,77582] 0 2024-04-26T23:01:33.393 INFO:tasks.workunit.client.0.smithi027.stdout:7/641: dwrite d0/f27 [0,4194304] 0 2024-04-26T23:01:33.394 INFO:tasks.workunit.client.0.smithi027.stdout:7/642: write d0/d3d/f43 [270731,106656] 0 2024-04-26T23:01:33.394 INFO:tasks.workunit.client.0.smithi027.stdout:3/612: mkdir d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d52/d5a/dcd 0 2024-04-26T23:01:33.395 INFO:tasks.workunit.client.0.smithi027.stdout:2/534: creat d2/d5/db/d1b/d1c/d25/d19/d23/f9a x:0 0 0 2024-04-26T23:01:33.396 INFO:tasks.workunit.client.0.smithi027.stdout:3/613: mkdir d6/dc/d34/d47/d7b/dce 0 2024-04-26T23:01:33.396 INFO:tasks.workunit.client.0.smithi027.stdout:3/614: chown d6/dc/fd 263 1 2024-04-26T23:01:33.397 INFO:tasks.workunit.client.0.smithi027.stdout:3/615: write d6/dc/d34/d2e/d9d/da7/dac/d74/da6/d52/d5a/f99 [528322,71115] 0 2024-04-26T23:01:33.398 INFO:tasks.workunit.client.0.smithi027.stdout:2/535: rename d2/d5/db/l54 to d2/d5/db/d1b/d1c/d25/d6a/d8a/l9b 0 2024-04-26T23:01:33.398 INFO:tasks.workunit.client.0.smithi027.stdout:3/616: symlink d6/dc/lcf 0 2024-04-26T23:01:33.400 INFO:tasks.workunit.client.0.smithi027.stdout:7/643: fsync d0/d28/d2b/d55/f95 0 2024-04-26T23:01:33.402 INFO:tasks.workunit.client.0.smithi027.stdout:3/617: creat d6/dc/d34/d78/da1/fd0 x:0 0 0 2024-04-26T23:01:33.403 INFO:tasks.workunit.client.0.smithi027.stdout:2/536: rmdir d2/d5/db/d1b/d1c/d25/d19/d44 39 2024-04-26T23:01:33.403 INFO:tasks.workunit.client.0.smithi027.stdout:9/525: read d4/d10/d40/d58/d75/d84/d86/fa5 [2866529,94014] 0 2024-04-26T23:01:33.403 INFO:tasks.workunit.client.0.smithi027.stdout:9/526: chown d4/d10/d67/l68 25 1 2024-04-26T23:01:33.404 INFO:tasks.workunit.client.0.smithi027.stdout:4/634: dwrite da/d40/f60 [0,4194304] 0 2024-04-26T23:01:33.406 INFO:tasks.workunit.client.0.smithi027.stdout:2/537: unlink d2/d5/db/d1b/d1c/f61 0 2024-04-26T23:01:33.407 INFO:tasks.workunit.client.0.smithi027.stdout:3/618: truncate d6/dc/d34/d2e/f31 4142739 0 2024-04-26T23:01:33.413 INFO:tasks.workunit.client.0.smithi027.stdout:9/527: mknod d4/d17/d2e/d46/cba 0 2024-04-26T23:01:33.413 INFO:tasks.workunit.client.0.smithi027.stdout:0/590: dread d0/df/d3f/d4a/f4d [4194304,4194304] 0 2024-04-26T23:01:33.416 INFO:tasks.workunit.client.0.smithi027.stdout:8/554: write d9/da/d2a/d5e/d72/d84/d4f/f79 [43232,114214] 0 2024-04-26T23:01:33.416 INFO:tasks.workunit.client.0.smithi027.stdout:8/555: chown d9/da/d2a/d5e/d72/d84/d5d 23063079 1 2024-04-26T23:01:33.425 INFO:tasks.workunit.client.0.smithi027.stdout:8/556: mkdir d9/da/d2a/dc5 0 2024-04-26T23:01:33.425 INFO:tasks.workunit.client.0.smithi027.stdout:2/538: symlink d2/d5/db/d1b/d1c/d25/d19/d44/d7e/l9c 0 2024-04-26T23:01:33.425 INFO:tasks.workunit.client.0.smithi027.stdout:2/539: truncate d2/d5/db/d1b/d1c/d25/d19/d23/f64 4415758 0 2024-04-26T23:01:33.427 INFO:tasks.workunit.client.0.smithi027.stdout:4/635: write da/dd/f56 [131532,95032] 0 2024-04-26T23:01:33.430 INFO:tasks.workunit.client.0.smithi027.stdout:3/619: dread d6/d5e/fa0 [0,4194304] 0 2024-04-26T23:01:33.434 INFO:tasks.workunit.client.0.smithi027.stdout:8/557: creat d9/da/d2a/dc5/fc6 x:0 0 0 2024-04-26T23:01:33.447 INFO:tasks.workunit.client.0.smithi027.stdout:5/607: sync 2024-04-26T23:01:33.447 INFO:tasks.workunit.client.0.smithi027.stdout:5/608: chown d4/d24/d22/dab/c85 1 1 2024-04-26T23:01:33.448 INFO:tasks.workunit.client.0.smithi027.stdout:5/609: stat d4/d24/d22/d29/d39/l52 0 2024-04-26T23:01:33.451 INFO:tasks.workunit.client.0.smithi027.stdout:5/610: rename d4/d1a/f1e to d4/fc1 0 2024-04-26T23:01:33.466 INFO:tasks.workunit.client.0.smithi027.stdout:7/644: dwrite d0/d28/d2b/d3a/d48/d56/d88/f3c [0,4194304] 0 2024-04-26T23:01:33.467 INFO:tasks.workunit.client.0.smithi027.stdout:7/645: stat d0/d28/d2b/l4a 0 2024-04-26T23:01:33.467 INFO:tasks.workunit.client.0.smithi027.stdout:7/646: fsync d0/d28/d2b/d55/d59/d6d/d82/fb9 0 2024-04-26T23:01:33.469 INFO:tasks.workunit.client.0.smithi027.stdout:7/647: rename d0/d28/d2b/d55/d59/d6d/d82 to d0/d28/d2b/d3a/d90/dcb 0 2024-04-26T23:01:33.471 INFO:tasks.workunit.client.0.smithi027.stdout:7/648: mknod d0/d28/d2b/d3a/d48/ccc 0 2024-04-26T23:01:33.472 INFO:tasks.workunit.client.0.smithi027.stdout:7/649: mknod d0/d28/d2b/d3a/d90/dcb/ccd 0 2024-04-26T23:01:33.473 INFO:tasks.workunit.client.0.smithi027.stdout:7/650: fdatasync d0/d28/d2b/d3a/d48/d56/fc7 0 2024-04-26T23:01:33.474 INFO:tasks.workunit.client.0.smithi027.stdout:7/651: fsync d0/d28/d2b/f5d 0 2024-04-26T23:01:33.474 INFO:tasks.workunit.client.0.smithi027.stdout:7/652: readlink d0/d28/d2b/d3a/d48/d56/lae 0 2024-04-26T23:01:33.474 INFO:tasks.workunit.client.0.smithi027.stdout:7/653: readlink d0/d28/d66/da3/lbe 0 2024-04-26T23:01:33.474 INFO:tasks.workunit.client.0.smithi027.stdout:7/654: chown d0/d28/d9f/db8/dbc 57 1 2024-04-26T23:01:33.474 INFO:tasks.workunit.client.0.smithi027.stdout:7/655: chown d0/d28/d2b/d3a/d48/d56/d88/c41 13 1 2024-04-26T23:01:33.475 INFO:tasks.workunit.client.0.smithi027.stdout:7/656: rename d0/d28/f6a to d0/d28/d2b/fce 0 2024-04-26T23:01:33.477 INFO:tasks.workunit.client.0.smithi027.stdout:5/611: dread d4/d24/d5f/dac/d40/d54/f75 [4194304,4194304] 0 2024-04-26T23:01:33.479 INFO:tasks.workunit.client.0.smithi027.stdout:7/657: rmdir d0/d28/d2b/d3a/d48/d56 39 2024-04-26T23:01:33.481 INFO:tasks.workunit.client.0.smithi027.stdout:5/612: rmdir d4/d1a/d2b 39 2024-04-26T23:01:33.483 INFO:tasks.workunit.client.0.smithi027.stdout:7/658: mknod d0/d28/d66/d67/ccf 0 2024-04-26T23:01:33.487 INFO:tasks.workunit.client.0.smithi027.stdout:7/659: creat d0/d28/d2b/d55/d69/fd0 x:0 0 0 2024-04-26T23:01:33.497 INFO:tasks.workunit.client.0.smithi027.stdout:9/528: dwrite d4/d17/f93 [0,4194304] 0 2024-04-26T23:01:33.498 INFO:tasks.workunit.client.0.smithi027.stdout:9/529: stat d4/d10/d25/la6 0 2024-04-26T23:01:33.498 INFO:tasks.workunit.client.0.smithi027.stdout:9/530: truncate d4/d10/d40/d58/f9c 901695 0 2024-04-26T23:01:33.498 INFO:tasks.workunit.client.0.smithi027.stdout:8/558: write d9/da/d33/f88 [891675,113044] 0 2024-04-26T23:01:33.498 INFO:tasks.workunit.client.0.smithi027.stdout:8/559: readlink d9/de/d2e/l4b 0 2024-04-26T23:01:33.498 INFO:tasks.workunit.client.0.smithi027.stdout:3/620: dwrite d6/d5e/f6d [0,4194304] 0 2024-04-26T23:01:33.499 INFO:tasks.workunit.client.0.smithi027.stdout:0/591: dwrite d0/d4/f3d [0,4194304] 0 2024-04-26T23:01:33.499 INFO:tasks.workunit.client.0.smithi027.stdout:0/592: chown d0/df/d3f/d64/l8d 338761772 1 2024-04-26T23:01:33.500 INFO:tasks.workunit.client.0.smithi027.stdout:4/636: dwrite da/dd/de/d14/d17/f2a [4194304,4194304] 0 2024-04-26T23:01:33.502 INFO:tasks.workunit.client.0.smithi027.stdout:4/637: getdents da/dd/de/d79/da8 0 2024-04-26T23:01:33.504 INFO:tasks.workunit.client.0.smithi027.stdout:4/638: creat da/d40/fc9 x:0 0 0 2024-04-26T23:01:33.510 INFO:tasks.workunit.client.0.smithi027.stdout:3/621: rename d6/dc/d34/d2e/d5f/l7a to d6/dc/d34/db3/db4/ld1 0 2024-04-26T23:01:33.512 INFO:tasks.workunit.client.0.smithi027.stdout:1/638: sync 2024-04-26T23:01:33.513 INFO:tasks.workunit.client.0.smithi027.stdout:8/560: dread d9/da/d2a/d5e/d72/d84/d5d/f7d [0,4194304] 0 2024-04-26T23:01:33.515 INFO:tasks.workunit.client.0.smithi027.stdout:3/622: rename d6/dc/d2f/d7e/l64 to d6/dc/d34/d2e/d9d/da7/dac/dca/ld2 0 2024-04-26T23:01:33.518 INFO:tasks.workunit.client.0.smithi027.stdout:6/532: sync 2024-04-26T23:01:33.518 INFO:tasks.workunit.client.0.smithi027.stdout:6/533: write d1/df/d3c/f75 [1260026,20035] 0 2024-04-26T23:01:33.518 INFO:tasks.workunit.client.0.smithi027.stdout:9/531: write d4/d10/d40/f5d [2955622,61620] 0 2024-04-26T23:01:33.519 INFO:tasks.workunit.client.0.smithi027.stdout:6/534: creat d1/df/d43/da8/da1/fb6 x:0 0 0 2024-04-26T23:01:33.519 INFO:tasks.workunit.client.0.smithi027.stdout:6/535: chown d1/df/d3c/d9a 1 1 2024-04-26T23:01:33.523 INFO:tasks.workunit.client.0.smithi027.stdout:7/660: dread d0/d28/d2b/d3a/f52 [0,4194304] 0 2024-04-26T23:01:33.524 INFO:tasks.workunit.client.0.smithi027.stdout:7/661: link d0/d28/d2b/d3a/d48/d56/d88/d73/l8e d0/d28/d2b/d55/d59/d6d/ld1 0 2024-04-26T23:01:33.524 INFO:tasks.workunit.client.0.smithi027.stdout:7/662: read - d0/d28/d2b/d55/f7e zero size 2024-04-26T23:01:33.526 INFO:tasks.workunit.client.0.smithi027.stdout:7/663: truncate d0/d28/d2b/d3a/d48/d56/d88/f3c 54214 0 2024-04-26T23:01:33.527 INFO:tasks.workunit.client.0.smithi027.stdout:7/664: mkdir d0/d28/d2b/d3a/d48/dd2 0 2024-04-26T23:01:33.529 INFO:tasks.workunit.client.0.smithi027.stdout:4/639: dread da/dd/de/d14/d21/d41/d59/d74/fa1 [0,4194304] 0 2024-04-26T23:01:33.530 INFO:tasks.workunit.client.0.smithi027.stdout:7/665: rename d0/d28/d2b/d55/fa8 to d0/d28/d66/fd3 0 2024-04-26T23:01:33.530 INFO:tasks.workunit.client.0.smithi027.stdout:7/666: fsync d0/d28/d2b/d55/d59/f6b 0 2024-04-26T23:01:33.530 INFO:tasks.workunit.client.0.smithi027.stdout:7/667: stat d0/d28/d66/da3/lbe 0 2024-04-26T23:01:33.530 INFO:tasks.workunit.client.0.smithi027.stdout:7/668: read d0/f30 [1208091,127934] 0 2024-04-26T23:01:33.530 INFO:tasks.workunit.client.0.smithi027.stdout:7/669: chown d0/d28/d2b/d55/d59/d6d/fb5 1 1 2024-04-26T23:01:33.531 INFO:tasks.workunit.client.0.smithi027.stdout:3/623: rename d6/dc to d6/dd3 0 2024-04-26T23:01:33.532 INFO:tasks.workunit.client.0.smithi027.stdout:4/640: link da/dd/de/d14/d21/d67/da6/lbe da/dd/de/d79/da8/dc5/lca 0 2024-04-26T23:01:33.533 INFO:tasks.workunit.client.0.smithi027.stdout:7/670: creat d0/d28/d2b/d3a/d48/d56/d8d/fd4 x:0 0 0 2024-04-26T23:01:33.535 INFO:tasks.workunit.client.0.smithi027.stdout:7/671: rmdir d0/d28/d2b/d55/d69 39 2024-04-26T23:01:33.535 INFO:tasks.workunit.client.0.smithi027.stdout:7/672: fdatasync d0/d2f/f31 0 2024-04-26T23:01:33.541 INFO:tasks.workunit.client.0.smithi027.stdout:3/624: dread d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d40/f8b [0,4194304] 0 2024-04-26T23:01:33.566 INFO:tasks.workunit.client.0.smithi027.stdout:1/639: dwrite d1/d5/d40/f78 [0,4194304] 0 2024-04-26T23:01:33.572 INFO:tasks.workunit.client.0.smithi027.stdout:1/640: dread d1/f94 [0,4194304] 0 2024-04-26T23:01:33.574 INFO:tasks.workunit.client.0.smithi027.stdout:3/625: unlink d6/dd3/d34/d47/d7b/f92 0 2024-04-26T23:01:33.574 INFO:tasks.workunit.client.0.smithi027.stdout:3/626: write d6/dd3/f55 [2775318,91369] 0 2024-04-26T23:01:33.575 INFO:tasks.workunit.client.0.smithi027.stdout:3/627: symlink d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/ld4 0 2024-04-26T23:01:33.582 INFO:tasks.workunit.client.0.smithi027.stdout:7/673: rename d0/d28/d2b/d3a/d48/f63 to d0/d28/d9f/db8/dbc/fd5 0 2024-04-26T23:01:33.594 INFO:tasks.workunit.client.0.smithi027.stdout:1/641: rmdir d1/d26/d6b/d42/d70 39 2024-04-26T23:01:33.594 INFO:tasks.workunit.client.0.smithi027.stdout:1/642: chown d1/d26/d86/d5c/l63 26000 1 2024-04-26T23:01:33.595 INFO:tasks.workunit.client.0.smithi027.stdout:1/643: stat d1/d26/d66/f77 0 2024-04-26T23:01:33.595 INFO:tasks.workunit.client.0.smithi027.stdout:7/674: mkdir d0/d28/d2b/d3a/d48/d56/d88/d73/dd6 0 2024-04-26T23:01:33.596 INFO:tasks.workunit.client.0.smithi027.stdout:7/675: truncate d0/d28/d2b/d3a/d48/fb2 902509 0 2024-04-26T23:01:33.597 INFO:tasks.workunit.client.0.smithi027.stdout:1/644: rename d1/d26/d66/d8d to d1/d26/dc8 0 2024-04-26T23:01:33.597 INFO:tasks.workunit.client.0.smithi027.stdout:1/645: chown d1/d26/d6b/fa1 2701 1 2024-04-26T23:01:33.598 INFO:tasks.workunit.client.0.smithi027.stdout:7/676: rename d0/d28/d2b/d55/d59/d6d/lbb to d0/d28/d66/d67/da6/ld7 0 2024-04-26T23:01:33.599 INFO:tasks.workunit.client.0.smithi027.stdout:7/677: symlink d0/d28/d2b/d55/d69/d75/ld8 0 2024-04-26T23:01:33.600 INFO:tasks.workunit.client.0.smithi027.stdout:7/678: truncate d0/d28/d2b/d3a/f52 3279112 0 2024-04-26T23:01:33.602 INFO:tasks.workunit.client.0.smithi027.stdout:7/679: symlink d0/d28/d2b/d55/d59/d6d/ld9 0 2024-04-26T23:01:33.603 INFO:tasks.workunit.client.0.smithi027.stdout:7/680: creat d0/d28/d2b/d3a/d48/d56/d88/fda x:0 0 0 2024-04-26T23:01:33.605 INFO:tasks.workunit.client.0.smithi027.stdout:7/681: symlink d0/d28/d2b/d3a/d48/d56/d88/d73/dd6/ldb 0 2024-04-26T23:01:33.606 INFO:tasks.workunit.client.0.smithi027.stdout:3/628: dwrite d6/dd3/d2f/f39 [0,4194304] 0 2024-04-26T23:01:33.607 INFO:tasks.workunit.client.0.smithi027.stdout:7/682: creat d0/d28/d2b/d3a/d48/d56/db6/fdc x:0 0 0 2024-04-26T23:01:33.607 INFO:tasks.workunit.client.0.smithi027.stdout:7/683: write d0/f30 [2529701,124962] 0 2024-04-26T23:01:33.607 INFO:tasks.workunit.client.0.smithi027.stdout:7/684: dread - d0/d28/d2b/d55/d69/da7/fc6 zero size 2024-04-26T23:01:33.608 INFO:tasks.workunit.client.0.smithi027.stdout:3/629: mknod d6/dd3/d88/cd5 0 2024-04-26T23:01:33.608 INFO:tasks.workunit.client.0.smithi027.stdout:3/630: fdatasync d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d40/f57 0 2024-04-26T23:01:33.609 INFO:tasks.workunit.client.0.smithi027.stdout:3/631: write d6/fbc [682343,16546] 0 2024-04-26T23:01:33.609 INFO:tasks.workunit.client.0.smithi027.stdout:3/632: truncate d6/dd3/d34/d47/f69 354694 0 2024-04-26T23:01:33.610 INFO:tasks.workunit.client.0.smithi027.stdout:3/633: mkdir d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/dd6 0 2024-04-26T23:01:33.612 INFO:tasks.workunit.client.0.smithi027.stdout:3/634: mknod d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/cd7 0 2024-04-26T23:01:33.613 INFO:tasks.workunit.client.0.smithi027.stdout:3/635: getdents d6/dd3/d34/d2e/d9d/da7/dac/dca 0 2024-04-26T23:01:33.619 INFO:tasks.workunit.client.0.smithi027.stdout:1/646: dread d1/d26/d36/d56/f6e [0,4194304] 0 2024-04-26T23:01:33.619 INFO:tasks.workunit.client.0.smithi027.stdout:1/647: chown d1/d26/d66/d73/dae 6 1 2024-04-26T23:01:33.619 INFO:tasks.workunit.client.0.smithi027.stdout:3/636: mknod d6/dd3/d34/d2e/d5f/d90/db9/cd8 0 2024-04-26T23:01:33.619 INFO:tasks.workunit.client.0.smithi027.stdout:3/637: read d6/dd3/d34/d2e/f31 [3146591,20604] 0 2024-04-26T23:01:33.623 INFO:tasks.workunit.client.0.smithi027.stdout:3/638: dread d6/dd3/d34/d2e/d70/d71/f9f [0,4194304] 0 2024-04-26T23:01:33.624 INFO:tasks.workunit.client.0.smithi027.stdout:3/639: mknod d6/d5e/cd9 0 2024-04-26T23:01:33.625 INFO:tasks.workunit.client.0.smithi027.stdout:1/648: dread d1/d26/d6b/f5a [0,4194304] 0 2024-04-26T23:01:33.625 INFO:tasks.workunit.client.0.smithi027.stdout:3/640: mknod d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/cda 0 2024-04-26T23:01:33.625 INFO:tasks.workunit.client.0.smithi027.stdout:3/641: stat d6/d5e 0 2024-04-26T23:01:33.625 INFO:tasks.workunit.client.0.smithi027.stdout:3/642: truncate d6/dd3/d34/d2e/d5f/d90/fa9 926432 0 2024-04-26T23:01:33.626 INFO:tasks.workunit.client.0.smithi027.stdout:1/649: link d1/d26/d7d/l33 d1/d26/d86/d5c/lc9 0 2024-04-26T23:01:33.626 INFO:tasks.workunit.client.0.smithi027.stdout:3/643: unlink d6/dd3/fd 0 2024-04-26T23:01:33.627 INFO:tasks.workunit.client.0.smithi027.stdout:1/650: mkdir d1/d26/d36/d88/dca 0 2024-04-26T23:01:33.627 INFO:tasks.workunit.client.0.smithi027.stdout:1/651: write d1/d26/d6b/f5a [1727540,102450] 0 2024-04-26T23:01:33.628 INFO:tasks.workunit.client.0.smithi027.stdout:1/652: rmdir d1/d26/d66/d73/dae/dbf 0 2024-04-26T23:01:33.629 INFO:tasks.workunit.client.0.smithi027.stdout:1/653: truncate d1/d26/d7d/f30 2544479 0 2024-04-26T23:01:33.630 INFO:tasks.workunit.client.0.smithi027.stdout:1/654: creat d1/d26/d66/d73/d9e/fcb x:0 0 0 2024-04-26T23:01:33.632 INFO:tasks.workunit.client.0.smithi027.stdout:1/655: mkdir d1/d26/d66/dcc 0 2024-04-26T23:01:33.634 INFO:tasks.workunit.client.0.smithi027.stdout:1/656: unlink d1/d5/d21/f34 0 2024-04-26T23:01:33.634 INFO:tasks.workunit.client.0.smithi027.stdout:1/657: write d1/f32 [3937150,86887] 0 2024-04-26T23:01:33.634 INFO:tasks.workunit.client.0.smithi027.stdout:1/658: write d1/d26/d36/d88/f99 [388858,106066] 0 2024-04-26T23:01:33.646 INFO:tasks.workunit.client.0.smithi027.stdout:3/644: dread d6/dd3/d2f/d7e/f6e [0,4194304] 0 2024-04-26T23:01:33.652 INFO:tasks.workunit.client.0.smithi027.stdout:3/645: rename d6/dd3/d34/d47/d7b to d6/dd3/d34/d2e/d5f/d90/ddb 0 2024-04-26T23:01:33.653 INFO:tasks.workunit.client.0.smithi027.stdout:1/659: dread d1/dc/fa9 [0,4194304] 0 2024-04-26T23:01:33.686 INFO:tasks.workunit.client.0.smithi027.stdout:3/646: dwrite d6/dd3/d2f/d7e/f6e [0,4194304] 0 2024-04-26T23:01:33.692 INFO:tasks.workunit.client.0.smithi027.stdout:1/660: dwrite d1/f55 [0,4194304] 0 2024-04-26T23:01:33.696 INFO:tasks.workunit.client.0.smithi027.stdout:1/661: creat d1/d26/d6b/d42/d79/fcd x:0 0 0 2024-04-26T23:01:33.697 INFO:tasks.workunit.client.0.smithi027.stdout:1/662: creat d1/d26/d6b/d42/d70/fce x:0 0 0 2024-04-26T23:01:33.697 INFO:tasks.workunit.client.0.smithi027.stdout:1/663: symlink d1/d26/d66/d73/lcf 0 2024-04-26T23:01:33.703 INFO:tasks.workunit.client.0.smithi027.stdout:3/647: dread d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/f97 [0,4194304] 0 2024-04-26T23:01:33.709 INFO:tasks.workunit.client.0.smithi027.stdout:3/648: read d6/dd3/d34/d2e/d70/d71/fc5 [77186,92834] 0 2024-04-26T23:01:33.709 INFO:tasks.workunit.client.0.smithi027.stdout:3/649: chown d6/dd3/d34/d2e/d5f/d90/ddb/fa4 54019628 1 2024-04-26T23:01:33.711 INFO:tasks.workunit.client.0.smithi027.stdout:3/650: getdents d6/dd3/d2f/d7e 0 2024-04-26T23:01:33.715 INFO:tasks.workunit.client.0.smithi027.stdout:1/664: write d1/d26/d6b/d42/d70/fc7 [995727,108225] 0 2024-04-26T23:01:33.716 INFO:tasks.workunit.client.0.smithi027.stdout:1/665: mkdir d1/d26/d66/d73/dae/dd0 0 2024-04-26T23:01:33.718 INFO:tasks.workunit.client.0.smithi027.stdout:1/666: creat d1/d5/d21/fd1 x:0 0 0 2024-04-26T23:01:33.719 INFO:tasks.workunit.client.0.smithi027.stdout:1/667: symlink d1/d26/d66/ld2 0 2024-04-26T23:01:33.721 INFO:tasks.workunit.client.0.smithi027.stdout:3/651: dread d6/dd3/d34/d2e/d5f/d90/ddb/fa4 [0,4194304] 0 2024-04-26T23:01:33.723 INFO:tasks.workunit.client.0.smithi027.stdout:3/652: link d6/dd3/d34/d9b/l9c d6/dd3/d34/db3/ldc 0 2024-04-26T23:01:33.743 INFO:tasks.workunit.client.0.smithi027.stdout:3/653: dread d6/dd3/f25 [0,4194304] 0 2024-04-26T23:01:33.744 INFO:tasks.workunit.client.0.smithi027.stdout:1/668: dread d1/d5/d21/f27 [0,4194304] 0 2024-04-26T23:01:33.744 INFO:tasks.workunit.client.0.smithi027.stdout:1/669: stat d1/d26/d7d/f30 0 2024-04-26T23:01:33.744 INFO:tasks.workunit.client.0.smithi027.stdout:5/613: sync 2024-04-26T23:01:33.744 INFO:tasks.workunit.client.0.smithi027.stdout:3/654: symlink d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/ldd 0 2024-04-26T23:01:33.744 INFO:tasks.workunit.client.0.smithi027.stdout:2/540: sync 2024-04-26T23:01:33.748 INFO:tasks.workunit.client.0.smithi027.stdout:5/614: creat d4/d1a/d2b/fc2 x:0 0 0 2024-04-26T23:01:33.750 INFO:tasks.workunit.client.0.smithi027.stdout:2/541: creat d2/d5/db/de/d21/f9d x:0 0 0 2024-04-26T23:01:33.751 INFO:tasks.workunit.client.0.smithi027.stdout:1/670: symlink d1/d26/d6b/d8e/dc5/ld3 0 2024-04-26T23:01:33.752 INFO:tasks.workunit.client.0.smithi027.stdout:5/615: symlink d4/d24/d22/d29/lc3 0 2024-04-26T23:01:33.752 INFO:tasks.workunit.client.0.smithi027.stdout:3/655: dread d6/dd3/d34/d2e/d70/d71/f9f [0,4194304] 0 2024-04-26T23:01:33.752 INFO:tasks.workunit.client.0.smithi027.stdout:3/656: truncate d6/dd3/d34/d2e/f66 4212762 0 2024-04-26T23:01:33.754 INFO:tasks.workunit.client.0.smithi027.stdout:3/657: mknod d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/d5a/cde 0 2024-04-26T23:01:33.755 INFO:tasks.workunit.client.0.smithi027.stdout:1/671: creat d1/d26/d86/d5c/fd4 x:0 0 0 2024-04-26T23:01:33.756 INFO:tasks.workunit.client.0.smithi027.stdout:3/658: creat d6/dd3/d34/db3/fdf x:0 0 0 2024-04-26T23:01:33.766 INFO:tasks.workunit.client.0.smithi027.stdout:3/659: dread d6/dd3/d88/f9a [0,4194304] 0 2024-04-26T23:01:33.766 INFO:tasks.workunit.client.0.smithi027.stdout:2/542: dread d2/d5/f34 [0,4194304] 0 2024-04-26T23:01:33.766 INFO:tasks.workunit.client.0.smithi027.stdout:2/543: chown d2/d5/db/d1b/d1c/d25/d6a/d8a 461817662 1 2024-04-26T23:01:33.767 INFO:tasks.workunit.client.0.smithi027.stdout:3/660: mkdir d6/dd3/d34/d9b/de0 0 2024-04-26T23:01:33.768 INFO:tasks.workunit.client.0.smithi027.stdout:3/661: mknod d6/dd3/d34/d2e/d5f/d90/ddb/dce/ce1 0 2024-04-26T23:01:33.769 INFO:tasks.workunit.client.0.smithi027.stdout:3/662: truncate d6/f18 272799 0 2024-04-26T23:01:33.769 INFO:tasks.workunit.client.0.smithi027.stdout:3/663: write d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/f33 [3492610,26622] 0 2024-04-26T23:01:33.773 INFO:tasks.workunit.client.0.smithi027.stdout:5/616: write d4/d24/f42 [3682394,86187] 0 2024-04-26T23:01:33.775 INFO:tasks.workunit.client.0.smithi027.stdout:3/664: truncate d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d40/f4d 294678 0 2024-04-26T23:01:33.775 INFO:tasks.workunit.client.0.smithi027.stdout:3/665: write d6/dd3/d34/d78/fb6 [1133507,120266] 0 2024-04-26T23:01:33.775 INFO:tasks.workunit.client.0.smithi027.stdout:5/617: link d4/d24/d5f/dac/d40/l9d d4/d24/d22/d29/d39/d84/lc4 0 2024-04-26T23:01:33.775 INFO:tasks.workunit.client.0.smithi027.stdout:5/618: mkdir d4/d9f/dc5 0 2024-04-26T23:01:33.776 INFO:tasks.workunit.client.0.smithi027.stdout:5/619: stat d4/d24/d22/d29/l49 0 2024-04-26T23:01:33.777 INFO:tasks.workunit.client.0.smithi027.stdout:5/620: mknod d4/cc6 0 2024-04-26T23:01:33.778 INFO:tasks.workunit.client.0.smithi027.stdout:5/621: truncate d4/d24/d5f/dac/f5a 358837 0 2024-04-26T23:01:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.799+0000 7efc1e400640 1 -- 172.21.15.27:0/3476492969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc18101700 msgr2=0x7efc18101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.799+0000 7efc1e400640 1 --2- 172.21.15.27:0/3476492969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc18101700 0x7efc18101ae0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efc04009900 tx=0x7efc0402a4a0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.799+0000 7efc1e400640 1 -- 172.21.15.27:0/3476492969 shutdown_connections 2024-04-26T23:01:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.799+0000 7efc1e400640 1 --2- 172.21.15.27:0/3476492969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc18106b00 0x7efc18106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.799+0000 7efc1e400640 1 --2- 172.21.15.27:0/3476492969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18102020 0x7efc181064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.799+0000 7efc1e400640 1 --2- 172.21.15.27:0/3476492969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc18101700 0x7efc18101ae0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.799+0000 7efc1e400640 1 -- 172.21.15.27:0/3476492969 >> 172.21.15.27:0/3476492969 conn(0x7efc180fd110 msgr2=0x7efc180ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.802+0000 7efc1e400640 1 -- 172.21.15.27:0/3476492969 shutdown_connections 2024-04-26T23:01:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.802+0000 7efc1e400640 1 -- 172.21.15.27:0/3476492969 wait complete. 2024-04-26T23:01:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.803+0000 7efc1e400640 1 Processor -- start 2024-04-26T23:01:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.804+0000 7efc1e400640 1 -- start start 2024-04-26T23:01:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.805+0000 7efc1e400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc18101700 0x7efc1819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.805+0000 7efc1e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18102020 0x7efc1819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.805+0000 7efc1e400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc18106b00 0x7efc181a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.805+0000 7efc1e400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc18119040 con 0x7efc18106b00 2024-04-26T23:01:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.805+0000 7efc1e400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efc18118ec0 con 0x7efc18101700 2024-04-26T23:01:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.805+0000 7efc1e400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc181191c0 con 0x7efc18102020 2024-04-26T23:01:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.806+0000 7efc17e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18102020 0x7efc1819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.806+0000 7efc17e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18102020 0x7efc1819fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:49720/0 (socket says 172.21.15.27:49720) 2024-04-26T23:01:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.806+0000 7efc17e00640 1 -- 172.21.15.27:0/1895688611 learned_addr learned my addr 172.21.15.27:0/1895688611 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.806+0000 7efc17e00640 1 -- 172.21.15.27:0/1895688611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc18101700 msgr2=0x7efc1819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.806+0000 7efc1d000640 1 --2- 172.21.15.27:0/1895688611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc18101700 0x7efc1819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.806+0000 7efc17e00640 1 --2- 172.21.15.27:0/1895688611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc18101700 0x7efc1819f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.806+0000 7efc17e00640 1 -- 172.21.15.27:0/1895688611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc18106b00 msgr2=0x7efc181a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.806+0000 7efc17e00640 1 --2- 172.21.15.27:0/1895688611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc18106b00 0x7efc181a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.806+0000 7efc17e00640 1 -- 172.21.15.27:0/1895688611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc181a45d0 con 0x7efc18102020 2024-04-26T23:01:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.807+0000 7efc17e00640 1 --2- 172.21.15.27:0/1895688611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18102020 0x7efc1819fb20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efc0800e9f0 tx=0x7efc0800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.807+0000 7efc15600640 1 -- 172.21.15.27:0/1895688611 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0800ccc0 con 0x7efc18102020 2024-04-26T23:01:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.807+0000 7efc15600640 1 -- 172.21.15.27:0/1895688611 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc080108e0 con 0x7efc18102020 2024-04-26T23:01:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.807+0000 7efc15600640 1 -- 172.21.15.27:0/1895688611 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc08018820 con 0x7efc18102020 2024-04-26T23:01:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.807+0000 7efc1e400640 1 -- 172.21.15.27:0/1895688611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc181a48c0 con 0x7efc18102020 2024-04-26T23:01:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.807+0000 7efc1e400640 1 -- 172.21.15.27:0/1895688611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc181ac150 con 0x7efc18102020 2024-04-26T23:01:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.815+0000 7efc2284c740 1 Processor -- start 2024-04-26T23:01:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.816+0000 7efc2284c740 1 -- start start 2024-04-26T23:01:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.816+0000 7efc2284c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 0x55e57bd593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.816+0000 7efc2284c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd59900 0x55e57bd63c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.816+0000 7efc2284c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd641c0 0x55e57bd665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.816+0000 7efc2284c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e57bcbe380 con 0x55e57bd641c0 2024-04-26T23:01:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.816+0000 7efc2284c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55e57bc5ac30 con 0x55e57bd58fe0 2024-04-26T23:01:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.816+0000 7efc2284c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e57bcba340 con 0x55e57bd59900 2024-04-26T23:01:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.817+0000 7efc17e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd641c0 0x55e57bd665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.817+0000 7efc17e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd641c0 0x55e57bd665b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58192/0 (socket says 172.21.15.27:58192) 2024-04-26T23:01:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.817+0000 7efc17e00640 1 -- 172.21.15.27:0/1851465378 learned_addr learned my addr 172.21.15.27:0/1851465378 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.817+0000 7efc1da00640 1 --2- 172.21.15.27:0/1851465378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 0x55e57bd593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.817+0000 7efc15600640 1 -- 172.21.15.27:0/1895688611 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efc0801f020 con 0x7efc18102020 2024-04-26T23:01:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.818+0000 7efc1da00640 1 -- 172.21.15.27:0/1851465378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd59900 msgr2=0x55e57bd63c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:01:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.818+0000 7efc1da00640 1 --2- 172.21.15.27:0/1851465378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd59900 0x55e57bd63c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.818+0000 7efc1da00640 1 -- 172.21.15.27:0/1851465378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd641c0 msgr2=0x55e57bd665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.818+0000 7efc1da00640 1 --2- 172.21.15.27:0/1851465378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd641c0 0x55e57bd665b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.818+0000 7efc1da00640 1 -- 172.21.15.27:0/1851465378 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e57bc7c940 con 0x55e57bd58fe0 2024-04-26T23:01:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.818+0000 7efc1da00640 1 --2- 172.21.15.27:0/1851465378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 0x55e57bd593c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efc10009cc0 tx=0x7efc1000b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47a0393e319c95fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.820+0000 7efc15600640 1 --2- 172.21.15.27:0/1895688611 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7efbfc0a3e00 0x7efbfc0a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.820+0000 7efc15600640 1 -- 172.21.15.27:0/1895688611 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7efc080d0e20 con 0x7efc18102020 2024-04-26T23:01:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.821+0000 7efc0e000640 1 -- 172.21.15.27:0/1851465378 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10018070 con 0x55e57bd58fe0 2024-04-26T23:01:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.821+0000 7efc0e000640 1 -- 172.21.15.27:0/1851465378 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc100138c0 con 0x55e57bd58fe0 2024-04-26T23:01:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.821+0000 7efc0e000640 1 -- 172.21.15.27:0/1851465378 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc100128b0 con 0x55e57bd58fe0 2024-04-26T23:01:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.821+0000 7efc1d000640 1 --2- 172.21.15.27:0/1895688611 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7efbfc0a3e00 0x7efbfc0a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.821+0000 7efc2284c740 1 -- 172.21.15.27:0/1851465378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 msgr2=0x55e57bd593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.821+0000 7efc2284c740 1 --2- 172.21.15.27:0/1851465378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 0x55e57bd593c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efc10009cc0 tx=0x7efc1000b7c0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.823 INFO:tasks.workunit.client.0.smithi027.stdout:2/544: dwrite d2/d5/f6e [0,4194304] 0 2024-04-26T23:01:33.823 INFO:tasks.workunit.client.0.smithi027.stdout:3/666: dwrite d6/dd3/d34/d78/da1/f7c [0,4194304] 0 2024-04-26T23:01:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.822+0000 7efc2284c740 1 -- 172.21.15.27:0/1851465378 shutdown_connections 2024-04-26T23:01:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.822+0000 7efc2284c740 1 --2- 172.21.15.27:0/1851465378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd641c0 0x55e57bd665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.822+0000 7efc2284c740 1 --2- 172.21.15.27:0/1851465378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd59900 0x55e57bd63c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.822+0000 7efc2284c740 1 --2- 172.21.15.27:0/1851465378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 0x55e57bd593c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.822+0000 7efc2284c740 1 -- 172.21.15.27:0/1851465378 >> 172.21.15.27:0/1851465378 conn(0x55e57bd554f0 msgr2=0x55e57bd558d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.822+0000 7efc2284c740 1 -- 172.21.15.27:0/1851465378 shutdown_connections 2024-04-26T23:01:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.822+0000 7efc2284c740 1 -- 172.21.15.27:0/1851465378 wait complete. 2024-04-26T23:01:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.822+0000 7efc1d000640 1 --2- 172.21.15.27:0/1895688611 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7efbfc0a3e00 0x7efbfc0a62c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc040315d0 tx=0x7efc04000bc0 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.823+0000 7efc2284c740 1 -- start start 2024-04-26T23:01:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.823+0000 7efc2284c740 1 Processor -- start 2024-04-26T23:01:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.824+0000 7efc2284c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 0x55e57bd611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.824+0000 7efc2284c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd59900 0x55e57bd616f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.824+0000 7efc2284c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd641c0 0x55e57bd61c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.825 INFO:tasks.workunit.client.0.smithi027.stdout:2/545: mknod d2/d5/db/de/c9e 0 2024-04-26T23:01:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.824+0000 7efc2284c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e57bd77e00 con 0x55e57bd641c0 2024-04-26T23:01:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.824+0000 7efc1d000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 0x55e57bd611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.824+0000 7efc1d000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 0x55e57bd611b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33256/0 (socket says 172.21.15.27:33256) 2024-04-26T23:01:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.824+0000 7efc1d000640 1 -- 172.21.15.27:0/2501776458 learned_addr learned my addr 172.21.15.27:0/2501776458 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.824+0000 7efc17e00640 1 --2- 172.21.15.27:0/2501776458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd59900 0x55e57bd616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.825+0000 7efc2284c740 1 -- 172.21.15.27:0/2501776458 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55e57bc45c60 con 0x55e57bd58fe0 2024-04-26T23:01:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.825+0000 7efc2284c740 1 -- 172.21.15.27:0/2501776458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e57bd77f80 con 0x55e57bd59900 2024-04-26T23:01:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.825+0000 7efc2284c740 10 client.0 _ll_register_callbacks cb 0x55e57bc71e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:01:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.826+0000 7efc1d000640 1 -- 172.21.15.27:0/2501776458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd59900 msgr2=0x55e57bd616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.826+0000 7efc1d000640 1 --2- 172.21.15.27:0/2501776458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd59900 0x55e57bd616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.826+0000 7efc1d000640 1 -- 172.21.15.27:0/2501776458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd641c0 msgr2=0x55e57bd61c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.826+0000 7efc1d000640 1 --2- 172.21.15.27:0/2501776458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd641c0 0x55e57bd61c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.826+0000 7efc1d000640 1 -- 172.21.15.27:0/2501776458 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e57bc54510 con 0x55e57bd58fe0 2024-04-26T23:01:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.826+0000 7efc17e00640 1 --2- 172.21.15.27:0/2501776458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd59900 0x55e57bd616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:01:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.826+0000 7efc1d000640 1 --2- 172.21.15.27:0/2501776458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 0x55e57bd611b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efc04009900 tx=0x7efc04009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be1695ec0141ea64 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.826+0000 7efc02a00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T23:01:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.826+0000 7efc02a00640 1 -- 172.21.15.27:0/2501776458 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc04041070 con 0x55e57bd58fe0 2024-04-26T23:01:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.826+0000 7efc02a00640 1 -- 172.21.15.27:0/2501776458 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7efc04004ad0 con 0x55e57bd58fe0 2024-04-26T23:01:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.826+0000 7efc2284c740 1 -- 172.21.15.27:0/2501776458 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc100138c0 con 0x55e57bd58fe0 2024-04-26T23:01:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.827+0000 7efc02a00640 1 -- 172.21.15.27:0/2501776458 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efc04035050 con 0x55e57bd58fe0 2024-04-26T23:01:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.827+0000 7efc2284c740 10 client.25685 fetch_fsmap learned FSMap version 27 2024-04-26T23:01:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.827+0000 7efc2284c740 1 -- 172.21.15.27:0/2501776458 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e57b9d0210 con 0x55e57bd58fe0 2024-04-26T23:01:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.828+0000 7efc02a00640 1 -- 172.21.15.27:0/2501776458 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc100138c0 con 0x55e57bd58fe0 2024-04-26T23:01:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.828+0000 7efc02a00640 1 -- 172.21.15.27:0/2501776458 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7efc040035f0 con 0x55e57bd58fe0 2024-04-26T23:01:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.828+0000 7efc2284c740 10 client.25685 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T23:01:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.828+0000 7efc2284c740 10 client.25685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:01:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.828+0000 7efc2284c740 20 client.25685 populate_metadata read hostname 'smithi027' 2024-04-26T23:01:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.828+0000 7efc2284c740 1 --2- 172.21.15.27:0/2501776458 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e57bd84fc0 0x55e57bd87410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.828+0000 7efc2284c740 4 client.25685 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:01:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.828+0000 7efc2284c740 1 -- 172.21.15.27:0/2501776458 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e57bccbf90 con 0x55e57bd84fc0 2024-04-26T23:01:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.828+0000 7efc17e00640 1 --2- 172.21.15.27:0/2501776458 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e57bd84fc0 0x55e57bd87410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.829+0000 7efc02a00640 10 client.25685 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:01:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.829+0000 7efc17e00640 1 --2- 172.21.15.27:0/2501776458 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e57bd84fc0 0x55e57bd87410 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ff5d361c85fb10 server_cookie=f6984c77372c9a1 in_seq=0 out_seq=0 2024-04-26T23:01:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.830+0000 7efc02a00640 1 -- 172.21.15.27:0/2501776458 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e57bccbf90 con 0x55e57bd84fc0 2024-04-26T23:01:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.830+0000 7efc02a00640 10 client.25685 handle_command_reply: tid=0 2024-04-26T23:01:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.830+0000 7efc2284c740 1 client.25685 shutdown 2024-04-26T23:01:33.831 INFO:tasks.workunit.client.0.smithi027.stdout:3/667: link d6/dd3/d88/f9a d6/dd3/d2f/fe2 0 2024-04-26T23:01:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.830+0000 7efc2284c740 1 -- 172.21.15.27:0/2501776458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 msgr2=0x55e57bd611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.830+0000 7efc2284c740 1 --2- 172.21.15.27:0/2501776458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 0x55e57bd611b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efc04009900 tx=0x7efc04009260 comp rx=0 tx=0).stop 2024-04-26T23:01:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.831+0000 7efc2284c740 1 -- 172.21.15.27:0/2501776458 shutdown_connections 2024-04-26T23:01:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.831+0000 7efc2284c740 1 --2- 172.21.15.27:0/2501776458 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e57bd84fc0 0x55e57bd87410 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.831+0000 7efc2284c740 1 --2- 172.21.15.27:0/2501776458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd641c0 0x55e57bd61c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.831+0000 7efc2284c740 1 --2- 172.21.15.27:0/2501776458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd59900 0x55e57bd616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.831+0000 7efc02a00640 0 client.25685 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:01:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.831+0000 7efc2284c740 1 --2- 172.21.15.27:0/2501776458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 0x55e57bd611b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.831+0000 7efc2284c740 1 -- 172.21.15.27:0/2501776458 >> 172.21.15.27:0/2501776458 conn(0x55e57bd554f0 msgr2=0x55e57bd5b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.831+0000 7efc2284c740 1 -- 172.21.15.27:0/2501776458 shutdown_connections 2024-04-26T23:01:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.831+0000 7efc2284c740 1 -- 172.21.15.27:0/2501776458 wait complete. 2024-04-26T23:01:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.831+0000 7efc2284c740 20 client.25685 trim_cache size 0 max 16384 2024-04-26T23:01:33.832 INFO:tasks.workunit.client.0.smithi027.stdout:3/668: truncate d6/dd3/d34/d2e/d5f/fc4 3369152 0 2024-04-26T23:01:33.837 INFO:tasks.workunit.client.0.smithi027.stdout:3/669: rename d6/dd3/d34/l5b to d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/dd6/le3 0 2024-04-26T23:01:33.837 INFO:tasks.workunit.client.0.smithi027.stdout:3/670: rename d6/dd3/d34/d2e to d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/de4 22 2024-04-26T23:01:33.839 INFO:tasks.workunit.client.0.smithi027.stdout:3/671: symlink d6/dd3/d34/d2e/d5f/d90/ddb/dce/le5 0 2024-04-26T23:01:33.841 INFO:tasks.workunit.client.0.smithi027.stdout:3/672: mknod d6/dd3/d34/d78/ce6 0 2024-04-26T23:01:33.841 INFO:tasks.workunit.client.0.smithi027.stdout:3/673: write d6/dd3/d34/d2e/d5f/d90/db9/fcc [227386,76511] 0 2024-04-26T23:01:33.841 INFO:tasks.workunit.client.0.smithi027.stdout:3/674: readlink d6/dd3/l14 0 2024-04-26T23:01:33.841 INFO:tasks.workunit.client.0.smithi027.stdout:2/546: dread d2/d5/db/d1b/d1c/d25/d19/d36/f43 [0,4194304] 0 2024-04-26T23:01:33.842 INFO:tasks.workunit.client.0.smithi027.stdout:2/547: chown d2/d5/db/d1b/d1c/d25/l90 139619 1 2024-04-26T23:01:33.845 INFO:tasks.workunit.client.0.smithi027.stdout:2/548: symlink d2/d5/db/de/d21/d8f/l9f 0 2024-04-26T23:01:33.846 INFO:tasks.workunit.client.0.smithi027.stdout:3/675: symlink d6/dd3/d34/d2e/d5f/d90/ddb/d98/le7 0 2024-04-26T23:01:33.846 INFO:tasks.workunit.client.0.smithi027.stdout:3/676: stat d6/dd3/lcf 0 2024-04-26T23:01:33.846 INFO:tasks.workunit.client.0.smithi027.stdout:3/677: dread - d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/fc2 zero size 2024-04-26T23:01:33.846 INFO:tasks.workunit.client.0.smithi027.stdout:3/678: creat d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/fe8 x:0 0 0 2024-04-26T23:01:33.851 INFO:tasks.workunit.client.0.smithi027.stdout:3/679: symlink d6/le9 0 2024-04-26T23:01:33.851 INFO:tasks.workunit.client.0.smithi027.stdout:3/680: readlink d6/dd3/d34/db3/db4/ld1 0 2024-04-26T23:01:33.851 INFO:tasks.workunit.client.0.smithi027.stdout:1/672: dwrite d1/dc/fa7 [0,4194304] 0 2024-04-26T23:01:33.851 INFO:tasks.workunit.client.0.smithi027.stdout:3/681: creat d6/dd3/d34/d2e/d9d/da3/fea x:0 0 0 2024-04-26T23:01:33.851 INFO:tasks.workunit.client.0.smithi027.stdout:3/682: chown d6/dd3/d34/d2e/d5f/d90/ddb/d98/le7 26865 1 2024-04-26T23:01:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.853+0000 7efc2284c740 1 Processor -- start 2024-04-26T23:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.853+0000 7efc2284c740 1 -- start start 2024-04-26T23:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.854+0000 7efc2284c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd58fe0 0x55e57bd5b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.854+0000 7efc2284c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd59900 0x55e57bd61a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.854+0000 7efc2284c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd641c0 0x55e57bd62fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.854+0000 7efc2284c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc181b05b0 con 0x55e57bd58fe0 2024-04-26T23:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.854+0000 7efc2284c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efc181ade20 con 0x55e57bd641c0 2024-04-26T23:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.854+0000 7efc2284c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc180691d0 con 0x55e57bd59900 2024-04-26T23:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.855+0000 7efc1da00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd641c0 0x55e57bd62fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.855+0000 7efc1da00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd641c0 0x55e57bd62fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:33268/0 (socket says 172.21.15.27:33268) 2024-04-26T23:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.855+0000 7efc1da00640 1 -- 172.21.15.27:0/2733659032 learned_addr learned my addr 172.21.15.27:0/2733659032 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.855+0000 7efc17e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd59900 0x55e57bd61a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.856+0000 7efc1d000640 1 --2- 172.21.15.27:0/2733659032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd58fe0 0x55e57bd5b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.856+0000 7efc1da00640 1 -- 172.21.15.27:0/2733659032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd59900 msgr2=0x55e57bd61a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.856+0000 7efc1da00640 1 --2- 172.21.15.27:0/2733659032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd59900 0x55e57bd61a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.856+0000 7efc1da00640 1 -- 172.21.15.27:0/2733659032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd58fe0 msgr2=0x55e57bd5b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.856+0000 7efc1da00640 1 --2- 172.21.15.27:0/2733659032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd58fe0 0x55e57bd5b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.856+0000 7efc1da00640 1 -- 172.21.15.27:0/2733659032 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e57ba9c620 con 0x55e57bd641c0 2024-04-26T23:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.856+0000 7efc1d000640 1 --2- 172.21.15.27:0/2733659032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd58fe0 0x55e57bd5b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.857+0000 7efc1da00640 1 --2- 172.21.15.27:0/2733659032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd641c0 0x55e57bd62fe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efc10016040 tx=0x7efc10004b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1c1c752cb067eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.857+0000 7efc0d600640 1 -- 172.21.15.27:0/2733659032 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10018040 con 0x55e57bd641c0 2024-04-26T23:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.857+0000 7efc0d600640 1 -- 172.21.15.27:0/2733659032 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc100098b0 con 0x55e57bd641c0 2024-04-26T23:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.857+0000 7efc0d600640 1 -- 172.21.15.27:0/2733659032 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1001e430 con 0x55e57bd641c0 2024-04-26T23:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.858+0000 7efc2284c740 1 -- 172.21.15.27:0/2733659032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd641c0 msgr2=0x55e57bd62fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.858+0000 7efc2284c740 1 --2- 172.21.15.27:0/2733659032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd641c0 0x55e57bd62fe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efc10016040 tx=0x7efc10004b30 comp rx=0 tx=0).stop 2024-04-26T23:01:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.859+0000 7efc2284c740 1 -- 172.21.15.27:0/2733659032 shutdown_connections 2024-04-26T23:01:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.859+0000 7efc2284c740 1 --2- 172.21.15.27:0/2733659032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd641c0 0x55e57bd62fe0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.859+0000 7efc2284c740 1 --2- 172.21.15.27:0/2733659032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd59900 0x55e57bd61a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.859+0000 7efc2284c740 1 --2- 172.21.15.27:0/2733659032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd58fe0 0x55e57bd5b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.859+0000 7efc2284c740 1 -- 172.21.15.27:0/2733659032 >> 172.21.15.27:0/2733659032 conn(0x55e57bd554f0 msgr2=0x55e57bd628d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.859+0000 7efc2284c740 1 -- 172.21.15.27:0/2733659032 shutdown_connections 2024-04-26T23:01:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.859+0000 7efc2284c740 1 -- 172.21.15.27:0/2733659032 wait complete. 2024-04-26T23:01:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.859+0000 7efc2284c740 1 -- start start 2024-04-26T23:01:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.860+0000 7efc2284c740 1 Processor -- start 2024-04-26T23:01:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.860+0000 7efc2284c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 0x55e57bd7d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.860+0000 7efc2284c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd59900 0x55e57bd78770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.860+0000 7efc2284c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd641c0 0x55e57bd78cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.860+0000 7efc2284c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc181ac380 con 0x55e57bd59900 2024-04-26T23:01:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.860+0000 7efc2284c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efc181ad750 con 0x55e57bd58fe0 2024-04-26T23:01:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.860+0000 7efc2284c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc181ac9b0 con 0x55e57bd641c0 2024-04-26T23:01:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.860+0000 7efc2284c740 10 client.0 _ll_register_callbacks cb 0x7efc08005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:01:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.861+0000 7efc1da00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd641c0 0x55e57bd78cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.861+0000 7efc1da00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd641c0 0x55e57bd78cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:49740/0 (socket says 172.21.15.27:49740) 2024-04-26T23:01:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.861+0000 7efc1da00640 1 -- 172.21.15.27:0/2864880853 learned_addr learned my addr 172.21.15.27:0/2864880853 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.861+0000 7efc17e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd59900 0x55e57bd78770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.861+0000 7efc1da00640 1 -- 172.21.15.27:0/2864880853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 msgr2=0x55e57bd7d300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:01:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.862+0000 7efc1d000640 1 --2- 172.21.15.27:0/2864880853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 0x55e57bd7d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.862+0000 7efc1da00640 1 --2- 172.21.15.27:0/2864880853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 0x55e57bd7d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.862+0000 7efc1da00640 1 -- 172.21.15.27:0/2864880853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd59900 msgr2=0x55e57bd78770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.862+0000 7efc1da00640 1 --2- 172.21.15.27:0/2864880853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd59900 0x55e57bd78770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.862+0000 7efc1da00640 1 -- 172.21.15.27:0/2864880853 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e57bc78cb0 con 0x55e57bd641c0 2024-04-26T23:01:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.862+0000 7efc17e00640 1 --2- 172.21.15.27:0/2864880853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd59900 0x55e57bd78770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:01:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.862+0000 7efc1da00640 1 --2- 172.21.15.27:0/2864880853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd641c0 0x55e57bd78cb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efc1000d010 tx=0x7efc10000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5d224b8178c9379 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.862+0000 7efc02a00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T23:01:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.863+0000 7efc02a00640 1 -- 172.21.15.27:0/2864880853 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1002d060 con 0x55e57bd641c0 2024-04-26T23:01:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.863+0000 7efc02a00640 1 -- 172.21.15.27:0/2864880853 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7efc10010040 con 0x55e57bd641c0 2024-04-26T23:01:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.863+0000 7efc2284c740 1 -- 172.21.15.27:0/2864880853 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc100098b0 con 0x55e57bd641c0 2024-04-26T23:01:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.863+0000 7efc02a00640 1 -- 172.21.15.27:0/2864880853 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efc10018040 con 0x55e57bd641c0 2024-04-26T23:01:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.863+0000 7efc2284c740 10 client.25735 fetch_fsmap learned FSMap version 27 2024-04-26T23:01:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.864+0000 7efc2284c740 1 -- 172.21.15.27:0/2864880853 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e57bdeb180 con 0x55e57bd641c0 2024-04-26T23:01:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.864+0000 7efc02a00640 1 -- 172.21.15.27:0/2864880853 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc100098b0 con 0x55e57bd641c0 2024-04-26T23:01:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.864+0000 7efc02a00640 1 -- 172.21.15.27:0/2864880853 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7efc10027070 con 0x55e57bd641c0 2024-04-26T23:01:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.864+0000 7efc2284c740 10 client.25735 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T23:01:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.864+0000 7efc2284c740 10 client.25735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:01:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.864+0000 7efc2284c740 20 client.25735 populate_metadata read hostname 'smithi027' 2024-04-26T23:01:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.864+0000 7efc2284c740 1 --2- 172.21.15.27:0/2864880853 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e57bdeb610 0x55e57bdeda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.864+0000 7efc2284c740 4 client.25735 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:01:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.864+0000 7efc2284c740 1 -- 172.21.15.27:0/2864880853 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e57bcc5e00 con 0x55e57bdeb610 2024-04-26T23:01:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.865+0000 7efc1d000640 1 --2- 172.21.15.27:0/2864880853 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e57bdeb610 0x55e57bdeda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.866+0000 7efc1d000640 1 --2- 172.21.15.27:0/2864880853 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e57bdeb610 0x55e57bdeda30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c140ddfc0e5af25 server_cookie=ec452ec2b2f0d036 in_seq=0 out_seq=0 2024-04-26T23:01:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:33.866+0000 7efc02a00640 10 client.25735 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:01:33.866 INFO:tasks.workunit.client.0.smithi027.stdout:5/622: dwrite d4/d24/d22/fa3 [0,4194304] 0 2024-04-26T23:01:33.867 INFO:tasks.workunit.client.0.smithi027.stdout:5/623: chown d4/d24/c11 9 1 2024-04-26T23:01:33.870 INFO:tasks.workunit.client.0.smithi027.stdout:5/624: read d4/d24/d5f/dac/f5a [304256,7732] 0 2024-04-26T23:01:33.883 INFO:tasks.workunit.client.0.smithi027.stdout:3/683: write d6/dd3/d2f/d7e/f6e [2942199,82412] 0 2024-04-26T23:01:33.884 INFO:tasks.workunit.client.0.smithi027.stdout:2/549: dwrite d2/d5/db/d1b/d1c/d25/d19/d23/f64 [4194304,4194304] 0 2024-04-26T23:01:33.887 INFO:tasks.workunit.client.0.smithi027.stdout:3/684: creat d6/dd3/d2f/feb x:0 0 0 2024-04-26T23:01:33.888 INFO:tasks.workunit.client.0.smithi027.stdout:3/685: getdents d6/dd3/d34/d9b 0 2024-04-26T23:01:33.888 INFO:tasks.workunit.client.0.smithi027.stdout:3/686: stat d6/dd3/d34/d78/fb6 0 2024-04-26T23:01:33.888 INFO:tasks.workunit.client.0.smithi027.stdout:1/673: dwrite d1/d5/d21/fd1 [0,4194304] 0 2024-04-26T23:01:33.889 INFO:tasks.workunit.client.0.smithi027.stdout:3/687: getdents d6/dd3/d34/d2e 0 2024-04-26T23:01:33.891 INFO:tasks.workunit.client.0.smithi027.stdout:3/688: creat d6/dd3/d34/d2e/d9d/da7/dac/dca/fec x:0 0 0 2024-04-26T23:01:33.893 INFO:tasks.workunit.client.0.smithi027.stdout:3/689: symlink d6/dd3/d34/d2e/d9d/da3/led 0 2024-04-26T23:01:33.894 INFO:tasks.workunit.client.0.smithi027.stdout:3/690: truncate d6/dd3/f25 7403501 0 2024-04-26T23:01:33.895 INFO:tasks.workunit.client.0.smithi027.stdout:3/691: creat d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d40/fee x:0 0 0 2024-04-26T23:01:33.895 INFO:tasks.workunit.client.0.smithi027.stdout:3/692: dread - d6/dd3/d34/d78/fbf zero size 2024-04-26T23:01:33.897 INFO:tasks.workunit.client.0.smithi027.stdout:3/693: write d6/f2c [1520232,130594] 0 2024-04-26T23:01:33.898 INFO:tasks.workunit.client.0.smithi027.stdout:3/694: mkdir d6/dd3/d34/d2e/d5f/def 0 2024-04-26T23:01:33.900 INFO:tasks.workunit.client.0.smithi027.stdout:3/695: rename d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/f97 to d6/dd3/d34/db3/ff0 0 2024-04-26T23:01:33.901 INFO:tasks.workunit.client.0.smithi027.stdout:3/696: link d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/dd6/le3 d6/dd3/d34/lf1 0 2024-04-26T23:01:33.902 INFO:tasks.workunit.client.0.smithi027.stdout:3/697: creat d6/d5e/ff2 x:0 0 0 2024-04-26T23:01:33.904 INFO:tasks.workunit.client.0.smithi027.stdout:3/698: creat d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/dd6/ff3 x:0 0 0 2024-04-26T23:01:33.905 INFO:tasks.workunit.client.0.smithi027.stdout:3/699: fdatasync d6/dd3/f11 0 2024-04-26T23:01:33.919 INFO:tasks.workunit.client.0.smithi027.stdout:3/700: dread d6/dd3/d34/d2e/f31 [0,4194304] 0 2024-04-26T23:01:33.921 INFO:tasks.workunit.client.0.smithi027.stdout:3/701: creat d6/dd3/d34/d2e/d70/d71/ff4 x:0 0 0 2024-04-26T23:01:33.922 INFO:tasks.workunit.client.0.smithi027.stdout:3/702: truncate d6/dd3/d34/d47/f69 122809 0 2024-04-26T23:01:33.923 INFO:tasks.workunit.client.0.smithi027.stdout:3/703: mkdir d6/dd3/d34/d2e/d5f/df5 0 2024-04-26T23:01:33.925 INFO:tasks.workunit.client.0.smithi027.stdout:3/704: creat d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/dd6/ff6 x:0 0 0 2024-04-26T23:01:33.925 INFO:tasks.workunit.client.0.smithi027.stdout:3/705: truncate d6/f18 322780 0 2024-04-26T23:01:33.955 INFO:tasks.workunit.client.0.smithi027.stdout:3/706: dwrite d6/dd3/f46 [0,4194304] 0 2024-04-26T23:01:33.956 INFO:tasks.workunit.client.0.smithi027.stdout:3/707: creat d6/dd3/d34/d2e/d9d/da7/dac/dca/ff7 x:0 0 0 2024-04-26T23:01:33.958 INFO:tasks.workunit.client.0.smithi027.stdout:3/708: rename d6/dd3/d88 to d6/dd3/d34/db3/df8 0 2024-04-26T23:01:33.958 INFO:tasks.workunit.client.0.smithi027.stdout:3/709: chown d6/dd3/d34/d2e/d5f/d90/fa9 1729999824 1 2024-04-26T23:01:33.959 INFO:tasks.workunit.client.0.smithi027.stdout:3/710: symlink d6/dd3/d34/d9b/de0/lf9 0 2024-04-26T23:01:33.960 INFO:tasks.workunit.client.0.smithi027.stdout:3/711: mknod d6/dd3/d34/d2e/d9d/da7/dac/dca/cfa 0 2024-04-26T23:01:33.961 INFO:tasks.workunit.client.0.smithi027.stdout:3/712: symlink d6/dd3/d34/d2e/d5f/d90/ddb/lfb 0 2024-04-26T23:01:33.984 INFO:tasks.workunit.client.0.smithi027.stdout:2/550: link d2/d5/db/d1b/f38 d2/d5/db/d1b/d1c/d25/d19/d36/fa0 0 2024-04-26T23:01:34.013 INFO:tasks.workunit.client.0.smithi027.stdout:1/674: creat d1/d26/d66/dab/fd5 x:0 0 0 2024-04-26T23:01:34.013 INFO:tasks.workunit.client.0.smithi027.stdout:1/675: stat d1/d2a/fbc 0 2024-04-26T23:01:34.086 INFO:tasks.workunit.client.0.smithi027.stdout:2/551: dread d2/d5/db/d1b/f80 [0,4194304] 0 2024-04-26T23:01:34.205 INFO:tasks.workunit.client.0.smithi027.stdout:2/552: dread d2/d5/db/de/f30 [0,4194304] 0 2024-04-26T23:01:34.221 INFO:tasks.workunit.client.0.smithi027.stdout:3/713: dwrite d6/f4f [0,4194304] 0 2024-04-26T23:01:34.255 INFO:tasks.workunit.client.0.smithi027.stdout:1/676: rmdir d1/d26/d36/d88 39 2024-04-26T23:01:34.271 INFO:tasks.workunit.client.0.smithi027.stdout:5/625: mkdir d4/d24/d22/d29/d39/dc7 0 2024-04-26T23:01:34.271 INFO:tasks.workunit.client.0.smithi027.stdout:2/553: mknod d2/d5/db/de/ca1 0 2024-04-26T23:01:34.272 INFO:tasks.workunit.client.0.smithi027.stdout:2/554: getdents d2/d5/db/d1b/d1c/d25/d6a/d8a/d8d 0 2024-04-26T23:01:34.272 INFO:tasks.workunit.client.0.smithi027.stdout:2/555: write d2/d5/db/d1b/f80 [4956080,46167] 0 2024-04-26T23:01:34.272 INFO:tasks.workunit.client.0.smithi027.stdout:5/626: creat d4/d24/d5f/dac/d40/fc8 x:0 0 0 2024-04-26T23:01:34.272 INFO:tasks.workunit.client.0.smithi027.stdout:5/627: write d4/d1a/d2b/fc2 [983680,38233] 0 2024-04-26T23:01:34.272 INFO:tasks.workunit.client.0.smithi027.stdout:5/628: dread - d4/d24/d5f/dac/d40/fc8 zero size 2024-04-26T23:01:34.276 INFO:tasks.workunit.client.0.smithi027.stdout:5/629: unlink d4/d24/d22/dab/cd 0 2024-04-26T23:01:34.276 INFO:tasks.workunit.client.0.smithi027.stdout:5/630: write d4/f64 [3435674,81140] 0 2024-04-26T23:01:34.306 INFO:tasks.workunit.client.0.smithi027.stdout:3/714: dwrite d6/dd3/d2f/fe2 [0,4194304] 0 2024-04-26T23:01:34.306 INFO:tasks.workunit.client.0.smithi027.stdout:2/556: dread d2/d5/db/d1b/d1c/d8b/f56 [0,4194304] 0 2024-04-26T23:01:34.307 INFO:tasks.workunit.client.0.smithi027.stdout:2/557: write d2/f6f [4426688,86898] 0 2024-04-26T23:01:34.309 INFO:tasks.workunit.client.0.smithi027.stdout:2/558: creat d2/d5/db/d1b/d1c/d25/d19/d44/fa2 x:0 0 0 2024-04-26T23:01:34.311 INFO:tasks.workunit.client.0.smithi027.stdout:2/559: creat d2/fa3 x:0 0 0 2024-04-26T23:01:34.312 INFO:tasks.workunit.client.0.smithi027.stdout:3/715: mkdir d6/dd3/d34/d2e/d5f/def/dfc 0 2024-04-26T23:01:34.312 INFO:tasks.workunit.client.0.smithi027.stdout:2/560: creat d2/d5/db/d1b/d1c/d25/fa4 x:0 0 0 2024-04-26T23:01:34.313 INFO:tasks.workunit.client.0.smithi027.stdout:5/631: write d4/fc1 [3511411,56256] 0 2024-04-26T23:01:34.317 INFO:tasks.workunit.client.0.smithi027.stdout:2/561: rename d2/d5/db/d1b/d1c/d25/d19/d44/l86 to d2/d5/db/d1b/d1c/d25/d6a/la5 0 2024-04-26T23:01:34.320 INFO:tasks.workunit.client.0.smithi027.stdout:5/632: link d4/d24/d22/d29/d2a/la0 d4/d24/d22/d29/d39/dc7/lc9 0 2024-04-26T23:01:34.327 INFO:tasks.workunit.client.0.smithi027.stdout:2/562: mknod d2/d5/db/de/d21/ca6 0 2024-04-26T23:01:34.327 INFO:tasks.workunit.client.0.smithi027.stdout:3/716: dread d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d40/f4d [0,4194304] 0 2024-04-26T23:01:34.328 INFO:tasks.workunit.client.0.smithi027.stdout:5/633: symlink d4/d1a/lca 0 2024-04-26T23:01:34.328 INFO:tasks.workunit.client.0.smithi027.stdout:5/634: fdatasync d4/f15 0 2024-04-26T23:01:34.329 INFO:tasks.workunit.client.0.smithi027.stdout:3/717: getdents d6/dd3/d34/d2e/d9d 0 2024-04-26T23:01:34.334 INFO:tasks.workunit.client.0.smithi027.stdout:2/563: mknod d2/d5/db/d1b/d1c/d25/d6a/d8a/d8d/ca7 0 2024-04-26T23:01:34.335 INFO:tasks.workunit.client.0.smithi027.stdout:5/635: mkdir d4/d1a/dcb 0 2024-04-26T23:01:34.338 INFO:tasks.workunit.client.0.smithi027.stdout:2/564: creat d2/d5/db/de/d21/fa8 x:0 0 0 2024-04-26T23:01:34.338 INFO:tasks.workunit.client.0.smithi027.stdout:2/565: dread - d2/d5/db/d1b/d1c/d25/d19/d36/d39/f8c zero size 2024-04-26T23:01:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.337+0000 7efc02a00640 1 -- 172.21.15.27:0/2864880853 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e57bcc5e00 con 0x55e57bdeb610 2024-04-26T23:01:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.338+0000 7efc02a00640 10 client.25735 handle_command_reply: tid=0 2024-04-26T23:01:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.338+0000 7efc2284c740 1 client.25735 shutdown 2024-04-26T23:01:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.338+0000 7efc2284c740 1 -- 172.21.15.27:0/2864880853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd641c0 msgr2=0x55e57bd78cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.338+0000 7efc2284c740 1 --2- 172.21.15.27:0/2864880853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd641c0 0x55e57bd78cb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efc1000d010 tx=0x7efc10000f40 comp rx=0 tx=0).stop 2024-04-26T23:01:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.338+0000 7efc2284c740 1 -- 172.21.15.27:0/2864880853 shutdown_connections 2024-04-26T23:01:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.338+0000 7efc2284c740 1 --2- 172.21.15.27:0/2864880853 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e57bdeb610 0x55e57bdeda30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.338+0000 7efc2284c740 1 --2- 172.21.15.27:0/2864880853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e57bd641c0 0x55e57bd78cb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.338+0000 7efc2284c740 1 --2- 172.21.15.27:0/2864880853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e57bd59900 0x55e57bd78770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.338+0000 7efc2284c740 1 --2- 172.21.15.27:0/2864880853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e57bd58fe0 0x55e57bd7d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.338+0000 7efc02a00640 0 client.25735 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:01:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.338+0000 7efc2284c740 1 -- 172.21.15.27:0/2864880853 >> 172.21.15.27:0/2864880853 conn(0x55e57bd554f0 msgr2=0x55e57bd5f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.338+0000 7efc2284c740 1 -- 172.21.15.27:0/2864880853 shutdown_connections 2024-04-26T23:01:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.339+0000 7efc2284c740 1 -- 172.21.15.27:0/2864880853 wait complete. 2024-04-26T23:01:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.339+0000 7efc2284c740 20 client.25735 trim_cache size 0 max 16384 2024-04-26T23:01:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T23:01:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T23:01:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T23:01:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T23:01:34.343 INFO:tasks.workunit.client.0.smithi027.stdout:2/566: creat d2/d5/db/d1b/d1c/d8b/fa9 x:0 0 0 2024-04-26T23:01:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.343+0000 7efc1e400640 1 -- 172.21.15.27:0/1895688611 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7efbfc0a3e00 msgr2=0x7efbfc0a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.343+0000 7efc1e400640 1 --2- 172.21.15.27:0/1895688611 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7efbfc0a3e00 0x7efbfc0a62c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc040315d0 tx=0x7efc04000bc0 comp rx=0 tx=0).stop 2024-04-26T23:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.343+0000 7efc1e400640 1 -- 172.21.15.27:0/1895688611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18102020 msgr2=0x7efc1819fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.343+0000 7efc1e400640 1 --2- 172.21.15.27:0/1895688611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18102020 0x7efc1819fb20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efc0800e9f0 tx=0x7efc0800eeb0 comp rx=0 tx=0).stop 2024-04-26T23:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.343+0000 7efc1e400640 1 -- 172.21.15.27:0/1895688611 shutdown_connections 2024-04-26T23:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.343+0000 7efc1e400640 1 --2- 172.21.15.27:0/1895688611 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7efbfc0a3e00 0x7efbfc0a62c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.343+0000 7efc1e400640 1 --2- 172.21.15.27:0/1895688611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc18106b00 0x7efc181a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.343+0000 7efc1e400640 1 --2- 172.21.15.27:0/1895688611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18102020 0x7efc1819fb20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.343+0000 7efc1e400640 1 --2- 172.21.15.27:0/1895688611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc18101700 0x7efc1819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.343+0000 7efc1e400640 1 -- 172.21.15.27:0/1895688611 >> 172.21.15.27:0/1895688611 conn(0x7efc180fd110 msgr2=0x7efc180fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.343+0000 7efc1e400640 1 -- 172.21.15.27:0/1895688611 shutdown_connections 2024-04-26T23:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:34.343+0000 7efc1e400640 1 -- 172.21.15.27:0/1895688611 wait complete. 2024-04-26T23:01:34.345 INFO:tasks.workunit.client.0.smithi027.stdout:2/567: stat d2/d5/db/d1b/d1c/d25/d6a/l8e 0 2024-04-26T23:01:34.345 INFO:tasks.workunit.client.0.smithi027.stdout:2/568: dread - d2/d5/db/de/d21/fa8 zero size 2024-04-26T23:01:34.345 INFO:tasks.workunit.client.0.smithi027.stdout:2/569: chown d2/d5/db/de/l85 9004961 1 2024-04-26T23:01:34.345 INFO:tasks.workunit.client.0.smithi027.stdout:2/570: write d2/f67 [932852,64879] 0 2024-04-26T23:01:34.347 INFO:tasks.workunit.client.0.smithi027.stdout:7/685: sync 2024-04-26T23:01:34.347 INFO:tasks.workunit.client.0.smithi027.stdout:4/641: sync 2024-04-26T23:01:34.347 INFO:tasks.workunit.client.0.smithi027.stdout:8/561: sync 2024-04-26T23:01:34.347 INFO:tasks.workunit.client.0.smithi027.stdout:9/532: sync 2024-04-26T23:01:34.348 INFO:tasks.workunit.client.0.smithi027.stdout:0/593: sync 2024-04-26T23:01:34.348 INFO:tasks.workunit.client.0.smithi027.stdout:6/536: sync 2024-04-26T23:01:34.348 INFO:tasks.workunit.client.0.smithi027.stdout:6/537: chown d1/d30/c60 2377 1 2024-04-26T23:01:34.348 INFO:tasks.workunit.client.0.smithi027.stdout:0/594: chown d0/d4/da/d1a/d6d/fa6 2498034 1 2024-04-26T23:01:34.348 INFO:tasks.workunit.client.0.smithi027.stdout:6/538: stat d1/f35 0 2024-04-26T23:01:34.348 INFO:tasks.workunit.client.0.smithi027.stdout:4/642: chown da/dd/de/d14/f44 0 1 2024-04-26T23:01:34.348 INFO:tasks.workunit.client.0.smithi027.stdout:0/595: chown d0/df/d9d/d9f 3497235 1 2024-04-26T23:01:34.348 INFO:tasks.workunit.client.0.smithi027.stdout:9/533: dread - d4/d17/d2e/d46/da0/d8d/fa8 zero size 2024-04-26T23:01:34.348 INFO:tasks.workunit.client.0.smithi027.stdout:6/539: fdatasync d1/df/f79 0 2024-04-26T23:01:34.348 INFO:tasks.workunit.client.0.smithi027.stdout:4/643: stat da/dd/de/d14/d21/d41/d59/d74/f7e 0 2024-04-26T23:01:34.348 INFO:tasks.workunit.client.0.smithi027.stdout:0/596: write d0/d4/da/d15/d26/f7c [5854707,105938] 0 2024-04-26T23:01:34.349 INFO:tasks.workunit.client.0.smithi027.stdout:7/686: mkdir d0/d28/d2b/d3a/d48/d56/d88/d73/d80/ddd 0 2024-04-26T23:01:34.354 INFO:tasks.workunit.client.0.smithi027.stdout:4/644: unlink c6 0 2024-04-26T23:01:34.354 INFO:tasks.workunit.client.0.smithi027.stdout:8/562: rmdir d9/da/d2a/d5e/d72/d84/d35 39 2024-04-26T23:01:34.357 INFO:tasks.workunit.client.0.smithi027.stdout:0/597: truncate d0/d4/f53 2760826 0 2024-04-26T23:01:34.357 INFO:tasks.workunit.client.0.smithi027.stdout:0/598: write d0/d4/da/d1a/d1e/fa5 [1688772,58273] 0 2024-04-26T23:01:34.359 INFO:tasks.workunit.client.0.smithi027.stdout:7/687: unlink d0/d28/d2b/d3a/d48/d56/fc7 0 2024-04-26T23:01:34.359 INFO:tasks.workunit.client.0.smithi027.stdout:7/688: chown d0/d2f 239 1 2024-04-26T23:01:34.364 INFO:tasks.workunit.client.0.smithi027.stdout:4/645: getdents da/dd/de/d14/d21/dad 0 2024-04-26T23:01:34.365 INFO:tasks.workunit.client.0.smithi027.stdout:4/646: truncate da/dd/de/d14/d17/d80/dab/d37/f58 815503 0 2024-04-26T23:01:34.365 INFO:tasks.workunit.client.0.smithi027.stdout:8/563: mkdir d9/da/d2a/d5e/d72/d84/d35/dc2/dc7 0 2024-04-26T23:01:34.365 INFO:tasks.workunit.client.0.smithi027.stdout:5/636: dread d4/d24/f4c [0,4194304] 0 2024-04-26T23:01:34.365 INFO:tasks.workunit.client.0.smithi027.stdout:8/564: rmdir d9/da/d2a/d5e/d72/d84/d35/d8d 39 2024-04-26T23:01:34.365 INFO:tasks.workunit.client.0.smithi027.stdout:8/565: fsync d9/da/d2a/d5e/d72/d84/f3f 0 2024-04-26T23:01:34.369 INFO:tasks.workunit.client.0.smithi027.stdout:0/599: symlink d0/d4/da/d1a/d1e/lae 0 2024-04-26T23:01:34.373 INFO:tasks.workunit.client.0.smithi027.stdout:7/689: dread d0/d3d/f43 [0,4194304] 0 2024-04-26T23:01:34.377 INFO:tasks.workunit.client.0.smithi027.stdout:5/637: creat d4/d9f/dc0/fcc x:0 0 0 2024-04-26T23:01:34.378 INFO:tasks.workunit.client.0.smithi027.stdout:9/534: dread d4/d17/d2e/f53 [0,4194304] 0 2024-04-26T23:01:34.378 INFO:tasks.workunit.client.0.smithi027.stdout:9/535: dread - d4/d10/d40/d58/d75/f8f zero size 2024-04-26T23:01:34.387 INFO:tasks.workunit.client.0.smithi027.stdout:7/690: dread d0/d28/d2b/d3a/d48/d56/d88/d73/f78 [0,4194304] 0 2024-04-26T23:01:34.396 INFO:tasks.workunit.client.0.smithi027.stdout:2/571: dwrite d2/d5/db/d1b/d1c/f77 [0,4194304] 0 2024-04-26T23:01:34.407 INFO:tasks.workunit.client.0.smithi027.stdout:4/647: dwrite da/dd/de/d79/da8/fbf [0,4194304] 0 2024-04-26T23:01:34.465 INFO:tasks.workunit.client.0.smithi027.stdout:9/536: rename d4/d10/d40/d58/d75/f8f to d4/d77/da2/daf/fbb 0 2024-04-26T23:01:34.465 INFO:tasks.workunit.client.0.smithi027.stdout:9/537: truncate d4/d17/d2e/d46/da0/d8d/fb5 630219 0 2024-04-26T23:01:34.465 INFO:tasks.workunit.client.0.smithi027.stdout:9/538: write d4/f2b [3196415,98798] 0 2024-04-26T23:01:34.465 INFO:tasks.workunit.client.0.smithi027.stdout:7/691: mknod d0/d28/d66/d67/da6/cde 0 2024-04-26T23:01:34.465 INFO:tasks.workunit.client.0.smithi027.stdout:7/692: fdatasync d0/d28/f18 0 2024-04-26T23:01:34.469 INFO:tasks.workunit.client.0.smithi027.stdout:4/648: creat da/dd/d5e/fcb x:0 0 0 2024-04-26T23:01:34.469 INFO:tasks.workunit.client.0.smithi027.stdout:4/649: fsync da/dd/de/d14/d21/d67/d8a/fa2 0 2024-04-26T23:01:34.472 INFO:tasks.workunit.client.0.smithi027.stdout:2/572: rename d2/d5/db/de/l93 to d2/d5/db/d1b/d1c/d25/d19/d36/d39/laa 0 2024-04-26T23:01:34.474 INFO:tasks.workunit.client.0.smithi027.stdout:9/539: link d4/d77/f7a d4/d17/d2e/d81/fbc 0 2024-04-26T23:01:34.475 INFO:tasks.workunit.client.0.smithi027.stdout:7/693: truncate d0/d28/d2b/d3a/d48/d56/d88/f39 4022992 0 2024-04-26T23:01:34.476 INFO:tasks.workunit.client.0.smithi027.stdout:4/650: link da/dd/de/d14/d17/f33 da/dd/d5f/fcc 0 2024-04-26T23:01:34.476 INFO:tasks.workunit.client.0.smithi027.stdout:4/651: write da/dd/d5f/fcc [1282209,63628] 0 2024-04-26T23:01:34.478 INFO:tasks.workunit.client.0.smithi027.stdout:9/540: creat d4/d17/d2e/d81/da7/db8/fbd x:0 0 0 2024-04-26T23:01:34.479 INFO:tasks.workunit.client.0.smithi027.stdout:7/694: symlink d0/d28/d2b/d3a/d48/d56/d8d/dc5/ldf 0 2024-04-26T23:01:34.480 INFO:tasks.workunit.client.0.smithi027.stdout:5/638: dread d4/d24/d22/d29/d2a/d3b/f3d [0,4194304] 0 2024-04-26T23:01:34.480 INFO:tasks.workunit.client.0.smithi027.stdout:5/639: write d4/d24/fb9 [92158,83096] 0 2024-04-26T23:01:34.481 INFO:tasks.workunit.client.0.smithi027.stdout:7/695: mknod d0/d28/d2b/ce0 0 2024-04-26T23:01:34.481 INFO:tasks.workunit.client.0.smithi027.stdout:7/696: chown d0/d28/d2b/d3a/d48/d56/db6 112 1 2024-04-26T23:01:34.484 INFO:tasks.workunit.client.0.smithi027.stdout:5/640: rename d4/d24/d5f/dac/d40/c74 to d4/d24/d22/d29/d2a/d3b/d50/ccd 0 2024-04-26T23:01:34.485 INFO:tasks.workunit.client.0.smithi027.stdout:7/697: mkdir d0/d28/d9f/de1 0 2024-04-26T23:01:34.485 INFO:tasks.workunit.client.0.smithi027.stdout:7/698: readlink d0/d28/d2b/d55/d59/d6d/l7c 0 2024-04-26T23:01:34.485 INFO:tasks.workunit.client.0.smithi027.stdout:2/573: write d2/f49 [1176630,15274] 0 2024-04-26T23:01:34.487 INFO:tasks.workunit.client.0.smithi027.stdout:2/574: creat d2/d5/db/d1b/d1c/d25/d19/d23/fab x:0 0 0 2024-04-26T23:01:34.488 INFO:tasks.workunit.client.0.smithi027.stdout:2/575: write d2/d5/db/d1b/d1c/d25/d6a/f4c [641627,117035] 0 2024-04-26T23:01:34.488 INFO:tasks.workunit.client.0.smithi027.stdout:2/576: write d2/d5/f7b [952524,120057] 0 2024-04-26T23:01:34.488 INFO:tasks.workunit.client.0.smithi027.stdout:2/577: chown d2/d5/db/de/d21/d8f/c4d 0 1 2024-04-26T23:01:34.490 INFO:tasks.workunit.client.0.smithi027.stdout:2/578: unlink d2/d5/db/de/d21/d8f/c4d 0 2024-04-26T23:01:34.490 INFO:tasks.workunit.client.0.smithi027.stdout:2/579: stat d2/d5/db/d1b/d1c/d25/d19/d44/d7e 0 2024-04-26T23:01:34.491 INFO:tasks.workunit.client.0.smithi027.stdout:2/580: creat d2/d5/db/d1b/d1c/d25/d19/d23/fac x:0 0 0 2024-04-26T23:01:34.492 INFO:tasks.workunit.client.0.smithi027.stdout:5/641: write d4/d24/d22/d29/d2a/d3b/f3d [339241,20796] 0 2024-04-26T23:01:34.497 INFO:tasks.workunit.client.0.smithi027.stdout:2/581: creat d2/d5/db/d1b/d1c/d25/d6a/d8a/fad x:0 0 0 2024-04-26T23:01:34.497 INFO:tasks.workunit.client.0.smithi027.stdout:5/642: symlink d4/d9f/dc5/lce 0 2024-04-26T23:01:34.497 INFO:tasks.workunit.client.0.smithi027.stdout:5/643: mknod d4/d9f/ccf 0 2024-04-26T23:01:34.497 INFO:tasks.workunit.client.0.smithi027.stdout:5/644: chown d4/d1a/d2b/daa 96 1 2024-04-26T23:01:34.497 INFO:tasks.workunit.client.0.smithi027.stdout:5/645: truncate d4/d24/f98 181807 0 2024-04-26T23:01:34.522 INFO:tasks.workunit.client.0.smithi027.stdout:4/652: dread da/dd/de/d14/d21/d41/f72 [0,4194304] 0 2024-04-26T23:01:34.524 INFO:tasks.workunit.client.0.smithi027.stdout:4/653: getdents da/dd/de/d79/da8/dc5 0 2024-04-26T23:01:34.525 INFO:tasks.workunit.client.0.smithi027.stdout:4/654: truncate da/dd/de/d14/d21/d67/d8a/f8c 820540 0 2024-04-26T23:01:34.527 INFO:tasks.workunit.client.0.smithi027.stdout:4/655: mkdir da/dd/de/d79/dcd 0 2024-04-26T23:01:34.530 INFO:tasks.workunit.client.0.smithi027.stdout:4/656: write da/dd/de/d14/d17/d80/dab/d37/f58 [15445,19384] 0 2024-04-26T23:01:34.537 INFO:tasks.workunit.client.0.smithi027.stdout:7/699: dwrite d0/d3d/f43 [0,4194304] 0 2024-04-26T23:01:34.537 INFO:tasks.workunit.client.0.smithi027.stdout:7/700: chown d0/d28/d2b/f9b 15072 1 2024-04-26T23:01:34.540 INFO:tasks.workunit.client.0.smithi027.stdout:9/541: dwrite d4/d17/d2e/fb7 [0,4194304] 0 2024-04-26T23:01:34.581 INFO:tasks.workunit.client.0.smithi027.stdout:2/582: dwrite d2/d5/db/de/f42 [0,4194304] 0 2024-04-26T23:01:34.581 INFO:tasks.workunit.client.0.smithi027.stdout:2/583: dread - d2/d5/db/d1b/d1c/d25/d19/d44/fa2 zero size 2024-04-26T23:01:34.581 INFO:tasks.workunit.client.0.smithi027.stdout:2/584: fsync d2/d5/db/de/d21/f65 0 2024-04-26T23:01:34.581 INFO:tasks.workunit.client.0.smithi027.stdout:2/585: stat d2/d5/db/d1b/d1c/d25/d19/d36 0 2024-04-26T23:01:34.581 INFO:tasks.workunit.client.0.smithi027.stdout:2/586: fsync d2/d5/db/f6c 0 2024-04-26T23:01:34.581 INFO:tasks.workunit.client.0.smithi027.stdout:9/542: dread d4/d17/f1d [0,4194304] 0 2024-04-26T23:01:34.581 INFO:tasks.workunit.client.0.smithi027.stdout:9/543: dread - d4/d17/f9e zero size 2024-04-26T23:01:34.584 INFO:tasks.workunit.client.0.smithi027.stdout:2/587: rename d2/d5/db/d1b/d1c/d25/d19/d36/d39/f73 to d2/d5/db/d1b/d1c/d25/d6a/d8a/d8d/fae 0 2024-04-26T23:01:34.587 INFO:tasks.workunit.client.0.smithi027.stdout:2/588: symlink d2/d5/db/d1b/d1c/d25/d19/d36/d39/laf 0 2024-04-26T23:01:34.589 INFO:tasks.workunit.client.0.smithi027.stdout:2/589: mkdir d2/d5/db/d1b/d1c/d25/d19/db0 0 2024-04-26T23:01:34.589 INFO:tasks.workunit.client.0.smithi027.stdout:2/590: fsync d2/d5/db/de/d21/d8f/f6d 0 2024-04-26T23:01:34.591 INFO:tasks.workunit.client.0.smithi027.stdout:2/591: creat d2/d5/db/d1b/d1c/d25/d19/d44/fb1 x:0 0 0 2024-04-26T23:01:34.607 INFO:tasks.workunit.client.0.smithi027.stdout:9/544: write d4/d17/d2e/fb7 [2857337,54220] 0 2024-04-26T23:01:34.607 INFO:tasks.workunit.client.0.smithi027.stdout:9/545: write d4/d10/d40/d58/f5a [4051567,38849] 0 2024-04-26T23:01:34.619 INFO:tasks.workunit.client.0.smithi027.stdout:5/646: dwrite d4/d24/d5f/dac/d40/f97 [0,4194304] 0 2024-04-26T23:01:34.621 INFO:tasks.workunit.client.0.smithi027.stdout:5/647: creat d4/d1a/d2b/fd0 x:0 0 0 2024-04-26T23:01:34.626 INFO:tasks.workunit.client.0.smithi027.stdout:4/657: dwrite da/d40/f60 [0,4194304] 0 2024-04-26T23:01:34.627 INFO:tasks.workunit.client.0.smithi027.stdout:4/658: stat da/dd/de/d79/da8/fbf 0 2024-04-26T23:01:34.628 INFO:tasks.workunit.client.0.smithi027.stdout:4/659: symlink da/lce 0 2024-04-26T23:01:34.628 INFO:tasks.workunit.client.0.smithi027.stdout:4/660: write da/dd/de/d14/d21/d41/f49 [2217431,105173] 0 2024-04-26T23:01:34.629 INFO:tasks.workunit.client.0.smithi027.stdout:9/546: dread d4/f11 [0,4194304] 0 2024-04-26T23:01:34.637 INFO:tasks.workunit.client.0.smithi027.stdout:9/547: rename d4/d17/d2e/d46/f7b to d4/d17/d2e/fbe 0 2024-04-26T23:01:34.638 INFO:tasks.workunit.client.0.smithi027.stdout:9/548: chown d4/d10/d40/f64 74 1 2024-04-26T23:01:34.638 INFO:tasks.workunit.client.0.smithi027.stdout:9/549: fsync d4/d10/d40/fab 0 2024-04-26T23:01:34.649 INFO:tasks.workunit.client.0.smithi027.stdout:2/592: dwrite d2/d5/db/d1b/d1c/d25/d19/f5b [0,4194304] 0 2024-04-26T23:01:34.661 INFO:tasks.workunit.client.0.smithi027.stdout:9/550: dread d4/d17/f1d [0,4194304] 0 2024-04-26T23:01:34.661 INFO:tasks.workunit.client.0.smithi027.stdout:9/551: dread - d4/d10/d40/d58/d75/f63 zero size 2024-04-26T23:01:34.662 INFO:tasks.workunit.client.0.smithi027.stdout:2/593: creat d2/d5/db/d1b/d1c/d25/d19/d23/fb2 x:0 0 0 2024-04-26T23:01:34.663 INFO:tasks.workunit.client.0.smithi027.stdout:9/552: link d4/d10/d67/l68 d4/d10/d25/lbf 0 2024-04-26T23:01:34.668 INFO:tasks.workunit.client.0.smithi027.stdout:2/594: creat d2/d5/db/d1b/d1c/d25/d19/d23/d7c/fb3 x:0 0 0 2024-04-26T23:01:34.668 INFO:tasks.workunit.client.0.smithi027.stdout:9/553: creat d4/d17/d2e/d46/da0/fc0 x:0 0 0 2024-04-26T23:01:34.668 INFO:tasks.workunit.client.0.smithi027.stdout:9/554: rename d4/d17/d2e/c54 to d4/d17/d2e/cc1 0 2024-04-26T23:01:34.668 INFO:tasks.workunit.client.0.smithi027.stdout:9/555: chown d4/d17/d2e/d46/cba 280010 1 2024-04-26T23:01:34.668 INFO:tasks.workunit.client.0.smithi027.stdout:9/556: stat d4/d10/d40/d58/l5b 0 2024-04-26T23:01:34.690 INFO:tasks.workunit.client.0.smithi027.stdout:5/648: dwrite d4/d24/d5f/dac/d40/d54/f73 [0,4194304] 0 2024-04-26T23:01:34.691 INFO:tasks.workunit.client.0.smithi027.stdout:5/649: fsync d4/d1a/d2b/fd0 0 2024-04-26T23:01:34.692 INFO:tasks.workunit.client.0.smithi027.stdout:5/650: mkdir d4/d1a/dcb/dd1 0 2024-04-26T23:01:34.694 INFO:tasks.workunit.client.0.smithi027.stdout:2/595: write d2/d5/db/de/d21/f5f [260166,96768] 0 2024-04-26T23:01:34.694 INFO:tasks.workunit.client.0.smithi027.stdout:2/596: chown d2/f89 3064075 1 2024-04-26T23:01:34.694 INFO:tasks.workunit.client.0.smithi027.stdout:2/597: chown d2/d5/db/de/d21/d8f/d88 551 1 2024-04-26T23:01:34.694 INFO:tasks.workunit.client.0.smithi027.stdout:2/598: stat d2 0 2024-04-26T23:01:34.695 INFO:tasks.workunit.client.0.smithi027.stdout:2/599: mknod d2/d5/db/de/cb4 0 2024-04-26T23:01:34.695 INFO:tasks.workunit.client.0.smithi027.stdout:2/600: chown d2/d5/db/de 0 1 2024-04-26T23:01:34.695 INFO:tasks.workunit.client.0.smithi027.stdout:2/601: dread - d2/d5/db/d1b/d1c/d25/d6a/d8a/fad zero size 2024-04-26T23:01:34.695 INFO:tasks.workunit.client.0.smithi027.stdout:2/602: chown d2/d5/db/d1b/d1c/d25/d19/d23/f64 91 1 2024-04-26T23:01:34.696 INFO:tasks.workunit.client.0.smithi027.stdout:2/603: stat d2/d5/db/d1b/d1c/l6b 0 2024-04-26T23:01:34.696 INFO:tasks.workunit.client.0.smithi027.stdout:2/604: chown d2/c4f 6 1 2024-04-26T23:01:34.697 INFO:tasks.workunit.client.0.smithi027.stdout:2/605: symlink d2/d5/db/d1b/d1c/d25/d19/lb5 0 2024-04-26T23:01:34.712 INFO:tasks.workunit.client.0.smithi027.stdout:9/557: dwrite d4/d17/d2e/d46/da0/d8d/fa8 [0,4194304] 0 2024-04-26T23:01:34.716 INFO:tasks.workunit.client.0.smithi027.stdout:9/558: creat d4/d17/d2e/d70/fc2 x:0 0 0 2024-04-26T23:01:34.717 INFO:tasks.workunit.client.0.smithi027.stdout:9/559: creat d4/d17/d2e/d46/da0/fc3 x:0 0 0 2024-04-26T23:01:34.744 INFO:tasks.workunit.client.0.smithi027.stdout:9/560: dwrite d4/d17/f9e [0,4194304] 0 2024-04-26T23:01:34.747 INFO:tasks.workunit.client.0.smithi027.stdout:9/561: mkdir d4/d10/d40/d58/d75/d84/dc4 0 2024-04-26T23:01:34.748 INFO:tasks.workunit.client.0.smithi027.stdout:9/562: mkdir d4/d10/d40/d58/d75/d84/d86/dc5 0 2024-04-26T23:01:34.762 INFO:tasks.workunit.client.0.smithi027.stdout:9/563: dread d4/d10/d67/fb6 [0,4194304] 0 2024-04-26T23:01:34.762 INFO:tasks.workunit.client.0.smithi027.stdout:9/564: truncate d4/d10/f2f 9686424 0 2024-04-26T23:01:35.132 INFO:tasks.workunit.client.0.smithi027.stdout:3/718: sync 2024-04-26T23:01:35.132 INFO:tasks.workunit.client.0.smithi027.stdout:7/701: sync 2024-04-26T23:01:35.133 INFO:tasks.workunit.client.0.smithi027.stdout:9/565: sync 2024-04-26T23:01:35.133 INFO:tasks.workunit.client.0.smithi027.stdout:5/651: sync 2024-04-26T23:01:35.133 INFO:tasks.workunit.client.0.smithi027.stdout:6/540: sync 2024-04-26T23:01:35.133 INFO:tasks.workunit.client.0.smithi027.stdout:9/566: write d4/d10/d40/fad [148758,45774] 0 2024-04-26T23:01:35.133 INFO:tasks.workunit.client.0.smithi027.stdout:9/567: chown d4/d17/d2e/d46/da0/f57 5919 1 2024-04-26T23:01:35.134 INFO:tasks.workunit.client.0.smithi027.stdout:2/606: sync 2024-04-26T23:01:35.135 INFO:tasks.workunit.client.0.smithi027.stdout:4/661: sync 2024-04-26T23:01:35.135 INFO:tasks.workunit.client.0.smithi027.stdout:6/541: write d1/df/d43/da8/da1/fb6 [169341,127060] 0 2024-04-26T23:01:35.135 INFO:tasks.workunit.client.0.smithi027.stdout:6/542: dread - d1/d30/fa7 zero size 2024-04-26T23:01:35.135 INFO:tasks.workunit.client.0.smithi027.stdout:4/662: fdatasync da/dd/d5f/f8b 0 2024-04-26T23:01:35.135 INFO:tasks.workunit.client.0.smithi027.stdout:8/566: sync 2024-04-26T23:01:35.135 INFO:tasks.workunit.client.0.smithi027.stdout:0/600: sync 2024-04-26T23:01:35.135 INFO:tasks.workunit.client.0.smithi027.stdout:7/702: creat d0/d28/d2b/d3a/d48/fe2 x:0 0 0 2024-04-26T23:01:35.135 INFO:tasks.workunit.client.0.smithi027.stdout:7/703: fsync d0/d28/d2b/f9b 0 2024-04-26T23:01:35.135 INFO:tasks.workunit.client.0.smithi027.stdout:7/704: chown d0/d28/d2b/d55/d59/d6d/ld1 69417 1 2024-04-26T23:01:35.135 INFO:tasks.workunit.client.0.smithi027.stdout:8/567: write d9/da/d2a/d5e/d72/d84/d35/d8d/da3/fc3 [372266,105461] 0 2024-04-26T23:01:35.136 INFO:tasks.workunit.client.0.smithi027.stdout:1/677: sync 2024-04-26T23:01:35.136 INFO:tasks.workunit.client.0.smithi027.stdout:9/568: symlink d4/d77/lc6 0 2024-04-26T23:01:35.136 INFO:tasks.workunit.client.0.smithi027.stdout:9/569: truncate d4/d10/d40/d58/d6e/f9f 430016 0 2024-04-26T23:01:35.136 INFO:tasks.workunit.client.0.smithi027.stdout:3/719: creat d6/dd3/d34/d47/ffd x:0 0 0 2024-04-26T23:01:35.136 INFO:tasks.workunit.client.0.smithi027.stdout:3/720: readlink d6/l1b 0 2024-04-26T23:01:35.136 INFO:tasks.workunit.client.0.smithi027.stdout:4/663: getdents da/dd/de/d14/d21/d41/d59/d74 0 2024-04-26T23:01:35.136 INFO:tasks.workunit.client.0.smithi027.stdout:6/543: truncate d1/d30/f36 747291 0 2024-04-26T23:01:35.136 INFO:tasks.workunit.client.0.smithi027.stdout:0/601: creat d0/d4/faf x:0 0 0 2024-04-26T23:01:35.137 INFO:tasks.workunit.client.0.smithi027.stdout:7/705: truncate d0/d28/d2b/d3a/d90/dcb/f96 503523 0 2024-04-26T23:01:35.138 INFO:tasks.workunit.client.0.smithi027.stdout:9/570: link d4/d10/d40/d58/d75/lb0 d4/d10/lc7 0 2024-04-26T23:01:35.141 INFO:tasks.workunit.client.0.smithi027.stdout:6/544: truncate d1/de/f28 2996549 0 2024-04-26T23:01:35.141 INFO:tasks.workunit.client.0.smithi027.stdout:6/545: dread - d1/d30/d6c/f99 zero size 2024-04-26T23:01:35.142 INFO:tasks.workunit.client.0.smithi027.stdout:6/546: fdatasync d1/d30/f83 0 2024-04-26T23:01:35.151 INFO:tasks.workunit.client.0.smithi027.stdout:4/664: mknod da/dd/de/d14/d17/d80/dab/ccf 0 2024-04-26T23:01:35.151 INFO:tasks.workunit.client.0.smithi027.stdout:0/602: rename d0/d4/da/d1a/d6d/fa6 to d0/d4/da/d1a/d1e/fb0 0 2024-04-26T23:01:35.151 INFO:tasks.workunit.client.0.smithi027.stdout:4/665: readlink da/dd/d5f/l99 0 2024-04-26T23:01:35.151 INFO:tasks.workunit.client.0.smithi027.stdout:0/603: dread - d0/df/d3f/d64/f71 zero size 2024-04-26T23:01:35.151 INFO:tasks.workunit.client.0.smithi027.stdout:7/706: getdents d0/d28/d2b/d55/d59 0 2024-04-26T23:01:35.151 INFO:tasks.workunit.client.0.smithi027.stdout:6/547: rmdir d1/df/d43/da8/da1 39 2024-04-26T23:01:35.152 INFO:tasks.workunit.client.0.smithi027.stdout:6/548: truncate d1/f72 921948 0 2024-04-26T23:01:35.152 INFO:tasks.workunit.client.0.smithi027.stdout:9/571: creat d4/d77/da2/daf/fc8 x:0 0 0 2024-04-26T23:01:35.152 INFO:tasks.workunit.client.0.smithi027.stdout:9/572: truncate d4/d10/d67/fb3 429492 0 2024-04-26T23:01:35.152 INFO:tasks.workunit.client.0.smithi027.stdout:0/604: creat d0/d4/da/d15/d26/fb1 x:0 0 0 2024-04-26T23:01:35.152 INFO:tasks.workunit.client.0.smithi027.stdout:7/707: mknod d0/d28/d2b/d3a/d90/ce3 0 2024-04-26T23:01:35.152 INFO:tasks.workunit.client.0.smithi027.stdout:7/708: truncate d0/d28/d2b/f9b 182850 0 2024-04-26T23:01:35.152 INFO:tasks.workunit.client.0.smithi027.stdout:7/709: chown d0/d2f 0 1 2024-04-26T23:01:35.152 INFO:tasks.workunit.client.0.smithi027.stdout:6/549: symlink d1/d30/d6c/lb7 0 2024-04-26T23:01:35.152 INFO:tasks.workunit.client.0.smithi027.stdout:9/573: mkdir d4/d10/d25/dc9 0 2024-04-26T23:01:35.152 INFO:tasks.workunit.client.0.smithi027.stdout:8/568: creat d9/da/d2a/d5e/d9e/fc8 x:0 0 0 2024-04-26T23:01:35.152 INFO:tasks.workunit.client.0.smithi027.stdout:8/569: chown d9/da/d33/f88 2829 1 2024-04-26T23:01:35.153 INFO:tasks.workunit.client.0.smithi027.stdout:0/605: truncate d0/d4/da/d1a/d2a/f74 2364143 0 2024-04-26T23:01:35.153 INFO:tasks.workunit.client.0.smithi027.stdout:7/710: creat d0/d28/d9f/db8/dbc/fe4 x:0 0 0 2024-04-26T23:01:35.154 INFO:tasks.workunit.client.0.smithi027.stdout:2/607: write d2/d5/db/d1b/d1c/f77 [719503,109577] 0 2024-04-26T23:01:35.154 INFO:tasks.workunit.client.0.smithi027.stdout:2/608: fsync d2/d5/db/d1b/d1c/d25/f59 0 2024-04-26T23:01:35.154 INFO:tasks.workunit.client.0.smithi027.stdout:6/550: unlink d1/de/f28 0 2024-04-26T23:01:35.155 INFO:tasks.workunit.client.0.smithi027.stdout:6/551: dread - d1/d30/d6c/f97 zero size 2024-04-26T23:01:35.155 INFO:tasks.workunit.client.0.smithi027.stdout:9/574: mkdir d4/d10/d40/d58/d75/d84/dca 0 2024-04-26T23:01:35.155 INFO:tasks.workunit.client.0.smithi027.stdout:8/570: symlink d9/da/d2a/d5e/d2b/lc9 0 2024-04-26T23:01:35.158 INFO:tasks.workunit.client.0.smithi027.stdout:7/711: rename d0/d28/d2b/d3a/d90/dcb/f96 to d0/d3d/dba/fe5 0 2024-04-26T23:01:35.160 INFO:tasks.workunit.client.0.smithi027.stdout:2/609: unlink d2/d5/c92 0 2024-04-26T23:01:35.160 INFO:tasks.workunit.client.0.smithi027.stdout:2/610: write d2/d5/db/d1b/d1c/d25/d19/d44/fb1 [662947,9704] 0 2024-04-26T23:01:35.161 INFO:tasks.workunit.client.0.smithi027.stdout:6/552: creat d1/df/d43/da8/d84/fb8 x:0 0 0 2024-04-26T23:01:35.161 INFO:tasks.workunit.client.0.smithi027.stdout:6/553: chown d1/de/d57/d8e/c9e 340119 1 2024-04-26T23:01:35.161 INFO:tasks.workunit.client.0.smithi027.stdout:7/712: creat d0/d28/d2b/d3a/d48/d56/db6/fe6 x:0 0 0 2024-04-26T23:01:35.161 INFO:tasks.workunit.client.0.smithi027.stdout:5/652: dread d4/d24/d22/d29/f58 [0,4194304] 0 2024-04-26T23:01:35.162 INFO:tasks.workunit.client.0.smithi027.stdout:1/678: dread d1/f1f [0,4194304] 0 2024-04-26T23:01:35.168 INFO:tasks.workunit.client.0.smithi027.stdout:9/575: fdatasync d4/d10/d40/fad 0 2024-04-26T23:01:35.169 INFO:tasks.workunit.client.0.smithi027.stdout:9/576: fsync d4/d17/d2e/d46/f62 0 2024-04-26T23:01:35.175 INFO:tasks.workunit.client.0.smithi027.stdout:2/611: unlink d2/d5/db/d1b/d1c/d25/d19/d44/f7f 0 2024-04-26T23:01:35.175 INFO:tasks.workunit.client.0.smithi027.stdout:2/612: dread - d2/d5/db/d1b/d1c/d25/d19/d23/f9a zero size 2024-04-26T23:01:35.175 INFO:tasks.workunit.client.0.smithi027.stdout:2/613: write d2/f6f [400187,60035] 0 2024-04-26T23:01:35.175 INFO:tasks.workunit.client.0.smithi027.stdout:7/713: unlink d0/d28/d66/d67/f81 0 2024-04-26T23:01:35.175 INFO:tasks.workunit.client.0.smithi027.stdout:6/554: symlink d1/d30/d6c/lb9 0 2024-04-26T23:01:35.175 INFO:tasks.workunit.client.0.smithi027.stdout:5/653: mkdir d4/d1a/d2b/dd2 0 2024-04-26T23:01:35.179 INFO:tasks.workunit.client.0.smithi027.stdout:1/679: symlink d1/d26/d66/dab/ld6 0 2024-04-26T23:01:35.179 INFO:tasks.workunit.client.0.smithi027.stdout:7/714: creat d0/d28/d2b/d55/fe7 x:0 0 0 2024-04-26T23:01:35.180 INFO:tasks.workunit.client.0.smithi027.stdout:6/555: creat d1/df/d3c/fba x:0 0 0 2024-04-26T23:01:35.181 INFO:tasks.workunit.client.0.smithi027.stdout:6/556: write d1/d30/f83 [492646,75161] 0 2024-04-26T23:01:35.182 INFO:tasks.workunit.client.0.smithi027.stdout:5/654: mkdir d4/d1a/d2b/dd3 0 2024-04-26T23:01:35.183 INFO:tasks.workunit.client.0.smithi027.stdout:7/715: creat d0/fe8 x:0 0 0 2024-04-26T23:01:35.183 INFO:tasks.workunit.client.0.smithi027.stdout:6/557: truncate d1/d30/d6c/d6f/f89 3183771 0 2024-04-26T23:01:35.183 INFO:tasks.workunit.client.0.smithi027.stdout:6/558: truncate d1/df/d43/da8/d84/fb8 660193 0 2024-04-26T23:01:35.183 INFO:tasks.workunit.client.0.smithi027.stdout:6/559: write d1/df/f29 [4572958,22742] 0 2024-04-26T23:01:35.183 INFO:tasks.workunit.client.0.smithi027.stdout:6/560: readlink d1/df/l3b 0 2024-04-26T23:01:35.187 INFO:tasks.workunit.client.0.smithi027.stdout:5/655: symlink d4/d24/d22/d29/d2a/d3b/ld4 0 2024-04-26T23:01:35.206 INFO:tasks.workunit.client.0.smithi027.stdout:4/666: dwrite da/dd/de/d14/d17/d80/dab/f16 [0,4194304] 0 2024-04-26T23:01:35.213 INFO:tasks.workunit.client.0.smithi027.stdout:4/667: link da/dd/de/d14/d17/d80/c82 da/dd/de/d14/d17/d80/dab/cd0 0 2024-04-26T23:01:35.213 INFO:tasks.workunit.client.0.smithi027.stdout:3/721: dwrite d6/f18 [0,4194304] 0 2024-04-26T23:01:35.214 INFO:tasks.workunit.client.0.smithi027.stdout:4/668: rmdir da/dd/de/d14/d21/d41/d59/d74 39 2024-04-26T23:01:35.215 INFO:tasks.workunit.client.0.smithi027.stdout:3/722: truncate d6/f8 5229431 0 2024-04-26T23:01:35.215 INFO:tasks.workunit.client.0.smithi027.stdout:3/723: dread - d6/dd3/d34/d2e/d9d/da7/dac/dca/fec zero size 2024-04-26T23:01:35.215 INFO:tasks.workunit.client.0.smithi027.stdout:3/724: readlink d6/dd3/d34/d2e/d9d/da3/led 0 2024-04-26T23:01:35.215 INFO:tasks.workunit.client.0.smithi027.stdout:4/669: symlink da/d40/ld1 0 2024-04-26T23:01:35.220 INFO:tasks.workunit.client.0.smithi027.stdout:9/577: dread d4/d17/d2e/d46/da0/d8d/fa8 [0,4194304] 0 2024-04-26T23:01:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:5/656: write d4/d24/f5d [2563456,109361] 0 2024-04-26T23:01:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:5/657: write d4/d24/d22/d29/d2a/d3b/fb7 [184365,101833] 0 2024-04-26T23:01:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:5/658: fdatasync d4/d24/d22/d29/d39/f9c 0 2024-04-26T23:01:35.236 INFO:tasks.workunit.client.0.smithi027.stdout:1/680: dread d1/dc/fa9 [0,4194304] 0 2024-04-26T23:01:35.236 INFO:tasks.workunit.client.0.smithi027.stdout:4/670: dread da/dd/de/d14/d21/d67/d8a/f8c [0,4194304] 0 2024-04-26T23:01:35.238 INFO:tasks.workunit.client.0.smithi027.stdout:0/606: dwrite d0/d4/da/d15/f5a [0,4194304] 0 2024-04-26T23:01:35.238 INFO:tasks.workunit.client.0.smithi027.stdout:3/725: rename d6/dd3/f10 to d6/dd3/d34/d2e/d5f/d90/ddb/dce/ffe 0 2024-04-26T23:01:35.241 INFO:tasks.workunit.client.0.smithi027.stdout:3/726: write d6/dd3/d2f/f39 [4716507,9503] 0 2024-04-26T23:01:35.242 INFO:tasks.workunit.client.0.smithi027.stdout:3/727: dread - d6/dd3/d34/d2e/d9d/da7/dac/dca/ff7 zero size 2024-04-26T23:01:35.244 INFO:tasks.workunit.client.0.smithi027.stdout:1/681: mkdir d1/d26/d86/d5c/dd7 0 2024-04-26T23:01:35.250 INFO:tasks.workunit.client.0.smithi027.stdout:4/671: creat da/dd/de/fd2 x:0 0 0 2024-04-26T23:01:35.250 INFO:tasks.workunit.client.0.smithi027.stdout:0/607: mknod d0/d4/da/d15/cb2 0 2024-04-26T23:01:35.250 INFO:tasks.workunit.client.0.smithi027.stdout:0/608: fdatasync d0/f37 0 2024-04-26T23:01:35.251 INFO:tasks.workunit.client.0.smithi027.stdout:2/614: dwrite d2/d5/db/d1b/d1c/d25/f3d [4194304,4194304] 0 2024-04-26T23:01:35.253 INFO:tasks.workunit.client.0.smithi027.stdout:2/615: mknod d2/d5/db/d1b/d1c/d25/cb6 0 2024-04-26T23:01:35.254 INFO:tasks.workunit.client.0.smithi027.stdout:3/728: link d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/ldd d6/dd3/d34/d9b/de0/lff 0 2024-04-26T23:01:35.256 INFO:tasks.workunit.client.0.smithi027.stdout:2/616: link d2/d5/db/d1b/d1c/l6b d2/d5/db/d1b/d1c/d25/d19/db0/lb7 0 2024-04-26T23:01:35.258 INFO:tasks.workunit.client.0.smithi027.stdout:2/617: creat d2/d5/db/d1b/fb8 x:0 0 0 2024-04-26T23:01:35.260 INFO:tasks.workunit.client.0.smithi027.stdout:6/561: dwrite d1/df/d43/f6e [0,4194304] 0 2024-04-26T23:01:35.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:35 smithi080 ceph-mon[32324]: pgmap v92: 129 pgs: 129 active+clean; 1.2 GiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 138 MiB/s rd, 165 MiB/s wr, 317 op/s 2024-04-26T23:01:35.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:35 smithi080 ceph-mon[32324]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:01:35.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:35 smithi094 ceph-mon[32889]: pgmap v92: 129 pgs: 129 active+clean; 1.2 GiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 138 MiB/s rd, 165 MiB/s wr, 317 op/s 2024-04-26T23:01:35.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:35 smithi094 ceph-mon[32889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:01:35.307 INFO:tasks.workunit.client.0.smithi027.stdout:4/672: dwrite da/dd/de/d14/d21/d41/f49 [4194304,4194304] 0 2024-04-26T23:01:35.307 INFO:tasks.workunit.client.0.smithi027.stdout:4/673: stat da/dd/d5f/cb1 0 2024-04-26T23:01:35.307 INFO:tasks.workunit.client.0.smithi027.stdout:6/562: read d1/df/d43/d56/d6b/f8b [916083,98313] 0 2024-04-26T23:01:35.307 INFO:tasks.workunit.client.0.smithi027.stdout:6/563: write d1/df/d43/f74 [1023789,58813] 0 2024-04-26T23:01:35.308 INFO:tasks.workunit.client.0.smithi027.stdout:1/682: dwrite d1/d5/f8b [0,4194304] 0 2024-04-26T23:01:35.308 INFO:tasks.workunit.client.0.smithi027.stdout:1/683: chown d1/dc/f37 1927508 1 2024-04-26T23:01:35.309 INFO:tasks.workunit.client.0.smithi027.stdout:6/564: rename d1/f35 to d1/d30/d6c/d6f/d7d/fbb 0 2024-04-26T23:01:35.313 INFO:tasks.workunit.client.0.smithi027.stdout:1/684: truncate d1/dc/f1d 4711881 0 2024-04-26T23:01:35.314 INFO:tasks.workunit.client.0.smithi027.stdout:2/618: dwrite d2/d5/db/de/d21/d8f/f2d [0,4194304] 0 2024-04-26T23:01:35.316 INFO:tasks.workunit.client.0.smithi027.stdout:6/565: unlink d1/de/d57/fb4 0 2024-04-26T23:01:35.316 INFO:tasks.workunit.client.0.smithi027.stdout:6/566: write d1/df/f2e [3162280,4126] 0 2024-04-26T23:01:35.316 INFO:tasks.workunit.client.0.smithi027.stdout:6/567: chown d1/df/l3b 27 1 2024-04-26T23:01:35.317 INFO:tasks.workunit.client.0.smithi027.stdout:4/674: truncate da/dd/de/d14/d21/d41/f49 4041667 0 2024-04-26T23:01:35.317 INFO:tasks.workunit.client.0.smithi027.stdout:8/571: rmdir d9/da/d2a/d5e/d72/d84/d5d/d6c 39 2024-04-26T23:01:35.317 INFO:tasks.workunit.client.0.smithi027.stdout:8/572: fsync d9/da/d33/f60 0 2024-04-26T23:01:35.318 INFO:tasks.workunit.client.0.smithi027.stdout:8/573: symlink d9/da/d33/lca 0 2024-04-26T23:01:35.321 INFO:tasks.workunit.client.0.smithi027.stdout:1/685: creat d1/d26/d66/dcc/fd8 x:0 0 0 2024-04-26T23:01:35.321 INFO:tasks.workunit.client.0.smithi027.stdout:8/574: getdents d9/da/d2a/d5e/d72/d84/d35/dc2 0 2024-04-26T23:01:35.322 INFO:tasks.workunit.client.0.smithi027.stdout:8/575: rename d9/da/d2a/d5e/d72/d84/d35/d8d/fb2 to d9/da/d2a/d5e/d72/d84/d5d/fcb 0 2024-04-26T23:01:35.327 INFO:tasks.workunit.client.0.smithi027.stdout:4/675: creat da/dd/de/d79/da8/fd3 x:0 0 0 2024-04-26T23:01:35.327 INFO:tasks.workunit.client.0.smithi027.stdout:4/676: write da/f8f [158130,3293] 0 2024-04-26T23:01:35.327 INFO:tasks.workunit.client.0.smithi027.stdout:8/576: link d9/da/d2a/d5e/d72/d84/d5d/d8b/fbb d9/fcc 0 2024-04-26T23:01:35.327 INFO:tasks.workunit.client.0.smithi027.stdout:2/619: mkdir d2/db9 0 2024-04-26T23:01:35.327 INFO:tasks.workunit.client.0.smithi027.stdout:2/620: dread - d2/d5/db/d1b/fb8 zero size 2024-04-26T23:01:35.340 INFO:tasks.workunit.client.0.smithi027.stdout:8/577: dread d9/da/d2a/fa2 [0,4194304] 0 2024-04-26T23:01:35.340 INFO:tasks.workunit.client.0.smithi027.stdout:8/578: write d9/da/d2a/d5e/f64 [3794547,50221] 0 2024-04-26T23:01:35.342 INFO:tasks.workunit.client.0.smithi027.stdout:9/578: dwrite d4/d10/d67/fb3 [0,4194304] 0 2024-04-26T23:01:35.350 INFO:tasks.workunit.client.0.smithi027.stdout:9/579: rmdir d4/d17/d2e/d46/da0/d8d 39 2024-04-26T23:01:35.350 INFO:tasks.workunit.client.0.smithi027.stdout:9/580: read d4/d17/d2e/f53 [1559397,84402] 0 2024-04-26T23:01:35.351 INFO:tasks.workunit.client.0.smithi027.stdout:9/581: fdatasync d4/d10/d40/f64 0 2024-04-26T23:01:35.351 INFO:tasks.workunit.client.0.smithi027.stdout:4/677: write da/dd/de/d14/d21/f75 [286723,81817] 0 2024-04-26T23:01:35.351 INFO:tasks.workunit.client.0.smithi027.stdout:4/678: dread - da/dd/de/d14/db3/fc6 zero size 2024-04-26T23:01:35.353 INFO:tasks.workunit.client.0.smithi027.stdout:4/679: rename da/dd/de/d14/d17/d80/dab/d37/f58 to da/dd/de/d14/d21/d41/d59/fd4 0 2024-04-26T23:01:35.354 INFO:tasks.workunit.client.0.smithi027.stdout:4/680: symlink da/ld5 0 2024-04-26T23:01:35.354 INFO:tasks.workunit.client.0.smithi027.stdout:4/681: truncate da/dd/de/d14/d21/d41/f52 4746213 0 2024-04-26T23:01:35.358 INFO:tasks.workunit.client.0.smithi027.stdout:4/682: truncate da/dd/de/d14/d21/d41/d59/f9c 1576905 0 2024-04-26T23:01:35.358 INFO:tasks.workunit.client.0.smithi027.stdout:1/686: dread d1/f94 [0,4194304] 0 2024-04-26T23:01:35.358 INFO:tasks.workunit.client.0.smithi027.stdout:1/687: chown d1/d26/d36/d88/lc6 11149411 1 2024-04-26T23:01:35.358 INFO:tasks.workunit.client.0.smithi027.stdout:8/579: mknod d9/da/d2a/d5e/d72/d84/d35/ccd 0 2024-04-26T23:01:35.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:35 smithi027 ceph-mon[28034]: pgmap v92: 129 pgs: 129 active+clean; 1.2 GiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 138 MiB/s rd, 165 MiB/s wr, 317 op/s 2024-04-26T23:01:35.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:35 smithi027 ceph-mon[28034]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:01:35.372 INFO:tasks.workunit.client.0.smithi027.stdout:8/580: link d9/da/d2a/dc5/fc6 d9/da/d2a/d5e/d92/fce 0 2024-04-26T23:01:35.372 INFO:tasks.workunit.client.0.smithi027.stdout:8/581: write d9/da/d2a/fa2 [734746,111186] 0 2024-04-26T23:01:35.375 INFO:tasks.workunit.client.0.smithi027.stdout:8/582: creat d9/da/d2a/d5e/d72/d84/d5d/d6c/dc0/fcf x:0 0 0 2024-04-26T23:01:35.375 INFO:tasks.workunit.client.0.smithi027.stdout:8/583: chown d9/da/d2a/d5e/d72/d84/c30 13 1 2024-04-26T23:01:35.376 INFO:tasks.workunit.client.0.smithi027.stdout:8/584: getdents d9/da/d2a 0 2024-04-26T23:01:35.377 INFO:tasks.workunit.client.0.smithi027.stdout:6/568: dwrite d1/df/d49/f4f [0,4194304] 0 2024-04-26T23:01:35.378 INFO:tasks.workunit.client.0.smithi027.stdout:6/569: write d1/df/d43/d56/d6b/f8b [4400214,49136] 0 2024-04-26T23:01:35.378 INFO:tasks.workunit.client.0.smithi027.stdout:6/570: chown d1/f2b 108383 1 2024-04-26T23:01:35.380 INFO:tasks.workunit.client.0.smithi027.stdout:0/609: dwrite d0/d4/da/d1a/d2a/f43 [0,4194304] 0 2024-04-26T23:01:35.380 INFO:tasks.workunit.client.0.smithi027.stdout:8/585: mknod d9/da/d2a/d5e/d72/d84/cd0 0 2024-04-26T23:01:35.385 INFO:tasks.workunit.client.0.smithi027.stdout:0/610: rename d0/d4/da/f14 to d0/d4/da/fb3 0 2024-04-26T23:01:35.387 INFO:tasks.workunit.client.0.smithi027.stdout:0/611: mkdir d0/df/d9d/db4 0 2024-04-26T23:01:35.394 INFO:tasks.workunit.client.0.smithi027.stdout:0/612: creat d0/df/d9d/fb5 x:0 0 0 2024-04-26T23:01:35.394 INFO:tasks.workunit.client.0.smithi027.stdout:0/613: write d0/d4/da/d1a/d59/f9c [188647,120151] 0 2024-04-26T23:01:35.396 INFO:tasks.workunit.client.0.smithi027.stdout:1/688: write d1/d2a/f62 [4006260,69574] 0 2024-04-26T23:01:35.396 INFO:tasks.workunit.client.0.smithi027.stdout:0/614: creat d0/df/d9d/db4/fb6 x:0 0 0 2024-04-26T23:01:35.396 INFO:tasks.workunit.client.0.smithi027.stdout:1/689: dread - d1/dc/fb1 zero size 2024-04-26T23:01:35.396 INFO:tasks.workunit.client.0.smithi027.stdout:1/690: dread - d1/d26/d86/fa2 zero size 2024-04-26T23:01:35.408 INFO:tasks.workunit.client.0.smithi027.stdout:8/586: read d9/da/d2a/d5e/d72/d84/db8/f9b [4028456,41464] 0 2024-04-26T23:01:35.409 INFO:tasks.workunit.client.0.smithi027.stdout:8/587: write d9/de/f18 [1159598,21724] 0 2024-04-26T23:01:35.409 INFO:tasks.workunit.client.0.smithi027.stdout:3/729: dwrite d6/dd3/d34/d2e/d5f/fc4 [0,4194304] 0 2024-04-26T23:01:35.414 INFO:tasks.workunit.client.0.smithi027.stdout:8/588: mknod d9/da/da1/cd1 0 2024-04-26T23:01:35.414 INFO:tasks.workunit.client.0.smithi027.stdout:8/589: mknod d9/da/da1/cd2 0 2024-04-26T23:01:35.415 INFO:tasks.workunit.client.0.smithi027.stdout:8/590: mkdir d9/da/d2a/d5e/d72/d84/d4f/dd3 0 2024-04-26T23:01:35.420 INFO:tasks.workunit.client.0.smithi027.stdout:8/591: rename d9/da/d2a/d5e/d72/d84/c63 to d9/da/d2a/d5e/d72/d84/d35/d8d/cd4 0 2024-04-26T23:01:35.421 INFO:tasks.workunit.client.0.smithi027.stdout:9/582: dwrite d4/d17/f9e [0,4194304] 0 2024-04-26T23:01:35.421 INFO:tasks.workunit.client.0.smithi027.stdout:8/592: rmdir d9/de/d2e 39 2024-04-26T23:01:35.421 INFO:tasks.workunit.client.0.smithi027.stdout:8/593: fdatasync d9/da/d33/f88 0 2024-04-26T23:01:35.424 INFO:tasks.workunit.client.0.smithi027.stdout:8/594: mknod d9/da/d2a/d5e/d72/d84/d35/d8d/cd5 0 2024-04-26T23:01:35.425 INFO:tasks.workunit.client.0.smithi027.stdout:8/595: getdents d9/da6 0 2024-04-26T23:01:35.427 INFO:tasks.workunit.client.0.smithi027.stdout:8/596: creat d9/da/d2a/d5e/d72/d84/d35/d8d/fd6 x:0 0 0 2024-04-26T23:01:35.432 INFO:tasks.workunit.client.0.smithi027.stdout:4/683: dwrite da/dd/de/d14/f53 [0,4194304] 0 2024-04-26T23:01:35.435 INFO:tasks.workunit.client.0.smithi027.stdout:4/684: rename da/dd/de/d79/da5 to da/dd/de/d14/d17/d80/dab/dd6 0 2024-04-26T23:01:35.436 INFO:tasks.workunit.client.0.smithi027.stdout:1/691: dread d1/d26/d6b/d42/d70/fc7 [0,4194304] 0 2024-04-26T23:01:35.436 INFO:tasks.workunit.client.0.smithi027.stdout:6/571: dwrite d1/d30/d6c/f99 [0,4194304] 0 2024-04-26T23:01:35.436 INFO:tasks.workunit.client.0.smithi027.stdout:4/685: mkdir da/d40/dd7 0 2024-04-26T23:01:35.438 INFO:tasks.workunit.client.0.smithi027.stdout:0/615: dwrite d0/d4/f7 [0,4194304] 0 2024-04-26T23:01:35.442 INFO:tasks.workunit.client.0.smithi027.stdout:1/692: rename d1/d5/d40/f78 to d1/d26/d6b/d42/d79/fd9 0 2024-04-26T23:01:35.442 INFO:tasks.workunit.client.0.smithi027.stdout:1/693: readlink d1/d26/d86/d5c/l63 0 2024-04-26T23:01:35.442 INFO:tasks.workunit.client.0.smithi027.stdout:6/572: truncate d1/df/d43/faa 7700030 0 2024-04-26T23:01:35.442 INFO:tasks.workunit.client.0.smithi027.stdout:4/686: creat da/dd/fd8 x:0 0 0 2024-04-26T23:01:35.443 INFO:tasks.workunit.client.0.smithi027.stdout:9/583: write d4/d10/f19 [11444283,9559] 0 2024-04-26T23:01:35.446 INFO:tasks.workunit.client.0.smithi027.stdout:0/616: getdents d0/df/d3f 0 2024-04-26T23:01:35.446 INFO:tasks.workunit.client.0.smithi027.stdout:0/617: truncate d0/d4/da/d15/f5a 4973619 0 2024-04-26T23:01:35.451 INFO:tasks.workunit.client.0.smithi027.stdout:6/573: getdents d1/df 0 2024-04-26T23:01:35.455 INFO:tasks.workunit.client.0.smithi027.stdout:0/618: dread d0/d4/da/f78 [0,4194304] 0 2024-04-26T23:01:35.462 INFO:tasks.workunit.client.0.smithi027.stdout:4/687: unlink da/dd/de/d79/da8/dc5/lca 0 2024-04-26T23:01:35.462 INFO:tasks.workunit.client.0.smithi027.stdout:9/584: creat d4/d77/da2/daf/fcb x:0 0 0 2024-04-26T23:01:35.462 INFO:tasks.workunit.client.0.smithi027.stdout:6/574: creat d1/df/d3c/d9a/fbc x:0 0 0 2024-04-26T23:01:35.462 INFO:tasks.workunit.client.0.smithi027.stdout:0/619: getdents d0/d4 0 2024-04-26T23:01:35.462 INFO:tasks.workunit.client.0.smithi027.stdout:0/620: chown d0/df/d3f/f61 367 1 2024-04-26T23:01:35.462 INFO:tasks.workunit.client.0.smithi027.stdout:6/575: creat d1/d30/d6c/fbd x:0 0 0 2024-04-26T23:01:35.462 INFO:tasks.workunit.client.0.smithi027.stdout:6/576: chown d1/df/d43/da8/da1/fb6 4 1 2024-04-26T23:01:35.462 INFO:tasks.workunit.client.0.smithi027.stdout:6/577: creat d1/df/d43/da8/da1/fbe x:0 0 0 2024-04-26T23:01:35.465 INFO:tasks.workunit.client.0.smithi027.stdout:3/730: dwrite d6/dd3/d34/d2e/d5f/d90/ddb/fa4 [0,4194304] 0 2024-04-26T23:01:35.465 INFO:tasks.workunit.client.0.smithi027.stdout:8/597: dread d9/da/d33/d71/fc1 [0,4194304] 0 2024-04-26T23:01:35.465 INFO:tasks.workunit.client.0.smithi027.stdout:3/731: chown d6/dd3/d34/d2e/d9d/da7/dac/dca/ff7 8830 1 2024-04-26T23:01:35.469 INFO:tasks.workunit.client.0.smithi027.stdout:8/598: mknod d9/da/d2a/d5e/d72/d84/d4f/dd3/cd7 0 2024-04-26T23:01:35.472 INFO:tasks.workunit.client.0.smithi027.stdout:3/732: mkdir d6/dd3/d34/d2e/d100 0 2024-04-26T23:01:35.477 INFO:tasks.workunit.client.0.smithi027.stdout:3/733: rename d6/dd3/d34/db3/db4/ld1 to d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/l101 0 2024-04-26T23:01:35.483 INFO:tasks.workunit.client.0.smithi027.stdout:9/585: write d4/d17/d2e/d46/da0/d8d/fa8 [1682391,93377] 0 2024-04-26T23:01:35.485 INFO:tasks.workunit.client.0.smithi027.stdout:9/586: creat d4/d17/d2e/fcc x:0 0 0 2024-04-26T23:01:35.488 INFO:tasks.workunit.client.0.smithi027.stdout:3/734: dread d6/dd3/d34/d2e/d5f/d90/db9/fcc [0,4194304] 0 2024-04-26T23:01:35.491 INFO:tasks.workunit.client.0.smithi027.stdout:3/735: read - d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/fc2 zero size 2024-04-26T23:01:35.492 INFO:tasks.workunit.client.0.smithi027.stdout:4/688: dread da/dd/de/d14/d17/d80/dab/f35 [0,4194304] 0 2024-04-26T23:01:35.494 INFO:tasks.workunit.client.0.smithi027.stdout:0/621: dread d0/d4/da/d15/d55/f77 [0,4194304] 0 2024-04-26T23:01:35.495 INFO:tasks.workunit.client.0.smithi027.stdout:4/689: mknod da/dd/d5f/dac/cd9 0 2024-04-26T23:01:35.495 INFO:tasks.workunit.client.0.smithi027.stdout:1/694: dwrite d1/d26/d66/d73/fbd [0,4194304] 0 2024-04-26T23:01:35.497 INFO:tasks.workunit.client.0.smithi027.stdout:4/690: getdents da/dd/de/d14/d21/d4a 0 2024-04-26T23:01:35.497 INFO:tasks.workunit.client.0.smithi027.stdout:1/695: creat d1/d26/d6b/d42/fda x:0 0 0 2024-04-26T23:01:35.499 INFO:tasks.workunit.client.0.smithi027.stdout:8/599: dread d9/da/d33/f88 [0,4194304] 0 2024-04-26T23:01:35.500 INFO:tasks.workunit.client.0.smithi027.stdout:0/622: write d0/d4/da/d15/f33 [750402,56363] 0 2024-04-26T23:01:35.502 INFO:tasks.workunit.client.0.smithi027.stdout:8/600: rmdir d9/da/da1 39 2024-04-26T23:01:35.503 INFO:tasks.workunit.client.0.smithi027.stdout:4/691: creat da/fda x:0 0 0 2024-04-26T23:01:35.504 INFO:tasks.workunit.client.0.smithi027.stdout:9/587: dread d4/d10/d40/d58/d6e/f7c [0,4194304] 0 2024-04-26T23:01:35.505 INFO:tasks.workunit.client.0.smithi027.stdout:9/588: stat d4/d10/d40/c71 0 2024-04-26T23:01:35.505 INFO:tasks.workunit.client.0.smithi027.stdout:9/589: chown d4/d17/d2e/d81/da7 29 1 2024-04-26T23:01:35.506 INFO:tasks.workunit.client.0.smithi027.stdout:6/578: dwrite d1/df/d3c/d66/f8d [0,4194304] 0 2024-04-26T23:01:35.509 INFO:tasks.workunit.client.0.smithi027.stdout:3/736: dread d6/dd3/d34/d2e/d9d/da7/dac/d74/f87 [0,4194304] 0 2024-04-26T23:01:35.512 INFO:tasks.workunit.client.0.smithi027.stdout:0/623: symlink d0/d4/da/d1a/d2a/lb7 0 2024-04-26T23:01:35.513 INFO:tasks.workunit.client.0.smithi027.stdout:4/692: link da/dd/de/d14/l34 da/d40/ldb 0 2024-04-26T23:01:35.516 INFO:tasks.workunit.client.0.smithi027.stdout:6/579: rename d1/de/c8c to d1/d6a/cbf 0 2024-04-26T23:01:35.516 INFO:tasks.workunit.client.0.smithi027.stdout:6/580: write d1/df/fa5 [970949,85435] 0 2024-04-26T23:01:35.517 INFO:tasks.workunit.client.0.smithi027.stdout:0/624: mknod d0/d4/da/d1a/cb8 0 2024-04-26T23:01:35.517 INFO:tasks.workunit.client.0.smithi027.stdout:0/625: readlink d0/d4/da/d15/l29 0 2024-04-26T23:01:35.521 INFO:tasks.workunit.client.0.smithi027.stdout:4/693: symlink da/dd/d5f/dac/ldc 0 2024-04-26T23:01:35.522 INFO:tasks.workunit.client.0.smithi027.stdout:6/581: truncate d1/f72 1424992 0 2024-04-26T23:01:35.526 INFO:tasks.workunit.client.0.smithi027.stdout:9/590: dread d4/f43 [0,4194304] 0 2024-04-26T23:01:35.527 INFO:tasks.workunit.client.0.smithi027.stdout:4/694: truncate da/dd/de/d79/f7b 1415175 0 2024-04-26T23:01:35.529 INFO:tasks.workunit.client.0.smithi027.stdout:6/582: unlink d1/df/d43/d56/c65 0 2024-04-26T23:01:35.531 INFO:tasks.workunit.client.0.smithi027.stdout:9/591: creat d4/fcd x:0 0 0 2024-04-26T23:01:35.531 INFO:tasks.workunit.client.0.smithi027.stdout:9/592: fdatasync d4/d17/f45 0 2024-04-26T23:01:35.531 INFO:tasks.workunit.client.0.smithi027.stdout:9/593: write d4/f2b [2470095,44209] 0 2024-04-26T23:01:35.535 INFO:tasks.workunit.client.0.smithi027.stdout:3/737: write d6/dd3/f46 [1707079,63216] 0 2024-04-26T23:01:35.538 INFO:tasks.workunit.client.0.smithi027.stdout:3/738: creat d6/dd3/d34/db3/f102 x:0 0 0 2024-04-26T23:01:35.538 INFO:tasks.workunit.client.0.smithi027.stdout:0/626: write d0/d4/f3d [2503688,120464] 0 2024-04-26T23:01:35.542 INFO:tasks.workunit.client.0.smithi027.stdout:8/601: dwrite d9/da/d2a/f48 [4194304,4194304] 0 2024-04-26T23:01:35.542 INFO:tasks.workunit.client.0.smithi027.stdout:8/602: chown d9/da/d2a/c34 3 1 2024-04-26T23:01:35.542 INFO:tasks.workunit.client.0.smithi027.stdout:8/603: write d9/de/fa5 [234525,55662] 0 2024-04-26T23:01:35.544 INFO:tasks.workunit.client.0.smithi027.stdout:0/627: write d0/d4/da/d1a/d59/f9c [294539,123169] 0 2024-04-26T23:01:35.546 INFO:tasks.workunit.client.0.smithi027.stdout:8/604: link d9/da/d2a/d5e/d2b/lc9 d9/da/d2a/d5e/d72/d84/d35/da7/ld8 0 2024-04-26T23:01:35.551 INFO:tasks.workunit.client.0.smithi027.stdout:4/695: read da/dd/de/d79/f7b [639313,126555] 0 2024-04-26T23:01:35.557 INFO:tasks.workunit.client.0.smithi027.stdout:4/696: creat da/dd/de/d14/d17/d80/dab/dd6/fdd x:0 0 0 2024-04-26T23:01:35.559 INFO:tasks.workunit.client.0.smithi027.stdout:4/697: creat da/dd/de/d79/da8/dc5/fde x:0 0 0 2024-04-26T23:01:35.573 INFO:tasks.workunit.client.0.smithi027.stdout:6/583: read d1/df/d43/f6e [3635011,28875] 0 2024-04-26T23:01:35.573 INFO:tasks.workunit.client.0.smithi027.stdout:6/584: write d1/f62 [1329551,64860] 0 2024-04-26T23:01:35.573 INFO:tasks.workunit.client.0.smithi027.stdout:6/585: write d1/df/d3c/f75 [1323318,51074] 0 2024-04-26T23:01:35.574 INFO:tasks.workunit.client.0.smithi027.stdout:4/698: dread da/dd/de/d14/d17/f20 [0,4194304] 0 2024-04-26T23:01:35.575 INFO:tasks.workunit.client.0.smithi027.stdout:4/699: rmdir da/dd/de/d14/d21/dad 39 2024-04-26T23:01:35.576 INFO:tasks.workunit.client.0.smithi027.stdout:1/696: dwrite d1/d26/d66/dab/fd5 [0,4194304] 0 2024-04-26T23:01:35.577 INFO:tasks.workunit.client.0.smithi027.stdout:1/697: truncate d1/d26/d66/d73/d9e/fba 4568416 0 2024-04-26T23:01:35.577 INFO:tasks.workunit.client.0.smithi027.stdout:1/698: fsync d1/d2a/f49 0 2024-04-26T23:01:35.577 INFO:tasks.workunit.client.0.smithi027.stdout:3/739: dread d6/dd3/f55 [0,4194304] 0 2024-04-26T23:01:35.577 INFO:tasks.workunit.client.0.smithi027.stdout:3/740: fdatasync d6/dd3/d34/d2e/d5f/fc4 0 2024-04-26T23:01:35.579 INFO:tasks.workunit.client.0.smithi027.stdout:4/700: getdents da 0 2024-04-26T23:01:35.580 INFO:tasks.workunit.client.0.smithi027.stdout:0/628: read d0/d4/da/d15/f6f [2378312,117262] 0 2024-04-26T23:01:35.582 INFO:tasks.workunit.client.0.smithi027.stdout:3/741: mknod d6/dd3/d34/d2e/d9d/da7/dac/c103 0 2024-04-26T23:01:35.582 INFO:tasks.workunit.client.0.smithi027.stdout:3/742: fsync d6/dd3/d34/d47/f69 0 2024-04-26T23:01:35.584 INFO:tasks.workunit.client.0.smithi027.stdout:3/743: mknod d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d40/c104 0 2024-04-26T23:01:35.585 INFO:tasks.workunit.client.0.smithi027.stdout:3/744: chown d6/dd3/d34/d2e/d5f/d90/ddb/d98/fc7 11 1 2024-04-26T23:01:35.586 INFO:tasks.workunit.client.0.smithi027.stdout:1/699: mknod d1/d26/d66/cdb 0 2024-04-26T23:01:35.587 INFO:tasks.workunit.client.0.smithi027.stdout:4/701: symlink da/dd/de/d14/db3/ldf 0 2024-04-26T23:01:35.588 INFO:tasks.workunit.client.0.smithi027.stdout:4/702: write da/dd/de/f3c [999407,102547] 0 2024-04-26T23:01:35.588 INFO:tasks.workunit.client.0.smithi027.stdout:4/703: chown da/dd/d5f/c9a 5666236 1 2024-04-26T23:01:35.588 INFO:tasks.workunit.client.0.smithi027.stdout:4/704: fdatasync da/dd/de/d14/d17/f2a 0 2024-04-26T23:01:35.588 INFO:tasks.workunit.client.0.smithi027.stdout:4/705: write da/dd/de/d14/d21/d41/d59/d74/fa1 [2610681,58048] 0 2024-04-26T23:01:35.588 INFO:tasks.workunit.client.0.smithi027.stdout:4/706: chown da/d40/dd7 22755441 1 2024-04-26T23:01:35.588 INFO:tasks.workunit.client.0.smithi027.stdout:4/707: read - da/fda zero size 2024-04-26T23:01:35.589 INFO:tasks.workunit.client.0.smithi027.stdout:4/708: write da/dd/de/d14/d21/d41/d59/d74/fa1 [3886334,112387] 0 2024-04-26T23:01:35.589 INFO:tasks.workunit.client.0.smithi027.stdout:4/709: dread - da/dd/de/d14/d17/d80/fb7 zero size 2024-04-26T23:01:35.589 INFO:tasks.workunit.client.0.smithi027.stdout:0/629: mknod d0/cb9 0 2024-04-26T23:01:35.596 INFO:tasks.workunit.client.0.smithi027.stdout:0/630: mkdir d0/d4/da/d1a/d1e/dba 0 2024-04-26T23:01:35.596 INFO:tasks.workunit.client.0.smithi027.stdout:0/631: read - d0/d4/da/d1a/d1e/f91 zero size 2024-04-26T23:01:35.596 INFO:tasks.workunit.client.0.smithi027.stdout:0/632: chown d0/df/d9d 609851 1 2024-04-26T23:01:35.596 INFO:tasks.workunit.client.0.smithi027.stdout:0/633: chown d0/df/d3f/c80 1 1 2024-04-26T23:01:35.596 INFO:tasks.workunit.client.0.smithi027.stdout:2/621: sync 2024-04-26T23:01:35.597 INFO:tasks.workunit.client.0.smithi027.stdout:2/622: mknod d2/d5/db/d1b/d1c/d25/d19/cba 0 2024-04-26T23:01:35.597 INFO:tasks.workunit.client.0.smithi027.stdout:2/623: write d2/d5/db/d1b/d1c/f77 [3688347,13500] 0 2024-04-26T23:01:35.598 INFO:tasks.workunit.client.0.smithi027.stdout:5/659: sync 2024-04-26T23:01:35.598 INFO:tasks.workunit.client.0.smithi027.stdout:7/716: sync 2024-04-26T23:01:35.598 INFO:tasks.workunit.client.0.smithi027.stdout:7/717: truncate d0/d28/d2b/d55/f95 5239037 0 2024-04-26T23:01:35.598 INFO:tasks.workunit.client.0.smithi027.stdout:7/718: readlink d0/d28/d2b/d55/d59/d6d/ld1 0 2024-04-26T23:01:35.599 INFO:tasks.workunit.client.0.smithi027.stdout:5/660: stat d4/d1a/d4a/l8d 0 2024-04-26T23:01:35.600 INFO:tasks.workunit.client.0.smithi027.stdout:5/661: write d4/d9f/dc0/fcc [999287,118001] 0 2024-04-26T23:01:35.601 INFO:tasks.workunit.client.0.smithi027.stdout:7/719: creat d0/d28/d2b/d3a/d90/dcb/fe9 x:0 0 0 2024-04-26T23:01:35.602 INFO:tasks.workunit.client.0.smithi027.stdout:7/720: dread - d0/d28/d2b/d3a/d48/d56/d88/f91 zero size 2024-04-26T23:01:35.602 INFO:tasks.workunit.client.0.smithi027.stdout:7/721: chown d0/d28/d9f/de1 46713 1 2024-04-26T23:01:35.607 INFO:tasks.workunit.client.0.smithi027.stdout:5/662: stat d4/d24/c11 0 2024-04-26T23:01:35.610 INFO:tasks.workunit.client.0.smithi027.stdout:3/745: dread d6/dd3/f11 [0,4194304] 0 2024-04-26T23:01:35.612 INFO:tasks.workunit.client.0.smithi027.stdout:5/663: truncate d4/f9 368488 0 2024-04-26T23:01:35.612 INFO:tasks.workunit.client.0.smithi027.stdout:4/710: dread da/dd/de/d14/f44 [4194304,4194304] 0 2024-04-26T23:01:35.612 INFO:tasks.workunit.client.0.smithi027.stdout:4/711: chown da/dd/de/d14/d21/dad 9815 1 2024-04-26T23:01:35.616 INFO:tasks.workunit.client.0.smithi027.stdout:3/746: mkdir d6/dd3/d34/d2e/d105 0 2024-04-26T23:01:35.616 INFO:tasks.workunit.client.0.smithi027.stdout:3/747: chown d6/dd3/d34/d2e/d5f/d90/ddb 5303 1 2024-04-26T23:01:35.621 INFO:tasks.workunit.client.0.smithi027.stdout:3/748: unlink d6/dd3/l6f 0 2024-04-26T23:01:35.621 INFO:tasks.workunit.client.0.smithi027.stdout:0/634: write d0/d4/da/d15/f5a [3171515,66630] 0 2024-04-26T23:01:35.623 INFO:tasks.workunit.client.0.smithi027.stdout:8/605: dwrite d9/da/d2a/d5e/d9e/fb5 [0,4194304] 0 2024-04-26T23:01:35.623 INFO:tasks.workunit.client.0.smithi027.stdout:3/749: mkdir d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d40/d106 0 2024-04-26T23:01:35.625 INFO:tasks.workunit.client.0.smithi027.stdout:4/712: mknod da/dd/de/d14/d17/d80/dab/dd6/ce0 0 2024-04-26T23:01:35.625 INFO:tasks.workunit.client.0.smithi027.stdout:4/713: stat da/dd/de/d79 0 2024-04-26T23:01:35.625 INFO:tasks.workunit.client.0.smithi027.stdout:5/664: creat d4/d1a/d2b/db8/fd5 x:0 0 0 2024-04-26T23:01:35.625 INFO:tasks.workunit.client.0.smithi027.stdout:5/665: chown d4/d1a/d2b/dd2 134129 1 2024-04-26T23:01:35.625 INFO:tasks.workunit.client.0.smithi027.stdout:5/666: readlink d4/d24/d22/dab/d16/d4b/d81/lb2 0 2024-04-26T23:01:35.626 INFO:tasks.workunit.client.0.smithi027.stdout:8/606: getdents d9/da/da1 0 2024-04-26T23:01:35.626 INFO:tasks.workunit.client.0.smithi027.stdout:8/607: truncate d9/da/d2a/d5e/d92/fae 751390 0 2024-04-26T23:01:35.626 INFO:tasks.workunit.client.0.smithi027.stdout:0/635: mkdir d0/df/d9d/d9f/dbb 0 2024-04-26T23:01:35.631 INFO:tasks.workunit.client.0.smithi027.stdout:3/750: truncate d6/dd3/d34/d47/f69 63557 0 2024-04-26T23:01:35.631 INFO:tasks.workunit.client.0.smithi027.stdout:3/751: fsync d6/d5e/ff2 0 2024-04-26T23:01:35.631 INFO:tasks.workunit.client.0.smithi027.stdout:3/752: getdents d6/dd3/d34/d2e/d9d 0 2024-04-26T23:01:35.631 INFO:tasks.workunit.client.0.smithi027.stdout:5/667: mknod d4/d24/d22/d29/d39/dc7/cd6 0 2024-04-26T23:01:35.631 INFO:tasks.workunit.client.0.smithi027.stdout:1/700: dread d1/dc/f1d [0,4194304] 0 2024-04-26T23:01:35.631 INFO:tasks.workunit.client.0.smithi027.stdout:0/636: getdents d0 0 2024-04-26T23:01:35.632 INFO:tasks.workunit.client.0.smithi027.stdout:8/608: rmdir d9/da/d2a/d5e/d72/d84/d5d 39 2024-04-26T23:01:35.634 INFO:tasks.workunit.client.0.smithi027.stdout:8/609: creat d9/da/d2a/d5e/d72/d84/d5d/d8b/fd9 x:0 0 0 2024-04-26T23:01:35.635 INFO:tasks.workunit.client.0.smithi027.stdout:1/701: link d1/d26/d86/fa2 d1/d26/d66/d73/d9e/db6/fdc 0 2024-04-26T23:01:35.635 INFO:tasks.workunit.client.0.smithi027.stdout:0/637: creat d0/d4/fbc x:0 0 0 2024-04-26T23:01:35.635 INFO:tasks.workunit.client.0.smithi027.stdout:0/638: truncate d0/d4/da/d15/d26/f7c 8507291 0 2024-04-26T23:01:35.637 INFO:tasks.workunit.client.0.smithi027.stdout:8/610: mknod d9/da/d2a/d5e/cda 0 2024-04-26T23:01:35.638 INFO:tasks.workunit.client.0.smithi027.stdout:8/611: creat d9/de/d22/fdb x:0 0 0 2024-04-26T23:01:35.639 INFO:tasks.workunit.client.0.smithi027.stdout:4/714: dread da/dd/de/d14/d21/d41/d59/fd4 [0,4194304] 0 2024-04-26T23:01:35.639 INFO:tasks.workunit.client.0.smithi027.stdout:0/639: symlink d0/d4/da/d15/d55/lbd 0 2024-04-26T23:01:35.639 INFO:tasks.workunit.client.0.smithi027.stdout:1/702: mknod d1/d26/d6b/d42/d70/cdd 0 2024-04-26T23:01:35.641 INFO:tasks.workunit.client.0.smithi027.stdout:0/640: rename d0/d4/da/d15/f33 to d0/d4/fbe 0 2024-04-26T23:01:35.643 INFO:tasks.workunit.client.0.smithi027.stdout:1/703: mknod d1/d26/d66/d73/dae/cde 0 2024-04-26T23:01:35.643 INFO:tasks.workunit.client.0.smithi027.stdout:1/704: read d1/fb [5288566,67814] 0 2024-04-26T23:01:35.645 INFO:tasks.workunit.client.0.smithi027.stdout:8/612: rmdir d9/da/d2a/d5e/d72/d84/d35/dc2 39 2024-04-26T23:01:35.646 INFO:tasks.workunit.client.0.smithi027.stdout:1/705: creat d1/d26/d86/d5c/fdf x:0 0 0 2024-04-26T23:01:35.647 INFO:tasks.workunit.client.0.smithi027.stdout:8/613: unlink d9/f1b 0 2024-04-26T23:01:35.647 INFO:tasks.workunit.client.0.smithi027.stdout:8/614: fdatasync d9/da/d2a/d5e/d72/d84/db8/fbe 0 2024-04-26T23:01:35.654 INFO:tasks.workunit.client.0.smithi027.stdout:8/615: creat d9/da/d2a/d5e/d72/d84/d5d/fdc x:0 0 0 2024-04-26T23:01:35.655 INFO:tasks.workunit.client.0.smithi027.stdout:8/616: readlink d9/da/d2a/lb1 0 2024-04-26T23:01:35.655 INFO:tasks.workunit.client.0.smithi027.stdout:8/617: fsync d9/da/d2a/d5e/d72/d84/db8/f5f 0 2024-04-26T23:01:35.657 INFO:tasks.workunit.client.0.smithi027.stdout:4/715: dread da/dd/de/f3c [0,4194304] 0 2024-04-26T23:01:35.659 INFO:tasks.workunit.client.0.smithi027.stdout:2/624: write d2/d5/db/d1b/d1c/d25/d19/f5b [212485,94961] 0 2024-04-26T23:01:35.659 INFO:tasks.workunit.client.0.smithi027.stdout:2/625: readlink d2/d5/db/d1b/d1c/l94 0 2024-04-26T23:01:35.659 INFO:tasks.workunit.client.0.smithi027.stdout:2/626: chown d2/d5/db/d1b/d1c/d25/d19/d23/fab 2 1 2024-04-26T23:01:35.659 INFO:tasks.workunit.client.0.smithi027.stdout:2/627: write d2/d5/db/f91 [399775,73046] 0 2024-04-26T23:01:35.659 INFO:tasks.workunit.client.0.smithi027.stdout:2/628: write d2/d5/db/de/d21/fa8 [476742,118163] 0 2024-04-26T23:01:35.669 INFO:tasks.workunit.client.0.smithi027.stdout:0/641: write d0/d4/da/f11 [1777285,30332] 0 2024-04-26T23:01:35.670 INFO:tasks.workunit.client.0.smithi027.stdout:6/586: dwrite d1/d30/d6c/d6f/f89 [0,4194304] 0 2024-04-26T23:01:35.671 INFO:tasks.workunit.client.0.smithi027.stdout:6/587: symlink d1/df/d3c/d66/lc0 0 2024-04-26T23:01:35.676 INFO:tasks.workunit.client.0.smithi027.stdout:8/618: rename d9/da/d2a/dc5 to d9/da/d2a/d5e/d46/ddd 0 2024-04-26T23:01:35.678 INFO:tasks.workunit.client.0.smithi027.stdout:7/722: dwrite d0/f62 [0,4194304] 0 2024-04-26T23:01:35.678 INFO:tasks.workunit.client.0.smithi027.stdout:3/753: dwrite d6/dd3/d34/d2e/d9d/da3/f89 [0,4194304] 0 2024-04-26T23:01:35.678 INFO:tasks.workunit.client.0.smithi027.stdout:3/754: chown d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/c77 23376 1 2024-04-26T23:01:35.678 INFO:tasks.workunit.client.0.smithi027.stdout:3/755: readlink d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/ld4 0 2024-04-26T23:01:35.679 INFO:tasks.workunit.client.0.smithi027.stdout:1/706: write d1/d26/d66/d73/d9e/f9f [934405,36568] 0 2024-04-26T23:01:35.681 INFO:tasks.workunit.client.0.smithi027.stdout:7/723: creat d0/d28/d2b/d3a/d48/d56/d88/d73/dd6/fea x:0 0 0 2024-04-26T23:01:35.681 INFO:tasks.workunit.client.0.smithi027.stdout:8/619: rmdir d9/da/d2a/d5e/d2b/d52 39 2024-04-26T23:01:35.681 INFO:tasks.workunit.client.0.smithi027.stdout:8/620: readlink d9/da/d2a/lb1 0 2024-04-26T23:01:35.685 INFO:tasks.workunit.client.0.smithi027.stdout:5/668: dwrite d4/d1a/d2b/f7b [0,4194304] 0 2024-04-26T23:01:35.712 INFO:tasks.workunit.client.0.smithi027.stdout:1/707: rename d1/d5/d21 to d1/d26/d66/dab/de0 0 2024-04-26T23:01:35.713 INFO:tasks.workunit.client.0.smithi027.stdout:1/708: symlink d1/d26/d66/le1 0 2024-04-26T23:01:35.714 INFO:tasks.workunit.client.0.smithi027.stdout:1/709: creat d1/dc/fe2 x:0 0 0 2024-04-26T23:01:35.716 INFO:tasks.workunit.client.0.smithi027.stdout:1/710: rename d1/d26/d7d/l38 to d1/d26/d36/d88/dca/le3 0 2024-04-26T23:01:35.719 INFO:tasks.workunit.client.0.smithi027.stdout:1/711: creat d1/d26/d6b/d42/d70/fe4 x:0 0 0 2024-04-26T23:01:35.724 INFO:tasks.workunit.client.0.smithi027.stdout:3/756: dwrite d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/fb8 [0,4194304] 0 2024-04-26T23:01:35.734 INFO:tasks.workunit.client.0.smithi027.stdout:4/716: dread da/dd/de/d14/d17/d80/dab/f19 [4194304,4194304] 0 2024-04-26T23:01:35.740 INFO:tasks.workunit.client.0.smithi027.stdout:6/588: write d1/f4e [1258460,92669] 0 2024-04-26T23:01:35.740 INFO:tasks.workunit.client.0.smithi027.stdout:6/589: chown d1/d30/l6d 44 1 2024-04-26T23:01:35.740 INFO:tasks.workunit.client.0.smithi027.stdout:6/590: dread - d1/df/d43/da8/da1/fbe zero size 2024-04-26T23:01:35.741 INFO:tasks.workunit.client.0.smithi027.stdout:6/591: readlink d1/d30/l38 0 2024-04-26T23:01:35.744 INFO:tasks.workunit.client.0.smithi027.stdout:6/592: mkdir d1/df/d3c/d66/dc1 0 2024-04-26T23:01:35.744 INFO:tasks.workunit.client.0.smithi027.stdout:7/724: write d0/d3d/f54 [1571425,6693] 0 2024-04-26T23:01:35.744 INFO:tasks.workunit.client.0.smithi027.stdout:6/593: chown d1/df/d43/f55 195855677 1 2024-04-26T23:01:35.748 INFO:tasks.workunit.client.0.smithi027.stdout:0/642: dwrite d0/d4/da/d1a/d1e/fa5 [0,4194304] 0 2024-04-26T23:01:35.748 INFO:tasks.workunit.client.0.smithi027.stdout:0/643: write d0/d4/faf [393450,1722] 0 2024-04-26T23:01:35.749 INFO:tasks.workunit.client.0.smithi027.stdout:0/644: symlink d0/d4/da/d1a/d6d/lbf 0 2024-04-26T23:01:35.757 INFO:tasks.workunit.client.0.smithi027.stdout:4/717: write da/fc2 [706452,103965] 0 2024-04-26T23:01:35.757 INFO:tasks.workunit.client.0.smithi027.stdout:4/718: readlink da/dd/de/d14/d21/d67/l90 0 2024-04-26T23:01:35.757 INFO:tasks.workunit.client.0.smithi027.stdout:8/621: dwrite d9/da/d2a/d5e/d72/d84/f29 [0,4194304] 0 2024-04-26T23:01:35.757 INFO:tasks.workunit.client.0.smithi027.stdout:8/622: write d9/da/d2a/d5e/d2b/d52/f7a [1503905,112600] 0 2024-04-26T23:01:35.759 INFO:tasks.workunit.client.0.smithi027.stdout:3/757: dwrite d6/dd3/d34/d2e/d70/f7d [0,4194304] 0 2024-04-26T23:01:35.760 INFO:tasks.workunit.client.0.smithi027.stdout:8/623: creat d9/da/d2a/d5e/d72/d84/d35/da7/fde x:0 0 0 2024-04-26T23:01:35.762 INFO:tasks.workunit.client.0.smithi027.stdout:1/712: read d1/d2a/f49 [2590963,44798] 0 2024-04-26T23:01:35.762 INFO:tasks.workunit.client.0.smithi027.stdout:0/645: symlink d0/df/d9d/d9f/dbb/lc0 0 2024-04-26T23:01:35.762 INFO:tasks.workunit.client.0.smithi027.stdout:7/725: symlink d0/d2f/leb 0 2024-04-26T23:01:35.763 INFO:tasks.workunit.client.0.smithi027.stdout:2/629: dwrite d2/d5/db/d1b/d1c/d25/f59 [0,4194304] 0 2024-04-26T23:01:35.764 INFO:tasks.workunit.client.0.smithi027.stdout:3/758: creat d6/dd3/d34/d78/da1/f107 x:0 0 0 2024-04-26T23:01:35.765 INFO:tasks.workunit.client.0.smithi027.stdout:6/594: read d1/f2 [3510266,92473] 0 2024-04-26T23:01:35.765 INFO:tasks.workunit.client.0.smithi027.stdout:4/719: rename da/dd/de/d14/d21/d67/d8a/lc1 to da/dd/le1 0 2024-04-26T23:01:35.766 INFO:tasks.workunit.client.0.smithi027.stdout:1/713: creat d1/d26/dc8/fe5 x:0 0 0 2024-04-26T23:01:35.766 INFO:tasks.workunit.client.0.smithi027.stdout:5/669: dwrite d4/d24/f42 [0,4194304] 0 2024-04-26T23:01:35.766 INFO:tasks.workunit.client.0.smithi027.stdout:5/670: fdatasync d4/d24/d22/d29/d2a/d3b/fa2 0 2024-04-26T23:01:35.768 INFO:tasks.workunit.client.0.smithi027.stdout:2/630: fdatasync d2/d5/db/de/d21/fa8 0 2024-04-26T23:01:35.770 INFO:tasks.workunit.client.0.smithi027.stdout:4/720: unlink da/dd/de/fc7 0 2024-04-26T23:01:35.770 INFO:tasks.workunit.client.0.smithi027.stdout:5/671: rename d4/d24/d22/d29/d2a/d3b/ld4 to d4/d9f/dc0/ld7 0 2024-04-26T23:01:35.770 INFO:tasks.workunit.client.0.smithi027.stdout:5/672: fdatasync d4/d1a/d2b/f7b 0 2024-04-26T23:01:35.770 INFO:tasks.workunit.client.0.smithi027.stdout:5/673: fdatasync d4/d1a/d2b/fc2 0 2024-04-26T23:01:35.770 INFO:tasks.workunit.client.0.smithi027.stdout:7/726: mknod d0/d28/d2b/d55/d69/da7/cec 0 2024-04-26T23:01:35.774 INFO:tasks.workunit.client.0.smithi027.stdout:3/759: mkdir d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d108 0 2024-04-26T23:01:35.775 INFO:tasks.workunit.client.0.smithi027.stdout:2/631: symlink d2/d5/db/d1b/d1c/d25/d6a/d8a/d8d/lbb 0 2024-04-26T23:01:35.775 INFO:tasks.workunit.client.0.smithi027.stdout:5/674: getdents d4/d24/d22/dab/d16/d4b 0 2024-04-26T23:01:35.775 INFO:tasks.workunit.client.0.smithi027.stdout:5/675: fsync d4/d24/d5f/dac/d40/d54/fa6 0 2024-04-26T23:01:35.775 INFO:tasks.workunit.client.0.smithi027.stdout:4/721: rename da/dd/d5f/l99 to da/dd/de/d14/db3/le2 0 2024-04-26T23:01:35.775 INFO:tasks.workunit.client.0.smithi027.stdout:4/722: chown da/dd/de/d14/d21/d4a/f6d 40607 1 2024-04-26T23:01:35.775 INFO:tasks.workunit.client.0.smithi027.stdout:4/723: stat da/dd/de/d14/d21/d67/d8a/f8c 0 2024-04-26T23:01:35.776 INFO:tasks.workunit.client.0.smithi027.stdout:2/632: getdents d2/d5/db/d1b/d1c/d25/d19/d36 0 2024-04-26T23:01:35.779 INFO:tasks.workunit.client.0.smithi027.stdout:8/624: dread d9/da/d2a/f89 [0,4194304] 0 2024-04-26T23:01:35.779 INFO:tasks.workunit.client.0.smithi027.stdout:8/625: chown d9/de/d2e/c44 7796 1 2024-04-26T23:01:35.783 INFO:tasks.workunit.client.0.smithi027.stdout:7/727: rename d0/d28/d2b/d55/f95 to d0/d28/fed 0 2024-04-26T23:01:35.788 INFO:tasks.workunit.client.0.smithi027.stdout:4/724: mkdir da/dd/d5e/de3 0 2024-04-26T23:01:35.794 INFO:tasks.workunit.client.0.smithi027.stdout:5/676: dread f1 [0,4194304] 0 2024-04-26T23:01:35.801 INFO:tasks.workunit.client.0.smithi027.stdout:2/633: rename d2/d5/db/d1b/d1c/d25/d19/d44/fa2 to d2/d5/db/d1b/d1c/fbc 0 2024-04-26T23:01:35.804 INFO:tasks.workunit.client.0.smithi027.stdout:5/677: rename d4/d1a/d4a/c82 to d4/d24/d22/d29/d2a/d3b/cd8 0 2024-04-26T23:01:35.806 INFO:tasks.workunit.client.0.smithi027.stdout:1/714: write d1/d26/d66/d73/fbd [3831670,116805] 0 2024-04-26T23:01:35.808 INFO:tasks.workunit.client.0.smithi027.stdout:3/760: dread d6/f2c [0,4194304] 0 2024-04-26T23:01:35.809 INFO:tasks.workunit.client.0.smithi027.stdout:1/715: rename d1/d26/d66/dab/fc3 to d1/d26/d66/d73/fe6 0 2024-04-26T23:01:35.810 INFO:tasks.workunit.client.0.smithi027.stdout:1/716: creat d1/d26/d6b/d8e/dc5/fe7 x:0 0 0 2024-04-26T23:01:35.813 INFO:tasks.workunit.client.0.smithi027.stdout:3/761: truncate d6/dd3/d34/d47/f65 493562 0 2024-04-26T23:01:35.814 INFO:tasks.workunit.client.0.smithi027.stdout:1/717: creat d1/d26/d66/d73/dae/fe8 x:0 0 0 2024-04-26T23:01:35.816 INFO:tasks.workunit.client.0.smithi027.stdout:1/718: mknod d1/d26/d66/d73/dae/dd0/ce9 0 2024-04-26T23:01:35.816 INFO:tasks.workunit.client.0.smithi027.stdout:1/719: write d1/d2a/fac [469679,40728] 0 2024-04-26T23:01:35.816 INFO:tasks.workunit.client.0.smithi027.stdout:4/725: read da/f1f [1850235,121901] 0 2024-04-26T23:01:35.816 INFO:tasks.workunit.client.0.smithi027.stdout:1/720: write d1/d26/d66/dab/de0/f44 [4324266,59517] 0 2024-04-26T23:01:35.816 INFO:tasks.workunit.client.0.smithi027.stdout:1/721: dread - d1/d26/d66/d73/dae/fe8 zero size 2024-04-26T23:01:35.817 INFO:tasks.workunit.client.0.smithi027.stdout:4/726: truncate da/dd/de/d79/f7b 672357 0 2024-04-26T23:01:35.817 INFO:tasks.workunit.client.0.smithi027.stdout:3/762: link d6/dd3/d34/l44 d6/dd3/d2f/d7e/l109 0 2024-04-26T23:01:35.817 INFO:tasks.workunit.client.0.smithi027.stdout:3/763: dread - d6/dd3/d34/d47/ffd zero size 2024-04-26T23:01:35.818 INFO:tasks.workunit.client.0.smithi027.stdout:4/727: rmdir da/dd/de/d14/db3 39 2024-04-26T23:01:35.822 INFO:tasks.workunit.client.0.smithi027.stdout:7/728: dread d0/d28/d66/f70 [0,4194304] 0 2024-04-26T23:01:35.823 INFO:tasks.workunit.client.0.smithi027.stdout:7/729: write d0/d28/d2b/d3a/d48/d56/d88/fda [655871,117281] 0 2024-04-26T23:01:35.823 INFO:tasks.workunit.client.0.smithi027.stdout:7/730: chown d0/d28/d2b/d3a/d48/d56/d88/l61 600666 1 2024-04-26T23:01:35.823 INFO:tasks.workunit.client.0.smithi027.stdout:7/731: read - d0/d28/d2b/d3a/d48/d56/d88/d73/dd6/fea zero size 2024-04-26T23:01:35.824 INFO:tasks.workunit.client.0.smithi027.stdout:1/722: rename d1/d26/d66/dcc/fd8 to d1/d26/d36/d56/fea 0 2024-04-26T23:01:35.826 INFO:tasks.workunit.client.0.smithi027.stdout:1/723: mkdir d1/d2a/deb 0 2024-04-26T23:01:35.826 INFO:tasks.workunit.client.0.smithi027.stdout:1/724: chown d1/d26/d6b/d42/d79 507919 1 2024-04-26T23:01:35.829 INFO:tasks.workunit.client.0.smithi027.stdout:4/728: rename da/dd/de/d14/d21/d67/da6/lbe to da/dd/de/d14/d21/le4 0 2024-04-26T23:01:35.829 INFO:tasks.workunit.client.0.smithi027.stdout:4/729: chown da/dd/de/d79/da8 17 1 2024-04-26T23:01:35.830 INFO:tasks.workunit.client.0.smithi027.stdout:1/725: getdents d1/d5 0 2024-04-26T23:01:35.830 INFO:tasks.workunit.client.0.smithi027.stdout:1/726: truncate d1/f94 765569 0 2024-04-26T23:01:35.830 INFO:tasks.workunit.client.0.smithi027.stdout:1/727: dread - d1/d26/d6b/d42/d70/fe4 zero size 2024-04-26T23:01:35.831 INFO:tasks.workunit.client.0.smithi027.stdout:4/730: truncate da/dd/de/d14/d21/d41/f72 3290122 0 2024-04-26T23:01:35.832 INFO:tasks.workunit.client.0.smithi027.stdout:4/731: stat da/dd/de/d79/da8/dc5 0 2024-04-26T23:01:35.834 INFO:tasks.workunit.client.0.smithi027.stdout:0/646: dwrite d0/d4/da/d1a/d1e/f7f [0,4194304] 0 2024-04-26T23:01:35.834 INFO:tasks.workunit.client.0.smithi027.stdout:0/647: read - d0/fac zero size 2024-04-26T23:01:35.840 INFO:tasks.workunit.client.0.smithi027.stdout:0/648: link d0/d4/c4f d0/d4/da/d1a/d1e/dba/cc1 0 2024-04-26T23:01:35.843 INFO:tasks.workunit.client.0.smithi027.stdout:0/649: creat d0/fc2 x:0 0 0 2024-04-26T23:01:35.856 INFO:tasks.workunit.client.0.smithi027.stdout:8/626: dwrite d9/da/d2a/d5e/d72/d84/d35/f99 [0,4194304] 0 2024-04-26T23:01:35.857 INFO:tasks.workunit.client.0.smithi027.stdout:0/650: mkdir d0/d4/da/d1a/dc3 0 2024-04-26T23:01:35.860 INFO:tasks.workunit.client.0.smithi027.stdout:0/651: mkdir d0/d4/da/d15/d26/dc4 0 2024-04-26T23:01:35.866 INFO:tasks.workunit.client.0.smithi027.stdout:1/728: rmdir d1/d26/d66/dcc 0 2024-04-26T23:01:35.866 INFO:tasks.workunit.client.0.smithi027.stdout:1/729: write d1/d2a/fbc [503684,52016] 0 2024-04-26T23:01:35.866 INFO:tasks.workunit.client.0.smithi027.stdout:8/627: creat d9/da/d2a/d5e/d92/fdf x:0 0 0 2024-04-26T23:01:35.866 INFO:tasks.workunit.client.0.smithi027.stdout:5/678: write d4/d24/f4c [844259,85879] 0 2024-04-26T23:01:35.866 INFO:tasks.workunit.client.0.smithi027.stdout:5/679: readlink d4/d1a/d2b/l9a 0 2024-04-26T23:01:35.880 INFO:tasks.workunit.client.0.smithi027.stdout:1/730: unlink d1/d26/l5f 0 2024-04-26T23:01:35.880 INFO:tasks.workunit.client.0.smithi027.stdout:1/731: chown d1/d2a 5430637 1 2024-04-26T23:01:35.880 INFO:tasks.workunit.client.0.smithi027.stdout:4/732: dwrite da/dd/de/d14/d17/d80/dab/f26 [0,4194304] 0 2024-04-26T23:01:35.881 INFO:tasks.workunit.client.0.smithi027.stdout:5/680: mkdir d4/d1a/dcb/dd1/dd9 0 2024-04-26T23:01:35.881 INFO:tasks.workunit.client.0.smithi027.stdout:5/681: readlink d4/d24/d5f/dac/d40/l4d 0 2024-04-26T23:01:35.881 INFO:tasks.workunit.client.0.smithi027.stdout:5/682: chown d4/d24/d22/d29/l2e 1157791 1 2024-04-26T23:01:35.882 INFO:tasks.workunit.client.0.smithi027.stdout:5/683: fsync d4/d24/d22/d29/d2a/d3b/fb7 0 2024-04-26T23:01:35.882 INFO:tasks.workunit.client.0.smithi027.stdout:5/684: write d4/d1a/d2b/db8/fd5 [366087,20138] 0 2024-04-26T23:01:35.883 INFO:tasks.workunit.client.0.smithi027.stdout:5/685: link d4/d24/d5f/dac/d40/d54/c8e d4/d1a/dcb/dd1/dd9/cda 0 2024-04-26T23:01:35.884 INFO:tasks.workunit.client.0.smithi027.stdout:0/652: write d0/df/d3f/d64/f70 [785438,72582] 0 2024-04-26T23:01:35.887 INFO:tasks.workunit.client.0.smithi027.stdout:4/733: rmdir da/dd/de/d14/d17/d80/dab 39 2024-04-26T23:01:35.888 INFO:tasks.workunit.client.0.smithi027.stdout:4/734: truncate da/f48 3606364 0 2024-04-26T23:01:35.888 INFO:tasks.workunit.client.0.smithi027.stdout:4/735: truncate da/dd/de/d14/d17/f45 931863 0 2024-04-26T23:01:35.888 INFO:tasks.workunit.client.0.smithi027.stdout:4/736: write da/dd/de/f3c [428490,10302] 0 2024-04-26T23:01:35.896 INFO:tasks.workunit.client.0.smithi027.stdout:2/634: dwrite d2/f89 [0,4194304] 0 2024-04-26T23:01:35.896 INFO:tasks.workunit.client.0.smithi027.stdout:1/732: write d1/d26/d66/d73/d9e/fba [2553570,16211] 0 2024-04-26T23:01:35.898 INFO:tasks.workunit.client.0.smithi027.stdout:2/635: creat d2/d5/db/d1b/d1c/d25/d19/d36/d39/fbd x:0 0 0 2024-04-26T23:01:35.903 INFO:tasks.workunit.client.0.smithi027.stdout:8/628: dread d9/da/d33/f60 [0,4194304] 0 2024-04-26T23:01:35.905 INFO:tasks.workunit.client.0.smithi027.stdout:7/732: dwrite d0/d28/d2b/d55/f7e [0,4194304] 0 2024-04-26T23:01:35.912 INFO:tasks.workunit.client.0.smithi027.stdout:8/629: write d9/da/d2a/f89 [55046,37010] 0 2024-04-26T23:01:35.915 INFO:tasks.workunit.client.0.smithi027.stdout:8/630: rename d9/da/d33/c5b to d9/da/d2a/d5e/d72/d84/d4f/dd3/ce0 0 2024-04-26T23:01:35.915 INFO:tasks.workunit.client.0.smithi027.stdout:8/631: stat d9/da/d2a/d5e/d46/f6f 0 2024-04-26T23:01:35.916 INFO:tasks.workunit.client.0.smithi027.stdout:0/653: read d0/d4/da/d1a/d1e/f35 [3466021,43079] 0 2024-04-26T23:01:35.917 INFO:tasks.workunit.client.0.smithi027.stdout:8/632: mknod d9/da/d2a/d5e/ce1 0 2024-04-26T23:01:35.918 INFO:tasks.workunit.client.0.smithi027.stdout:8/633: mkdir d9/da/d2a/d6b/de2 0 2024-04-26T23:01:35.919 INFO:tasks.workunit.client.0.smithi027.stdout:8/634: creat d9/de/d22/fe3 x:0 0 0 2024-04-26T23:01:35.921 INFO:tasks.workunit.client.0.smithi027.stdout:8/635: link d9/de/d22/c75 d9/da6/ce4 0 2024-04-26T23:01:35.922 INFO:tasks.workunit.client.0.smithi027.stdout:8/636: symlink d9/da/d2a/d5e/d2b/d52/le5 0 2024-04-26T23:01:35.924 INFO:tasks.workunit.client.0.smithi027.stdout:8/637: rmdir d9/da/d2a/d6b/de2 0 2024-04-26T23:01:35.924 INFO:tasks.workunit.client.0.smithi027.stdout:5/686: dread d4/d24/d5f/dac/d40/d54/f75 [0,4194304] 0 2024-04-26T23:01:35.928 INFO:tasks.workunit.client.0.smithi027.stdout:1/733: read d1/d26/d66/dab/fbb [213792,116148] 0 2024-04-26T23:01:35.929 INFO:tasks.workunit.client.0.smithi027.stdout:7/733: getdents d0/d3d/dba 0 2024-04-26T23:01:35.929 INFO:tasks.workunit.client.0.smithi027.stdout:9/594: sync 2024-04-26T23:01:35.947 INFO:tasks.workunit.client.0.smithi027.stdout:0/654: mknod d0/df/d3f/d64/da8/cc5 0 2024-04-26T23:01:35.947 INFO:tasks.workunit.client.0.smithi027.stdout:7/734: mkdir d0/d28/d2b/d3a/d48/d56/dee 0 2024-04-26T23:01:35.947 INFO:tasks.workunit.client.0.smithi027.stdout:9/595: link d4/d10/c16 d4/d17/d2e/d81/da7/cce 0 2024-04-26T23:01:35.947 INFO:tasks.workunit.client.0.smithi027.stdout:1/734: creat d1/d26/d66/d73/d9e/db6/fec x:0 0 0 2024-04-26T23:01:35.947 INFO:tasks.workunit.client.0.smithi027.stdout:5/687: symlink d4/d1a/dcb/dd1/dd9/ldb 0 2024-04-26T23:01:35.947 INFO:tasks.workunit.client.0.smithi027.stdout:5/688: chown d4/d24/c41 2933535 1 2024-04-26T23:01:35.947 INFO:tasks.workunit.client.0.smithi027.stdout:9/596: mkdir d4/d17/d2e/dcf 0 2024-04-26T23:01:35.947 INFO:tasks.workunit.client.0.smithi027.stdout:9/597: chown d4/d10/d40/d58/l5b 520576151 1 2024-04-26T23:01:35.947 INFO:tasks.workunit.client.0.smithi027.stdout:9/598: fdatasync d4/d17/d2e/d46/da0/d72/f79 0 2024-04-26T23:01:35.947 INFO:tasks.workunit.client.0.smithi027.stdout:5/689: creat d4/d1a/d2b/dd3/fdc x:0 0 0 2024-04-26T23:01:35.947 INFO:tasks.workunit.client.0.smithi027.stdout:0/655: mkdir d0/d4/da/d15/d26/dc4/dc6 0 2024-04-26T23:01:35.948 INFO:tasks.workunit.client.0.smithi027.stdout:0/656: readlink d0/d4/da/d1a/d59/lab 0 2024-04-26T23:01:35.948 INFO:tasks.workunit.client.0.smithi027.stdout:5/690: creat d4/d24/d55/d63/fdd x:0 0 0 2024-04-26T23:01:35.950 INFO:tasks.workunit.client.0.smithi027.stdout:4/737: dwrite f2 [0,4194304] 0 2024-04-26T23:01:35.950 INFO:tasks.workunit.client.0.smithi027.stdout:4/738: fsync da/dd/f2b 0 2024-04-26T23:01:35.950 INFO:tasks.workunit.client.0.smithi027.stdout:4/739: fsync da/f1f 0 2024-04-26T23:01:35.955 INFO:tasks.workunit.client.0.smithi027.stdout:0/657: link d0/d4/da/d15/l29 d0/d4/da/d15/d55/lc7 0 2024-04-26T23:01:35.956 INFO:tasks.workunit.client.0.smithi027.stdout:4/740: truncate da/dd/f56 59361 0 2024-04-26T23:01:35.956 INFO:tasks.workunit.client.0.smithi027.stdout:4/741: write da/dd/de/d14/d21/d41/d59/f78 [2354095,72321] 0 2024-04-26T23:01:35.956 INFO:tasks.workunit.client.0.smithi027.stdout:4/742: truncate da/dd/d5f/dac/f96 424989 0 2024-04-26T23:01:35.956 INFO:tasks.workunit.client.0.smithi027.stdout:4/743: chown da/f1f 43012 1 2024-04-26T23:01:35.961 INFO:tasks.workunit.client.0.smithi027.stdout:4/744: dread da/d40/f66 [0,4194304] 0 2024-04-26T23:01:35.961 INFO:tasks.workunit.client.0.smithi027.stdout:4/745: getdents da/dd/de/d79/dcd 0 2024-04-26T23:01:35.980 INFO:tasks.workunit.client.0.smithi027.stdout:8/638: dwrite d9/da/d2a/d5e/d72/d84/d5d/d8b/fbb [0,4194304] 0 2024-04-26T23:01:35.983 INFO:tasks.workunit.client.0.smithi027.stdout:1/735: write d1/d26/d86/f4c [375497,130308] 0 2024-04-26T23:01:35.983 INFO:tasks.workunit.client.0.smithi027.stdout:1/736: chown d1/dc/c9b 0 1 2024-04-26T23:01:35.983 INFO:tasks.workunit.client.0.smithi027.stdout:4/746: creat da/dd/d5e/fe5 x:0 0 0 2024-04-26T23:01:35.986 INFO:tasks.workunit.client.0.smithi027.stdout:1/737: fdatasync d1/d26/d6b/f7f 0 2024-04-26T23:01:35.986 INFO:tasks.workunit.client.0.smithi027.stdout:4/747: rename da/dd/de/d79/f83 to da/dd/de/d14/d21/d41/d59/d74/fe6 0 2024-04-26T23:01:35.987 INFO:tasks.workunit.client.0.smithi027.stdout:0/658: dread d0/d4/da/f2f [0,4194304] 0 2024-04-26T23:01:35.987 INFO:tasks.workunit.client.0.smithi027.stdout:0/659: chown d0/d4/da/d1a/d1e/fb0 16287 1 2024-04-26T23:01:35.997 INFO:tasks.workunit.client.0.smithi027.stdout:1/738: link d1/d26/f29 d1/d26/d66/d91/fed 0 2024-04-26T23:01:36.000 INFO:tasks.workunit.client.0.smithi027.stdout:1/739: link d1/d26/d86/l4f d1/d26/dc8/lee 0 2024-04-26T23:01:36.026 INFO:tasks.workunit.client.0.smithi027.stdout:7/735: dwrite d0/d28/d2b/f4e [4194304,4194304] 0 2024-04-26T23:01:36.027 INFO:tasks.workunit.client.0.smithi027.stdout:5/691: dread d4/d24/f4c [0,4194304] 0 2024-04-26T23:01:36.027 INFO:tasks.workunit.client.0.smithi027.stdout:7/736: chown d0/d28/d2b/d3a/d48/d56/d88 1038 1 2024-04-26T23:01:36.027 INFO:tasks.workunit.client.0.smithi027.stdout:7/737: stat d0/d28/c21 0 2024-04-26T23:01:36.027 INFO:tasks.workunit.client.0.smithi027.stdout:7/738: fdatasync d0/d28/d2b/d3a/fb1 0 2024-04-26T23:01:36.027 INFO:tasks.workunit.client.0.smithi027.stdout:5/692: write d4/d1a/d2b/dd3/fdc [431811,105699] 0 2024-04-26T23:01:36.027 INFO:tasks.workunit.client.0.smithi027.stdout:5/693: link d4/d24/d22/d29/d2a/d3b/f3d d4/d24/d22/d29/d39/d61/fde 0 2024-04-26T23:01:36.027 INFO:tasks.workunit.client.0.smithi027.stdout:1/740: write d1/d26/d36/d56/f6e [4175922,86157] 0 2024-04-26T23:01:36.028 INFO:tasks.workunit.client.0.smithi027.stdout:5/694: mknod d4/d24/d22/cdf 0 2024-04-26T23:01:36.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/741: creat d1/d26/fef x:0 0 0 2024-04-26T23:01:36.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/742: truncate d1/f1b 2879345 0 2024-04-26T23:01:36.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/743: dread - d1/d26/fef zero size 2024-04-26T23:01:36.030 INFO:tasks.workunit.client.0.smithi027.stdout:5/695: write d4/f64 [5592599,79212] 0 2024-04-26T23:01:36.030 INFO:tasks.workunit.client.0.smithi027.stdout:0/660: dwrite d0/d4/faf [0,4194304] 0 2024-04-26T23:01:36.031 INFO:tasks.workunit.client.0.smithi027.stdout:1/744: symlink d1/d26/dc8/lf0 0 2024-04-26T23:01:36.032 INFO:tasks.workunit.client.0.smithi027.stdout:1/745: truncate d1/dc/fb1 819728 0 2024-04-26T23:01:36.034 INFO:tasks.workunit.client.0.smithi027.stdout:5/696: rename c0 to d4/d24/d5f/ce0 0 2024-04-26T23:01:36.034 INFO:tasks.workunit.client.0.smithi027.stdout:9/599: dwrite d4/d17/f1d [0,4194304] 0 2024-04-26T23:01:36.034 INFO:tasks.workunit.client.0.smithi027.stdout:8/639: dread d9/de/f18 [0,4194304] 0 2024-04-26T23:01:36.035 INFO:tasks.workunit.client.0.smithi027.stdout:5/697: rmdir d4/d24/d5f/dac/d40/d54 39 2024-04-26T23:01:36.036 INFO:tasks.workunit.client.0.smithi027.stdout:5/698: chown d4/fc1 27604538 1 2024-04-26T23:01:36.042 INFO:tasks.workunit.client.0.smithi027.stdout:8/640: truncate d9/da/d2a/d5e/d2b/f76 867188 0 2024-04-26T23:01:36.042 INFO:tasks.workunit.client.0.smithi027.stdout:8/641: chown d9/da/d33/lca 195029787 1 2024-04-26T23:01:36.042 INFO:tasks.workunit.client.0.smithi027.stdout:5/699: creat d4/d24/d22/d29/d2a/fe1 x:0 0 0 2024-04-26T23:01:36.042 INFO:tasks.workunit.client.0.smithi027.stdout:1/746: read d1/d5/f2e [106728,12619] 0 2024-04-26T23:01:36.049 INFO:tasks.workunit.client.0.smithi027.stdout:1/747: mkdir d1/d26/d36/d56/df1 0 2024-04-26T23:01:36.050 INFO:tasks.workunit.client.0.smithi027.stdout:1/748: mkdir d1/d26/d36/d56/df1/df2 0 2024-04-26T23:01:36.054 INFO:tasks.workunit.client.0.smithi027.stdout:9/600: write d4/d17/d2e/d46/da0/d72/f79 [3090162,22713] 0 2024-04-26T23:01:36.056 INFO:tasks.workunit.client.0.smithi027.stdout:9/601: unlink d4/d10/d40/d58/d75/f7d 0 2024-04-26T23:01:36.057 INFO:tasks.workunit.client.0.smithi027.stdout:9/602: readlink d4/d17/d2e/d46/l60 0 2024-04-26T23:01:36.057 INFO:tasks.workunit.client.0.smithi027.stdout:5/700: dread d4/d24/f1b [0,4194304] 0 2024-04-26T23:01:36.058 INFO:tasks.workunit.client.0.smithi027.stdout:9/603: creat d4/d17/d2e/d46/da0/d72/fd0 x:0 0 0 2024-04-26T23:01:36.059 INFO:tasks.workunit.client.0.smithi027.stdout:5/701: symlink d4/d1a/d2b/dd3/le2 0 2024-04-26T23:01:36.060 INFO:tasks.workunit.client.0.smithi027.stdout:9/604: mkdir d4/d17/d2e/d46/dd1 0 2024-04-26T23:01:36.062 INFO:tasks.workunit.client.0.smithi027.stdout:5/702: creat d4/d24/d22/dab/d16/d4b/d81/fe3 x:0 0 0 2024-04-26T23:01:36.064 INFO:tasks.workunit.client.0.smithi027.stdout:9/605: creat d4/d10/d40/d58/d75/d84/d86/fd2 x:0 0 0 2024-04-26T23:01:36.065 INFO:tasks.workunit.client.0.smithi027.stdout:9/606: creat d4/d17/d2e/d46/fd3 x:0 0 0 2024-04-26T23:01:36.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/607: creat d4/fd4 x:0 0 0 2024-04-26T23:01:36.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/608: chown d4/d17/d2e/d46/da0/f41 0 1 2024-04-26T23:01:36.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/609: fsync d4/d10/d40/f66 0 2024-04-26T23:01:36.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/610: symlink d4/d10/d25/dc9/ld5 0 2024-04-26T23:01:36.068 INFO:tasks.workunit.client.0.smithi027.stdout:9/611: readlink d4/d17/lb2 0 2024-04-26T23:01:36.069 INFO:tasks.workunit.client.0.smithi027.stdout:9/612: creat d4/d17/d2e/d46/da0/d72/fd6 x:0 0 0 2024-04-26T23:01:36.070 INFO:tasks.workunit.client.0.smithi027.stdout:9/613: creat d4/d10/d40/d58/d75/d84/dc4/fd7 x:0 0 0 2024-04-26T23:01:36.070 INFO:tasks.workunit.client.0.smithi027.stdout:9/614: truncate d4/d10/d40/d58/d75/f47 342959 0 2024-04-26T23:01:36.072 INFO:tasks.workunit.client.0.smithi027.stdout:9/615: symlink d4/d77/da2/daf/ld8 0 2024-04-26T23:01:36.072 INFO:tasks.workunit.client.0.smithi027.stdout:9/616: truncate d4/d77/da2/daf/fbb 603499 0 2024-04-26T23:01:36.074 INFO:tasks.workunit.client.0.smithi027.stdout:9/617: rename d4/d17/d2e/d81/c92 to d4/d77/da2/daf/cd9 0 2024-04-26T23:01:36.075 INFO:tasks.workunit.client.0.smithi027.stdout:9/618: link d4/d10/d40/d58/d75/f63 d4/d17/d2e/dcf/fda 0 2024-04-26T23:01:36.094 INFO:tasks.workunit.client.0.smithi027.stdout:1/749: dwrite d1/d26/d66/dab/de0/f22 [4194304,4194304] 0 2024-04-26T23:01:36.095 INFO:tasks.workunit.client.0.smithi027.stdout:1/750: creat d1/d26/d66/d91/ff3 x:0 0 0 2024-04-26T23:01:36.114 INFO:tasks.workunit.client.0.smithi027.stdout:1/751: write d1/d26/d66/dab/fb9 [4062137,71705] 0 2024-04-26T23:01:36.114 INFO:tasks.workunit.client.0.smithi027.stdout:9/619: dwrite d4/fd4 [0,4194304] 0 2024-04-26T23:01:36.120 INFO:tasks.workunit.client.0.smithi027.stdout:1/752: link d1/d26/d66/d73/d9e/db6/lbe d1/d26/d66/dab/de0/lf4 0 2024-04-26T23:01:36.120 INFO:tasks.workunit.client.0.smithi027.stdout:1/753: write d1/d26/d86/d5c/fd4 [8339,45476] 0 2024-04-26T23:01:36.121 INFO:tasks.workunit.client.0.smithi027.stdout:1/754: creat d1/d26/dc8/ff5 x:0 0 0 2024-04-26T23:01:36.131 INFO:tasks.workunit.client.0.smithi027.stdout:9/620: write d4/d17/f29 [2956363,120989] 0 2024-04-26T23:01:36.131 INFO:tasks.workunit.client.0.smithi027.stdout:9/621: dread - d4/d10/d40/f9a zero size 2024-04-26T23:01:36.131 INFO:tasks.workunit.client.0.smithi027.stdout:9/622: chown d4/d17/d2e/d81/da7 132 1 2024-04-26T23:01:36.171 INFO:tasks.workunit.client.0.smithi027.stdout:1/755: dwrite d1/d26/d66/d91/ff3 [0,4194304] 0 2024-04-26T23:01:36.199 INFO:tasks.workunit.client.0.smithi027.stdout:9/623: dwrite d4/d17/d2e/d46/f5f [0,4194304] 0 2024-04-26T23:01:36.202 INFO:tasks.workunit.client.0.smithi027.stdout:9/624: link d4/d17/d2e/cc1 d4/cdb 0 2024-04-26T23:01:36.203 INFO:tasks.workunit.client.0.smithi027.stdout:9/625: creat d4/d17/d2e/d46/da0/d8d/fdc x:0 0 0 2024-04-26T23:01:36.204 INFO:tasks.workunit.client.0.smithi027.stdout:4/748: sync 2024-04-26T23:01:36.205 INFO:tasks.workunit.client.0.smithi027.stdout:6/595: sync 2024-04-26T23:01:36.205 INFO:tasks.workunit.client.0.smithi027.stdout:3/764: sync 2024-04-26T23:01:36.205 INFO:tasks.workunit.client.0.smithi027.stdout:2/636: sync 2024-04-26T23:01:36.205 INFO:tasks.workunit.client.0.smithi027.stdout:9/626: unlink d4/d17/l4c 0 2024-04-26T23:01:36.206 INFO:tasks.workunit.client.0.smithi027.stdout:6/596: mkdir d1/df/dc2 0 2024-04-26T23:01:36.236 INFO:tasks.workunit.client.0.smithi027.stdout:1/756: dwrite d1/d26/d86/d5c/fd4 [0,4194304] 0 2024-04-26T23:01:36.242 INFO:tasks.workunit.client.0.smithi027.stdout:1/757: fdatasync d1/d26/d6b/fad 0 2024-04-26T23:01:36.242 INFO:tasks.workunit.client.0.smithi027.stdout:4/749: write da/dd/de/d14/d17/d80/dab/f3a [716497,88523] 0 2024-04-26T23:01:36.244 INFO:tasks.workunit.client.0.smithi027.stdout:8/642: sync 2024-04-26T23:01:36.244 INFO:tasks.workunit.client.0.smithi027.stdout:5/703: sync 2024-04-26T23:01:36.244 INFO:tasks.workunit.client.0.smithi027.stdout:7/739: sync 2024-04-26T23:01:36.244 INFO:tasks.workunit.client.0.smithi027.stdout:7/740: dread - d0/d28/d2b/d55/d69/d75/f89 zero size 2024-04-26T23:01:36.245 INFO:tasks.workunit.client.0.smithi027.stdout:0/661: sync 2024-04-26T23:01:36.247 INFO:tasks.workunit.client.0.smithi027.stdout:8/643: symlink d9/da/d2a/d5e/d72/d84/d5d/le6 0 2024-04-26T23:01:36.256 INFO:tasks.workunit.client.0.smithi027.stdout:1/758: creat d1/d26/d66/d73/dae/dd0/ff6 x:0 0 0 2024-04-26T23:01:36.256 INFO:tasks.workunit.client.0.smithi027.stdout:7/741: symlink d0/d28/d2b/d3a/d48/d56/d8d/dc5/lef 0 2024-04-26T23:01:36.257 INFO:tasks.workunit.client.0.smithi027.stdout:4/750: rename da/dd/de/d14/d17/d80/dab/f19 to da/d40/daf/fe7 0 2024-04-26T23:01:36.257 INFO:tasks.workunit.client.0.smithi027.stdout:4/751: chown da/dd/de 128079 1 2024-04-26T23:01:36.257 INFO:tasks.workunit.client.0.smithi027.stdout:4/752: chown da/d40/daf/fe7 622551 1 2024-04-26T23:01:36.257 INFO:tasks.workunit.client.0.smithi027.stdout:4/753: stat da/c2e 0 2024-04-26T23:01:36.257 INFO:tasks.workunit.client.0.smithi027.stdout:2/637: dread d2/d5/f6e [0,4194304] 0 2024-04-26T23:01:36.258 INFO:tasks.workunit.client.0.smithi027.stdout:3/765: dwrite d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/f2b [0,4194304] 0 2024-04-26T23:01:36.263 INFO:tasks.workunit.client.0.smithi027.stdout:3/766: mknod d6/d5e/c10a 0 2024-04-26T23:01:36.263 INFO:tasks.workunit.client.0.smithi027.stdout:9/627: read d4/fd4 [1637391,25031] 0 2024-04-26T23:01:36.264 INFO:tasks.workunit.client.0.smithi027.stdout:6/597: dwrite d1/df/d43/da8/d84/fb8 [0,4194304] 0 2024-04-26T23:01:36.265 INFO:tasks.workunit.client.0.smithi027.stdout:6/598: write d1/df/d43/da8/da1/fbe [15324,120624] 0 2024-04-26T23:01:36.265 INFO:tasks.workunit.client.0.smithi027.stdout:0/662: creat d0/d4/da/d1a/d2a/fc8 x:0 0 0 2024-04-26T23:01:36.265 INFO:tasks.workunit.client.0.smithi027.stdout:1/759: creat d1/d2a/ff7 x:0 0 0 2024-04-26T23:01:36.266 INFO:tasks.workunit.client.0.smithi027.stdout:4/754: mkdir da/dd/de/d79/da8/dc5/de8 0 2024-04-26T23:01:36.268 INFO:tasks.workunit.client.0.smithi027.stdout:2/638: unlink d2/d5/db/d1b/d1c/d25/d19/d23/fac 0 2024-04-26T23:01:36.269 INFO:tasks.workunit.client.0.smithi027.stdout:9/628: unlink d4/d10/f28 0 2024-04-26T23:01:36.271 INFO:tasks.workunit.client.0.smithi027.stdout:8/644: dread d9/da/d33/f41 [0,4194304] 0 2024-04-26T23:01:36.272 INFO:tasks.workunit.client.0.smithi027.stdout:1/760: mknod d1/d2a/cf8 0 2024-04-26T23:01:36.272 INFO:tasks.workunit.client.0.smithi027.stdout:1/761: readlink d1/l12 0 2024-04-26T23:01:36.273 INFO:tasks.workunit.client.0.smithi027.stdout:5/704: write d4/d24/d22/d29/d39/f7f [2841671,95896] 0 2024-04-26T23:01:36.274 INFO:tasks.workunit.client.0.smithi027.stdout:2/639: rename d2/d5/db/d1b/d1c/l76 to d2/d5/db/de/d21/lbe 0 2024-04-26T23:01:36.276 INFO:tasks.workunit.client.0.smithi027.stdout:6/599: rmdir d1/de 39 2024-04-26T23:01:36.279 INFO:tasks.workunit.client.0.smithi027.stdout:9/629: link d4/d10/c16 d4/d17/d2e/cdd 0 2024-04-26T23:01:36.279 INFO:tasks.workunit.client.0.smithi027.stdout:1/762: symlink d1/d26/d66/d73/lf9 0 2024-04-26T23:01:36.280 INFO:tasks.workunit.client.0.smithi027.stdout:8/645: mknod d9/da/d2a/d5e/d72/d84/d5d/d6c/ce7 0 2024-04-26T23:01:36.281 INFO:tasks.workunit.client.0.smithi027.stdout:2/640: read d2/d5/db/d1b/d1c/d25/d19/d44/fb1 [561601,9768] 0 2024-04-26T23:01:36.281 INFO:tasks.workunit.client.0.smithi027.stdout:8/646: read d9/da/d2a/f48 [728702,82093] 0 2024-04-26T23:01:36.281 INFO:tasks.workunit.client.0.smithi027.stdout:8/647: dread - d9/da/d2a/d5e/d92/fdf zero size 2024-04-26T23:01:36.283 INFO:tasks.workunit.client.0.smithi027.stdout:6/600: rmdir d1/df/d3c/d66/dc1 0 2024-04-26T23:01:36.285 INFO:tasks.workunit.client.0.smithi027.stdout:9/630: mknod d4/d17/d2e/d46/da0/d8d/cde 0 2024-04-26T23:01:36.286 INFO:tasks.workunit.client.0.smithi027.stdout:2/641: creat d2/d5/db/de/d21/d8f/fbf x:0 0 0 2024-04-26T23:01:36.286 INFO:tasks.workunit.client.0.smithi027.stdout:2/642: chown d2/d5/db/d1b/d1c/d25/d6a/l8e 1749 1 2024-04-26T23:01:36.286 INFO:tasks.workunit.client.0.smithi027.stdout:6/601: symlink d1/df/d49/lc3 0 2024-04-26T23:01:36.287 INFO:tasks.workunit.client.0.smithi027.stdout:6/602: fdatasync d1/df/d43/faa 0 2024-04-26T23:01:36.287 INFO:tasks.workunit.client.0.smithi027.stdout:9/631: mknod d4/d10/d40/d58/d6e/cdf 0 2024-04-26T23:01:36.287 INFO:tasks.workunit.client.0.smithi027.stdout:4/755: write da/dd/de/f94 [3477830,125203] 0 2024-04-26T23:01:36.287 INFO:tasks.workunit.client.0.smithi027.stdout:4/756: write da/d40/fc9 [716239,43313] 0 2024-04-26T23:01:36.287 INFO:tasks.workunit.client.0.smithi027.stdout:4/757: chown da/dd/d5f/dac/cc3 60457 1 2024-04-26T23:01:36.288 INFO:tasks.workunit.client.0.smithi027.stdout:6/603: truncate d1/df/f39 3193313 0 2024-04-26T23:01:36.289 INFO:tasks.workunit.client.0.smithi027.stdout:6/604: write d1/df/d43/d56/d6b/f76 [3795432,34980] 0 2024-04-26T23:01:36.289 INFO:tasks.workunit.client.0.smithi027.stdout:9/632: mknod d4/d10/d40/ce0 0 2024-04-26T23:01:36.293 INFO:tasks.workunit.client.0.smithi027.stdout:6/605: mkdir d1/df/d43/d56/d6b/dc4 0 2024-04-26T23:01:36.296 INFO:tasks.workunit.client.0.smithi027.stdout:6/606: rename d1/de/d57/c87 to d1/d6a/cc5 0 2024-04-26T23:01:36.296 INFO:tasks.workunit.client.0.smithi027.stdout:6/607: stat d1/df/f29 0 2024-04-26T23:01:36.310 INFO:tasks.workunit.client.0.smithi027.stdout:7/742: dwrite d0/f27 [0,4194304] 0 2024-04-26T23:01:36.310 INFO:tasks.workunit.client.0.smithi027.stdout:7/743: truncate d0/fac 54061 0 2024-04-26T23:01:36.310 INFO:tasks.workunit.client.0.smithi027.stdout:7/744: readlink d0/d28/d2b/d3a/d48/d56/d88/l58 0 2024-04-26T23:01:36.311 INFO:tasks.workunit.client.0.smithi027.stdout:7/745: rmdir d0 39 2024-04-26T23:01:36.313 INFO:tasks.workunit.client.0.smithi027.stdout:7/746: link d0/d28/d2b/d3a/fb1 d0/d28/d2b/d3a/d48/d56/db6/ff0 0 2024-04-26T23:01:36.336 INFO:tasks.workunit.client.0.smithi027.stdout:1/763: dread d1/d2a/f62 [0,4194304] 0 2024-04-26T23:01:36.343 INFO:tasks.workunit.client.0.smithi027.stdout:2/643: dwrite d2/d5/db/d1b/d1c/d25/f59 [4194304,4194304] 0 2024-04-26T23:01:36.344 INFO:tasks.workunit.client.0.smithi027.stdout:6/608: dread d1/de/f42 [0,4194304] 0 2024-04-26T23:01:36.345 INFO:tasks.workunit.client.0.smithi027.stdout:7/747: dread d0/d28/d2b/d3a/d48/d56/d88/f39 [0,4194304] 0 2024-04-26T23:01:36.345 INFO:tasks.workunit.client.0.smithi027.stdout:2/644: mkdir d2/d5/db/d1b/d1c/d25/dc0 0 2024-04-26T23:01:36.345 INFO:tasks.workunit.client.0.smithi027.stdout:2/645: readlink d2/d5/db/de/l2a 0 2024-04-26T23:01:36.346 INFO:tasks.workunit.client.0.smithi027.stdout:7/748: unlink d0/d28/d2b/d3a/f8b 0 2024-04-26T23:01:36.347 INFO:tasks.workunit.client.0.smithi027.stdout:2/646: rename d2/d5/db/d1b/d1c/d25/d19/d44/l81 to d2/d5/lc1 0 2024-04-26T23:01:36.349 INFO:tasks.workunit.client.0.smithi027.stdout:7/749: creat d0/d28/d2b/d3a/d48/d56/d88/d73/d80/ddd/ff1 x:0 0 0 2024-04-26T23:01:36.350 INFO:tasks.workunit.client.0.smithi027.stdout:5/705: dread d4/d24/d22/d29/d39/f6d [0,4194304] 0 2024-04-26T23:01:36.350 INFO:tasks.workunit.client.0.smithi027.stdout:3/767: dwrite d6/dd3/d34/db3/ff0 [0,4194304] 0 2024-04-26T23:01:36.350 INFO:tasks.workunit.client.0.smithi027.stdout:8/648: dread d9/da/d2a/d5e/d72/d84/db8/f5f [0,4194304] 0 2024-04-26T23:01:36.352 INFO:tasks.workunit.client.0.smithi027.stdout:7/750: symlink d0/d28/d2b/d3a/d48/d56/d8d/lf2 0 2024-04-26T23:01:36.353 INFO:tasks.workunit.client.0.smithi027.stdout:3/768: mkdir d6/dd3/d34/d2e/d10b 0 2024-04-26T23:01:36.353 INFO:tasks.workunit.client.0.smithi027.stdout:3/769: fdatasync d6/dd3/d34/d78/fbf 0 2024-04-26T23:01:36.353 INFO:tasks.workunit.client.0.smithi027.stdout:9/633: dwrite d4/d10/d67/fb3 [0,4194304] 0 2024-04-26T23:01:36.353 INFO:tasks.workunit.client.0.smithi027.stdout:7/751: mknod d0/d2f/cf3 0 2024-04-26T23:01:36.355 INFO:tasks.workunit.client.0.smithi027.stdout:9/634: creat d4/d10/d40/d58/d75/fe1 x:0 0 0 2024-04-26T23:01:36.355 INFO:tasks.workunit.client.0.smithi027.stdout:9/635: stat d4/d17/d2e/d46/da0/fc0 0 2024-04-26T23:01:36.355 INFO:tasks.workunit.client.0.smithi027.stdout:9/636: chown d4/d10/d40/d58/d75/f78 212062839 1 2024-04-26T23:01:36.355 INFO:tasks.workunit.client.0.smithi027.stdout:9/637: write d4/d17/d2e/d46/da0/f57 [4628496,19672] 0 2024-04-26T23:01:36.361 INFO:tasks.workunit.client.0.smithi027.stdout:8/649: rename d9/da/d2a/d5e/ce1 to d9/da/d2a/d5e/d2b/d52/ce8 0 2024-04-26T23:01:36.361 INFO:tasks.workunit.client.0.smithi027.stdout:8/650: getdents d9/da/d2a/d5e/d72/d84/d35/dc2/dc7 0 2024-04-26T23:01:36.361 INFO:tasks.workunit.client.0.smithi027.stdout:8/651: truncate d9/da/d2a/d5e/d72/f8a 932110 0 2024-04-26T23:01:36.361 INFO:tasks.workunit.client.0.smithi027.stdout:7/752: fsync d0/d2f/f84 0 2024-04-26T23:01:36.361 INFO:tasks.workunit.client.0.smithi027.stdout:7/753: write d0/d28/d2b/d3a/d48/d56/d88/d73/dd6/fea [188734,33079] 0 2024-04-26T23:01:36.361 INFO:tasks.workunit.client.0.smithi027.stdout:7/754: chown d0/fac 15815779 1 2024-04-26T23:01:36.364 INFO:tasks.workunit.client.0.smithi027.stdout:7/755: mknod d0/d28/d2b/d55/d69/d75/cf4 0 2024-04-26T23:01:36.364 INFO:tasks.workunit.client.0.smithi027.stdout:1/764: dread d1/d2a/f49 [0,4194304] 0 2024-04-26T23:01:36.368 INFO:tasks.workunit.client.0.smithi027.stdout:7/756: mkdir d0/d28/d2b/d3a/d48/d56/d8d/dc5/df5 0 2024-04-26T23:01:36.368 INFO:tasks.workunit.client.0.smithi027.stdout:1/765: write d1/dc/fb1 [46589,80561] 0 2024-04-26T23:01:36.372 INFO:tasks.workunit.client.0.smithi027.stdout:1/766: creat d1/d26/dc8/ffa x:0 0 0 2024-04-26T23:01:36.373 INFO:tasks.workunit.client.0.smithi027.stdout:1/767: creat d1/d26/d36/ffb x:0 0 0 2024-04-26T23:01:36.373 INFO:tasks.workunit.client.0.smithi027.stdout:1/768: truncate d1/d26/d66/dab/de0/f22 8740457 0 2024-04-26T23:01:36.374 INFO:tasks.workunit.client.0.smithi027.stdout:3/770: write d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d40/f4d [637949,30243] 0 2024-04-26T23:01:36.376 INFO:tasks.workunit.client.0.smithi027.stdout:8/652: write d9/da/d2a/d5e/d72/d84/f29 [1009966,108865] 0 2024-04-26T23:01:36.376 INFO:tasks.workunit.client.0.smithi027.stdout:8/653: fsync d9/da/d2a/d5e/d9e/fb5 0 2024-04-26T23:01:36.377 INFO:tasks.workunit.client.0.smithi027.stdout:1/769: getdents d1/d26/d36/d88/dca 0 2024-04-26T23:01:36.377 INFO:tasks.workunit.client.0.smithi027.stdout:1/770: truncate d1/d2a/ff7 628988 0 2024-04-26T23:01:36.379 INFO:tasks.workunit.client.0.smithi027.stdout:8/654: creat d9/da/d2a/d5e/d46/fe9 x:0 0 0 2024-04-26T23:01:36.387 INFO:tasks.workunit.client.0.smithi027.stdout:8/655: unlink d9/da/d2a/d6b/f9c 0 2024-04-26T23:01:36.387 INFO:tasks.workunit.client.0.smithi027.stdout:7/757: dread d0/f13 [0,4194304] 0 2024-04-26T23:01:36.387 INFO:tasks.workunit.client.0.smithi027.stdout:6/609: dread d1/f2 [0,4194304] 0 2024-04-26T23:01:36.387 INFO:tasks.workunit.client.0.smithi027.stdout:6/610: chown d1/df/d43/da8 2 1 2024-04-26T23:01:36.394 INFO:tasks.workunit.client.0.smithi027.stdout:1/771: read d1/d2a/ff7 [24444,38501] 0 2024-04-26T23:01:36.400 INFO:tasks.workunit.client.0.smithi027.stdout:5/706: dwrite d4/d24/d22/d29/f33 [0,4194304] 0 2024-04-26T23:01:36.400 INFO:tasks.workunit.client.0.smithi027.stdout:6/611: getdents d1/df/d43/d56/d6b 0 2024-04-26T23:01:36.400 INFO:tasks.workunit.client.0.smithi027.stdout:5/707: chown d4/d24/d22/cdf 276 1 2024-04-26T23:01:36.401 INFO:tasks.workunit.client.0.smithi027.stdout:1/772: rename d1/d26/d86/f4c to d1/d2a/deb/ffc 0 2024-04-26T23:01:36.402 INFO:tasks.workunit.client.0.smithi027.stdout:6/612: creat d1/fc6 x:0 0 0 2024-04-26T23:01:36.403 INFO:tasks.workunit.client.0.smithi027.stdout:5/708: rename d4/d1a/d4a/l8d to d4/d1a/dcb/le4 0 2024-04-26T23:01:36.404 INFO:tasks.workunit.client.0.smithi027.stdout:1/773: creat d1/d5/d40/ffd x:0 0 0 2024-04-26T23:01:36.406 INFO:tasks.workunit.client.0.smithi027.stdout:2/647: dwrite d2/d5/db/de/f30 [8388608,4194304] 0 2024-04-26T23:01:36.407 INFO:tasks.workunit.client.0.smithi027.stdout:2/648: write d2/d5/db/de/d21/f9d [285678,66969] 0 2024-04-26T23:01:36.414 INFO:tasks.workunit.client.0.smithi027.stdout:6/613: read d1/df/d3c/f75 [1040264,31574] 0 2024-04-26T23:01:36.415 INFO:tasks.workunit.client.0.smithi027.stdout:5/709: creat d4/d24/d22/d29/d39/fe5 x:0 0 0 2024-04-26T23:01:36.415 INFO:tasks.workunit.client.0.smithi027.stdout:5/710: write d4/d24/d22/d29/d2a/d3b/d50/fbb [238974,123843] 0 2024-04-26T23:01:36.421 INFO:tasks.workunit.client.0.smithi027.stdout:5/711: rename d4/d1a/dcb/le4 to d4/d9f/dc5/le6 0 2024-04-26T23:01:36.421 INFO:tasks.workunit.client.0.smithi027.stdout:5/712: stat d4/d24/d55/d63 0 2024-04-26T23:01:36.423 INFO:tasks.workunit.client.0.smithi027.stdout:5/713: truncate d4/fa5 384118 0 2024-04-26T23:01:36.423 INFO:tasks.workunit.client.0.smithi027.stdout:5/714: fdatasync d4/d24/d55/d63/fdd 0 2024-04-26T23:01:36.433 INFO:tasks.workunit.client.0.smithi027.stdout:3/771: dwrite d6/dd3/d34/d2e/f22 [0,4194304] 0 2024-04-26T23:01:36.433 INFO:tasks.workunit.client.0.smithi027.stdout:3/772: chown d6/d5e/cd9 811387 1 2024-04-26T23:01:36.434 INFO:tasks.workunit.client.0.smithi027.stdout:3/773: fdatasync d6/dd3/d2f/f39 0 2024-04-26T23:01:36.440 INFO:tasks.workunit.client.0.smithi027.stdout:1/774: dwrite d1/d5/f46 [0,4194304] 0 2024-04-26T23:01:36.443 INFO:tasks.workunit.client.0.smithi027.stdout:1/775: mkdir d1/d26/d66/dab/de0/dfe 0 2024-04-26T23:01:36.443 INFO:tasks.workunit.client.0.smithi027.stdout:1/776: chown d1/d5/c23 20704111 1 2024-04-26T23:01:36.444 INFO:tasks.workunit.client.0.smithi027.stdout:1/777: creat d1/d26/d6b/d42/d79/fff x:0 0 0 2024-04-26T23:01:36.452 INFO:tasks.workunit.client.0.smithi027.stdout:1/778: dread d1/dc/fb1 [0,4194304] 0 2024-04-26T23:01:36.453 INFO:tasks.workunit.client.0.smithi027.stdout:1/779: truncate d1/d26/d7d/f30 2425443 0 2024-04-26T23:01:36.455 INFO:tasks.workunit.client.0.smithi027.stdout:1/780: creat d1/d26/d6b/d8e/dc5/f100 x:0 0 0 2024-04-26T23:01:36.457 INFO:tasks.workunit.client.0.smithi027.stdout:1/781: link d1/d2a/cf8 d1/d26/d6b/d8e/dc5/c101 0 2024-04-26T23:01:36.459 INFO:tasks.workunit.client.0.smithi027.stdout:1/782: mkdir d1/d5/d40/d102 0 2024-04-26T23:01:36.459 INFO:tasks.workunit.client.0.smithi027.stdout:1/783: fsync d1/d2a/deb/ffc 0 2024-04-26T23:01:36.459 INFO:tasks.workunit.client.0.smithi027.stdout:8/656: dwrite d9/de/d2e/f38 [0,4194304] 0 2024-04-26T23:01:36.469 INFO:tasks.workunit.client.0.smithi027.stdout:5/715: dwrite d4/d24/d22/d29/d39/f6d [0,4194304] 0 2024-04-26T23:01:36.469 INFO:tasks.workunit.client.0.smithi027.stdout:5/716: readlink d4/d24/d5f/dac/d40/l4d 0 2024-04-26T23:01:36.469 INFO:tasks.workunit.client.0.smithi027.stdout:5/717: write d4/d24/d5f/dac/d40/fc8 [145939,86670] 0 2024-04-26T23:01:36.471 INFO:tasks.workunit.client.0.smithi027.stdout:5/718: symlink d4/d24/d22/d29/d39/d61/le7 0 2024-04-26T23:01:36.471 INFO:tasks.workunit.client.0.smithi027.stdout:5/719: stat d4/d24/d22/d29/d39/d84 0 2024-04-26T23:01:36.471 INFO:tasks.workunit.client.0.smithi027.stdout:5/720: write d4/fc1 [820044,64484] 0 2024-04-26T23:01:36.473 INFO:tasks.workunit.client.0.smithi027.stdout:5/721: mknod d4/d24/d55/d63/ce8 0 2024-04-26T23:01:36.476 INFO:tasks.workunit.client.0.smithi027.stdout:8/657: mkdir d9/da/d2a/d5e/d72/d84/d35/dea 0 2024-04-26T23:01:36.476 INFO:tasks.workunit.client.0.smithi027.stdout:8/658: dread - d9/da/d2a/d5e/d46/fe9 zero size 2024-04-26T23:01:36.479 INFO:tasks.workunit.client.0.smithi027.stdout:8/659: creat d9/da/d2a/d5e/d46/feb x:0 0 0 2024-04-26T23:01:36.481 INFO:tasks.workunit.client.0.smithi027.stdout:8/660: unlink d9/da/d2a/d5e/d72/d84/db8/f4c 0 2024-04-26T23:01:36.488 INFO:tasks.workunit.client.0.smithi027.stdout:7/758: dwrite d0/f13 [0,4194304] 0 2024-04-26T23:01:36.489 INFO:tasks.workunit.client.0.smithi027.stdout:6/614: dread d1/f4e [0,4194304] 0 2024-04-26T23:01:36.489 INFO:tasks.workunit.client.0.smithi027.stdout:6/615: write d1/df/d43/da8/da1/fbe [806539,58352] 0 2024-04-26T23:01:36.494 INFO:tasks.workunit.client.0.smithi027.stdout:7/759: getdents d0/d28/d9f 0 2024-04-26T23:01:36.494 INFO:tasks.workunit.client.0.smithi027.stdout:7/760: chown d0/d28/d2b/d55/f7d 1 1 2024-04-26T23:01:36.495 INFO:tasks.workunit.client.0.smithi027.stdout:6/616: getdents d1/df/d43/da8/d84 0 2024-04-26T23:01:36.497 INFO:tasks.workunit.client.0.smithi027.stdout:7/761: mkdir d0/d3d/df6 0 2024-04-26T23:01:36.497 INFO:tasks.workunit.client.0.smithi027.stdout:7/762: write d0/d28/d2b/d3a/fb1 [788885,14788] 0 2024-04-26T23:01:36.498 INFO:tasks.workunit.client.0.smithi027.stdout:6/617: creat d1/de/d57/d98/fc7 x:0 0 0 2024-04-26T23:01:36.500 INFO:tasks.workunit.client.0.smithi027.stdout:5/722: dread d4/d1a/d2b/f7b [0,4194304] 0 2024-04-26T23:01:36.502 INFO:tasks.workunit.client.0.smithi027.stdout:5/723: rmdir d4/d24/d22/d29/d39/dc7 39 2024-04-26T23:01:36.502 INFO:tasks.workunit.client.0.smithi027.stdout:3/774: dwrite d6/fbc [0,4194304] 0 2024-04-26T23:01:36.504 INFO:tasks.workunit.client.0.smithi027.stdout:5/724: mkdir d4/d1a/d2b/dd2/de9 0 2024-04-26T23:01:36.504 INFO:tasks.workunit.client.0.smithi027.stdout:5/725: write d4/d24/d22/dab/d16/f9b [650905,13258] 0 2024-04-26T23:01:36.504 INFO:tasks.workunit.client.0.smithi027.stdout:1/784: dwrite d1/d26/d6b/d42/d70/fc7 [0,4194304] 0 2024-04-26T23:01:36.504 INFO:tasks.workunit.client.0.smithi027.stdout:1/785: chown d1/d26/d86/l8f 27 1 2024-04-26T23:01:36.505 INFO:tasks.workunit.client.0.smithi027.stdout:1/786: fsync d1/d26/d6b/d42/d79/fcd 0 2024-04-26T23:01:36.506 INFO:tasks.workunit.client.0.smithi027.stdout:5/726: read d4/fa5 [158331,75677] 0 2024-04-26T23:01:36.506 INFO:tasks.workunit.client.0.smithi027.stdout:5/727: chown d4/d24/f4c 8 1 2024-04-26T23:01:36.506 INFO:tasks.workunit.client.0.smithi027.stdout:5/728: read d4/d24/f5d [1569580,89082] 0 2024-04-26T23:01:36.507 INFO:tasks.workunit.client.0.smithi027.stdout:1/787: creat d1/d26/d66/f103 x:0 0 0 2024-04-26T23:01:36.507 INFO:tasks.workunit.client.0.smithi027.stdout:7/763: dread d0/fac [0,4194304] 0 2024-04-26T23:01:36.510 INFO:tasks.workunit.client.0.smithi027.stdout:1/788: creat d1/d26/d86/d5c/f104 x:0 0 0 2024-04-26T23:01:36.511 INFO:tasks.workunit.client.0.smithi027.stdout:7/764: getdents d0/d28/d2b/d3a/d48/d56/d8d/dc5 0 2024-04-26T23:01:36.511 INFO:tasks.workunit.client.0.smithi027.stdout:7/765: creat d0/d3d/ff7 x:0 0 0 2024-04-26T23:01:36.511 INFO:tasks.workunit.client.0.smithi027.stdout:7/766: fsync d0/f11 0 2024-04-26T23:01:36.519 INFO:tasks.workunit.client.0.smithi027.stdout:7/767: rename d0/d28/d66/d67/da6 to d0/d3d/dba/df8 0 2024-04-26T23:01:36.522 INFO:tasks.workunit.client.0.smithi027.stdout:9/638: sync 2024-04-26T23:01:36.523 INFO:tasks.workunit.client.0.smithi027.stdout:0/663: sync 2024-04-26T23:01:36.524 INFO:tasks.workunit.client.0.smithi027.stdout:4/758: sync 2024-04-26T23:01:36.527 INFO:tasks.workunit.client.0.smithi027.stdout:7/768: mkdir d0/d28/d2b/d55/df9 0 2024-04-26T23:01:36.527 INFO:tasks.workunit.client.0.smithi027.stdout:7/769: getdents d0/d28/d2b/d55/d69/db4 0 2024-04-26T23:01:36.530 INFO:tasks.workunit.client.0.smithi027.stdout:0/664: truncate d0/d4/da/d15/d26/f7c 6235040 0 2024-04-26T23:01:36.530 INFO:tasks.workunit.client.0.smithi027.stdout:0/665: chown d0/d4/da/d1a/d6d/fa7 3 1 2024-04-26T23:01:36.531 INFO:tasks.workunit.client.0.smithi027.stdout:0/666: mkdir d0/d4/da/d1a/d6d/dc9 0 2024-04-26T23:01:36.532 INFO:tasks.workunit.client.0.smithi027.stdout:9/639: getdents d4/d10/d40/d58 0 2024-04-26T23:01:36.532 INFO:tasks.workunit.client.0.smithi027.stdout:9/640: write d4/d17/d2e/d46/da0/fb1 [420768,3369] 0 2024-04-26T23:01:36.532 INFO:tasks.workunit.client.0.smithi027.stdout:4/759: truncate da/dd/de/d14/f44 69617 0 2024-04-26T23:01:36.533 INFO:tasks.workunit.client.0.smithi027.stdout:0/667: getdents d0/d4/da/d1a/d59 0 2024-04-26T23:01:36.534 INFO:tasks.workunit.client.0.smithi027.stdout:9/641: link d4/d10/d40/d58/d75/f63 d4/d77/da2/daf/fe2 0 2024-04-26T23:01:36.539 INFO:tasks.workunit.client.0.smithi027.stdout:8/661: dwrite d9/da/d2a/d5e/d46/f6f [0,4194304] 0 2024-04-26T23:01:36.539 INFO:tasks.workunit.client.0.smithi027.stdout:9/642: symlink d4/d10/d67/le3 0 2024-04-26T23:01:36.544 INFO:tasks.workunit.client.0.smithi027.stdout:8/662: creat d9/da/d2a/d5e/d46/ddd/fec x:0 0 0 2024-04-26T23:01:36.581 INFO:tasks.workunit.client.0.smithi027.stdout:9/643: dread d4/d10/f7e [0,4194304] 0 2024-04-26T23:01:36.584 INFO:tasks.workunit.client.0.smithi027.stdout:9/644: mkdir d4/d10/d40/d58/d75/d84/dc4/de4 0 2024-04-26T23:01:36.584 INFO:tasks.workunit.client.0.smithi027.stdout:9/645: dread - d4/d17/d2e/d46/da0/d8d/fdc zero size 2024-04-26T23:01:36.585 INFO:tasks.workunit.client.0.smithi027.stdout:9/646: creat d4/d17/d2e/d70/fe5 x:0 0 0 2024-04-26T23:01:36.587 INFO:tasks.workunit.client.0.smithi027.stdout:4/760: dwrite da/d40/f60 [0,4194304] 0 2024-04-26T23:01:36.616 INFO:tasks.workunit.client.0.smithi027.stdout:0/668: write d0/d4/da/d15/f5a [4832288,69216] 0 2024-04-26T23:01:36.616 INFO:tasks.workunit.client.0.smithi027.stdout:0/669: write d0/d4/da/d1a/f3b [5887033,50485] 0 2024-04-26T23:01:36.624 INFO:tasks.workunit.client.0.smithi027.stdout:6/618: dwrite d1/df/d43/f5d [0,4194304] 0 2024-04-26T23:01:36.628 INFO:tasks.workunit.client.0.smithi027.stdout:6/619: getdents d1/d30 0 2024-04-26T23:01:36.630 INFO:tasks.workunit.client.0.smithi027.stdout:1/789: dwrite d1/d26/d86/d5c/fd4 [0,4194304] 0 2024-04-26T23:01:36.633 INFO:tasks.workunit.client.0.smithi027.stdout:1/790: mkdir d1/d26/d66/d73/db5/d105 0 2024-04-26T23:01:36.633 INFO:tasks.workunit.client.0.smithi027.stdout:1/791: fsync d1/d26/d66/dab/de0/fd1 0 2024-04-26T23:01:36.636 INFO:tasks.workunit.client.0.smithi027.stdout:1/792: symlink d1/d2a/deb/l106 0 2024-04-26T23:01:36.636 INFO:tasks.workunit.client.0.smithi027.stdout:1/793: chown d1/d26/d66/dab/de0/f44 4719 1 2024-04-26T23:01:36.637 INFO:tasks.workunit.client.0.smithi027.stdout:1/794: stat d1/d26/d66/dab/de0/l6f 0 2024-04-26T23:01:36.643 INFO:tasks.workunit.client.0.smithi027.stdout:3/775: dwrite d6/dd3/d2f/fe2 [0,4194304] 0 2024-04-26T23:01:36.649 INFO:tasks.workunit.client.0.smithi027.stdout:9/647: dwrite d4/fcd [0,4194304] 0 2024-04-26T23:01:36.649 INFO:tasks.workunit.client.0.smithi027.stdout:9/648: truncate d4/d17/d2e/d46/da0/d72/fd0 185051 0 2024-04-26T23:01:36.653 INFO:tasks.workunit.client.0.smithi027.stdout:4/761: dwrite da/dd/de/d14/d17/d80/dab/dd6/fdd [0,4194304] 0 2024-04-26T23:01:36.657 INFO:tasks.workunit.client.0.smithi027.stdout:9/649: mknod d4/d10/d40/d58/ce6 0 2024-04-26T23:01:36.666 INFO:tasks.workunit.client.0.smithi027.stdout:4/762: creat da/dd/de/d14/d21/d41/d59/fe9 x:0 0 0 2024-04-26T23:01:36.668 INFO:tasks.workunit.client.0.smithi027.stdout:6/620: dread d1/df/d49/f4f [0,4194304] 0 2024-04-26T23:01:36.668 INFO:tasks.workunit.client.0.smithi027.stdout:9/650: rename d4/d10/d40/d58/d6e/l73 to d4/d17/d2e/dcf/le7 0 2024-04-26T23:01:36.669 INFO:tasks.workunit.client.0.smithi027.stdout:4/763: mknod da/d40/cea 0 2024-04-26T23:01:36.672 INFO:tasks.workunit.client.0.smithi027.stdout:6/621: mknod d1/df/d43/da8/d84/cc8 0 2024-04-26T23:01:36.673 INFO:tasks.workunit.client.0.smithi027.stdout:9/651: creat d4/d10/d40/d58/d75/d84/d86/dc5/fe8 x:0 0 0 2024-04-26T23:01:36.673 INFO:tasks.workunit.client.0.smithi027.stdout:6/622: write d1/df/f9b [245589,81255] 0 2024-04-26T23:01:36.673 INFO:tasks.workunit.client.0.smithi027.stdout:3/776: dread d6/f82 [0,4194304] 0 2024-04-26T23:01:36.673 INFO:tasks.workunit.client.0.smithi027.stdout:3/777: chown d6/dd3/d34/d2e/d9d/da7/dac/dca/ld2 0 1 2024-04-26T23:01:36.675 INFO:tasks.workunit.client.0.smithi027.stdout:3/778: creat d6/dd3/f10c x:0 0 0 2024-04-26T23:01:36.675 INFO:tasks.workunit.client.0.smithi027.stdout:4/764: creat da/dd/de/d14/d17/d80/dab/dd6/feb x:0 0 0 2024-04-26T23:01:36.675 INFO:tasks.workunit.client.0.smithi027.stdout:4/765: write da/dd/de/d14/d17/f81 [162939,8433] 0 2024-04-26T23:01:36.675 INFO:tasks.workunit.client.0.smithi027.stdout:9/652: rmdir d4/d10/d25 39 2024-04-26T23:01:36.675 INFO:tasks.workunit.client.0.smithi027.stdout:4/766: fdatasync da/dd/de/f94 0 2024-04-26T23:01:36.676 INFO:tasks.workunit.client.0.smithi027.stdout:3/779: creat d6/dd3/d34/d2e/d9d/da3/f10d x:0 0 0 2024-04-26T23:01:36.676 INFO:tasks.workunit.client.0.smithi027.stdout:3/780: write d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/f33 [944349,79553] 0 2024-04-26T23:01:36.676 INFO:tasks.workunit.client.0.smithi027.stdout:3/781: read d6/dd3/f25 [3516334,59985] 0 2024-04-26T23:01:36.676 INFO:tasks.workunit.client.0.smithi027.stdout:3/782: chown d6/dd3/d34/db3/fc1 287184 1 2024-04-26T23:01:36.677 INFO:tasks.workunit.client.0.smithi027.stdout:9/653: unlink d4/d17/d2e/d70/fc2 0 2024-04-26T23:01:36.678 INFO:tasks.workunit.client.0.smithi027.stdout:9/654: write d4/d17/d2e/d46/da0/fc0 [481174,59537] 0 2024-04-26T23:01:36.678 INFO:tasks.workunit.client.0.smithi027.stdout:3/783: mknod d6/dd3/d34/d2e/d5f/d90/ddb/c10e 0 2024-04-26T23:01:36.680 INFO:tasks.workunit.client.0.smithi027.stdout:3/784: mknod d6/dd3/d34/db3/df8/c10f 0 2024-04-26T23:01:36.680 INFO:tasks.workunit.client.0.smithi027.stdout:3/785: dread - d6/dd3/f10c zero size 2024-04-26T23:01:36.680 INFO:tasks.workunit.client.0.smithi027.stdout:3/786: fdatasync d6/dd3/f72 0 2024-04-26T23:01:36.680 INFO:tasks.workunit.client.0.smithi027.stdout:3/787: write d6/dd3/d34/d78/fbf [798738,68029] 0 2024-04-26T23:01:36.680 INFO:tasks.workunit.client.0.smithi027.stdout:1/795: dwrite d1/d26/d66/f77 [0,4194304] 0 2024-04-26T23:01:36.684 INFO:tasks.workunit.client.0.smithi027.stdout:4/767: dread da/dd/d5f/f8b [0,4194304] 0 2024-04-26T23:01:36.684 INFO:tasks.workunit.client.0.smithi027.stdout:4/768: truncate da/dd/de/d14/d21/d41/fa4 765427 0 2024-04-26T23:01:36.685 INFO:tasks.workunit.client.0.smithi027.stdout:1/796: getdents d1/d26/d66/d73/d9e/db6 0 2024-04-26T23:01:36.686 INFO:tasks.workunit.client.0.smithi027.stdout:3/788: symlink d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/l110 0 2024-04-26T23:01:36.686 INFO:tasks.workunit.client.0.smithi027.stdout:3/789: truncate d6/dd3/d34/d2e/d5f/d90/ddb/d98/fc7 126664 0 2024-04-26T23:01:36.686 INFO:tasks.workunit.client.0.smithi027.stdout:3/790: fsync d6/f82 0 2024-04-26T23:01:36.686 INFO:tasks.workunit.client.0.smithi027.stdout:1/797: symlink d1/d26/d66/d73/d9e/l107 0 2024-04-26T23:01:36.686 INFO:tasks.workunit.client.0.smithi027.stdout:1/798: chown d1/d26/d66/d73/d9e/db6 241 1 2024-04-26T23:01:36.689 INFO:tasks.workunit.client.0.smithi027.stdout:1/799: rename d1/d26/d66/cdb to d1/d26/d6b/d42/d79/c108 0 2024-04-26T23:01:36.691 INFO:tasks.workunit.client.0.smithi027.stdout:1/800: rename d1/d26/d6b/d42/d79/fcd to d1/d26/d66/d91/f109 0 2024-04-26T23:01:36.691 INFO:tasks.workunit.client.0.smithi027.stdout:1/801: chown d1/d5/c23 11577 1 2024-04-26T23:01:36.693 INFO:tasks.workunit.client.0.smithi027.stdout:1/802: rename d1/d26/d66/f74 to d1/d26/d6b/d42/d79/f10a 0 2024-04-26T23:01:36.700 INFO:tasks.workunit.client.0.smithi027.stdout:1/803: dread d1/d26/d86/f9c [0,4194304] 0 2024-04-26T23:01:36.701 INFO:tasks.workunit.client.0.smithi027.stdout:1/804: creat d1/d26/d6b/d8e/f10b x:0 0 0 2024-04-26T23:01:36.701 INFO:tasks.workunit.client.0.smithi027.stdout:1/805: truncate d1/d2a/fbc 1514447 0 2024-04-26T23:01:36.702 INFO:tasks.workunit.client.0.smithi027.stdout:1/806: mkdir d1/d26/d66/d10c 0 2024-04-26T23:01:36.702 INFO:tasks.workunit.client.0.smithi027.stdout:1/807: chown d1/d26/d66/d73/dae/cde 750801 1 2024-04-26T23:01:36.702 INFO:tasks.workunit.client.0.smithi027.stdout:1/808: fsync d1/d26/d36/d88/f99 0 2024-04-26T23:01:36.702 INFO:tasks.workunit.client.0.smithi027.stdout:1/809: truncate d1/d26/d6b/d42/d79/fff 491 0 2024-04-26T23:01:36.703 INFO:tasks.workunit.client.0.smithi027.stdout:1/810: read d1/d26/d6b/f5a [258293,125213] 0 2024-04-26T23:01:36.704 INFO:tasks.workunit.client.0.smithi027.stdout:1/811: symlink d1/d26/d36/d88/dca/l10d 0 2024-04-26T23:01:36.704 INFO:tasks.workunit.client.0.smithi027.stdout:1/812: stat d1/d26/d7d/f30 0 2024-04-26T23:01:36.705 INFO:tasks.workunit.client.0.smithi027.stdout:1/813: unlink d1/d5/f8b 0 2024-04-26T23:01:36.708 INFO:tasks.workunit.client.0.smithi027.stdout:3/791: dread d6/dd3/d34/d2e/f31 [0,4194304] 0 2024-04-26T23:01:36.715 INFO:tasks.workunit.client.0.smithi027.stdout:4/769: dwrite da/dd/de/d79/da8/fbf [0,4194304] 0 2024-04-26T23:01:36.716 INFO:tasks.workunit.client.0.smithi027.stdout:4/770: mkdir da/dd/d5e/de3/dec 0 2024-04-26T23:01:36.718 INFO:tasks.workunit.client.0.smithi027.stdout:4/771: mkdir da/dd/de/d14/d21/d67/d8a/ded 0 2024-04-26T23:01:36.719 INFO:tasks.workunit.client.0.smithi027.stdout:4/772: symlink da/dd/de/d14/d21/d67/d8a/ded/lee 0 2024-04-26T23:01:36.720 INFO:tasks.workunit.client.0.smithi027.stdout:4/773: truncate da/dd/de/d14/d21/f46 456649 0 2024-04-26T23:01:36.720 INFO:tasks.workunit.client.0.smithi027.stdout:4/774: read da/dd/de/d14/d17/f2a [379064,72149] 0 2024-04-26T23:01:36.747 INFO:tasks.workunit.client.0.smithi027.stdout:1/814: write d1/d26/d6b/d42/d79/fd9 [3972440,37834] 0 2024-04-26T23:01:36.747 INFO:tasks.workunit.client.0.smithi027.stdout:1/815: dread - d1/d26/d6b/d8e/dc5/f100 zero size 2024-04-26T23:01:36.748 INFO:tasks.workunit.client.0.smithi027.stdout:4/775: dwrite da/dd/de/d14/d21/d67/d8a/f8c [0,4194304] 0 2024-04-26T23:01:36.756 INFO:tasks.workunit.client.0.smithi027.stdout:1/816: getdents d1/d26/d36/d88/dca 0 2024-04-26T23:01:36.757 INFO:tasks.workunit.client.0.smithi027.stdout:3/792: dwrite d6/dd3/d34/d2e/d9d/da7/fbe [0,4194304] 0 2024-04-26T23:01:36.760 INFO:tasks.workunit.client.0.smithi027.stdout:3/793: mknod d6/dd3/d34/d9b/c111 0 2024-04-26T23:01:36.771 INFO:tasks.workunit.client.0.smithi027.stdout:4/776: write da/dd/d5f/fb6 [2984803,66145] 0 2024-04-26T23:01:36.774 INFO:tasks.workunit.client.0.smithi027.stdout:2/649: sync 2024-04-26T23:01:36.775 INFO:tasks.workunit.client.0.smithi027.stdout:2/650: fsync d2/d5/db/d1b/d1c/fbc 0 2024-04-26T23:01:36.775 INFO:tasks.workunit.client.0.smithi027.stdout:1/817: dread d1/dc/f37 [0,4194304] 0 2024-04-26T23:01:36.775 INFO:tasks.workunit.client.0.smithi027.stdout:1/818: write d1/d26/d66/d73/d9e/db6/fec [147215,93817] 0 2024-04-26T23:01:36.777 INFO:tasks.workunit.client.0.smithi027.stdout:2/651: link d2/d5/db/d1b/d1c/d8b/fa9 d2/d5/db/de/d21/fc2 0 2024-04-26T23:01:36.778 INFO:tasks.workunit.client.0.smithi027.stdout:2/652: mkdir d2/d5/db/d1b/d1c/d25/dc3 0 2024-04-26T23:01:36.778 INFO:tasks.workunit.client.0.smithi027.stdout:2/653: dread - d2/d5/db/d1b/d1c/fbc zero size 2024-04-26T23:01:36.786 INFO:tasks.workunit.client.0.smithi027.stdout:2/654: rename d2/d5/db/d1b/d1c/d25/d19/d36 to d2/d5/db/de/d21/dc4 0 2024-04-26T23:01:36.787 INFO:tasks.workunit.client.0.smithi027.stdout:2/655: unlink d2/d5/db/d1b/d1c/d25/d19/cba 0 2024-04-26T23:01:36.791 INFO:tasks.workunit.client.0.smithi027.stdout:2/656: truncate d2/f49 547395 0 2024-04-26T23:01:36.791 INFO:tasks.workunit.client.0.smithi027.stdout:2/657: write d2/d5/db/d1b/d1c/d25/d19/d44/fb1 [79162,94233] 0 2024-04-26T23:01:36.792 INFO:tasks.workunit.client.0.smithi027.stdout:4/777: dread da/dd/de/d14/f53 [0,4194304] 0 2024-04-26T23:01:36.795 INFO:tasks.workunit.client.0.smithi027.stdout:4/778: mkdir da/dd/de/d79/dcd/def 0 2024-04-26T23:01:36.795 INFO:tasks.workunit.client.0.smithi027.stdout:2/658: truncate d2/f6f 2503968 0 2024-04-26T23:01:36.796 INFO:tasks.workunit.client.0.smithi027.stdout:1/819: dread d1/f1b [0,4194304] 0 2024-04-26T23:01:36.796 INFO:tasks.workunit.client.0.smithi027.stdout:1/820: chown d1/dc/fe2 66514 1 2024-04-26T23:01:36.796 INFO:tasks.workunit.client.0.smithi027.stdout:3/794: dwrite d6/dd3/d34/d2e/d50/fb2 [0,4194304] 0 2024-04-26T23:01:36.797 INFO:tasks.workunit.client.0.smithi027.stdout:3/795: chown d6/dd3/d34/f58 2 1 2024-04-26T23:01:36.798 INFO:tasks.workunit.client.0.smithi027.stdout:4/779: creat da/dd/de/d14/d21/d67/ff0 x:0 0 0 2024-04-26T23:01:36.798 INFO:tasks.workunit.client.0.smithi027.stdout:4/780: chown da/dd/d92/l98 1449 1 2024-04-26T23:01:36.798 INFO:tasks.workunit.client.0.smithi027.stdout:4/781: fsync da/dd/de/d14/d17/f45 0 2024-04-26T23:01:36.800 INFO:tasks.workunit.client.0.smithi027.stdout:2/659: creat d2/d5/db/d1b/d1c/d25/d19/d44/fc5 x:0 0 0 2024-04-26T23:01:36.800 INFO:tasks.workunit.client.0.smithi027.stdout:2/660: chown d2/d5/db/d1b/d1c/l78 935652 1 2024-04-26T23:01:36.800 INFO:tasks.workunit.client.0.smithi027.stdout:4/782: getdents da/dd/de/d14/d21/d67/d8a 0 2024-04-26T23:01:36.803 INFO:tasks.workunit.client.0.smithi027.stdout:3/796: rmdir d6/dd3/d34/d2e/d5f/def/dfc 0 2024-04-26T23:01:36.803 INFO:tasks.workunit.client.0.smithi027.stdout:3/797: stat d6/fbc 0 2024-04-26T23:01:36.803 INFO:tasks.workunit.client.0.smithi027.stdout:4/783: link da/dd/de/d14/d17/d80/dab/dd6/ce0 da/dd/de/d79/da8/dc5/cf1 0 2024-04-26T23:01:36.804 INFO:tasks.workunit.client.0.smithi027.stdout:4/784: symlink da/lf2 0 2024-04-26T23:01:36.806 INFO:tasks.workunit.client.0.smithi027.stdout:4/785: rename da/dd/de/d14/db3/le2 to da/dd/de/d14/db3/lf3 0 2024-04-26T23:01:36.822 INFO:tasks.workunit.client.0.smithi027.stdout:3/798: dread d6/dd3/d34/d2e/d70/f7d [0,4194304] 0 2024-04-26T23:01:36.826 INFO:tasks.workunit.client.0.smithi027.stdout:3/799: write d6/dd3/d34/d78/fbf [293537,116701] 0 2024-04-26T23:01:36.827 INFO:tasks.workunit.client.0.smithi027.stdout:3/800: write d6/f96 [522945,1239] 0 2024-04-26T23:01:36.827 INFO:tasks.workunit.client.0.smithi027.stdout:3/801: fdatasync d6/dd3/d34/d2e/d50/f8a 0 2024-04-26T23:01:36.827 INFO:tasks.workunit.client.0.smithi027.stdout:3/802: write d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/dd6/ff3 [39253,79051] 0 2024-04-26T23:01:36.827 INFO:tasks.workunit.client.0.smithi027.stdout:3/803: write d6/dd3/d34/db3/fdf [449586,67080] 0 2024-04-26T23:01:36.827 INFO:tasks.workunit.client.0.smithi027.stdout:3/804: fsync d6/dd3/d34/d2e/d70/d71/ff4 0 2024-04-26T23:01:36.857 INFO:tasks.workunit.client.0.smithi027.stdout:3/805: dwrite d6/dd3/d34/d78/da1/fa8 [0,4194304] 0 2024-04-26T23:01:36.859 INFO:tasks.workunit.client.0.smithi027.stdout:3/806: mknod d6/dd3/d34/db3/c112 0 2024-04-26T23:01:36.860 INFO:tasks.workunit.client.0.smithi027.stdout:3/807: mkdir d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/d113 0 2024-04-26T23:01:36.863 INFO:tasks.workunit.client.0.smithi027.stdout:1/821: dwrite d1/d2a/f49 [0,4194304] 0 2024-04-26T23:01:36.865 INFO:tasks.workunit.client.0.smithi027.stdout:1/822: rmdir d1/d26/d66/d73/db5 39 2024-04-26T23:01:36.867 INFO:tasks.workunit.client.0.smithi027.stdout:1/823: symlink d1/d26/d36/d56/df1/l10e 0 2024-04-26T23:01:36.878 INFO:tasks.workunit.client.0.smithi027.stdout:2/661: dwrite d2/d5/f34 [0,4194304] 0 2024-04-26T23:01:36.879 INFO:tasks.workunit.client.0.smithi027.stdout:2/662: mkdir d2/db9/dc6 0 2024-04-26T23:01:36.886 INFO:tasks.workunit.client.0.smithi027.stdout:2/663: rename d2/d5/db/d1b/d1c/fbc to d2/d5/db/d1b/d1c/d25/d6a/d8a/fc7 0 2024-04-26T23:01:36.888 INFO:tasks.workunit.client.0.smithi027.stdout:2/664: mknod d2/d5/db/de/d21/d8f/cc8 0 2024-04-26T23:01:36.888 INFO:tasks.workunit.client.0.smithi027.stdout:2/665: write d2/d5/db/d1b/d1c/d25/d19/f79 [109128,49483] 0 2024-04-26T23:01:36.888 INFO:tasks.workunit.client.0.smithi027.stdout:2/666: write d2/d5/db/d1b/d1c/d25/d6a/d8a/fc7 [324410,124885] 0 2024-04-26T23:01:36.890 INFO:tasks.workunit.client.0.smithi027.stdout:2/667: mknod d2/db9/cc9 0 2024-04-26T23:01:36.890 INFO:tasks.workunit.client.0.smithi027.stdout:2/668: write d2/d5/db/d1b/d1c/d25/d19/d23/fab [873901,67659] 0 2024-04-26T23:01:36.895 INFO:tasks.workunit.client.0.smithi027.stdout:1/824: dread d1/d26/d6b/d42/d79/fd9 [0,4194304] 0 2024-04-26T23:01:36.895 INFO:tasks.workunit.client.0.smithi027.stdout:1/825: write d1/dc/fe2 [502530,27772] 0 2024-04-26T23:01:36.895 INFO:tasks.workunit.client.0.smithi027.stdout:4/786: dwrite da/dd/fd8 [0,4194304] 0 2024-04-26T23:01:36.902 INFO:tasks.workunit.client.0.smithi027.stdout:4/787: creat da/dd/de/d14/d21/dad/ff4 x:0 0 0 2024-04-26T23:01:36.903 INFO:tasks.workunit.client.0.smithi027.stdout:1/826: dread d1/d26/d36/d88/f99 [0,4194304] 0 2024-04-26T23:01:36.905 INFO:tasks.workunit.client.0.smithi027.stdout:5/729: sync 2024-04-26T23:01:36.905 INFO:tasks.workunit.client.0.smithi027.stdout:5/730: write d4/d9f/dc0/fcc [1149345,55509] 0 2024-04-26T23:01:36.907 INFO:tasks.workunit.client.0.smithi027.stdout:5/731: symlink d4/d24/d5f/dac/lea 0 2024-04-26T23:01:36.908 INFO:tasks.workunit.client.0.smithi027.stdout:5/732: write d4/d24/d22/d29/d39/fe5 [421673,96669] 0 2024-04-26T23:01:36.908 INFO:tasks.workunit.client.0.smithi027.stdout:5/733: stat d4/d24/d22/dab/d16/d4b/d81 0 2024-04-26T23:01:36.909 INFO:tasks.workunit.client.0.smithi027.stdout:2/669: read d2/d5/db/de/d21/f51 [11349721,81815] 0 2024-04-26T23:01:36.909 INFO:tasks.workunit.client.0.smithi027.stdout:2/670: chown d2/d5/db/d1b/d1c/d25/d19/d23/fb2 1609894 1 2024-04-26T23:01:36.909 INFO:tasks.workunit.client.0.smithi027.stdout:2/671: readlink d2/d5/db/de/d21/dc4/l69 0 2024-04-26T23:01:36.910 INFO:tasks.workunit.client.0.smithi027.stdout:5/734: rmdir d4/d24/d22/d29/d39/d61 39 2024-04-26T23:01:36.911 INFO:tasks.workunit.client.0.smithi027.stdout:5/735: truncate d4/d24/d22/fa3 4069285 0 2024-04-26T23:01:36.913 INFO:tasks.workunit.client.0.smithi027.stdout:5/736: mknod d4/d1a/d2b/dd3/ceb 0 2024-04-26T23:01:36.926 INFO:tasks.workunit.client.0.smithi027.stdout:1/827: dread d1/d26/d6b/f5a [0,4194304] 0 2024-04-26T23:01:36.926 INFO:tasks.workunit.client.0.smithi027.stdout:4/788: dread da/dd/de/d14/d17/d80/dab/f3a [0,4194304] 0 2024-04-26T23:01:36.926 INFO:tasks.workunit.client.0.smithi027.stdout:4/789: fsync da/dd/de/d14/db3/fbd 0 2024-04-26T23:01:36.926 INFO:tasks.workunit.client.0.smithi027.stdout:4/790: chown da/d40/ld1 39022 1 2024-04-26T23:01:36.928 INFO:tasks.workunit.client.0.smithi027.stdout:1/828: symlink d1/d26/d36/d56/df1/l10f 0 2024-04-26T23:01:36.928 INFO:tasks.workunit.client.0.smithi027.stdout:1/829: fsync d1/d26/d66/d73/d9e/db6/fdc 0 2024-04-26T23:01:36.934 INFO:tasks.workunit.client.0.smithi027.stdout:4/791: mknod da/dd/de/d14/d21/d67/da6/cf5 0 2024-04-26T23:01:36.934 INFO:tasks.workunit.client.0.smithi027.stdout:1/830: symlink d1/d26/d36/l110 0 2024-04-26T23:01:36.935 INFO:tasks.workunit.client.0.smithi027.stdout:4/792: creat da/d40/ff6 x:0 0 0 2024-04-26T23:01:36.935 INFO:tasks.workunit.client.0.smithi027.stdout:4/793: fdatasync da/dd/de/d14/f43 0 2024-04-26T23:01:36.936 INFO:tasks.workunit.client.0.smithi027.stdout:4/794: mkdir da/dd/de/d14/d21/d41/df7 0 2024-04-26T23:01:36.937 INFO:tasks.workunit.client.0.smithi027.stdout:4/795: symlink da/dd/d5e/de3/lf8 0 2024-04-26T23:01:36.938 INFO:tasks.workunit.client.0.smithi027.stdout:4/796: creat da/dd/de/d14/d21/d41/df7/ff9 x:0 0 0 2024-04-26T23:01:36.939 INFO:tasks.workunit.client.0.smithi027.stdout:4/797: mknod da/dd/de/d14/cfa 0 2024-04-26T23:01:36.939 INFO:tasks.workunit.client.0.smithi027.stdout:4/798: fdatasync da/dd/de/d14/d17/d80/dab/f3e 0 2024-04-26T23:01:36.939 INFO:tasks.workunit.client.0.smithi027.stdout:4/799: read da/dd/f56 [25987,105407] 0 2024-04-26T23:01:36.939 INFO:tasks.workunit.client.0.smithi027.stdout:4/800: fdatasync da/dd/de/d14/d21/d41/d59/d74/f7e 0 2024-04-26T23:01:36.939 INFO:tasks.workunit.client.0.smithi027.stdout:4/801: write da/dd/de/d14/d21/d41/d59/f78 [1440969,54450] 0 2024-04-26T23:01:36.944 INFO:tasks.workunit.client.0.smithi027.stdout:1/831: dread d1/d26/d66/dab/fbb [0,4194304] 0 2024-04-26T23:01:36.946 INFO:tasks.workunit.client.0.smithi027.stdout:1/832: rename d1/d26/d86/l4f to d1/d26/d6b/d8e/dc5/l111 0 2024-04-26T23:01:36.953 INFO:tasks.workunit.client.0.smithi027.stdout:5/737: dwrite d4/d24/f42 [4194304,4194304] 0 2024-04-26T23:01:36.977 INFO:tasks.workunit.client.0.smithi027.stdout:5/738: dread d4/d24/d5f/dac/d40/f6c [4194304,4194304] 0 2024-04-26T23:01:36.978 INFO:tasks.workunit.client.0.smithi027.stdout:5/739: stat d4/d24/d22/d29/d2a/d3b/d50/l71 0 2024-04-26T23:01:36.999 INFO:tasks.workunit.client.0.smithi027.stdout:1/833: dwrite d1/d26/dc8/ff5 [0,4194304] 0 2024-04-26T23:01:37.001 INFO:tasks.workunit.client.0.smithi027.stdout:1/834: unlink d1/d2a/f62 0 2024-04-26T23:01:37.002 INFO:tasks.workunit.client.0.smithi027.stdout:1/835: read - d1/d26/d6b/d42/d70/fce zero size 2024-04-26T23:01:37.003 INFO:tasks.workunit.client.0.smithi027.stdout:1/836: mknod d1/d5/d40/d102/c112 0 2024-04-26T23:01:37.004 INFO:tasks.workunit.client.0.smithi027.stdout:1/837: chown d1/d26/d66/d91/fed 0 1 2024-04-26T23:01:37.004 INFO:tasks.workunit.client.0.smithi027.stdout:1/838: chown d1/d26/d6b/d8e/lb7 1 1 2024-04-26T23:01:37.007 INFO:tasks.workunit.client.0.smithi027.stdout:4/802: dwrite da/dd/de/d14/d21/f46 [0,4194304] 0 2024-04-26T23:01:37.007 INFO:tasks.workunit.client.0.smithi027.stdout:4/803: write da/dd/de/d14/d17/f33 [2558333,65572] 0 2024-04-26T23:01:37.009 INFO:tasks.workunit.client.0.smithi027.stdout:4/804: creat da/dd/de/d14/d21/dad/ffb x:0 0 0 2024-04-26T23:01:37.019 INFO:tasks.workunit.client.0.smithi027.stdout:5/740: dwrite d4/d24/d22/d29/d2a/d3b/fa2 [0,4194304] 0 2024-04-26T23:01:37.026 INFO:tasks.workunit.client.0.smithi027.stdout:5/741: rename d4/d1a/d2b/dd3/le2 to d4/d1a/dcb/lec 0 2024-04-26T23:01:37.028 INFO:tasks.workunit.client.0.smithi027.stdout:5/742: symlink d4/d1a/d2b/db8/led 0 2024-04-26T23:01:37.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/839: write d1/d5/f46 [4181716,10465] 0 2024-04-26T23:01:37.030 INFO:tasks.workunit.client.0.smithi027.stdout:5/743: mkdir d4/d24/d22/d29/d2a/dee 0 2024-04-26T23:01:37.030 INFO:tasks.workunit.client.0.smithi027.stdout:5/744: stat d4/d24/d22/d29/d2a/d3b/f3d 0 2024-04-26T23:01:37.030 INFO:tasks.workunit.client.0.smithi027.stdout:1/840: dread - d1/d26/d6b/fa1 zero size 2024-04-26T23:01:37.031 INFO:tasks.workunit.client.0.smithi027.stdout:1/841: mknod d1/d26/d36/d88/dca/c113 0 2024-04-26T23:01:37.031 INFO:tasks.workunit.client.0.smithi027.stdout:1/842: fsync d1/dc/f37 0 2024-04-26T23:01:37.032 INFO:tasks.workunit.client.0.smithi027.stdout:1/843: write d1/d26/d86/d5c/f104 [937461,89778] 0 2024-04-26T23:01:37.032 INFO:tasks.workunit.client.0.smithi027.stdout:1/844: fsync d1/d5/d40/ffd 0 2024-04-26T23:01:37.032 INFO:tasks.workunit.client.0.smithi027.stdout:1/845: write d1/d26/d66/d73/d9e/fba [5066130,110970] 0 2024-04-26T23:01:37.033 INFO:tasks.workunit.client.0.smithi027.stdout:1/846: creat d1/d26/d66/d73/db5/f114 x:0 0 0 2024-04-26T23:01:37.033 INFO:tasks.workunit.client.0.smithi027.stdout:1/847: write d1/d26/d66/d73/d9e/fba [5658016,55460] 0 2024-04-26T23:01:37.033 INFO:tasks.workunit.client.0.smithi027.stdout:1/848: chown d1/d5/l1a 822162 1 2024-04-26T23:01:37.035 INFO:tasks.workunit.client.0.smithi027.stdout:1/849: creat d1/d26/d66/d73/dae/dd0/f115 x:0 0 0 2024-04-26T23:01:37.035 INFO:tasks.workunit.client.0.smithi027.stdout:1/850: mknod d1/d26/d66/d91/c116 0 2024-04-26T23:01:37.037 INFO:tasks.workunit.client.0.smithi027.stdout:5/745: dread d4/d24/fb9 [0,4194304] 0 2024-04-26T23:01:37.052 INFO:tasks.workunit.client.0.smithi027.stdout:5/746: dread d4/d24/d22/dab/d16/f67 [0,4194304] 0 2024-04-26T23:01:37.052 INFO:tasks.workunit.client.0.smithi027.stdout:5/747: chown d4/d24/d22/d29/d2a/d3b/fa2 0 1 2024-04-26T23:01:37.052 INFO:tasks.workunit.client.0.smithi027.stdout:5/748: readlink d4/d24/d22/d29/d39/d61/le7 0 2024-04-26T23:01:37.054 INFO:tasks.workunit.client.0.smithi027.stdout:4/805: dwrite da/dd/de/d14/d17/d80/dab/f26 [0,4194304] 0 2024-04-26T23:01:37.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/655: sync 2024-04-26T23:01:37.067 INFO:tasks.workunit.client.0.smithi027.stdout:7/770: sync 2024-04-26T23:01:37.067 INFO:tasks.workunit.client.0.smithi027.stdout:6/623: sync 2024-04-26T23:01:37.068 INFO:tasks.workunit.client.0.smithi027.stdout:3/808: sync 2024-04-26T23:01:37.069 INFO:tasks.workunit.client.0.smithi027.stdout:6/624: truncate d1/df/d43/f6e 5062684 0 2024-04-26T23:01:37.069 INFO:tasks.workunit.client.0.smithi027.stdout:0/670: sync 2024-04-26T23:01:37.069 INFO:tasks.workunit.client.0.smithi027.stdout:0/671: chown d0/df/d3f/c63 268346 1 2024-04-26T23:01:37.069 INFO:tasks.workunit.client.0.smithi027.stdout:8/663: sync 2024-04-26T23:01:37.069 INFO:tasks.workunit.client.0.smithi027.stdout:8/664: fdatasync d9/de/f14 0 2024-04-26T23:01:37.069 INFO:tasks.workunit.client.0.smithi027.stdout:7/771: creat d0/d28/d2b/d3a/d48/ffa x:0 0 0 2024-04-26T23:01:37.070 INFO:tasks.workunit.client.0.smithi027.stdout:7/772: fsync d0/d28/d2b/d3a/d48/d56/f74 0 2024-04-26T23:01:37.070 INFO:tasks.workunit.client.0.smithi027.stdout:6/625: mkdir d1/df/d3c/d66/dc9 0 2024-04-26T23:01:37.071 INFO:tasks.workunit.client.0.smithi027.stdout:0/672: getdents d0/d4/da/d1a/d1e 0 2024-04-26T23:01:37.072 INFO:tasks.workunit.client.0.smithi027.stdout:7/773: creat d0/d28/d2b/d3a/ffb x:0 0 0 2024-04-26T23:01:37.078 INFO:tasks.workunit.client.0.smithi027.stdout:0/673: link d0/d4/da/d1a/d1e/l46 d0/df/d3f/d64/lca 0 2024-04-26T23:01:37.078 INFO:tasks.workunit.client.0.smithi027.stdout:7/774: symlink d0/lfc 0 2024-04-26T23:01:37.078 INFO:tasks.workunit.client.0.smithi027.stdout:3/809: unlink d6/c42 0 2024-04-26T23:01:37.080 INFO:tasks.workunit.client.0.smithi027.stdout:3/810: write d6/dd3/d34/d2e/d9d/da7/dac/dca/ff7 [813336,2421] 0 2024-04-26T23:01:37.080 INFO:tasks.workunit.client.0.smithi027.stdout:3/811: readlink d6/dd3/lba 0 2024-04-26T23:01:37.080 INFO:tasks.workunit.client.0.smithi027.stdout:6/626: rename d1/d30/fa7 to d1/df/d3c/d66/dc9/fca 0 2024-04-26T23:01:37.080 INFO:tasks.workunit.client.0.smithi027.stdout:3/812: getdents d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/d113 0 2024-04-26T23:01:37.080 INFO:tasks.workunit.client.0.smithi027.stdout:6/627: creat d1/df/d43/da8/d84/fcb x:0 0 0 2024-04-26T23:01:37.084 INFO:tasks.workunit.client.0.smithi027.stdout:6/628: creat d1/d30/d6c/db3/fcc x:0 0 0 2024-04-26T23:01:37.085 INFO:tasks.workunit.client.0.smithi027.stdout:0/674: rmdir d0/d4 39 2024-04-26T23:01:37.085 INFO:tasks.workunit.client.0.smithi027.stdout:5/749: dread d4/d24/d22/d29/f36 [0,4194304] 0 2024-04-26T23:01:37.087 INFO:tasks.workunit.client.0.smithi027.stdout:7/775: dread d0/d28/d9f/fa1 [0,4194304] 0 2024-04-26T23:01:37.087 INFO:tasks.workunit.client.0.smithi027.stdout:7/776: chown d0/d28/f18 59896 1 2024-04-26T23:01:37.087 INFO:tasks.workunit.client.0.smithi027.stdout:0/675: symlink d0/d4/lcb 0 2024-04-26T23:01:37.089 INFO:tasks.workunit.client.0.smithi027.stdout:5/750: creat d4/d24/d22/dab/d16/d4b/fef x:0 0 0 2024-04-26T23:01:37.089 INFO:tasks.workunit.client.0.smithi027.stdout:7/777: creat d0/d28/d66/d67/ffd x:0 0 0 2024-04-26T23:01:37.089 INFO:tasks.workunit.client.0.smithi027.stdout:7/778: write d0/d28/d2b/d3a/f3f [5963323,101201] 0 2024-04-26T23:01:37.090 INFO:tasks.workunit.client.0.smithi027.stdout:5/751: mknod d4/d24/d5f/cf0 0 2024-04-26T23:01:37.093 INFO:tasks.workunit.client.0.smithi027.stdout:7/779: rename d0/c4f to d0/d28/d2b/d55/d69/d75/cfe 0 2024-04-26T23:01:37.095 INFO:tasks.workunit.client.0.smithi027.stdout:5/752: unlink d4/d24/d22/d29/d39/dc7/cd6 0 2024-04-26T23:01:37.096 INFO:tasks.workunit.client.0.smithi027.stdout:7/780: creat d0/d28/d2b/d55/d59/d6d/fff x:0 0 0 2024-04-26T23:01:37.099 INFO:tasks.workunit.client.0.smithi027.stdout:5/753: creat d4/d1a/dcb/ff1 x:0 0 0 2024-04-26T23:01:37.101 INFO:tasks.workunit.client.0.smithi027.stdout:8/665: dread d9/da/d2a/d5e/d46/f6f [0,4194304] 0 2024-04-26T23:01:37.101 INFO:tasks.workunit.client.0.smithi027.stdout:8/666: chown d9/da/d2a/d5e/d72/d84/d35/f99 7 1 2024-04-26T23:01:37.101 INFO:tasks.workunit.client.0.smithi027.stdout:5/754: creat d4/d24/d22/ff2 x:0 0 0 2024-04-26T23:01:37.106 INFO:tasks.workunit.client.0.smithi027.stdout:8/667: rename d9/da/d2a/d5e/d72/d84/l5c to d9/da/d2a/d5e/d72/d84/d35/d8d/da3/led 0 2024-04-26T23:01:37.110 INFO:tasks.workunit.client.0.smithi027.stdout:8/668: creat d9/da/d2a/d5e/d72/d84/d35/dc2/fee x:0 0 0 2024-04-26T23:01:37.110 INFO:tasks.workunit.client.0.smithi027.stdout:8/669: readlink d9/da/d2a/d5e/d2b/lc9 0 2024-04-26T23:01:37.113 INFO:tasks.workunit.client.0.smithi027.stdout:8/670: creat d9/da/d33/fef x:0 0 0 2024-04-26T23:01:37.122 INFO:tasks.workunit.client.0.smithi027.stdout:3/813: dread d6/dd3/d2f/f39 [0,4194304] 0 2024-04-26T23:01:37.122 INFO:tasks.workunit.client.0.smithi027.stdout:4/806: dwrite da/dd/de/d14/f44 [0,4194304] 0 2024-04-26T23:01:37.122 INFO:tasks.workunit.client.0.smithi027.stdout:4/807: rename da/dd/d5f/dac/cbb to da/dd/de/d14/d21/d67/cfc 0 2024-04-26T23:01:37.123 INFO:tasks.workunit.client.0.smithi027.stdout:4/808: rename da/lf2 to da/dd/de/d14/db3/lfd 0 2024-04-26T23:01:37.124 INFO:tasks.workunit.client.0.smithi027.stdout:4/809: stat da/dd/de/d14/d21/d41 0 2024-04-26T23:01:37.128 INFO:tasks.workunit.client.0.smithi027.stdout:4/810: mknod da/dd/de/d14/d17/d80/dab/dd6/cfe 0 2024-04-26T23:01:37.128 INFO:tasks.workunit.client.0.smithi027.stdout:4/811: read da/dd/de/d14/d21/f75 [1490031,62900] 0 2024-04-26T23:01:37.128 INFO:tasks.workunit.client.0.smithi027.stdout:7/781: dread d0/d28/d2b/d3a/d48/d56/d88/d73/f78 [0,4194304] 0 2024-04-26T23:01:37.128 INFO:tasks.workunit.client.0.smithi027.stdout:7/782: fdatasync d0/d3d/f44 0 2024-04-26T23:01:37.130 INFO:tasks.workunit.client.0.smithi027.stdout:4/812: creat da/dd/de/d14/d21/d41/df7/fff x:0 0 0 2024-04-26T23:01:37.132 INFO:tasks.workunit.client.0.smithi027.stdout:7/783: rename d0/d28/d2b/d3a/d48/d56/d88/d73/d80/ddd to d0/d28/d2b/d3a/d48/d56/dee/d100 0 2024-04-26T23:01:37.132 INFO:tasks.workunit.client.0.smithi027.stdout:7/784: truncate d0/d28/d66/d67/ffd 45181 0 2024-04-26T23:01:37.137 INFO:tasks.workunit.client.0.smithi027.stdout:9/656: dwrite d4/f24 [0,4194304] 0 2024-04-26T23:01:37.137 INFO:tasks.workunit.client.0.smithi027.stdout:0/676: dwrite d0/d4/da/d15/d55/f77 [0,4194304] 0 2024-04-26T23:01:37.139 INFO:tasks.workunit.client.0.smithi027.stdout:3/814: write d6/dd3/d2f/f39 [545261,102708] 0 2024-04-26T23:01:37.139 INFO:tasks.workunit.client.0.smithi027.stdout:3/815: write d6/dd3/d34/d2e/d5f/fc4 [4310820,79555] 0 2024-04-26T23:01:37.143 INFO:tasks.workunit.client.0.smithi027.stdout:6/629: read d1/de/f48 [3522851,46681] 0 2024-04-26T23:01:37.143 INFO:tasks.workunit.client.0.smithi027.stdout:1/851: dwrite d1/d5/f19 [0,4194304] 0 2024-04-26T23:01:37.143 INFO:tasks.workunit.client.0.smithi027.stdout:3/816: rename d6/dd3/d34/d2e/d5f/d90/fb0 to d6/dd3/d2f/f114 0 2024-04-26T23:01:37.143 INFO:tasks.workunit.client.0.smithi027.stdout:1/852: link d1/d26/d66/d73/dae/caf d1/d26/d66/d73/d9e/c117 0 2024-04-26T23:01:37.143 INFO:tasks.workunit.client.0.smithi027.stdout:1/853: chown d1/d26/d66/d91/ff3 21 1 2024-04-26T23:01:37.144 INFO:tasks.workunit.client.0.smithi027.stdout:1/854: creat d1/d26/d66/d73/dae/dd0/f118 x:0 0 0 2024-04-26T23:01:37.144 INFO:tasks.workunit.client.0.smithi027.stdout:1/855: dread - d1/d26/d66/d91/f109 zero size 2024-04-26T23:01:37.146 INFO:tasks.workunit.client.0.smithi027.stdout:3/817: getdents d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/d5a 0 2024-04-26T23:01:37.149 INFO:tasks.workunit.client.0.smithi027.stdout:9/657: write d4/f52 [201383,80977] 0 2024-04-26T23:01:37.161 INFO:tasks.workunit.client.0.smithi027.stdout:4/813: dread da/dd/de/d14/d17/d80/dab/dd6/fdd [0,4194304] 0 2024-04-26T23:01:37.161 INFO:tasks.workunit.client.0.smithi027.stdout:4/814: write da/dd/de/d14/d21/d41/df7/fff [163947,114661] 0 2024-04-26T23:01:37.169 INFO:tasks.workunit.client.0.smithi027.stdout:4/815: symlink da/dd/de/d14/d21/d41/df7/l100 0 2024-04-26T23:01:37.170 INFO:tasks.workunit.client.0.smithi027.stdout:4/816: getdents da/dd/d92 0 2024-04-26T23:01:37.173 INFO:tasks.workunit.client.0.smithi027.stdout:4/817: creat da/dd/f101 x:0 0 0 2024-04-26T23:01:37.175 INFO:tasks.workunit.client.0.smithi027.stdout:5/755: dwrite d4/d24/f7e [0,4194304] 0 2024-04-26T23:01:37.177 INFO:tasks.workunit.client.0.smithi027.stdout:5/756: creat d4/d24/d5f/dac/d40/ff3 x:0 0 0 2024-04-26T23:01:37.177 INFO:tasks.workunit.client.0.smithi027.stdout:5/757: chown d4/d24/d22/cdf 443 1 2024-04-26T23:01:37.178 INFO:tasks.workunit.client.0.smithi027.stdout:5/758: creat d4/d1a/d2b/db8/ff4 x:0 0 0 2024-04-26T23:01:37.179 INFO:tasks.workunit.client.0.smithi027.stdout:5/759: write d4/fa5 [1111435,42611] 0 2024-04-26T23:01:37.185 INFO:tasks.workunit.client.0.smithi027.stdout:1/856: write d1/f55 [3711222,15265] 0 2024-04-26T23:01:37.189 INFO:tasks.workunit.client.0.smithi027.stdout:5/760: rename d4/d24/d22/dab/d16/c65 to d4/d24/d22/cf5 0 2024-04-26T23:01:37.190 INFO:tasks.workunit.client.0.smithi027.stdout:1/857: getdents d1/d26/d66/d73/db5 0 2024-04-26T23:01:37.191 INFO:tasks.workunit.client.0.smithi027.stdout:7/785: dwrite d0/d6c/f8c [0,4194304] 0 2024-04-26T23:01:37.191 INFO:tasks.workunit.client.0.smithi027.stdout:7/786: fdatasync d0/d3d/dba/fe5 0 2024-04-26T23:01:37.191 INFO:tasks.workunit.client.0.smithi027.stdout:5/761: mkdir d4/d24/d22/d29/d39/df6 0 2024-04-26T23:01:37.193 INFO:tasks.workunit.client.0.smithi027.stdout:7/787: creat d0/d28/d2b/d3a/d48/d56/d88/f101 x:0 0 0 2024-04-26T23:01:37.193 INFO:tasks.workunit.client.0.smithi027.stdout:5/762: mknod d4/d1a/cf7 0 2024-04-26T23:01:37.195 INFO:tasks.workunit.client.0.smithi027.stdout:8/671: dread d9/da/d2a/d5e/f57 [4194304,4194304] 0 2024-04-26T23:01:37.195 INFO:tasks.workunit.client.0.smithi027.stdout:7/788: creat d0/d3d/dba/df8/f102 x:0 0 0 2024-04-26T23:01:37.196 INFO:tasks.workunit.client.0.smithi027.stdout:6/630: write d1/d30/d6c/d6f/d7d/f80 [3070378,80996] 0 2024-04-26T23:01:37.200 INFO:tasks.workunit.client.0.smithi027.stdout:4/818: write da/dd/de/d14/d21/d41/fa4 [361609,13120] 0 2024-04-26T23:01:37.201 INFO:tasks.workunit.client.0.smithi027.stdout:2/672: sync 2024-04-26T23:01:37.205 INFO:tasks.workunit.client.0.smithi027.stdout:6/631: symlink d1/d30/d6c/lcd 0 2024-04-26T23:01:37.206 INFO:tasks.workunit.client.0.smithi027.stdout:0/677: dread d0/d4/da/d1a/f3b [8388608,4194304] 0 2024-04-26T23:01:37.208 INFO:tasks.workunit.client.0.smithi027.stdout:0/678: link d0/d4/f24 d0/d4/da/d1a/d2a/fcc 0 2024-04-26T23:01:37.210 INFO:tasks.workunit.client.0.smithi027.stdout:0/679: mkdir d0/d4/da/d1a/d59/d92/dcd 0 2024-04-26T23:01:37.211 INFO:tasks.workunit.client.0.smithi027.stdout:0/680: rmdir d0/df/d9d/d9f 39 2024-04-26T23:01:37.212 INFO:tasks.workunit.client.0.smithi027.stdout:0/681: creat d0/d4/da/d1a/d6d/dc9/fce x:0 0 0 2024-04-26T23:01:37.212 INFO:tasks.workunit.client.0.smithi027.stdout:0/682: write d0/df/d9d/db4/fb6 [314494,91664] 0 2024-04-26T23:01:37.218 INFO:tasks.workunit.client.0.smithi027.stdout:0/683: link d0/d4/da/d1a/d1e/f8c d0/d4/da/d1a/dc3/fcf 0 2024-04-26T23:01:37.223 INFO:tasks.workunit.client.0.smithi027.stdout:3/818: dwrite d6/f96 [0,4194304] 0 2024-04-26T23:01:37.223 INFO:tasks.workunit.client.0.smithi027.stdout:0/684: unlink d0/d4/da/d15/d26/c9e 0 2024-04-26T23:01:37.223 INFO:tasks.workunit.client.0.smithi027.stdout:0/685: mknod d0/cd0 0 2024-04-26T23:01:37.223 INFO:tasks.workunit.client.0.smithi027.stdout:0/686: chown d0/d4/fbe 495920334 1 2024-04-26T23:01:37.227 INFO:tasks.workunit.client.0.smithi027.stdout:3/819: mknod d6/dd3/d34/d2e/d100/c115 0 2024-04-26T23:01:37.228 INFO:tasks.workunit.client.0.smithi027.stdout:7/789: dread d0/d28/d2b/d3a/d48/d56/db6/ff0 [0,4194304] 0 2024-04-26T23:01:37.228 INFO:tasks.workunit.client.0.smithi027.stdout:6/632: dread d1/df/f5f [0,4194304] 0 2024-04-26T23:01:37.228 INFO:tasks.workunit.client.0.smithi027.stdout:7/790: write d0/f11 [308430,62510] 0 2024-04-26T23:01:37.230 INFO:tasks.workunit.client.0.smithi027.stdout:7/791: mkdir d0/d28/d2b/d3a/d90/d103 0 2024-04-26T23:01:37.230 INFO:tasks.workunit.client.0.smithi027.stdout:3/820: creat d6/dd3/d34/d2e/d9d/f116 x:0 0 0 2024-04-26T23:01:37.232 INFO:tasks.workunit.client.0.smithi027.stdout:7/792: creat d0/d28/d2b/d3a/d48/d56/d88/d73/f104 x:0 0 0 2024-04-26T23:01:37.233 INFO:tasks.workunit.client.0.smithi027.stdout:7/793: creat d0/d28/d2b/d3a/d48/d56/dee/d100/f105 x:0 0 0 2024-04-26T23:01:37.233 INFO:tasks.workunit.client.0.smithi027.stdout:7/794: stat d0/c53 0 2024-04-26T23:01:37.234 INFO:tasks.workunit.client.0.smithi027.stdout:7/795: readlink d0/d2f/l45 0 2024-04-26T23:01:37.234 INFO:tasks.workunit.client.0.smithi027.stdout:7/796: fsync d0/d6c/f93 0 2024-04-26T23:01:37.248 INFO:tasks.workunit.client.0.smithi027.stdout:8/672: dread d9/da/d2a/d5e/d72/f8a [0,4194304] 0 2024-04-26T23:01:37.262 INFO:tasks.workunit.client.0.smithi027.stdout:2/673: dwrite d2/f67 [0,4194304] 0 2024-04-26T23:01:37.262 INFO:tasks.workunit.client.0.smithi027.stdout:2/674: dread - d2/d5/db/de/d21/d8f/f6d zero size 2024-04-26T23:01:37.262 INFO:tasks.workunit.client.0.smithi027.stdout:4/819: dread da/dd/de/d14/f44 [0,4194304] 0 2024-04-26T23:01:37.264 INFO:tasks.workunit.client.0.smithi027.stdout:4/820: creat da/dd/d5f/dac/f102 x:0 0 0 2024-04-26T23:01:37.267 INFO:tasks.workunit.client.0.smithi027.stdout:2/675: rename d2/d5/db/d1b/d1c/d25/d6a/l74 to d2/d5/db/lca 0 2024-04-26T23:01:37.269 INFO:tasks.workunit.client.0.smithi027.stdout:4/821: mkdir da/d40/daf/d103 0 2024-04-26T23:01:37.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:37 smithi080 ceph-mon[32324]: pgmap v93: 129 pgs: 129 active+clean; 1.2 GiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 138 MiB/s rd, 163 MiB/s wr, 218 op/s 2024-04-26T23:01:37.271 INFO:tasks.workunit.client.0.smithi027.stdout:1/858: dwrite d1/d26/d66/dab/fd5 [4194304,4194304] 0 2024-04-26T23:01:37.272 INFO:tasks.workunit.client.0.smithi027.stdout:2/676: creat d2/d5/db/d1b/fcb x:0 0 0 2024-04-26T23:01:37.272 INFO:tasks.workunit.client.0.smithi027.stdout:2/677: write d2/d5/db/d1b/f38 [1567593,114782] 0 2024-04-26T23:01:37.275 INFO:tasks.workunit.client.0.smithi027.stdout:0/687: dwrite d0/d4/da/d1a/f90 [0,4194304] 0 2024-04-26T23:01:37.278 INFO:tasks.workunit.client.0.smithi027.stdout:7/797: dread d0/d28/fed [0,4194304] 0 2024-04-26T23:01:37.278 INFO:tasks.workunit.client.0.smithi027.stdout:7/798: truncate d0/d28/d2b/d3a/f3f 6272299 0 2024-04-26T23:01:37.278 INFO:tasks.workunit.client.0.smithi027.stdout:1/859: link d1/l45 d1/d26/d66/dab/de0/l119 0 2024-04-26T23:01:37.278 INFO:tasks.workunit.client.0.smithi027.stdout:5/763: dwrite d4/d9f/dc0/fcc [0,4194304] 0 2024-04-26T23:01:37.282 INFO:tasks.workunit.client.0.smithi027.stdout:4/822: dread da/dd/de/f3c [0,4194304] 0 2024-04-26T23:01:37.282 INFO:tasks.workunit.client.0.smithi027.stdout:4/823: write da/d40/f60 [4695587,92784] 0 2024-04-26T23:01:37.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:37 smithi094 ceph-mon[32889]: pgmap v93: 129 pgs: 129 active+clean; 1.2 GiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 138 MiB/s rd, 163 MiB/s wr, 218 op/s 2024-04-26T23:01:37.285 INFO:tasks.workunit.client.0.smithi027.stdout:0/688: link d0/d4/da/d15/d55/f96 d0/d4/da/d1a/d2a/fd1 0 2024-04-26T23:01:37.285 INFO:tasks.workunit.client.0.smithi027.stdout:3/821: dwrite d6/dd3/d34/d78/da1/fd0 [0,4194304] 0 2024-04-26T23:01:37.285 INFO:tasks.workunit.client.0.smithi027.stdout:3/822: readlink d6/dd3/d2f/d7e/l109 0 2024-04-26T23:01:37.292 INFO:tasks.workunit.client.0.smithi027.stdout:4/824: read da/dd/de/d14/d21/d67/d8a/fa2 [885477,48998] 0 2024-04-26T23:01:37.292 INFO:tasks.workunit.client.0.smithi027.stdout:4/825: stat da/dd/d5f/dac/cb9 0 2024-04-26T23:01:37.292 INFO:tasks.workunit.client.0.smithi027.stdout:4/826: readlink da/d40/ld1 0 2024-04-26T23:01:37.296 INFO:tasks.workunit.client.0.smithi027.stdout:1/860: truncate d1/d26/d66/dab/de0/f44 652331 0 2024-04-26T23:01:37.296 INFO:tasks.workunit.client.0.smithi027.stdout:1/861: dread - d1/d26/d6b/f7f zero size 2024-04-26T23:01:37.296 INFO:tasks.workunit.client.0.smithi027.stdout:2/678: link d2/d5/db/d1b/d1c/d25/f3d d2/fcc 0 2024-04-26T23:01:37.300 INFO:tasks.workunit.client.0.smithi027.stdout:4/827: creat da/dd/de/d79/da8/dc5/de8/f104 x:0 0 0 2024-04-26T23:01:37.307 INFO:tasks.workunit.client.0.smithi027.stdout:2/679: unlink d2/d5/db/d1b/d1c/d25/f3d 0 2024-04-26T23:01:37.307 INFO:tasks.workunit.client.0.smithi027.stdout:4/828: rmdir da/dd/de/d79/dcd 39 2024-04-26T23:01:37.309 INFO:tasks.workunit.client.0.smithi027.stdout:1/862: mknod d1/d26/d7d/c11a 0 2024-04-26T23:01:37.309 INFO:tasks.workunit.client.0.smithi027.stdout:1/863: read - d1/d26/d6b/d8e/f10b zero size 2024-04-26T23:01:37.311 INFO:tasks.workunit.client.0.smithi027.stdout:1/864: creat d1/d26/d6b/d42/d79/f11b x:0 0 0 2024-04-26T23:01:37.314 INFO:tasks.workunit.client.0.smithi027.stdout:1/865: symlink d1/d26/d6b/l11c 0 2024-04-26T23:01:37.314 INFO:tasks.workunit.client.0.smithi027.stdout:1/866: truncate d1/d26/d6b/d42/d79/f11b 742714 0 2024-04-26T23:01:37.314 INFO:tasks.workunit.client.0.smithi027.stdout:1/867: write d1/d26/d6b/d42/d79/f11b [1354845,103320] 0 2024-04-26T23:01:37.324 INFO:tasks.workunit.client.0.smithi027.stdout:8/673: dwrite d9/da/d2a/d5e/d72/d84/d5d/f7d [0,4194304] 0 2024-04-26T23:01:37.324 INFO:tasks.workunit.client.0.smithi027.stdout:3/823: dread d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/f2b [0,4194304] 0 2024-04-26T23:01:37.328 INFO:tasks.workunit.client.0.smithi027.stdout:8/674: mkdir d9/da/d2a/d5e/d72/d84/d35/df0 0 2024-04-26T23:01:37.329 INFO:tasks.workunit.client.0.smithi027.stdout:3/824: mkdir d6/dd3/d34/d47/d117 0 2024-04-26T23:01:37.330 INFO:tasks.workunit.client.0.smithi027.stdout:3/825: read - d6/dd3/d2f/feb zero size 2024-04-26T23:01:37.333 INFO:tasks.workunit.client.0.smithi027.stdout:8/675: truncate d9/de/f18 226908 0 2024-04-26T23:01:37.333 INFO:tasks.workunit.client.0.smithi027.stdout:3/826: mknod d6/dd3/d34/d2e/c118 0 2024-04-26T23:01:37.333 INFO:tasks.workunit.client.0.smithi027.stdout:3/827: truncate d6/dd3/d34/d2e/d9d/da3/f10d 150060 0 2024-04-26T23:01:37.333 INFO:tasks.workunit.client.0.smithi027.stdout:3/828: write d6/dd3/d2f/d7e/f6e [1817547,31060] 0 2024-04-26T23:01:37.337 INFO:tasks.workunit.client.0.smithi027.stdout:3/829: write d6/dd3/d34/d2e/d5f/d90/ddb/d98/fc7 [805025,28003] 0 2024-04-26T23:01:37.337 INFO:tasks.workunit.client.0.smithi027.stdout:8/676: creat d9/da/d2a/d5e/d72/d84/d4f/ff1 x:0 0 0 2024-04-26T23:01:37.341 INFO:tasks.workunit.client.0.smithi027.stdout:7/799: dwrite d0/d6c/f8c [4194304,4194304] 0 2024-04-26T23:01:37.341 INFO:tasks.workunit.client.0.smithi027.stdout:7/800: write d0/d28/d2b/d3a/f52 [3955203,61662] 0 2024-04-26T23:01:37.351 INFO:tasks.workunit.client.0.smithi027.stdout:8/677: dread d9/da/d2a/d5e/d72/d84/d5d/d8b/f9a [0,4194304] 0 2024-04-26T23:01:37.351 INFO:tasks.workunit.client.0.smithi027.stdout:8/678: fsync d9/da/d2a/d5e/d72/d84/d4f/f79 0 2024-04-26T23:01:37.352 INFO:tasks.workunit.client.0.smithi027.stdout:0/689: dwrite d0/d4/da/d1a/d2a/fc8 [0,4194304] 0 2024-04-26T23:01:37.352 INFO:tasks.workunit.client.0.smithi027.stdout:0/690: chown d0/d4/da/d1a/d1e/ca9 1215368417 1 2024-04-26T23:01:37.356 INFO:tasks.workunit.client.0.smithi027.stdout:0/691: rename d0/d4/da/d1a/c42 to d0/df/cd2 0 2024-04-26T23:01:37.359 INFO:tasks.workunit.client.0.smithi027.stdout:0/692: unlink d0/d4/da/d15/d26/c5c 0 2024-04-26T23:01:37.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:37 smithi027 ceph-mon[28034]: pgmap v93: 129 pgs: 129 active+clean; 1.2 GiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 138 MiB/s rd, 163 MiB/s wr, 218 op/s 2024-04-26T23:01:37.365 INFO:tasks.workunit.client.0.smithi027.stdout:1/868: dwrite d1/dc/fa9 [4194304,4194304] 0 2024-04-26T23:01:37.365 INFO:tasks.workunit.client.0.smithi027.stdout:1/869: stat d1/d26/d66/l68 0 2024-04-26T23:01:37.365 INFO:tasks.workunit.client.0.smithi027.stdout:1/870: fdatasync d1/f32 0 2024-04-26T23:01:37.365 INFO:tasks.workunit.client.0.smithi027.stdout:7/801: write d0/d28/fed [175513,118484] 0 2024-04-26T23:01:37.365 INFO:tasks.workunit.client.0.smithi027.stdout:8/679: dread d9/da/d2a/d5e/d92/fae [0,4194304] 0 2024-04-26T23:01:37.372 INFO:tasks.workunit.client.0.smithi027.stdout:1/871: creat d1/d5/d40/d102/f11d x:0 0 0 2024-04-26T23:01:37.372 INFO:tasks.workunit.client.0.smithi027.stdout:1/872: stat d1/d26/d86/d5c/l63 0 2024-04-26T23:01:37.372 INFO:tasks.workunit.client.0.smithi027.stdout:8/680: symlink d9/da/d2a/d5e/d9e/lf2 0 2024-04-26T23:01:37.372 INFO:tasks.workunit.client.0.smithi027.stdout:2/680: dwrite d2/d5/db/f6c [0,4194304] 0 2024-04-26T23:01:37.373 INFO:tasks.workunit.client.0.smithi027.stdout:8/681: unlink d9/da/d33/lca 0 2024-04-26T23:01:37.373 INFO:tasks.workunit.client.0.smithi027.stdout:8/682: write d9/de/d2e/f1f [738244,75952] 0 2024-04-26T23:01:37.378 INFO:tasks.workunit.client.0.smithi027.stdout:2/681: mknod d2/d5/db/de/d21/ccd 0 2024-04-26T23:01:37.379 INFO:tasks.workunit.client.0.smithi027.stdout:8/683: link d9/de/fa5 d9/da/d2a/d5e/d72/d84/db8/ff3 0 2024-04-26T23:01:37.379 INFO:tasks.workunit.client.0.smithi027.stdout:8/684: chown d9/da/d2a/d5e/d46/feb 6107 1 2024-04-26T23:01:37.379 INFO:tasks.workunit.client.0.smithi027.stdout:8/685: write d9/da/d33/f60 [3719506,98608] 0 2024-04-26T23:01:37.379 INFO:tasks.workunit.client.0.smithi027.stdout:8/686: chown d9/da/d2a/d5e/d46/feb 251064 1 2024-04-26T23:01:37.392 INFO:tasks.workunit.client.0.smithi027.stdout:8/687: symlink d9/da/d2a/d5e/d2b/lf4 0 2024-04-26T23:01:37.392 INFO:tasks.workunit.client.0.smithi027.stdout:8/688: chown d9/da/d2a/d5e/d72/d84/db8/f9b 573419 1 2024-04-26T23:01:37.412 INFO:tasks.workunit.client.0.smithi027.stdout:0/693: dwrite d0/d4/da/d15/d55/f96 [0,4194304] 0 2024-04-26T23:01:37.413 INFO:tasks.workunit.client.0.smithi027.stdout:3/830: dwrite d6/dd3/d34/db3/ff0 [0,4194304] 0 2024-04-26T23:01:37.416 INFO:tasks.workunit.client.0.smithi027.stdout:0/694: unlink d0/d4/caa 0 2024-04-26T23:01:37.416 INFO:tasks.workunit.client.0.smithi027.stdout:0/695: getdents d0/df/d3f 0 2024-04-26T23:01:37.418 INFO:tasks.workunit.client.0.smithi027.stdout:0/696: mkdir d0/d4/dd3 0 2024-04-26T23:01:37.418 INFO:tasks.workunit.client.0.smithi027.stdout:0/697: fsync d0/d4/da/d1a/d2a/fcc 0 2024-04-26T23:01:37.420 INFO:tasks.workunit.client.0.smithi027.stdout:0/698: creat d0/d4/da/d1a/d59/d92/dcd/fd4 x:0 0 0 2024-04-26T23:01:37.426 INFO:tasks.workunit.client.0.smithi027.stdout:1/873: dwrite d1/d26/d66/d73/d9e/fba [4194304,4194304] 0 2024-04-26T23:01:37.429 INFO:tasks.workunit.client.0.smithi027.stdout:1/874: mknod d1/d26/d6b/d8e/c11e 0 2024-04-26T23:01:37.449 INFO:tasks.workunit.client.0.smithi027.stdout:3/831: dwrite d6/f2d [0,4194304] 0 2024-04-26T23:01:37.451 INFO:tasks.workunit.client.0.smithi027.stdout:3/832: truncate d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/fb8 79262 0 2024-04-26T23:01:37.451 INFO:tasks.workunit.client.0.smithi027.stdout:3/833: fdatasync d6/dd3/d34/db3/fc1 0 2024-04-26T23:01:37.452 INFO:tasks.workunit.client.0.smithi027.stdout:3/834: creat d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/dd6/f119 x:0 0 0 2024-04-26T23:01:37.454 INFO:tasks.workunit.client.0.smithi027.stdout:3/835: creat d6/dd3/d34/d47/d117/f11a x:0 0 0 2024-04-26T23:01:37.474 INFO:tasks.workunit.client.0.smithi027.stdout:3/836: dread d6/fbc [0,4194304] 0 2024-04-26T23:01:37.476 INFO:tasks.workunit.client.0.smithi027.stdout:2/682: dwrite d2/d5/db/d1b/d1c/d25/d19/d23/fb2 [0,4194304] 0 2024-04-26T23:01:37.476 INFO:tasks.workunit.client.0.smithi027.stdout:8/689: dread d9/da/d2a/d5e/d72/d84/d35/f56 [0,4194304] 0 2024-04-26T23:01:37.476 INFO:tasks.workunit.client.0.smithi027.stdout:3/837: creat d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d40/f11b x:0 0 0 2024-04-26T23:01:37.481 INFO:tasks.workunit.client.0.smithi027.stdout:3/838: mkdir d6/dd3/d2f/d11c 0 2024-04-26T23:01:37.482 INFO:tasks.workunit.client.0.smithi027.stdout:0/699: dwrite d0/d4/da/d15/d26/f79 [0,4194304] 0 2024-04-26T23:01:37.482 INFO:tasks.workunit.client.0.smithi027.stdout:0/700: chown d0/d4/da/d1a/l5f 2811 1 2024-04-26T23:01:37.484 INFO:tasks.workunit.client.0.smithi027.stdout:0/701: creat d0/df/d3f/fd5 x:0 0 0 2024-04-26T23:01:37.493 INFO:tasks.workunit.client.0.smithi027.stdout:2/683: write d2/d5/db/d1b/d1c/d25/d19/d23/f47 [3092411,57286] 0 2024-04-26T23:01:37.498 INFO:tasks.workunit.client.0.smithi027.stdout:9/658: sync 2024-04-26T23:01:37.498 INFO:tasks.workunit.client.0.smithi027.stdout:6/633: sync 2024-04-26T23:01:37.512 INFO:tasks.workunit.client.0.smithi027.stdout:9/659: write d4/d10/d40/d58/d75/d84/d86/f94 [336434,84311] 0 2024-04-26T23:01:37.515 INFO:tasks.workunit.client.0.smithi027.stdout:0/702: write d0/df/d3f/d64/f6a [1854412,38303] 0 2024-04-26T23:01:37.515 INFO:tasks.workunit.client.0.smithi027.stdout:0/703: fsync d0/d4/faf 0 2024-04-26T23:01:37.515 INFO:tasks.workunit.client.0.smithi027.stdout:0/704: chown d0/d4/da/d1a/d2a/fc8 213546079 1 2024-04-26T23:01:37.516 INFO:tasks.workunit.client.0.smithi027.stdout:6/634: mknod d1/d30/d6c/d6f/cce 0 2024-04-26T23:01:37.517 INFO:tasks.workunit.client.0.smithi027.stdout:9/660: mknod d4/d17/d2e/d46/da0/ce9 0 2024-04-26T23:01:37.517 INFO:tasks.workunit.client.0.smithi027.stdout:9/661: write d4/d77/f7a [5003563,6703] 0 2024-04-26T23:01:37.517 INFO:tasks.workunit.client.0.smithi027.stdout:9/662: chown d4/d10/f4a 638 1 2024-04-26T23:01:37.518 INFO:tasks.workunit.client.0.smithi027.stdout:2/684: creat d2/db9/dc6/fce x:0 0 0 2024-04-26T23:01:37.518 INFO:tasks.workunit.client.0.smithi027.stdout:0/705: creat d0/d4/da/d1a/d1e/dba/fd6 x:0 0 0 2024-04-26T23:01:37.518 INFO:tasks.workunit.client.0.smithi027.stdout:0/706: chown d0/d4/da/d1a/d59/d92/dcd 5 1 2024-04-26T23:01:37.520 INFO:tasks.workunit.client.0.smithi027.stdout:6/635: unlink d1/de/l46 0 2024-04-26T23:01:37.521 INFO:tasks.workunit.client.0.smithi027.stdout:8/690: dwrite d9/da/d2a/d5e/d9e/fc8 [0,4194304] 0 2024-04-26T23:01:37.522 INFO:tasks.workunit.client.0.smithi027.stdout:9/663: mknod d4/d17/d2e/d46/da0/d72/cea 0 2024-04-26T23:01:37.522 INFO:tasks.workunit.client.0.smithi027.stdout:2/685: mknod d2/d5/db/d1b/d1c/d25/d6a/ccf 0 2024-04-26T23:01:37.522 INFO:tasks.workunit.client.0.smithi027.stdout:0/707: rename d0/df/d3f/d64/da8/cc5 to d0/d4/cd7 0 2024-04-26T23:01:37.523 INFO:tasks.workunit.client.0.smithi027.stdout:0/708: chown d0/d4/da/d1a/l5f 7312189 1 2024-04-26T23:01:37.523 INFO:tasks.workunit.client.0.smithi027.stdout:6/636: getdents d1/d30/d6c/db3 0 2024-04-26T23:01:37.524 INFO:tasks.workunit.client.0.smithi027.stdout:2/686: creat d2/d5/fd0 x:0 0 0 2024-04-26T23:01:37.524 INFO:tasks.workunit.client.0.smithi027.stdout:2/687: dread - d2/d5/db/d1b/d1c/d25/fa4 zero size 2024-04-26T23:01:37.525 INFO:tasks.workunit.client.0.smithi027.stdout:6/637: creat d1/df/d3c/d9a/fcf x:0 0 0 2024-04-26T23:01:37.527 INFO:tasks.workunit.client.0.smithi027.stdout:6/638: rename d1/df/d3c/d66 to d1/de/d57/d8e/dd0 0 2024-04-26T23:01:37.532 INFO:tasks.workunit.client.0.smithi027.stdout:6/639: write d1/df/d43/da8/da1/fb6 [49449,123313] 0 2024-04-26T23:01:37.536 INFO:tasks.workunit.client.0.smithi027.stdout:0/709: dread d0/d4/da/d1a/d59/f9c [0,4194304] 0 2024-04-26T23:01:37.539 INFO:tasks.workunit.client.0.smithi027.stdout:6/640: read d1/d30/f36 [461910,95836] 0 2024-04-26T23:01:37.541 INFO:tasks.workunit.client.0.smithi027.stdout:9/664: dread d4/d17/d2e/d46/da0/d8d/fa8 [0,4194304] 0 2024-04-26T23:01:37.541 INFO:tasks.workunit.client.0.smithi027.stdout:9/665: write d4/d17/d2e/d46/da0/d8d/fdc [332157,78874] 0 2024-04-26T23:01:37.549 INFO:tasks.workunit.client.0.smithi027.stdout:2/688: read d2/d5/db/de/f30 [395180,68995] 0 2024-04-26T23:01:37.567 INFO:tasks.workunit.client.0.smithi027.stdout:3/839: dwrite d6/f7 [0,4194304] 0 2024-04-26T23:01:37.571 INFO:tasks.workunit.client.0.smithi027.stdout:8/691: dwrite d9/da/d2a/d5e/d72/d84/d5d/d6c/fa9 [0,4194304] 0 2024-04-26T23:01:37.571 INFO:tasks.workunit.client.0.smithi027.stdout:8/692: stat d9/da/d2a/d5e/d72/d84/f29 0 2024-04-26T23:01:37.571 INFO:tasks.workunit.client.0.smithi027.stdout:8/693: write d9/da/d33/f41 [2794402,38895] 0 2024-04-26T23:01:37.572 INFO:tasks.workunit.client.0.smithi027.stdout:8/694: dread - d9/de/d22/fdb zero size 2024-04-26T23:01:37.572 INFO:tasks.workunit.client.0.smithi027.stdout:8/695: rename d9/da/d2a/d5e/d72/d84 to d9/da/d2a/d5e/d72/d84/d35/dea/df5 22 2024-04-26T23:01:37.573 INFO:tasks.workunit.client.0.smithi027.stdout:0/710: dread d0/df/d3f/d64/f6a [0,4194304] 0 2024-04-26T23:01:37.573 INFO:tasks.workunit.client.0.smithi027.stdout:8/696: getdents d9/da/d33/d71 0 2024-04-26T23:01:37.577 INFO:tasks.workunit.client.0.smithi027.stdout:0/711: truncate d0/d4/f3d 1430518 0 2024-04-26T23:01:37.577 INFO:tasks.workunit.client.0.smithi027.stdout:8/697: symlink d9/da/d2a/d5e/d72/d84/d4f/dd3/lf6 0 2024-04-26T23:01:37.583 INFO:tasks.workunit.client.0.smithi027.stdout:8/698: creat d9/da/d2a/d5e/d72/d84/d35/dc2/dc7/ff7 x:0 0 0 2024-04-26T23:01:37.583 INFO:tasks.workunit.client.0.smithi027.stdout:6/641: dwrite d1/f4e [0,4194304] 0 2024-04-26T23:01:37.583 INFO:tasks.workunit.client.0.smithi027.stdout:5/764: sync 2024-04-26T23:01:37.583 INFO:tasks.workunit.client.0.smithi027.stdout:1/875: sync 2024-04-26T23:01:37.583 INFO:tasks.workunit.client.0.smithi027.stdout:5/765: write d4/d24/d5f/dac/d40/ff3 [98701,115961] 0 2024-04-26T23:01:37.587 INFO:tasks.workunit.client.0.smithi027.stdout:7/802: sync 2024-04-26T23:01:37.587 INFO:tasks.workunit.client.0.smithi027.stdout:4/829: sync 2024-04-26T23:01:37.591 INFO:tasks.workunit.client.0.smithi027.stdout:7/803: getdents d0/d28/d2b/d3a/d48/d56/d8d 0 2024-04-26T23:01:37.591 INFO:tasks.workunit.client.0.smithi027.stdout:7/804: write d0/d28/d2b/d3a/d90/dcb/fe9 [229710,69772] 0 2024-04-26T23:01:37.591 INFO:tasks.workunit.client.0.smithi027.stdout:3/840: dread d6/d5e/f6d [0,4194304] 0 2024-04-26T23:01:37.591 INFO:tasks.workunit.client.0.smithi027.stdout:3/841: write d6/dd3/d2f/d7e/f6e [378823,28004] 0 2024-04-26T23:01:37.591 INFO:tasks.workunit.client.0.smithi027.stdout:3/842: truncate d6/dd3/d34/db3/fdf 1038772 0 2024-04-26T23:01:37.591 INFO:tasks.workunit.client.0.smithi027.stdout:8/699: dread d9/de/f14 [0,4194304] 0 2024-04-26T23:01:37.592 INFO:tasks.workunit.client.0.smithi027.stdout:8/700: fsync d9/da/d2a/d5e/d2b/d52/f7a 0 2024-04-26T23:01:37.593 INFO:tasks.workunit.client.0.smithi027.stdout:8/701: chown d9/da/d2a/d6b/l98 14 1 2024-04-26T23:01:37.593 INFO:tasks.workunit.client.0.smithi027.stdout:7/805: mknod d0/d28/d2b/d3a/d90/d103/c106 0 2024-04-26T23:01:37.593 INFO:tasks.workunit.client.0.smithi027.stdout:7/806: fsync d0/d28/d66/d67/fb7 0 2024-04-26T23:01:37.593 INFO:tasks.workunit.client.0.smithi027.stdout:7/807: write d0/d28/d2b/d3a/d48/d56/d88/d73/dd6/fea [526718,105479] 0 2024-04-26T23:01:37.593 INFO:tasks.workunit.client.0.smithi027.stdout:7/808: dread - d0/d3d/ff7 zero size 2024-04-26T23:01:37.593 INFO:tasks.workunit.client.0.smithi027.stdout:0/712: dread d0/f7e [4194304,4194304] 0 2024-04-26T23:01:37.594 INFO:tasks.workunit.client.0.smithi027.stdout:0/713: unlink d0/df/d3f/d64/c82 0 2024-04-26T23:01:37.595 INFO:tasks.workunit.client.0.smithi027.stdout:0/714: dread - d0/d4/da/d1a/d6d/dc9/fce zero size 2024-04-26T23:01:37.596 INFO:tasks.workunit.client.0.smithi027.stdout:4/830: creat da/dd/de/d14/d21/d67/f105 x:0 0 0 2024-04-26T23:01:37.600 INFO:tasks.workunit.client.0.smithi027.stdout:8/702: mknod d9/da/d2a/d5e/d72/d84/d35/cf8 0 2024-04-26T23:01:37.602 INFO:tasks.workunit.client.0.smithi027.stdout:0/715: rename d0/d4/da/d1a/d59/d92/f94 to d0/df/d3f/d64/fd8 0 2024-04-26T23:01:37.602 INFO:tasks.workunit.client.0.smithi027.stdout:4/831: symlink da/dd/de/d79/l106 0 2024-04-26T23:01:37.602 INFO:tasks.workunit.client.0.smithi027.stdout:8/703: creat d9/da/d2a/d5e/d72/d84/d35/df0/ff9 x:0 0 0 2024-04-26T23:01:37.602 INFO:tasks.workunit.client.0.smithi027.stdout:8/704: dread - d9/da/d2a/d5e/d46/ddd/fc6 zero size 2024-04-26T23:01:37.606 INFO:tasks.workunit.client.0.smithi027.stdout:0/716: rename d0/d4/da/d1a/d59/d92/l99 to d0/d4/dd3/ld9 0 2024-04-26T23:01:37.637 INFO:tasks.workunit.client.0.smithi027.stdout:0/717: write d0/df/d3f/d64/f5e [16643,37624] 0 2024-04-26T23:01:37.637 INFO:tasks.workunit.client.0.smithi027.stdout:3/843: dread d6/dd3/d34/f36 [0,4194304] 0 2024-04-26T23:01:37.640 INFO:tasks.workunit.client.0.smithi027.stdout:3/844: mkdir d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d11d 0 2024-04-26T23:01:37.643 INFO:tasks.workunit.client.0.smithi027.stdout:3/845: creat d6/dd3/f11e x:0 0 0 2024-04-26T23:01:37.643 INFO:tasks.workunit.client.0.smithi027.stdout:3/846: fdatasync d6/f7 0 2024-04-26T23:01:37.643 INFO:tasks.workunit.client.0.smithi027.stdout:3/847: dread - d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/f86 zero size 2024-04-26T23:01:37.645 INFO:tasks.workunit.client.0.smithi027.stdout:6/642: dwrite d1/df/d43/da8/da1/fb6 [0,4194304] 0 2024-04-26T23:01:37.646 INFO:tasks.workunit.client.0.smithi027.stdout:3/848: mkdir d6/dd3/d34/d2e/d10b/d11f 0 2024-04-26T23:01:37.646 INFO:tasks.workunit.client.0.smithi027.stdout:3/849: chown d6/dd3/f11e 3542617 1 2024-04-26T23:01:37.646 INFO:tasks.workunit.client.0.smithi027.stdout:3/850: write d6/d5e/fa0 [5069076,76080] 0 2024-04-26T23:01:37.647 INFO:tasks.workunit.client.0.smithi027.stdout:6/643: mknod d1/de/d57/d8e/dd0/dc9/cd1 0 2024-04-26T23:01:37.648 INFO:tasks.workunit.client.0.smithi027.stdout:3/851: truncate d6/dd3/d34/d78/fbf 642409 0 2024-04-26T23:01:37.648 INFO:tasks.workunit.client.0.smithi027.stdout:3/852: chown d6/dd3/d34/d2e/d100 1845247689 1 2024-04-26T23:01:37.648 INFO:tasks.workunit.client.0.smithi027.stdout:3/853: truncate d6/dd3/d34/d2e/d9d/da7/dac/dca/fec 680059 0 2024-04-26T23:01:37.649 INFO:tasks.workunit.client.0.smithi027.stdout:6/644: symlink d1/de/d57/d8e/dd0/ld2 0 2024-04-26T23:01:37.649 INFO:tasks.workunit.client.0.smithi027.stdout:6/645: write d1/d30/d6c/f97 [45529,47222] 0 2024-04-26T23:01:37.651 INFO:tasks.workunit.client.0.smithi027.stdout:6/646: truncate d1/f69 834647 0 2024-04-26T23:01:37.651 INFO:tasks.workunit.client.0.smithi027.stdout:6/647: fsync d1/d30/faf 0 2024-04-26T23:01:37.651 INFO:tasks.workunit.client.0.smithi027.stdout:6/648: chown d1/de/c2c 5 1 2024-04-26T23:01:37.655 INFO:tasks.workunit.client.0.smithi027.stdout:7/809: dwrite d0/d28/f18 [0,4194304] 0 2024-04-26T23:01:37.658 INFO:tasks.workunit.client.0.smithi027.stdout:6/649: truncate d1/df/f2e 868133 0 2024-04-26T23:01:37.658 INFO:tasks.workunit.client.0.smithi027.stdout:6/650: write d1/de/d57/d98/fae [737376,114732] 0 2024-04-26T23:01:37.658 INFO:tasks.workunit.client.0.smithi027.stdout:3/854: creat d6/dd3/d34/d2e/d5f/d90/db9/f120 x:0 0 0 2024-04-26T23:01:37.658 INFO:tasks.workunit.client.0.smithi027.stdout:3/855: truncate d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/dd6/ff6 1012550 0 2024-04-26T23:01:37.658 INFO:tasks.workunit.client.0.smithi027.stdout:3/856: stat d6/dd3/d34/d2e/d9d/da7 0 2024-04-26T23:01:37.658 INFO:tasks.workunit.client.0.smithi027.stdout:3/857: fdatasync d6/dd3/d34/d2e/d50/fb2 0 2024-04-26T23:01:37.659 INFO:tasks.workunit.client.0.smithi027.stdout:6/651: mkdir d1/d30/d6c/d6f/dd3 0 2024-04-26T23:01:37.661 INFO:tasks.workunit.client.0.smithi027.stdout:3/858: truncate d6/dd3/d34/f36 1423531 0 2024-04-26T23:01:37.661 INFO:tasks.workunit.client.0.smithi027.stdout:6/652: unlink d1/de/d57/d98/fc7 0 2024-04-26T23:01:37.662 INFO:tasks.workunit.client.0.smithi027.stdout:0/718: dread d0/d4/da/d1a/f3b [0,4194304] 0 2024-04-26T23:01:37.662 INFO:tasks.workunit.client.0.smithi027.stdout:6/653: creat d1/df/d43/d56/fd4 x:0 0 0 2024-04-26T23:01:37.664 INFO:tasks.workunit.client.0.smithi027.stdout:1/876: dwrite d1/d2a/fac [0,4194304] 0 2024-04-26T23:01:37.664 INFO:tasks.workunit.client.0.smithi027.stdout:6/654: mknod d1/df/d43/d56/cd5 0 2024-04-26T23:01:37.664 INFO:tasks.workunit.client.0.smithi027.stdout:1/877: chown d1/d26/dc8 1 1 2024-04-26T23:01:37.665 INFO:tasks.workunit.client.0.smithi027.stdout:6/655: rmdir d1/de/d57/d98 39 2024-04-26T23:01:37.667 INFO:tasks.workunit.client.0.smithi027.stdout:1/878: unlink d1/d26/d66/d73/dae/dd0/ce9 0 2024-04-26T23:01:37.679 INFO:tasks.workunit.client.0.smithi027.stdout:6/656: dread d1/df/d43/d56/d6b/f93 [0,4194304] 0 2024-04-26T23:01:37.682 INFO:tasks.workunit.client.0.smithi027.stdout:5/766: dwrite d4/d24/d5f/dac/d40/ff3 [0,4194304] 0 2024-04-26T23:01:37.687 INFO:tasks.workunit.client.0.smithi027.stdout:5/767: link d4/d24/d22/d29/d2a/d3b/c6f d4/d9f/dc5/cf8 0 2024-04-26T23:01:37.691 INFO:tasks.workunit.client.0.smithi027.stdout:1/879: dread d1/dc/f1d [0,4194304] 0 2024-04-26T23:01:37.692 INFO:tasks.workunit.client.0.smithi027.stdout:1/880: mknod d1/d26/d6b/d42/d70/c11f 0 2024-04-26T23:01:37.694 INFO:tasks.workunit.client.0.smithi027.stdout:1/881: creat d1/d26/d36/d88/f120 x:0 0 0 2024-04-26T23:01:37.694 INFO:tasks.workunit.client.0.smithi027.stdout:1/882: mknod d1/d26/d66/d73/d9e/c121 0 2024-04-26T23:01:37.702 INFO:tasks.workunit.client.0.smithi027.stdout:6/657: write d1/df/d43/f74 [908930,101165] 0 2024-04-26T23:01:37.705 INFO:tasks.workunit.client.0.smithi027.stdout:5/768: dread d4/d1a/f94 [0,4194304] 0 2024-04-26T23:01:37.708 INFO:tasks.workunit.client.0.smithi027.stdout:5/769: creat d4/d9f/dc0/ff9 x:0 0 0 2024-04-26T23:01:37.713 INFO:tasks.workunit.client.0.smithi027.stdout:3/859: dwrite d6/dd3/d34/db3/f102 [0,4194304] 0 2024-04-26T23:01:37.715 INFO:tasks.workunit.client.0.smithi027.stdout:7/810: dwrite d0/d28/d2b/d3a/d90/fbd [0,4194304] 0 2024-04-26T23:01:37.717 INFO:tasks.workunit.client.0.smithi027.stdout:3/860: link d6/dd3/d34/d2e/d9d/da3/f89 d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d108/f121 0 2024-04-26T23:01:37.720 INFO:tasks.workunit.client.0.smithi027.stdout:3/861: mknod d6/dd3/d34/d2e/c122 0 2024-04-26T23:01:37.720 INFO:tasks.workunit.client.0.smithi027.stdout:3/862: fsync d6/fbc 0 2024-04-26T23:01:37.724 INFO:tasks.workunit.client.0.smithi027.stdout:9/666: sync 2024-04-26T23:01:37.724 INFO:tasks.workunit.client.0.smithi027.stdout:2/689: sync 2024-04-26T23:01:37.726 INFO:tasks.workunit.client.0.smithi027.stdout:2/690: creat d2/d5/db/d1b/fd1 x:0 0 0 2024-04-26T23:01:37.727 INFO:tasks.workunit.client.0.smithi027.stdout:9/667: creat d4/d17/d2e/d81/da7/db8/feb x:0 0 0 2024-04-26T23:01:37.727 INFO:tasks.workunit.client.0.smithi027.stdout:9/668: chown d4/d77/da2/daf/fcb 450481084 1 2024-04-26T23:01:37.727 INFO:tasks.workunit.client.0.smithi027.stdout:2/691: fsync d2/d5/db/d1b/d1c/d25/d19/f45 0 2024-04-26T23:01:37.728 INFO:tasks.workunit.client.0.smithi027.stdout:7/811: dread d0/d28/d2b/fce [0,4194304] 0 2024-04-26T23:01:37.728 INFO:tasks.workunit.client.0.smithi027.stdout:7/812: chown d0/d28/d2b/d3a/d90/dcb 31648033 1 2024-04-26T23:01:37.728 INFO:tasks.workunit.client.0.smithi027.stdout:7/813: write d0/d28/f37 [2496168,25672] 0 2024-04-26T23:01:37.728 INFO:tasks.workunit.client.0.smithi027.stdout:7/814: fsync d0/d28/d2b/f9b 0 2024-04-26T23:01:37.728 INFO:tasks.workunit.client.0.smithi027.stdout:7/815: truncate d0/d28/d2b/d55/f7d 535329 0 2024-04-26T23:01:37.728 INFO:tasks.workunit.client.0.smithi027.stdout:7/816: write d0/d28/d9f/fa1 [887324,87573] 0 2024-04-26T23:01:37.728 INFO:tasks.workunit.client.0.smithi027.stdout:9/669: mknod d4/d10/d40/d58/d75/d84/dc4/cec 0 2024-04-26T23:01:37.729 INFO:tasks.workunit.client.0.smithi027.stdout:1/883: dread d1/d2a/fac [0,4194304] 0 2024-04-26T23:01:37.732 INFO:tasks.workunit.client.0.smithi027.stdout:2/692: chown d2/fa3 3827783 1 2024-04-26T23:01:37.732 INFO:tasks.workunit.client.0.smithi027.stdout:7/817: mkdir d0/d28/d2b/d3a/d107 0 2024-04-26T23:01:37.732 INFO:tasks.workunit.client.0.smithi027.stdout:5/770: dread d4/d24/d5f/dac/d40/fae [0,4194304] 0 2024-04-26T23:01:37.733 INFO:tasks.workunit.client.0.smithi027.stdout:1/884: mknod d1/d26/d66/d73/dae/c122 0 2024-04-26T23:01:37.733 INFO:tasks.workunit.client.0.smithi027.stdout:1/885: chown d1/d26/d66/d73/dae/c122 121290 1 2024-04-26T23:01:37.733 INFO:tasks.workunit.client.0.smithi027.stdout:1/886: fsync d1/d26/dc8/ff5 0 2024-04-26T23:01:37.734 INFO:tasks.workunit.client.0.smithi027.stdout:2/693: mkdir d2/d5/db/de/d21/dc4/d39/dd2 0 2024-04-26T23:01:37.735 INFO:tasks.workunit.client.0.smithi027.stdout:2/694: creat d2/d5/db/d1b/d1c/d25/d19/d23/fd3 x:0 0 0 2024-04-26T23:01:37.736 INFO:tasks.workunit.client.0.smithi027.stdout:2/695: getdents d2/d5/db/d1b/d1c/d25/d6a/d8a/d8d 0 2024-04-26T23:01:37.737 INFO:tasks.workunit.client.0.smithi027.stdout:2/696: rmdir d2/d5/db/d1b/d1c/d25/d19/d23 39 2024-04-26T23:01:37.743 INFO:tasks.workunit.client.0.smithi027.stdout:3/863: dread d6/dd3/d34/d2e/d70/f7d [0,4194304] 0 2024-04-26T23:01:37.743 INFO:tasks.workunit.client.0.smithi027.stdout:3/864: readlink d6/dd3/d34/d47/l43 0 2024-04-26T23:01:37.745 INFO:tasks.workunit.client.0.smithi027.stdout:3/865: symlink d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d40/l123 0 2024-04-26T23:01:37.746 INFO:tasks.workunit.client.0.smithi027.stdout:3/866: write d6/dd3/d2f/f39 [5024586,111584] 0 2024-04-26T23:01:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:2/697: dread d2/d5/db/de/d21/fa8 [0,4194304] 0 2024-04-26T23:01:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:9/670: dread d4/d17/d2e/d46/da0/d8d/fb5 [0,4194304] 0 2024-04-26T23:01:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:2/698: symlink d2/d5/db/d1b/d1c/d25/dc3/ld4 0 2024-04-26T23:01:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:9/671: fsync d4/d10/d40/f64 0 2024-04-26T23:01:37.767 INFO:tasks.workunit.client.0.smithi027.stdout:2/699: creat d2/d5/db/d1b/d1c/d25/d19/d23/d82/fd5 x:0 0 0 2024-04-26T23:01:37.769 INFO:tasks.workunit.client.0.smithi027.stdout:9/672: getdents d4/d10/d40/d58/d75/d84/dc4 0 2024-04-26T23:01:37.769 INFO:tasks.workunit.client.0.smithi027.stdout:9/673: readlink d4/l36 0 2024-04-26T23:01:37.771 INFO:tasks.workunit.client.0.smithi027.stdout:9/674: mkdir d4/d10/d40/d58/d75/d84/dc4/ded 0 2024-04-26T23:01:37.774 INFO:tasks.workunit.client.0.smithi027.stdout:7/818: dwrite d0/d28/d2b/d3a/d48/fb2 [0,4194304] 0 2024-04-26T23:01:37.774 INFO:tasks.workunit.client.0.smithi027.stdout:7/819: chown d0/d3d/dba 980 1 2024-04-26T23:01:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:1/887: dwrite d1/d26/d6b/d42/d70/fce [0,4194304] 0 2024-04-26T23:01:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:1/888: fdatasync d1/f3 0 2024-04-26T23:01:37.780 INFO:tasks.workunit.client.0.smithi027.stdout:5/771: dwrite d4/d24/d22/d29/d39/d61/fde [0,4194304] 0 2024-04-26T23:01:37.780 INFO:tasks.workunit.client.0.smithi027.stdout:5/772: dread - d4/d24/d22/ff2 zero size 2024-04-26T23:01:37.786 INFO:tasks.workunit.client.0.smithi027.stdout:9/675: mknod d4/d77/da2/cee 0 2024-04-26T23:01:37.789 INFO:tasks.workunit.client.0.smithi027.stdout:7/820: rename d0/d28/d2b/d3a/d48/d56/db6/ff0 to d0/d28/d9f/f108 0 2024-04-26T23:01:37.791 INFO:tasks.workunit.client.0.smithi027.stdout:1/889: getdents d1/d5/d40/d102 0 2024-04-26T23:01:37.795 INFO:tasks.workunit.client.0.smithi027.stdout:5/773: write d4/d24/d22/d29/d39/f6d [1831840,75437] 0 2024-04-26T23:01:37.796 INFO:tasks.workunit.client.0.smithi027.stdout:9/676: rename d4/d17/d2e/cdd to d4/d10/d40/d58/cef 0 2024-04-26T23:01:37.798 INFO:tasks.workunit.client.0.smithi027.stdout:1/890: mkdir d1/d5/d40/d102/d123 0 2024-04-26T23:01:37.801 INFO:tasks.workunit.client.0.smithi027.stdout:5/774: mkdir d4/d1a/d2b/dd3/dfa 0 2024-04-26T23:01:37.801 INFO:tasks.workunit.client.0.smithi027.stdout:9/677: rmdir d4/d17/d2e/dcf 39 2024-04-26T23:01:37.801 INFO:tasks.workunit.client.0.smithi027.stdout:9/678: dread - d4/d10/d40/d58/d75/d84/d86/dc5/fe8 zero size 2024-04-26T23:01:37.802 INFO:tasks.workunit.client.0.smithi027.stdout:7/821: read d0/d3d/f44 [1125778,33732] 0 2024-04-26T23:01:37.802 INFO:tasks.workunit.client.0.smithi027.stdout:1/891: symlink d1/d26/d66/dab/de0/l124 0 2024-04-26T23:01:37.802 INFO:tasks.workunit.client.0.smithi027.stdout:1/892: write d1/d26/d6b/d42/d79/fff [487671,3530] 0 2024-04-26T23:01:37.803 INFO:tasks.workunit.client.0.smithi027.stdout:5/775: rename d4/d24/d5f/dac/d40/fae to d4/d24/d22/dab/ffb 0 2024-04-26T23:01:37.805 INFO:tasks.workunit.client.0.smithi027.stdout:5/776: creat d4/d24/d22/dab/d16/ffc x:0 0 0 2024-04-26T23:01:37.806 INFO:tasks.workunit.client.0.smithi027.stdout:5/777: truncate d4/d24/f98 842927 0 2024-04-26T23:01:37.807 INFO:tasks.workunit.client.0.smithi027.stdout:5/778: mkdir d4/d24/dfd 0 2024-04-26T23:01:37.808 INFO:tasks.workunit.client.0.smithi027.stdout:5/779: getdents d4/d1a/dcb 0 2024-04-26T23:01:37.840 INFO:tasks.workunit.client.0.smithi027.stdout:9/679: dwrite d4/d77/da2/daf/fbb [0,4194304] 0 2024-04-26T23:01:37.841 INFO:tasks.workunit.client.0.smithi027.stdout:9/680: write d4/d10/d40/f5d [2972395,6646] 0 2024-04-26T23:01:37.850 INFO:tasks.workunit.client.0.smithi027.stdout:9/681: dread d4/d10/d40/d58/d75/f47 [0,4194304] 0 2024-04-26T23:01:37.852 INFO:tasks.workunit.client.0.smithi027.stdout:9/682: symlink d4/d10/d40/d58/lf0 0 2024-04-26T23:01:37.852 INFO:tasks.workunit.client.0.smithi027.stdout:9/683: fsync d4/d17/d2e/f53 0 2024-04-26T23:01:37.853 INFO:tasks.workunit.client.0.smithi027.stdout:7/822: dwrite d0/d28/d2b/d3a/d48/d56/db6/fdc [0,4194304] 0 2024-04-26T23:01:37.854 INFO:tasks.workunit.client.0.smithi027.stdout:2/700: dwrite d2/d5/db/d1b/f38 [4194304,4194304] 0 2024-04-26T23:01:37.854 INFO:tasks.workunit.client.0.smithi027.stdout:9/684: rename d4/d10/lc7 to d4/d17/d2e/d46/lf1 0 2024-04-26T23:01:37.855 INFO:tasks.workunit.client.0.smithi027.stdout:7/823: symlink d0/d28/d2b/d3a/d48/d56/dee/d100/l109 0 2024-04-26T23:01:37.857 INFO:tasks.workunit.client.0.smithi027.stdout:2/701: rename d2/d5/db/d1b/d1c/d25/d6a/l8e to d2/db9/ld6 0 2024-04-26T23:01:37.858 INFO:tasks.workunit.client.0.smithi027.stdout:9/685: mkdir d4/d17/d2e/d46/da0/d8d/df2 0 2024-04-26T23:01:37.860 INFO:tasks.workunit.client.0.smithi027.stdout:2/702: creat d2/d5/db/de/d21/fd7 x:0 0 0 2024-04-26T23:01:37.861 INFO:tasks.workunit.client.0.smithi027.stdout:9/686: mkdir d4/d10/d25/dc9/df3 0 2024-04-26T23:01:37.862 INFO:tasks.workunit.client.0.smithi027.stdout:9/687: fdatasync d4/d10/d40/d58/d75/d84/dc4/fd7 0 2024-04-26T23:01:37.866 INFO:tasks.workunit.client.0.smithi027.stdout:2/703: creat d2/d5/db/d1b/d1c/d25/d19/d23/fd8 x:0 0 0 2024-04-26T23:01:37.867 INFO:tasks.workunit.client.0.smithi027.stdout:2/704: fsync d2/d5/db/d1b/d1c/d25/d6a/f4c 0 2024-04-26T23:01:37.867 INFO:tasks.workunit.client.0.smithi027.stdout:2/705: write d2/d5/db/d1b/f38 [2734768,69724] 0 2024-04-26T23:01:37.873 INFO:tasks.workunit.client.0.smithi027.stdout:2/706: dread d2/d5/db/de/d21/dc4/d39/f96 [0,4194304] 0 2024-04-26T23:01:37.873 INFO:tasks.workunit.client.0.smithi027.stdout:2/707: chown d2/d5/db/d1b/d1c/d25/d19/d44/fc5 13 1 2024-04-26T23:01:37.874 INFO:tasks.workunit.client.0.smithi027.stdout:2/708: symlink d2/d5/db/de/d21/d8f/d88/ld9 0 2024-04-26T23:01:37.875 INFO:tasks.workunit.client.0.smithi027.stdout:1/893: dwrite d1/f94 [0,4194304] 0 2024-04-26T23:01:37.875 INFO:tasks.workunit.client.0.smithi027.stdout:1/894: fsync d1/d5/f19 0 2024-04-26T23:01:37.877 INFO:tasks.workunit.client.0.smithi027.stdout:2/709: symlink d2/d5/db/de/d21/dc4/d39/dd2/lda 0 2024-04-26T23:01:37.877 INFO:tasks.workunit.client.0.smithi027.stdout:1/895: creat d1/d2a/f125 x:0 0 0 2024-04-26T23:01:37.878 INFO:tasks.workunit.client.0.smithi027.stdout:2/710: mknod d2/db9/cdb 0 2024-04-26T23:01:37.893 INFO:tasks.workunit.client.0.smithi027.stdout:4/832: sync 2024-04-26T23:01:37.894 INFO:tasks.workunit.client.0.smithi027.stdout:8/705: sync 2024-04-26T23:01:37.894 INFO:tasks.workunit.client.0.smithi027.stdout:8/706: fsync d9/da/d2a/d5e/d72/d84/db8/fbe 0 2024-04-26T23:01:37.896 INFO:tasks.workunit.client.0.smithi027.stdout:7/824: dwrite d0/d28/d2b/d3a/d48/d56/dee/d100/ff1 [0,4194304] 0 2024-04-26T23:01:37.897 INFO:tasks.workunit.client.0.smithi027.stdout:8/707: truncate d9/da/d2a/f48 967969 0 2024-04-26T23:01:37.898 INFO:tasks.workunit.client.0.smithi027.stdout:8/708: creat d9/da/d2a/d5e/d9e/dba/ffa x:0 0 0 2024-04-26T23:01:37.898 INFO:tasks.workunit.client.0.smithi027.stdout:8/709: fdatasync d9/da/f13 0 2024-04-26T23:01:37.899 INFO:tasks.workunit.client.0.smithi027.stdout:4/833: creat da/dd/de/db8/f107 x:0 0 0 2024-04-26T23:01:37.899 INFO:tasks.workunit.client.0.smithi027.stdout:8/710: mknod d9/da/d2a/d5e/cfb 0 2024-04-26T23:01:37.902 INFO:tasks.workunit.client.0.smithi027.stdout:7/825: getdents d0/d28/d2b/d3a/d48/d56/dee 0 2024-04-26T23:01:37.902 INFO:tasks.workunit.client.0.smithi027.stdout:7/826: chown d0/d28/d2b/d3a/d48/d56/d88/f101 351693758 1 2024-04-26T23:01:37.902 INFO:tasks.workunit.client.0.smithi027.stdout:7/827: chown d0/d28/d66 2 1 2024-04-26T23:01:37.902 INFO:tasks.workunit.client.0.smithi027.stdout:7/828: read - d0/fe8 zero size 2024-04-26T23:01:37.902 INFO:tasks.workunit.client.0.smithi027.stdout:4/834: symlink da/dd/de/d14/d21/d67/da6/l108 0 2024-04-26T23:01:37.906 INFO:tasks.workunit.client.0.smithi027.stdout:2/711: dread d2/d5/fa [4194304,4194304] 0 2024-04-26T23:01:37.907 INFO:tasks.workunit.client.0.smithi027.stdout:9/688: dwrite d4/d10/d40/d58/d75/d84/d86/f94 [0,4194304] 0 2024-04-26T23:01:37.908 INFO:tasks.workunit.client.0.smithi027.stdout:4/835: creat da/dd/de/d14/d21/d67/da6/f109 x:0 0 0 2024-04-26T23:01:37.908 INFO:tasks.workunit.client.0.smithi027.stdout:4/836: stat da/dd/de/d14/db3/fc6 0 2024-04-26T23:01:37.909 INFO:tasks.workunit.client.0.smithi027.stdout:2/712: creat d2/fdc x:0 0 0 2024-04-26T23:01:37.909 INFO:tasks.workunit.client.0.smithi027.stdout:2/713: dread - d2/db9/dc6/fce zero size 2024-04-26T23:01:37.911 INFO:tasks.workunit.client.0.smithi027.stdout:2/714: mknod d2/db9/cdd 0 2024-04-26T23:01:37.915 INFO:tasks.workunit.client.0.smithi027.stdout:7/829: dread d0/d28/d2b/d3a/d48/d56/d88/d73/dd6/fea [0,4194304] 0 2024-04-26T23:01:37.921 INFO:tasks.workunit.client.0.smithi027.stdout:0/719: sync 2024-04-26T23:01:37.922 INFO:tasks.workunit.client.0.smithi027.stdout:7/830: dread d0/d28/d2b/d3a/d48/d56/f74 [4194304,4194304] 0 2024-04-26T23:01:37.923 INFO:tasks.workunit.client.0.smithi027.stdout:0/720: rename d0/d4/da/d1a/d2a/c86 to d0/d4/da/d1a/cda 0 2024-04-26T23:01:37.925 INFO:tasks.workunit.client.0.smithi027.stdout:0/721: symlink d0/d4/da/d1a/d59/ldb 0 2024-04-26T23:01:37.925 INFO:tasks.workunit.client.0.smithi027.stdout:1/896: dwrite d1/d26/d36/d56/f6e [0,4194304] 0 2024-04-26T23:01:37.926 INFO:tasks.workunit.client.0.smithi027.stdout:1/897: write d1/d26/d66/d73/dae/fe8 [322026,59167] 0 2024-04-26T23:01:37.928 INFO:tasks.workunit.client.0.smithi027.stdout:8/711: dread d9/de/d22/f4d [0,4194304] 0 2024-04-26T23:01:37.929 INFO:tasks.workunit.client.0.smithi027.stdout:1/898: fdatasync d1/d26/d6b/f4d 0 2024-04-26T23:01:37.935 INFO:tasks.workunit.client.0.smithi027.stdout:5/780: sync 2024-04-26T23:01:37.935 INFO:tasks.workunit.client.0.smithi027.stdout:6/658: sync 2024-04-26T23:01:37.935 INFO:tasks.workunit.client.0.smithi027.stdout:6/659: chown d1/df/d43/d56/d6b/dc4 484453 1 2024-04-26T23:01:37.935 INFO:tasks.workunit.client.0.smithi027.stdout:9/689: write d4/f24 [3233006,74675] 0 2024-04-26T23:01:37.935 INFO:tasks.workunit.client.0.smithi027.stdout:3/867: sync 2024-04-26T23:01:37.935 INFO:tasks.workunit.client.0.smithi027.stdout:9/690: fdatasync d4/d17/d2e/d46/fd3 0 2024-04-26T23:01:37.935 INFO:tasks.workunit.client.0.smithi027.stdout:3/868: write d6/dd3/d34/d2e/d5f/d90/ddb/dce/ffe [1503283,28734] 0 2024-04-26T23:01:37.938 INFO:tasks.workunit.client.0.smithi027.stdout:9/691: write d4/d77/da2/fa4 [5480665,76578] 0 2024-04-26T23:01:37.938 INFO:tasks.workunit.client.0.smithi027.stdout:9/692: write d4/d17/d2e/d46/da0/fc3 [1020162,46366] 0 2024-04-26T23:01:37.939 INFO:tasks.workunit.client.0.smithi027.stdout:3/869: unlink d6/dd3/d34/d47/f3c 0 2024-04-26T23:01:37.939 INFO:tasks.workunit.client.0.smithi027.stdout:3/870: fsync d6/dd3/d34/db3/df8/f9a 0 2024-04-26T23:01:37.940 INFO:tasks.workunit.client.0.smithi027.stdout:9/693: symlink d4/d10/d40/d58/lf4 0 2024-04-26T23:01:37.946 INFO:tasks.workunit.client.0.smithi027.stdout:3/871: creat d6/dd3/d34/d2e/d9d/f124 x:0 0 0 2024-04-26T23:01:37.947 INFO:tasks.workunit.client.0.smithi027.stdout:0/722: dread d0/df/d3f/d64/f70 [0,4194304] 0 2024-04-26T23:01:37.947 INFO:tasks.workunit.client.0.smithi027.stdout:0/723: stat d0/d4/da/d1a/d6d/c85 0 2024-04-26T23:01:37.949 INFO:tasks.workunit.client.0.smithi027.stdout:2/715: write d2/d5/db/d1b/d1c/f20 [5704656,127655] 0 2024-04-26T23:01:37.949 INFO:tasks.workunit.client.0.smithi027.stdout:0/724: mkdir d0/d4/da/d15/ddc 0 2024-04-26T23:01:37.950 INFO:tasks.workunit.client.0.smithi027.stdout:7/831: dread d0/d28/d66/f70 [4194304,4194304] 0 2024-04-26T23:01:37.950 INFO:tasks.workunit.client.0.smithi027.stdout:7/832: fsync d0/d3d/f43 0 2024-04-26T23:01:37.954 INFO:tasks.workunit.client.0.smithi027.stdout:7/833: mkdir d0/d3d/d10a 0 2024-04-26T23:01:37.963 INFO:tasks.workunit.client.0.smithi027.stdout:1/899: dread d1/f32 [0,4194304] 0 2024-04-26T23:01:37.980 INFO:tasks.workunit.client.0.smithi027.stdout:6/660: dread d1/df/f31 [0,4194304] 0 2024-04-26T23:01:37.980 INFO:tasks.workunit.client.0.smithi027.stdout:6/661: truncate d1/df/f5f 1085094 0 2024-04-26T23:01:37.983 INFO:tasks.workunit.client.0.smithi027.stdout:1/900: creat d1/d26/d7d/f126 x:0 0 0 2024-04-26T23:01:37.985 INFO:tasks.workunit.client.0.smithi027.stdout:1/901: truncate d1/d26/d6b/d42/d70/fc7 3478492 0 2024-04-26T23:01:37.987 INFO:tasks.workunit.client.0.smithi027.stdout:9/694: dread d4/d17/d2e/d46/da0/d72/f79 [0,4194304] 0 2024-04-26T23:01:37.989 INFO:tasks.workunit.client.0.smithi027.stdout:9/695: creat d4/d10/d40/d58/d75/d84/dc4/ded/ff5 x:0 0 0 2024-04-26T23:01:37.989 INFO:tasks.workunit.client.0.smithi027.stdout:9/696: dread - d4/d77/da2/daf/fcb zero size 2024-04-26T23:01:37.989 INFO:tasks.workunit.client.0.smithi027.stdout:9/697: dread - d4/d10/d40/d58/d75/d84/dc4/fd7 zero size 2024-04-26T23:01:37.996 INFO:tasks.workunit.client.0.smithi027.stdout:8/712: dwrite d9/da/d33/fef [0,4194304] 0 2024-04-26T23:01:37.996 INFO:tasks.workunit.client.0.smithi027.stdout:8/713: write d9/da/d33/f88 [3864602,82032] 0 2024-04-26T23:01:37.996 INFO:tasks.workunit.client.0.smithi027.stdout:5/781: dread d4/d24/d22/dab/f23 [0,4194304] 0 2024-04-26T23:01:37.997 INFO:tasks.workunit.client.0.smithi027.stdout:9/698: creat d4/d17/ff6 x:0 0 0 2024-04-26T23:01:37.997 INFO:tasks.workunit.client.0.smithi027.stdout:6/662: read d1/df/d49/f4f [1613152,109914] 0 2024-04-26T23:01:38.000 INFO:tasks.workunit.client.0.smithi027.stdout:5/782: creat d4/d9f/dc5/ffe x:0 0 0 2024-04-26T23:01:38.004 INFO:tasks.workunit.client.0.smithi027.stdout:9/699: rename d4/d77/c97 to d4/cf7 0 2024-04-26T23:01:38.004 INFO:tasks.workunit.client.0.smithi027.stdout:6/663: creat d1/de/d57/d8e/dd0/dc9/fd6 x:0 0 0 2024-04-26T23:01:38.004 INFO:tasks.workunit.client.0.smithi027.stdout:6/664: readlink d1/lb1 0 2024-04-26T23:01:38.005 INFO:tasks.workunit.client.0.smithi027.stdout:9/700: symlink d4/d17/d2e/dcf/lf8 0 2024-04-26T23:01:38.005 INFO:tasks.workunit.client.0.smithi027.stdout:2/716: write d2/d5/db/d1b/d1c/d25/f59 [456715,86787] 0 2024-04-26T23:01:38.006 INFO:tasks.workunit.client.0.smithi027.stdout:3/872: dwrite d6/dd3/d34/f58 [0,4194304] 0 2024-04-26T23:01:38.006 INFO:tasks.workunit.client.0.smithi027.stdout:3/873: stat d6/dd3/d34/d2e/c3f 0 2024-04-26T23:01:38.008 INFO:tasks.workunit.client.0.smithi027.stdout:9/701: rename d4/d17/f45 to d4/d10/d40/d58/d75/ff9 0 2024-04-26T23:01:38.010 INFO:tasks.workunit.client.0.smithi027.stdout:3/874: getdents d6/dd3/d34/d2e/d5f/d90/ddb/dce 0 2024-04-26T23:01:38.011 INFO:tasks.workunit.client.0.smithi027.stdout:9/702: mknod d4/d10/d40/d58/d75/cfa 0 2024-04-26T23:01:38.016 INFO:tasks.workunit.client.0.smithi027.stdout:3/875: mkdir d6/dd3/d34/db3/d125 0 2024-04-26T23:01:38.016 INFO:tasks.workunit.client.0.smithi027.stdout:2/717: mknod d2/d5/db/d1b/d1c/cde 0 2024-04-26T23:01:38.016 INFO:tasks.workunit.client.0.smithi027.stdout:2/718: creat d2/db9/fdf x:0 0 0 2024-04-26T23:01:38.018 INFO:tasks.workunit.client.0.smithi027.stdout:8/714: dread d9/da/d2a/d5e/d86/f8c [0,4194304] 0 2024-04-26T23:01:38.019 INFO:tasks.workunit.client.0.smithi027.stdout:2/719: rename d2/d5/db/de/cb4 to d2/d5/db/d1b/ce0 0 2024-04-26T23:01:38.021 INFO:tasks.workunit.client.0.smithi027.stdout:2/720: creat d2/db9/fe1 x:0 0 0 2024-04-26T23:01:38.023 INFO:tasks.workunit.client.0.smithi027.stdout:1/902: write d1/d2a/f49 [3448886,2124] 0 2024-04-26T23:01:38.024 INFO:tasks.workunit.client.0.smithi027.stdout:2/721: creat d2/d5/db/d1b/d1c/d25/d19/db0/fe2 x:0 0 0 2024-04-26T23:01:38.024 INFO:tasks.workunit.client.0.smithi027.stdout:2/722: write d2/d5/db/d1b/d1c/d25/fa4 [2628,97172] 0 2024-04-26T23:01:38.025 INFO:tasks.workunit.client.0.smithi027.stdout:3/876: dread d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/dd6/ff6 [0,4194304] 0 2024-04-26T23:01:38.026 INFO:tasks.workunit.client.0.smithi027.stdout:0/725: dwrite d0/d4/fbe [0,4194304] 0 2024-04-26T23:01:38.026 INFO:tasks.workunit.client.0.smithi027.stdout:0/726: chown d0/d4/da/d1a/d6d 41 1 2024-04-26T23:01:38.028 INFO:tasks.workunit.client.0.smithi027.stdout:1/903: rename d1/l3a to d1/d5/d40/d102/d123/l127 0 2024-04-26T23:01:38.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/904: write d1/d2a/fbc [1468545,67772] 0 2024-04-26T23:01:38.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/905: fsync d1/f3 0 2024-04-26T23:01:38.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/906: chown d1/d26/d6b/d42/d70/cdd 50 1 2024-04-26T23:01:38.032 INFO:tasks.workunit.client.0.smithi027.stdout:2/723: link d2/d5/db/d1b/d1c/d25/d6a/d8a/l9b d2/db9/le3 0 2024-04-26T23:01:38.033 INFO:tasks.workunit.client.0.smithi027.stdout:6/665: read d1/de/f13 [2389258,42687] 0 2024-04-26T23:01:38.034 INFO:tasks.workunit.client.0.smithi027.stdout:0/727: mknod d0/d4/dd3/cdd 0 2024-04-26T23:01:38.037 INFO:tasks.workunit.client.0.smithi027.stdout:2/724: symlink d2/d5/db/de/le4 0 2024-04-26T23:01:38.039 INFO:tasks.workunit.client.0.smithi027.stdout:0/728: mknod d0/df/d9d/cde 0 2024-04-26T23:01:38.041 INFO:tasks.workunit.client.0.smithi027.stdout:1/907: rmdir d1 39 2024-04-26T23:01:38.042 INFO:tasks.workunit.client.0.smithi027.stdout:5/783: dread d4/f9 [0,4194304] 0 2024-04-26T23:01:38.045 INFO:tasks.workunit.client.0.smithi027.stdout:2/725: getdents d2/d5/db/d1b/d1c/d25/d19/d23/d7c 0 2024-04-26T23:01:38.049 INFO:tasks.workunit.client.0.smithi027.stdout:5/784: symlink d4/d24/d22/d29/d2a/dee/lff 0 2024-04-26T23:01:38.058 INFO:tasks.workunit.client.0.smithi027.stdout:3/877: read d6/dd3/d34/d2e/d9d/da7/fbe [3112087,61980] 0 2024-04-26T23:01:38.060 INFO:tasks.workunit.client.0.smithi027.stdout:3/878: creat d6/dd3/d34/d2e/d70/d71/f126 x:0 0 0 2024-04-26T23:01:38.062 INFO:tasks.workunit.client.0.smithi027.stdout:6/666: dread d1/df/d43/da8/d84/fb8 [0,4194304] 0 2024-04-26T23:01:38.062 INFO:tasks.workunit.client.0.smithi027.stdout:6/667: fdatasync d1/de/f1e 0 2024-04-26T23:01:38.064 INFO:tasks.workunit.client.0.smithi027.stdout:6/668: creat d1/df/d43/d56/d6b/fd7 x:0 0 0 2024-04-26T23:01:38.064 INFO:tasks.workunit.client.0.smithi027.stdout:6/669: write d1/df/d3c/d9a/fcf [467403,32305] 0 2024-04-26T23:01:38.065 INFO:tasks.workunit.client.0.smithi027.stdout:6/670: write d1/f59 [1106442,28240] 0 2024-04-26T23:01:38.077 INFO:tasks.workunit.client.0.smithi027.stdout:0/729: dread d0/d4/f53 [0,4194304] 0 2024-04-26T23:01:38.081 INFO:tasks.workunit.client.0.smithi027.stdout:8/715: dread d9/de/d2e/f38 [0,4194304] 0 2024-04-26T23:01:38.081 INFO:tasks.workunit.client.0.smithi027.stdout:9/703: dwrite d4/d17/d2e/d46/da0/d72/f90 [0,4194304] 0 2024-04-26T23:01:38.082 INFO:tasks.workunit.client.0.smithi027.stdout:8/716: creat d9/da/d2a/d5e/d72/d84/d5d/d6c/ffc x:0 0 0 2024-04-26T23:01:38.083 INFO:tasks.workunit.client.0.smithi027.stdout:8/717: symlink d9/da/d2a/d5e/d92/lfd 0 2024-04-26T23:01:38.083 INFO:tasks.workunit.client.0.smithi027.stdout:8/718: chown d9/de/d22/fe3 27964737 1 2024-04-26T23:01:38.084 INFO:tasks.workunit.client.0.smithi027.stdout:8/719: chown d9/da/d2a/d5e/d46/ddd 79787 1 2024-04-26T23:01:38.086 INFO:tasks.workunit.client.0.smithi027.stdout:0/730: write d0/d4/f20 [4492142,35181] 0 2024-04-26T23:01:38.087 INFO:tasks.workunit.client.0.smithi027.stdout:9/704: creat d4/d17/d2e/d46/da0/d72/ffb x:0 0 0 2024-04-26T23:01:38.087 INFO:tasks.workunit.client.0.smithi027.stdout:9/705: chown d4/d10/d40/d58/f9c 0 1 2024-04-26T23:01:38.087 INFO:tasks.workunit.client.0.smithi027.stdout:9/706: read d4/d10/f7e [833376,19392] 0 2024-04-26T23:01:38.088 INFO:tasks.workunit.client.0.smithi027.stdout:0/731: truncate d0/d4/da/d15/d26/f79 487091 0 2024-04-26T23:01:38.088 INFO:tasks.workunit.client.0.smithi027.stdout:0/732: fdatasync d0/d4/f53 0 2024-04-26T23:01:38.088 INFO:tasks.workunit.client.0.smithi027.stdout:5/785: dread d4/d24/d22/fa3 [0,4194304] 0 2024-04-26T23:01:38.089 INFO:tasks.workunit.client.0.smithi027.stdout:9/707: creat d4/d17/d2e/d46/da0/d8d/df2/ffc x:0 0 0 2024-04-26T23:01:38.089 INFO:tasks.workunit.client.0.smithi027.stdout:9/708: dread - d4/d17/ff6 zero size 2024-04-26T23:01:38.089 INFO:tasks.workunit.client.0.smithi027.stdout:9/709: readlink d4/d17/d2e/d81/da7/lac 0 2024-04-26T23:01:38.095 INFO:tasks.workunit.client.0.smithi027.stdout:5/786: mknod d4/d24/d22/c100 0 2024-04-26T23:01:38.096 INFO:tasks.workunit.client.0.smithi027.stdout:8/720: dread d9/da/d2a/d5e/d72/d84/db8/f5f [0,4194304] 0 2024-04-26T23:01:38.099 INFO:tasks.workunit.client.0.smithi027.stdout:8/721: mkdir d9/da/d2a/d5e/d72/d84/d35/d8d/da3/dfe 0 2024-04-26T23:01:38.103 INFO:tasks.workunit.client.0.smithi027.stdout:0/733: dread d0/d4/f53 [0,4194304] 0 2024-04-26T23:01:38.116 INFO:tasks.workunit.client.0.smithi027.stdout:1/908: dwrite d1/d26/d66/d73/fe6 [0,4194304] 0 2024-04-26T23:01:38.117 INFO:tasks.workunit.client.0.smithi027.stdout:0/734: symlink d0/df/d3f/d64/da8/ldf 0 2024-04-26T23:01:38.120 INFO:tasks.workunit.client.0.smithi027.stdout:3/879: dwrite d6/f96 [4194304,4194304] 0 2024-04-26T23:01:38.125 INFO:tasks.workunit.client.0.smithi027.stdout:3/880: creat d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/f127 x:0 0 0 2024-04-26T23:01:38.126 INFO:tasks.workunit.client.0.smithi027.stdout:1/909: rename d1/d26/d66/d73/d9e/db6/lbe to d1/d26/d36/d56/l128 0 2024-04-26T23:01:38.127 INFO:tasks.workunit.client.0.smithi027.stdout:1/910: rename d1/d26/d66/c82 to d1/d26/d6b/d42/c129 0 2024-04-26T23:01:38.141 INFO:tasks.workunit.client.0.smithi027.stdout:8/722: dwrite d9/da/d2a/d5e/d46/f6f [0,4194304] 0 2024-04-26T23:01:38.143 INFO:tasks.workunit.client.0.smithi027.stdout:4/837: sync 2024-04-26T23:01:38.145 INFO:tasks.workunit.client.0.smithi027.stdout:4/838: rmdir da/dd/de/d14/d21/d67 39 2024-04-26T23:01:38.147 INFO:tasks.workunit.client.0.smithi027.stdout:4/839: creat da/dd/de/d14/d17/d80/f10a x:0 0 0 2024-04-26T23:01:38.148 INFO:tasks.workunit.client.0.smithi027.stdout:4/840: creat da/dd/de/db8/f10b x:0 0 0 2024-04-26T23:01:38.148 INFO:tasks.workunit.client.0.smithi027.stdout:4/841: write da/d40/fc9 [1768059,122531] 0 2024-04-26T23:01:38.148 INFO:tasks.workunit.client.0.smithi027.stdout:4/842: write da/fda [509496,66751] 0 2024-04-26T23:01:38.149 INFO:tasks.workunit.client.0.smithi027.stdout:4/843: dread - da/dd/d5e/fe5 zero size 2024-04-26T23:01:38.150 INFO:tasks.workunit.client.0.smithi027.stdout:4/844: mknod da/dd/de/d14/d21/d67/c10c 0 2024-04-26T23:01:38.152 INFO:tasks.workunit.client.0.smithi027.stdout:4/845: rmdir da/dd/d5e/de3 39 2024-04-26T23:01:38.159 INFO:tasks.workunit.client.0.smithi027.stdout:3/881: write d6/dd3/d34/d2e/f22 [2319657,19] 0 2024-04-26T23:01:38.161 INFO:tasks.workunit.client.0.smithi027.stdout:3/882: mknod d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/d5a/dcd/c128 0 2024-04-26T23:01:38.161 INFO:tasks.workunit.client.0.smithi027.stdout:5/787: dread d4/d24/f4c [0,4194304] 0 2024-04-26T23:01:38.162 INFO:tasks.workunit.client.0.smithi027.stdout:9/710: dwrite d4/d17/d2e/d46/da0/d72/f90 [0,4194304] 0 2024-04-26T23:01:38.164 INFO:tasks.workunit.client.0.smithi027.stdout:5/788: mkdir d4/d1a/d2b/d101 0 2024-04-26T23:01:38.165 INFO:tasks.workunit.client.0.smithi027.stdout:3/883: mknod d6/dd3/d34/db3/df8/c129 0 2024-04-26T23:01:38.165 INFO:tasks.workunit.client.0.smithi027.stdout:3/884: write d6/dd3/d34/d2e/d5f/d90/db9/f120 [592518,52319] 0 2024-04-26T23:01:38.166 INFO:tasks.workunit.client.0.smithi027.stdout:5/789: write d4/d24/d22/d29/d39/d84/fb3 [866404,42339] 0 2024-04-26T23:01:38.167 INFO:tasks.workunit.client.0.smithi027.stdout:0/735: dwrite d0/d4/da/f19 [0,4194304] 0 2024-04-26T23:01:38.167 INFO:tasks.workunit.client.0.smithi027.stdout:3/885: symlink d6/dd3/d34/d2e/d5f/d90/l12a 0 2024-04-26T23:01:38.170 INFO:tasks.workunit.client.0.smithi027.stdout:5/790: mknod d4/d1a/d2b/d101/c102 0 2024-04-26T23:01:38.174 INFO:tasks.workunit.client.0.smithi027.stdout:1/911: dwrite d1/d26/d86/d5c/fdf [0,4194304] 0 2024-04-26T23:01:38.182 INFO:tasks.workunit.client.0.smithi027.stdout:0/736: mknod d0/df/d9d/d9f/dbb/ce0 0 2024-04-26T23:01:38.182 INFO:tasks.workunit.client.0.smithi027.stdout:5/791: stat d4/d24/d22/d29/d2a/la0 0 2024-04-26T23:01:38.182 INFO:tasks.workunit.client.0.smithi027.stdout:5/792: readlink d4/d24/d5f/dac/l78 0 2024-04-26T23:01:38.182 INFO:tasks.workunit.client.0.smithi027.stdout:5/793: readlink d4/d24/d22/dab/l79 0 2024-04-26T23:01:38.184 INFO:tasks.workunit.client.0.smithi027.stdout:1/912: link d1/d26/d7d/c11a d1/d26/d66/d73/dae/c12a 0 2024-04-26T23:01:38.184 INFO:tasks.workunit.client.0.smithi027.stdout:1/913: dread - d1/d26/d6b/fa1 zero size 2024-04-26T23:01:38.185 INFO:tasks.workunit.client.0.smithi027.stdout:0/737: mkdir d0/d4/da/d1a/de1 0 2024-04-26T23:01:38.185 INFO:tasks.workunit.client.0.smithi027.stdout:0/738: chown d0/d4/dd3 3 1 2024-04-26T23:01:38.187 INFO:tasks.workunit.client.0.smithi027.stdout:1/914: rename d1/d26/d66/d73/dae/caf to d1/d2a/deb/c12b 0 2024-04-26T23:01:38.187 INFO:tasks.workunit.client.0.smithi027.stdout:1/915: chown d1/d5/d40/f52 103 1 2024-04-26T23:01:38.188 INFO:tasks.workunit.client.0.smithi027.stdout:0/739: symlink d0/d4/da/d1a/d1e/le2 0 2024-04-26T23:01:38.191 INFO:tasks.workunit.client.0.smithi027.stdout:1/916: symlink d1/d26/d66/d73/d9e/db6/l12c 0 2024-04-26T23:01:38.191 INFO:tasks.workunit.client.0.smithi027.stdout:8/723: write d9/da/d2a/d5e/d72/d84/d5d/d8b/fbb [1833339,80164] 0 2024-04-26T23:01:38.193 INFO:tasks.workunit.client.0.smithi027.stdout:1/917: truncate d1/d26/d6b/f5a 118679 0 2024-04-26T23:01:38.198 INFO:tasks.workunit.client.0.smithi027.stdout:1/918: creat d1/d26/dc8/f12d x:0 0 0 2024-04-26T23:01:38.213 INFO:tasks.workunit.client.0.smithi027.stdout:1/919: unlink d1/d26/d6b/fad 0 2024-04-26T23:01:38.215 INFO:tasks.workunit.client.0.smithi027.stdout:1/920: creat d1/d26/d66/dab/de0/f12e x:0 0 0 2024-04-26T23:01:38.215 INFO:tasks.workunit.client.0.smithi027.stdout:1/921: stat d1/d26/d86/f9c 0 2024-04-26T23:01:38.217 INFO:tasks.workunit.client.0.smithi027.stdout:1/922: creat d1/d26/d66/f12f x:0 0 0 2024-04-26T23:01:38.219 INFO:tasks.workunit.client.0.smithi027.stdout:1/923: rename d1/l12 to d1/d26/d66/d73/dae/l130 0 2024-04-26T23:01:38.231 INFO:tasks.workunit.client.0.smithi027.stdout:4/846: dwrite da/dd/de/d14/d17/d80/f10a [0,4194304] 0 2024-04-26T23:01:38.236 INFO:tasks.workunit.client.0.smithi027.stdout:9/711: dwrite d4/d17/d2e/d46/da0/f57 [4194304,4194304] 0 2024-04-26T23:01:38.236 INFO:tasks.workunit.client.0.smithi027.stdout:9/712: fsync d4/d10/d40/fab 0 2024-04-26T23:01:38.237 INFO:tasks.workunit.client.0.smithi027.stdout:4/847: truncate da/dd/de/f94 900711 0 2024-04-26T23:01:38.237 INFO:tasks.workunit.client.0.smithi027.stdout:4/848: fdatasync da/dd/de/f3c 0 2024-04-26T23:01:38.237 INFO:tasks.workunit.client.0.smithi027.stdout:4/849: write da/dd/de/d14/d21/d41/d59/f8d [4797517,90170] 0 2024-04-26T23:01:38.243 INFO:tasks.workunit.client.0.smithi027.stdout:9/713: write d4/d10/d40/d58/d6e/f7c [700011,86879] 0 2024-04-26T23:01:38.244 INFO:tasks.workunit.client.0.smithi027.stdout:9/714: creat d4/d17/d2e/d70/ffd x:0 0 0 2024-04-26T23:01:38.249 INFO:tasks.workunit.client.0.smithi027.stdout:9/715: getdents d4/d17/d2e/d46/da0/d8d/df2 0 2024-04-26T23:01:38.249 INFO:tasks.workunit.client.0.smithi027.stdout:9/716: write d4/d17/d2e/d46/da0/d72/ffb [279862,62803] 0 2024-04-26T23:01:38.262 INFO:tasks.workunit.client.0.smithi027.stdout:5/794: dwrite d4/fa5 [0,4194304] 0 2024-04-26T23:01:38.264 INFO:tasks.workunit.client.0.smithi027.stdout:3/886: dwrite d6/dd3/d34/d78/da1/f107 [0,4194304] 0 2024-04-26T23:01:38.265 INFO:tasks.workunit.client.0.smithi027.stdout:8/724: dwrite d9/da/d2a/d5e/d46/ddd/fc6 [0,4194304] 0 2024-04-26T23:01:38.265 INFO:tasks.workunit.client.0.smithi027.stdout:5/795: creat d4/d24/dfd/f103 x:0 0 0 2024-04-26T23:01:38.265 INFO:tasks.workunit.client.0.smithi027.stdout:8/725: chown d9/da/d2a/d5e/d72/d84/d35/d8d/cd5 421 1 2024-04-26T23:01:38.267 INFO:tasks.workunit.client.0.smithi027.stdout:0/740: dwrite d0/d4/da/fb3 [0,4194304] 0 2024-04-26T23:01:38.267 INFO:tasks.workunit.client.0.smithi027.stdout:0/741: truncate d0/df/d3f/d4a/f6c 943876 0 2024-04-26T23:01:38.268 INFO:tasks.workunit.client.0.smithi027.stdout:9/717: dread d4/d17/d2e/d46/da0/d72/f79 [0,4194304] 0 2024-04-26T23:01:38.268 INFO:tasks.workunit.client.0.smithi027.stdout:5/796: mkdir d4/d24/d22/d29/d39/dc7/d104 0 2024-04-26T23:01:38.269 INFO:tasks.workunit.client.0.smithi027.stdout:9/718: creat d4/d17/d2e/d46/dd1/ffe x:0 0 0 2024-04-26T23:01:38.275 INFO:tasks.workunit.client.0.smithi027.stdout:7/834: sync 2024-04-26T23:01:38.275 INFO:tasks.workunit.client.0.smithi027.stdout:8/726: rmdir d9/da/d2a/d6b 39 2024-04-26T23:01:38.275 INFO:tasks.workunit.client.0.smithi027.stdout:8/727: write d9/da/d2a/d5e/d72/d84/d5d/fcb [492895,47326] 0 2024-04-26T23:01:38.275 INFO:tasks.workunit.client.0.smithi027.stdout:5/797: rmdir d4/d1a/d2b 39 2024-04-26T23:01:38.276 INFO:tasks.workunit.client.0.smithi027.stdout:8/728: creat d9/da/d2a/d5e/d72/d84/d5d/d8b/fff x:0 0 0 2024-04-26T23:01:38.276 INFO:tasks.workunit.client.0.smithi027.stdout:8/729: truncate d9/da/d2a/d5e/d92/fb9 942611 0 2024-04-26T23:01:38.276 INFO:tasks.workunit.client.0.smithi027.stdout:5/798: mkdir d4/d9f/d105 0 2024-04-26T23:01:38.276 INFO:tasks.workunit.client.0.smithi027.stdout:5/799: readlink d4/d24/d5f/dac/d40/l4d 0 2024-04-26T23:01:38.276 INFO:tasks.workunit.client.0.smithi027.stdout:5/800: chown d4/d24/d22/fa3 49233705 1 2024-04-26T23:01:38.277 INFO:tasks.workunit.client.0.smithi027.stdout:5/801: dread - d4/d24/d22/dab/d16/d4b/d81/fe3 zero size 2024-04-26T23:01:38.277 INFO:tasks.workunit.client.0.smithi027.stdout:8/730: symlink d9/da/d33/l100 0 2024-04-26T23:01:38.277 INFO:tasks.workunit.client.0.smithi027.stdout:8/731: truncate d9/de/d22/fdb 941896 0 2024-04-26T23:01:38.278 INFO:tasks.workunit.client.0.smithi027.stdout:5/802: creat d4/d9f/dc5/f106 x:0 0 0 2024-04-26T23:01:38.279 INFO:tasks.workunit.client.0.smithi027.stdout:8/732: creat d9/da/da1/f101 x:0 0 0 2024-04-26T23:01:38.284 INFO:tasks.workunit.client.0.smithi027.stdout:3/887: dread d6/dd3/d34/d2e/d9d/da7/dac/d74/f87 [0,4194304] 0 2024-04-26T23:01:38.284 INFO:tasks.workunit.client.0.smithi027.stdout:3/888: stat d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/d5a/f99 0 2024-04-26T23:01:38.285 INFO:tasks.workunit.client.0.smithi027.stdout:8/733: unlink d9/da/d2a/d5e/d72/d84/d5d/fdc 0 2024-04-26T23:01:38.286 INFO:tasks.workunit.client.0.smithi027.stdout:0/742: dread d0/d4/da/d15/d26/f79 [0,4194304] 0 2024-04-26T23:01:38.288 INFO:tasks.workunit.client.0.smithi027.stdout:9/719: dread d4/d17/d2e/d81/fbc [4194304,4194304] 0 2024-04-26T23:01:38.288 INFO:tasks.workunit.client.0.smithi027.stdout:3/889: creat d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/d113/f12b x:0 0 0 2024-04-26T23:01:38.288 INFO:tasks.workunit.client.0.smithi027.stdout:8/734: rmdir d9/da/d2a/d5e/d72/d84/d5d/d6c/dc0 39 2024-04-26T23:01:38.289 INFO:tasks.workunit.client.0.smithi027.stdout:6/671: sync 2024-04-26T23:01:38.289 INFO:tasks.workunit.client.0.smithi027.stdout:2/726: sync 2024-04-26T23:01:38.289 INFO:tasks.workunit.client.0.smithi027.stdout:2/727: fsync d2/d5/db/de/d21/dc4/d39/fbd 0 2024-04-26T23:01:38.294 INFO:tasks.workunit.client.0.smithi027.stdout:3/890: creat d6/f12c x:0 0 0 2024-04-26T23:01:38.295 INFO:tasks.workunit.client.0.smithi027.stdout:8/735: rename d9/da/d2a/d5e/d72/d84/d35/dc2/fee to d9/da/d2a/d5e/d9e/dba/f102 0 2024-04-26T23:01:38.296 INFO:tasks.workunit.client.0.smithi027.stdout:0/743: link d0/d4/c6e d0/d4/da/ce3 0 2024-04-26T23:01:38.296 INFO:tasks.workunit.client.0.smithi027.stdout:8/736: mknod d9/de/c103 0 2024-04-26T23:01:38.297 INFO:tasks.workunit.client.0.smithi027.stdout:4/850: dwrite da/dd/f5d [0,4194304] 0 2024-04-26T23:01:38.297 INFO:tasks.workunit.client.0.smithi027.stdout:4/851: chown da/dd/de/d14/d17/f28 97 1 2024-04-26T23:01:38.301 INFO:tasks.workunit.client.0.smithi027.stdout:0/744: link d0/d4/da/d1a/d1e/ca9 d0/ce4 0 2024-04-26T23:01:38.301 INFO:tasks.workunit.client.0.smithi027.stdout:0/745: chown d0/d4/l27 23533 1 2024-04-26T23:01:38.301 INFO:tasks.workunit.client.0.smithi027.stdout:2/728: getdents d2/d5/db/d1b/d1c/d25/d19/d44 0 2024-04-26T23:01:38.302 INFO:tasks.workunit.client.0.smithi027.stdout:9/720: creat d4/d10/d25/dc9/fff x:0 0 0 2024-04-26T23:01:38.302 INFO:tasks.workunit.client.0.smithi027.stdout:9/721: write d4/d10/d40/d58/d75/f63 [752026,110975] 0 2024-04-26T23:01:38.306 INFO:tasks.workunit.client.0.smithi027.stdout:4/852: mknod da/dd/de/d79/da8/dc5/c10d 0 2024-04-26T23:01:38.306 INFO:tasks.workunit.client.0.smithi027.stdout:4/853: fdatasync da/f1f 0 2024-04-26T23:01:38.306 INFO:tasks.workunit.client.0.smithi027.stdout:0/746: getdents d0/d4/da/d15/d26 0 2024-04-26T23:01:38.307 INFO:tasks.workunit.client.0.smithi027.stdout:2/729: chown d2/d5/db/d1b/d1c/l48 42319634 1 2024-04-26T23:01:38.307 INFO:tasks.workunit.client.0.smithi027.stdout:2/730: chown d2/d5/db/de/d21/dc4/d39/laf 29 1 2024-04-26T23:01:38.307 INFO:tasks.workunit.client.0.smithi027.stdout:2/731: readlink d2/d5/db/de/l84 0 2024-04-26T23:01:38.307 INFO:tasks.workunit.client.0.smithi027.stdout:3/891: dread d6/dd3/f25 [4194304,4194304] 0 2024-04-26T23:01:38.308 INFO:tasks.workunit.client.0.smithi027.stdout:4/854: rename da/dd/de/d14/c6b to da/dd/de/d14/d21/d41/d59/d74/c10e 0 2024-04-26T23:01:38.309 INFO:tasks.workunit.client.0.smithi027.stdout:0/747: mkdir d0/df/d9d/de5 0 2024-04-26T23:01:38.310 INFO:tasks.workunit.client.0.smithi027.stdout:2/732: rename d2/d5/db/d1b/d1c/d25/f4a to d2/d5/db/d1b/d1c/d8b/fe5 0 2024-04-26T23:01:38.310 INFO:tasks.workunit.client.0.smithi027.stdout:2/733: fsync d2/d5/db/d1b/d1c/d25/d19/d23/fd8 0 2024-04-26T23:01:38.311 INFO:tasks.workunit.client.0.smithi027.stdout:4/855: link da/dd/de/d14/f43 da/dd/d5f/dac/f10f 0 2024-04-26T23:01:38.312 INFO:tasks.workunit.client.0.smithi027.stdout:0/748: creat d0/df/d9d/d9f/fe6 x:0 0 0 2024-04-26T23:01:38.314 INFO:tasks.workunit.client.0.smithi027.stdout:2/734: creat d2/d5/db/de/d21/d8f/d88/fe6 x:0 0 0 2024-04-26T23:01:38.314 INFO:tasks.workunit.client.0.smithi027.stdout:2/735: stat d2/d5/db/d1b/d1c/f20 0 2024-04-26T23:01:38.314 INFO:tasks.workunit.client.0.smithi027.stdout:4/856: symlink da/dd/de/d79/dcd/l110 0 2024-04-26T23:01:38.319 INFO:tasks.workunit.client.0.smithi027.stdout:0/749: creat d0/d4/dd3/fe7 x:0 0 0 2024-04-26T23:01:38.319 INFO:tasks.workunit.client.0.smithi027.stdout:6/672: read d1/df/d43/f5d [2507377,108244] 0 2024-04-26T23:01:38.319 INFO:tasks.workunit.client.0.smithi027.stdout:2/736: mkdir d2/d5/db/de/d21/d8f/d88/de7 0 2024-04-26T23:01:38.319 INFO:tasks.workunit.client.0.smithi027.stdout:2/737: fdatasync d2/d5/db/de/d21/d8f/d88/fe6 0 2024-04-26T23:01:38.319 INFO:tasks.workunit.client.0.smithi027.stdout:2/738: readlink d2/d5/db/de/d21/d8f/d88/ld9 0 2024-04-26T23:01:38.319 INFO:tasks.workunit.client.0.smithi027.stdout:4/857: mkdir da/dd/de/d14/d17/d111 0 2024-04-26T23:01:38.322 INFO:tasks.workunit.client.0.smithi027.stdout:8/737: write d9/de/f2c [2510178,7176] 0 2024-04-26T23:01:38.324 INFO:tasks.workunit.client.0.smithi027.stdout:4/858: creat da/dd/de/d79/dcd/f112 x:0 0 0 2024-04-26T23:01:38.324 INFO:tasks.workunit.client.0.smithi027.stdout:3/892: creat d6/dd3/d34/d2e/d50/f12d x:0 0 0 2024-04-26T23:01:38.334 INFO:tasks.workunit.client.0.smithi027.stdout:5/803: dwrite d4/d24/d5f/dac/d40/d54/f75 [0,4194304] 0 2024-04-26T23:01:38.334 INFO:tasks.workunit.client.0.smithi027.stdout:5/804: getdents d4/d24/d55 0 2024-04-26T23:01:38.334 INFO:tasks.workunit.client.0.smithi027.stdout:3/893: unlink d6/dd3/d34/db3/fdf 0 2024-04-26T23:01:38.334 INFO:tasks.workunit.client.0.smithi027.stdout:3/894: mkdir d6/dd3/d34/d2e/d9d/da7/dac/dc0/d12e 0 2024-04-26T23:01:38.334 INFO:tasks.workunit.client.0.smithi027.stdout:3/895: chown d6/dd3/d34/d2e/d9d/da7/dac/dca/cfa 12785820 1 2024-04-26T23:01:38.334 INFO:tasks.workunit.client.0.smithi027.stdout:3/896: chown d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/d5a/dcd/c128 0 1 2024-04-26T23:01:38.338 INFO:tasks.workunit.client.0.smithi027.stdout:7/835: dwrite d0/d28/d2b/d3a/d48/d56/d88/fda [0,4194304] 0 2024-04-26T23:01:38.338 INFO:tasks.workunit.client.0.smithi027.stdout:0/750: write d0/d4/da/fb3 [2957644,57255] 0 2024-04-26T23:01:38.338 INFO:tasks.workunit.client.0.smithi027.stdout:9/722: write d4/d17/d2e/d46/da0/f41 [3438443,22019] 0 2024-04-26T23:01:38.338 INFO:tasks.workunit.client.0.smithi027.stdout:9/723: chown d4/d17/d2e/d46/cba 20344 1 2024-04-26T23:01:38.339 INFO:tasks.workunit.client.0.smithi027.stdout:2/739: dread d2/d5/db/de/d21/dc4/d39/f96 [0,4194304] 0 2024-04-26T23:01:38.346 INFO:tasks.workunit.client.0.smithi027.stdout:7/836: rename d0/d28/d9f/de1 to d0/d3d/dba/df8/d10b 0 2024-04-26T23:01:38.346 INFO:tasks.workunit.client.0.smithi027.stdout:9/724: truncate d4/d17/d2e/fbe 669159 0 2024-04-26T23:01:38.346 INFO:tasks.workunit.client.0.smithi027.stdout:0/751: mkdir d0/d4/da/d1a/d1e/de8 0 2024-04-26T23:01:38.349 INFO:tasks.workunit.client.0.smithi027.stdout:9/725: mknod d4/d17/d2e/dcf/c100 0 2024-04-26T23:01:38.353 INFO:tasks.workunit.client.0.smithi027.stdout:0/752: mkdir d0/df/d3f/d4a/de9 0 2024-04-26T23:01:38.354 INFO:tasks.workunit.client.0.smithi027.stdout:2/740: fsync d2/d5/db/de/d21/d8f/d88/fe6 0 2024-04-26T23:01:38.355 INFO:tasks.workunit.client.0.smithi027.stdout:0/753: rename d0/c4c to d0/d4/da/d1a/d1e/dba/cea 0 2024-04-26T23:01:38.363 INFO:tasks.workunit.client.0.smithi027.stdout:0/754: rmdir d0/d4/da/d15/d55 39 2024-04-26T23:01:38.370 INFO:tasks.workunit.client.0.smithi027.stdout:7/837: write d0/d28/d2b/d55/d59/d6d/fad [2819284,107654] 0 2024-04-26T23:01:38.371 INFO:tasks.workunit.client.0.smithi027.stdout:7/838: write d0/d28/d2b/d55/d59/d6d/fff [742253,44905] 0 2024-04-26T23:01:38.372 INFO:tasks.workunit.client.0.smithi027.stdout:7/839: mknod d0/d3d/c10c 0 2024-04-26T23:01:38.373 INFO:tasks.workunit.client.0.smithi027.stdout:7/840: mknod d0/d3d/dba/df8/d10b/c10d 0 2024-04-26T23:01:38.395 INFO:tasks.workunit.client.0.smithi027.stdout:5/805: dwrite d4/d24/d22/d29/d39/f6d [0,4194304] 0 2024-04-26T23:01:38.395 INFO:tasks.workunit.client.0.smithi027.stdout:5/806: readlink d4/d9f/dc0/ld7 0 2024-04-26T23:01:38.396 INFO:tasks.workunit.client.0.smithi027.stdout:5/807: write d4/d1a/d2b/db8/ff4 [716467,114045] 0 2024-04-26T23:01:38.397 INFO:tasks.workunit.client.0.smithi027.stdout:8/738: dwrite d9/da/d2a/d5e/d72/d84/d5d/d8b/fbb [0,4194304] 0 2024-04-26T23:01:38.399 INFO:tasks.workunit.client.0.smithi027.stdout:8/739: mknod d9/da/d2a/d5e/d2b/c104 0 2024-04-26T23:01:38.400 INFO:tasks.workunit.client.0.smithi027.stdout:8/740: creat d9/da/d33/f105 x:0 0 0 2024-04-26T23:01:38.416 INFO:tasks.workunit.client.0.smithi027.stdout:4/859: dwrite da/dd/de/d14/d17/d80/dab/dd6/feb [0,4194304] 0 2024-04-26T23:01:38.416 INFO:tasks.workunit.client.0.smithi027.stdout:6/673: dwrite d1/d30/d6c/d6f/d7d/f80 [0,4194304] 0 2024-04-26T23:01:38.418 INFO:tasks.workunit.client.0.smithi027.stdout:4/860: creat da/dd/d92/f113 x:0 0 0 2024-04-26T23:01:38.418 INFO:tasks.workunit.client.0.smithi027.stdout:4/861: dread - da/dd/de/d14/d17/d80/fb7 zero size 2024-04-26T23:01:38.419 INFO:tasks.workunit.client.0.smithi027.stdout:8/741: write d9/da/d2a/d5e/d46/f6f [554791,101462] 0 2024-04-26T23:01:38.419 INFO:tasks.workunit.client.0.smithi027.stdout:8/742: fsync d9/da/d2a/d6b/f81 0 2024-04-26T23:01:38.421 INFO:tasks.workunit.client.0.smithi027.stdout:8/743: mkdir d9/de/d106 0 2024-04-26T23:01:38.422 INFO:tasks.workunit.client.0.smithi027.stdout:8/744: symlink d9/da/d2a/d5e/d72/d84/d35/df0/l107 0 2024-04-26T23:01:38.422 INFO:tasks.workunit.client.0.smithi027.stdout:8/745: fdatasync d9/da/d2a/d5e/d72/d84/d5d/d6c/fc4 0 2024-04-26T23:01:38.429 INFO:tasks.workunit.client.0.smithi027.stdout:0/755: dwrite d0/d4/da/d1a/d1e/f91 [0,4194304] 0 2024-04-26T23:01:38.429 INFO:tasks.workunit.client.0.smithi027.stdout:5/808: write d4/d24/d5f/dac/d40/f6c [3934573,68649] 0 2024-04-26T23:01:38.430 INFO:tasks.workunit.client.0.smithi027.stdout:8/746: write d9/de/f14 [721977,120948] 0 2024-04-26T23:01:38.430 INFO:tasks.workunit.client.0.smithi027.stdout:8/747: dread - d9/da/d2a/d5e/d46/fe9 zero size 2024-04-26T23:01:38.430 INFO:tasks.workunit.client.0.smithi027.stdout:8/748: readlink d9/de/d2e/l95 0 2024-04-26T23:01:38.431 INFO:tasks.workunit.client.0.smithi027.stdout:0/756: getdents d0/df/d3f/d64/da8 0 2024-04-26T23:01:38.431 INFO:tasks.workunit.client.0.smithi027.stdout:0/757: fsync d0/d4/f53 0 2024-04-26T23:01:38.433 INFO:tasks.workunit.client.0.smithi027.stdout:5/809: unlink d4/d1a/l7a 0 2024-04-26T23:01:38.435 INFO:tasks.workunit.client.0.smithi027.stdout:0/758: mkdir d0/df/d9d/db4/deb 0 2024-04-26T23:01:38.436 INFO:tasks.workunit.client.0.smithi027.stdout:5/810: truncate d4/d1a/d4a/f76 2673396 0 2024-04-26T23:01:38.438 INFO:tasks.workunit.client.0.smithi027.stdout:9/726: dwrite d4/d10/f7e [0,4194304] 0 2024-04-26T23:01:38.438 INFO:tasks.workunit.client.0.smithi027.stdout:9/727: write d4/d77/da2/daf/fc8 [583622,13079] 0 2024-04-26T23:01:38.444 INFO:tasks.workunit.client.0.smithi027.stdout:9/728: creat d4/d10/d40/d58/f101 x:0 0 0 2024-04-26T23:01:38.444 INFO:tasks.workunit.client.0.smithi027.stdout:9/729: chown d4/d10/c23 526413780 1 2024-04-26T23:01:38.448 INFO:tasks.workunit.client.0.smithi027.stdout:8/749: dread d9/da/d33/f60 [0,4194304] 0 2024-04-26T23:01:38.448 INFO:tasks.workunit.client.0.smithi027.stdout:8/750: write d9/da/d2a/d5e/d46/fe9 [413027,43902] 0 2024-04-26T23:01:38.457 INFO:tasks.workunit.client.0.smithi027.stdout:0/759: dread d0/f12 [0,4194304] 0 2024-04-26T23:01:38.458 INFO:tasks.workunit.client.0.smithi027.stdout:8/751: unlink d9/da/d2a/d5e/d72/d84/cd0 0 2024-04-26T23:01:38.459 INFO:tasks.workunit.client.0.smithi027.stdout:0/760: creat d0/df/d9d/de5/fec x:0 0 0 2024-04-26T23:01:38.459 INFO:tasks.workunit.client.0.smithi027.stdout:0/761: chown d0/d4/cd7 258706500 1 2024-04-26T23:01:38.459 INFO:tasks.workunit.client.0.smithi027.stdout:8/752: truncate d9/da/d2a/f89 21251 0 2024-04-26T23:01:38.459 INFO:tasks.workunit.client.0.smithi027.stdout:8/753: chown d9/da/d33/d71/l85 18237468 1 2024-04-26T23:01:38.460 INFO:tasks.workunit.client.0.smithi027.stdout:8/754: mkdir d9/da/d2a/d5e/d2b/d52/d108 0 2024-04-26T23:01:38.478 INFO:tasks.workunit.client.0.smithi027.stdout:0/762: dread d0/d4/da/f11 [0,4194304] 0 2024-04-26T23:01:38.478 INFO:tasks.workunit.client.0.smithi027.stdout:0/763: dread - d0/df/d9d/d9f/fe6 zero size 2024-04-26T23:01:38.478 INFO:tasks.workunit.client.0.smithi027.stdout:0/764: chown d0/d4/da/d1a/d59/c75 843 1 2024-04-26T23:01:38.478 INFO:tasks.workunit.client.0.smithi027.stdout:0/765: fdatasync d0/d4/dd3/fe7 0 2024-04-26T23:01:38.478 INFO:tasks.workunit.client.0.smithi027.stdout:0/766: fdatasync d0/d4/da/d1a/d1e/dba/fd6 0 2024-04-26T23:01:38.500 INFO:tasks.workunit.client.0.smithi027.stdout:5/811: dwrite d4/d24/d22/d29/d2a/d3b/d50/f70 [8388608,4194304] 0 2024-04-26T23:01:38.502 INFO:tasks.workunit.client.0.smithi027.stdout:0/767: write d0/d4/da/d1a/d2a/f43 [1862828,114621] 0 2024-04-26T23:01:38.504 INFO:tasks.workunit.client.0.smithi027.stdout:5/812: mknod d4/d1a/d4a/c107 0 2024-04-26T23:01:38.510 INFO:tasks.workunit.client.0.smithi027.stdout:9/730: dwrite d4/d10/d40/d58/d75/d84/d86/dc5/fe8 [0,4194304] 0 2024-04-26T23:01:38.511 INFO:tasks.workunit.client.0.smithi027.stdout:8/755: dwrite d9/da/d2a/d5e/d72/d84/f40 [0,4194304] 0 2024-04-26T23:01:38.512 INFO:tasks.workunit.client.0.smithi027.stdout:9/731: mkdir d4/d10/d40/d58/d75/d84/d102 0 2024-04-26T23:01:38.518 INFO:tasks.workunit.client.0.smithi027.stdout:8/756: link d9/de/d22/c36 d9/da/c109 0 2024-04-26T23:01:38.518 INFO:tasks.workunit.client.0.smithi027.stdout:8/757: chown d9/de/d2e 1 1 2024-04-26T23:01:38.518 INFO:tasks.workunit.client.0.smithi027.stdout:8/758: write d9/da/d2a/d5e/d72/d84/d35/d8d/fd6 [590115,53776] 0 2024-04-26T23:01:38.518 INFO:tasks.workunit.client.0.smithi027.stdout:8/759: chown d9/da/d2a/d5e/d2b/lf4 0 1 2024-04-26T23:01:38.521 INFO:tasks.workunit.client.0.smithi027.stdout:8/760: creat d9/da/d2a/d5e/d72/d84/d35/d8d/da3/f10a x:0 0 0 2024-04-26T23:01:38.528 INFO:tasks.workunit.client.0.smithi027.stdout:5/813: dread d4/d24/d5f/dac/d40/f97 [0,4194304] 0 2024-04-26T23:01:38.529 INFO:tasks.workunit.client.0.smithi027.stdout:5/814: symlink d4/d24/d22/d29/d39/dc7/d104/l108 0 2024-04-26T23:01:38.530 INFO:tasks.workunit.client.0.smithi027.stdout:1/924: sync 2024-04-26T23:01:38.533 INFO:tasks.workunit.client.0.smithi027.stdout:5/815: symlink d4/d1a/l109 0 2024-04-26T23:01:38.543 INFO:tasks.workunit.client.0.smithi027.stdout:5/816: dread d4/d24/d22/d29/d2a/d3b/fb7 [0,4194304] 0 2024-04-26T23:01:38.544 INFO:tasks.workunit.client.0.smithi027.stdout:0/768: dwrite d0/d4/da/d1a/d2a/fcc [0,4194304] 0 2024-04-26T23:01:38.546 INFO:tasks.workunit.client.0.smithi027.stdout:9/732: dwrite d4/d17/d2e/d46/da0/fc3 [0,4194304] 0 2024-04-26T23:01:38.546 INFO:tasks.workunit.client.0.smithi027.stdout:9/733: chown d4/d10/c32 1197133252 1 2024-04-26T23:01:38.546 INFO:tasks.workunit.client.0.smithi027.stdout:5/817: truncate d4/d24/d5f/dac/d40/fc8 227198 0 2024-04-26T23:01:38.546 INFO:tasks.workunit.client.0.smithi027.stdout:0/769: creat d0/d4/da/d15/d26/fed x:0 0 0 2024-04-26T23:01:38.547 INFO:tasks.workunit.client.0.smithi027.stdout:0/770: write d0/d4/da/d15/d26/f8a [534762,51373] 0 2024-04-26T23:01:38.547 INFO:tasks.workunit.client.0.smithi027.stdout:0/771: chown d0/df/d3f/d64/l8d 21316 1 2024-04-26T23:01:38.547 INFO:tasks.workunit.client.0.smithi027.stdout:0/772: write d0/d4/da/d1a/d2a/f74 [4616517,111694] 0 2024-04-26T23:01:38.547 INFO:tasks.workunit.client.0.smithi027.stdout:0/773: write d0/d4/da/d15/f6f [3025372,48021] 0 2024-04-26T23:01:38.547 INFO:tasks.workunit.client.0.smithi027.stdout:0/774: write d0/df/d3f/d64/f6a [2157586,118827] 0 2024-04-26T23:01:38.550 INFO:tasks.workunit.client.0.smithi027.stdout:9/734: link d4/l15 d4/d17/d2e/d81/da7/db8/l103 0 2024-04-26T23:01:38.551 INFO:tasks.workunit.client.0.smithi027.stdout:5/818: creat d4/f10a x:0 0 0 2024-04-26T23:01:38.551 INFO:tasks.workunit.client.0.smithi027.stdout:5/819: stat d4/d24/f1b 0 2024-04-26T23:01:38.553 INFO:tasks.workunit.client.0.smithi027.stdout:0/775: creat d0/d4/da/d1a/d6d/dc9/fee x:0 0 0 2024-04-26T23:01:38.554 INFO:tasks.workunit.client.0.smithi027.stdout:0/776: write d0/d4/da/d1a/d1e/fb0 [831074,92120] 0 2024-04-26T23:01:38.554 INFO:tasks.workunit.client.0.smithi027.stdout:9/735: unlink d4/d10/d25/l8b 0 2024-04-26T23:01:38.555 INFO:tasks.workunit.client.0.smithi027.stdout:5/820: creat d4/d24/d22/d29/d2a/d3b/d50/f10b x:0 0 0 2024-04-26T23:01:38.555 INFO:tasks.workunit.client.0.smithi027.stdout:5/821: dread - d4/d24/d55/d63/fdd zero size 2024-04-26T23:01:38.558 INFO:tasks.workunit.client.0.smithi027.stdout:0/777: unlink d0/df/f4e 0 2024-04-26T23:01:38.558 INFO:tasks.workunit.client.0.smithi027.stdout:8/761: dwrite d9/da/d2a/d5e/d72/d84/d5d/d8b/fbb [0,4194304] 0 2024-04-26T23:01:38.558 INFO:tasks.workunit.client.0.smithi027.stdout:9/736: truncate d4/d17/f29 2246089 0 2024-04-26T23:01:38.558 INFO:tasks.workunit.client.0.smithi027.stdout:9/737: readlink d4/l87 0 2024-04-26T23:01:38.559 INFO:tasks.workunit.client.0.smithi027.stdout:9/738: truncate d4/d10/d40/f64 2303432 0 2024-04-26T23:01:38.559 INFO:tasks.workunit.client.0.smithi027.stdout:9/739: chown d4/d17/lb9 237451 1 2024-04-26T23:01:38.559 INFO:tasks.workunit.client.0.smithi027.stdout:9/740: read - d4/d17/d2e/d46/dd1/ffe zero size 2024-04-26T23:01:38.559 INFO:tasks.workunit.client.0.smithi027.stdout:9/741: read - d4/d17/d2e/fcc zero size 2024-04-26T23:01:38.559 INFO:tasks.workunit.client.0.smithi027.stdout:9/742: creat d4/d10/d25/dc9/f104 x:0 0 0 2024-04-26T23:01:38.562 INFO:tasks.workunit.client.0.smithi027.stdout:9/743: rename d4/d10/cb4 to d4/d10/d40/d58/d75/d84/c105 0 2024-04-26T23:01:38.564 INFO:tasks.workunit.client.0.smithi027.stdout:9/744: rename d4/d17/d2e/d46/da0/d8d/cde to d4/d17/d2e/d81/da7/c106 0 2024-04-26T23:01:38.574 INFO:tasks.workunit.client.0.smithi027.stdout:9/745: creat d4/d10/d40/d58/d75/f107 x:0 0 0 2024-04-26T23:01:38.574 INFO:tasks.workunit.client.0.smithi027.stdout:9/746: stat d4/f52 0 2024-04-26T23:01:38.574 INFO:tasks.workunit.client.0.smithi027.stdout:8/762: dread d9/da/d2a/d5e/d72/d84/f40 [0,4194304] 0 2024-04-26T23:01:38.575 INFO:tasks.workunit.client.0.smithi027.stdout:8/763: truncate d9/da/d33/d71/fc1 2945411 0 2024-04-26T23:01:38.575 INFO:tasks.workunit.client.0.smithi027.stdout:9/747: creat d4/d10/d40/d58/d75/d84/dc4/de4/f108 x:0 0 0 2024-04-26T23:01:38.575 INFO:tasks.workunit.client.0.smithi027.stdout:8/764: write d9/da/d2a/d5e/d72/f7e [1027390,60816] 0 2024-04-26T23:01:38.576 INFO:tasks.workunit.client.0.smithi027.stdout:8/765: creat d9/da6/f10b x:0 0 0 2024-04-26T23:01:38.578 INFO:tasks.workunit.client.0.smithi027.stdout:9/748: mkdir d4/d10/d40/d58/d75/d109 0 2024-04-26T23:01:38.578 INFO:tasks.workunit.client.0.smithi027.stdout:8/766: write d9/da/d2a/d5e/d72/f7e [504635,54215] 0 2024-04-26T23:01:38.578 INFO:tasks.workunit.client.0.smithi027.stdout:8/767: write d9/da/d2a/d5e/d2b/f76 [1333762,46384] 0 2024-04-26T23:01:38.580 INFO:tasks.workunit.client.0.smithi027.stdout:8/768: mknod d9/da/d2a/d5e/d72/d84/d35/dc2/c10c 0 2024-04-26T23:01:38.581 INFO:tasks.workunit.client.0.smithi027.stdout:9/749: creat d4/d17/d2e/d81/da7/db8/f10a x:0 0 0 2024-04-26T23:01:38.589 INFO:tasks.workunit.client.0.smithi027.stdout:5/822: dwrite d4/d1a/d4a/f76 [0,4194304] 0 2024-04-26T23:01:38.615 INFO:tasks.workunit.client.0.smithi027.stdout:0/778: dwrite d0/d4/da/d15/f8e [0,4194304] 0 2024-04-26T23:01:38.618 INFO:tasks.workunit.client.0.smithi027.stdout:0/779: rmdir d0/d4/da/d15/d26 39 2024-04-26T23:01:38.618 INFO:tasks.workunit.client.0.smithi027.stdout:0/780: chown d0/df/d3f/d4a/f60 981363786 1 2024-04-26T23:01:38.618 INFO:tasks.workunit.client.0.smithi027.stdout:0/781: write d0/df/d3f/d64/f5e [636913,116781] 0 2024-04-26T23:01:38.620 INFO:tasks.workunit.client.0.smithi027.stdout:0/782: creat d0/df/d9d/d9f/dbb/fef x:0 0 0 2024-04-26T23:01:38.621 INFO:tasks.workunit.client.0.smithi027.stdout:0/783: symlink d0/d4/da/d1a/d2a/lf0 0 2024-04-26T23:01:38.621 INFO:tasks.workunit.client.0.smithi027.stdout:0/784: fsync d0/f12 0 2024-04-26T23:01:38.623 INFO:tasks.workunit.client.0.smithi027.stdout:0/785: rename d0/d4/da/d15/f93 to d0/df/d9d/ff1 0 2024-04-26T23:01:38.623 INFO:tasks.workunit.client.0.smithi027.stdout:0/786: readlink d0/d4/da/d1a/d6d/lbf 0 2024-04-26T23:01:38.624 INFO:tasks.workunit.client.0.smithi027.stdout:0/787: mknod d0/d4/da/d15/ddc/cf2 0 2024-04-26T23:01:38.624 INFO:tasks.workunit.client.0.smithi027.stdout:0/788: chown d0/d4/fbe 0 1 2024-04-26T23:01:38.626 INFO:tasks.workunit.client.0.smithi027.stdout:0/789: mknod d0/d4/da/d15/d26/dc4/cf3 0 2024-04-26T23:01:38.628 INFO:tasks.workunit.client.0.smithi027.stdout:0/790: symlink d0/d4/da/d15/lf4 0 2024-04-26T23:01:38.631 INFO:tasks.workunit.client.0.smithi027.stdout:9/750: dwrite d4/d17/d2e/d46/fd3 [0,4194304] 0 2024-04-26T23:01:38.634 INFO:tasks.workunit.client.0.smithi027.stdout:9/751: mknod d4/d17/c10b 0 2024-04-26T23:01:38.636 INFO:tasks.workunit.client.0.smithi027.stdout:9/752: creat d4/d10/d40/d58/d75/d84/dc4/de4/f10c x:0 0 0 2024-04-26T23:01:38.636 INFO:tasks.workunit.client.0.smithi027.stdout:9/753: stat d4/d17/d2e/d70/fe5 0 2024-04-26T23:01:38.636 INFO:tasks.workunit.client.0.smithi027.stdout:9/754: chown d4/d17/d2e/dcf 14 1 2024-04-26T23:01:38.637 INFO:tasks.workunit.client.0.smithi027.stdout:9/755: symlink d4/d17/d2e/d46/da0/l10d 0 2024-04-26T23:01:38.637 INFO:tasks.workunit.client.0.smithi027.stdout:9/756: fsync d4/d17/d2e/d46/da0/fc3 0 2024-04-26T23:01:38.644 INFO:tasks.workunit.client.0.smithi027.stdout:8/769: dwrite d9/da/d2a/d5e/d72/d84/d35/da7/fb7 [0,4194304] 0 2024-04-26T23:01:38.650 INFO:tasks.workunit.client.0.smithi027.stdout:0/791: dread d0/df/d3f/d64/f5e [0,4194304] 0 2024-04-26T23:01:38.657 INFO:tasks.workunit.client.0.smithi027.stdout:5/823: dwrite d4/d24/d22/d29/d39/f9c [0,4194304] 0 2024-04-26T23:01:38.657 INFO:tasks.workunit.client.0.smithi027.stdout:5/824: read d4/d24/f5d [941161,129032] 0 2024-04-26T23:01:38.659 INFO:tasks.workunit.client.0.smithi027.stdout:5/825: mkdir d4/d1a/d2b/daa/d10c 0 2024-04-26T23:01:38.659 INFO:tasks.workunit.client.0.smithi027.stdout:5/826: write d4/f15 [4703768,1118] 0 2024-04-26T23:01:38.661 INFO:tasks.workunit.client.0.smithi027.stdout:5/827: symlink d4/d9f/d105/l10d 0 2024-04-26T23:01:38.661 INFO:tasks.workunit.client.0.smithi027.stdout:5/828: dread - d4/d24/d22/dab/d16/d4b/d81/fe3 zero size 2024-04-26T23:01:38.662 INFO:tasks.workunit.client.0.smithi027.stdout:5/829: symlink d4/d1a/d2b/l10e 0 2024-04-26T23:01:38.663 INFO:tasks.workunit.client.0.smithi027.stdout:5/830: getdents d4/d1a/d2b/dd3 0 2024-04-26T23:01:38.665 INFO:tasks.workunit.client.0.smithi027.stdout:5/831: mknod d4/d24/d22/d29/d2a/dee/c10f 0 2024-04-26T23:01:38.682 INFO:tasks.workunit.client.0.smithi027.stdout:9/757: dwrite d4/d10/d25/f39 [0,4194304] 0 2024-04-26T23:01:38.682 INFO:tasks.workunit.client.0.smithi027.stdout:8/770: dread d9/da/d2a/d5e/d72/d84/db8/ff3 [0,4194304] 0 2024-04-26T23:01:38.686 INFO:tasks.workunit.client.0.smithi027.stdout:8/771: mknod d9/da/d2a/d5e/d72/d84/d35/d8d/c10d 0 2024-04-26T23:01:38.686 INFO:tasks.workunit.client.0.smithi027.stdout:8/772: write d9/da/d33/f60 [4040112,7560] 0 2024-04-26T23:01:38.686 INFO:tasks.workunit.client.0.smithi027.stdout:0/792: dwrite d0/d4/da/d1a/d1e/f91 [0,4194304] 0 2024-04-26T23:01:38.686 INFO:tasks.workunit.client.0.smithi027.stdout:5/832: dread d4/d24/d22/d29/d2a/d3b/d50/f70 [8388608,4194304] 0 2024-04-26T23:01:38.692 INFO:tasks.workunit.client.0.smithi027.stdout:8/773: dread d9/da/d2a/d6b/f81 [0,4194304] 0 2024-04-26T23:01:38.694 INFO:tasks.workunit.client.0.smithi027.stdout:8/774: link d9/de/d22/l27 d9/da/d2a/d5e/d72/d84/d35/d8d/da3/dfe/l10e 0 2024-04-26T23:01:38.694 INFO:tasks.workunit.client.0.smithi027.stdout:8/775: chown d9/da/d2a/d5e/d72/d84/d35/ccd 11097727 1 2024-04-26T23:01:38.696 INFO:tasks.workunit.client.0.smithi027.stdout:8/776: creat d9/da/d2a/d5e/f10f x:0 0 0 2024-04-26T23:01:38.705 INFO:tasks.workunit.client.0.smithi027.stdout:0/793: dread d0/d4/da/d15/d55/f96 [0,4194304] 0 2024-04-26T23:01:38.706 INFO:tasks.workunit.client.0.smithi027.stdout:4/862: sync 2024-04-26T23:01:38.706 INFO:tasks.workunit.client.0.smithi027.stdout:2/741: sync 2024-04-26T23:01:38.706 INFO:tasks.workunit.client.0.smithi027.stdout:6/674: sync 2024-04-26T23:01:38.706 INFO:tasks.workunit.client.0.smithi027.stdout:1/925: sync 2024-04-26T23:01:38.706 INFO:tasks.workunit.client.0.smithi027.stdout:2/742: readlink d2/d5/db/de/l85 0 2024-04-26T23:01:38.707 INFO:tasks.workunit.client.0.smithi027.stdout:1/926: chown d1/d5/d40 520 1 2024-04-26T23:01:38.707 INFO:tasks.workunit.client.0.smithi027.stdout:6/675: write d1/df/f29 [445312,24247] 0 2024-04-26T23:01:38.707 INFO:tasks.workunit.client.0.smithi027.stdout:7/841: sync 2024-04-26T23:01:38.707 INFO:tasks.workunit.client.0.smithi027.stdout:3/897: sync 2024-04-26T23:01:38.708 INFO:tasks.workunit.client.0.smithi027.stdout:1/927: getdents d1/d26/d66/dab 0 2024-04-26T23:01:38.708 INFO:tasks.workunit.client.0.smithi027.stdout:1/928: stat d1/d2a/deb/l106 0 2024-04-26T23:01:38.708 INFO:tasks.workunit.client.0.smithi027.stdout:1/929: dread - d1/d26/d66/d91/f109 zero size 2024-04-26T23:01:38.708 INFO:tasks.workunit.client.0.smithi027.stdout:3/898: getdents d6/dd3/d34/d2e/d5f/d90/ddb/d98 0 2024-04-26T23:01:38.708 INFO:tasks.workunit.client.0.smithi027.stdout:3/899: stat d6/dd3/f72 0 2024-04-26T23:01:38.708 INFO:tasks.workunit.client.0.smithi027.stdout:7/842: symlink d0/d28/d2b/d3a/d48/d56/d88/d73/dd6/l10e 0 2024-04-26T23:01:38.709 INFO:tasks.workunit.client.0.smithi027.stdout:4/863: mkdir da/dd/de/d14/db3/d114 0 2024-04-26T23:01:38.711 INFO:tasks.workunit.client.0.smithi027.stdout:4/864: truncate da/dd/de/d14/d17/f2a 8683027 0 2024-04-26T23:01:38.711 INFO:tasks.workunit.client.0.smithi027.stdout:4/865: write da/d40/ff6 [953773,71568] 0 2024-04-26T23:01:38.711 INFO:tasks.workunit.client.0.smithi027.stdout:4/866: fdatasync da/dd/de/d14/d21/d41/d59/d74/fe6 0 2024-04-26T23:01:38.711 INFO:tasks.workunit.client.0.smithi027.stdout:4/867: write da/dd/de/d79/dcd/f112 [921638,105428] 0 2024-04-26T23:01:38.711 INFO:tasks.workunit.client.0.smithi027.stdout:4/868: write da/dd/de/d14/d17/d80/fa9 [4826340,44814] 0 2024-04-26T23:01:38.716 INFO:tasks.workunit.client.0.smithi027.stdout:8/777: rename d9/da/d2a/d5e/d72/d84/d35 to d9/de/d2e/d110 0 2024-04-26T23:01:38.716 INFO:tasks.workunit.client.0.smithi027.stdout:8/778: write d9/de/d22/f3e [3983682,85310] 0 2024-04-26T23:01:38.722 INFO:tasks.workunit.client.0.smithi027.stdout:2/743: rename d2/d5/db/f91 to d2/d5/db/d1b/d1c/d25/d19/d44/fe8 0 2024-04-26T23:01:38.722 INFO:tasks.workunit.client.0.smithi027.stdout:8/779: rename d9/de/d2e/d110/da7/fb7 to d9/de/d2e/d110/dc2/f111 0 2024-04-26T23:01:38.722 INFO:tasks.workunit.client.0.smithi027.stdout:8/780: dread - d9/da/d2a/d5e/d72/d84/d5d/d6c/fc4 zero size 2024-04-26T23:01:38.722 INFO:tasks.workunit.client.0.smithi027.stdout:8/781: write d9/da/da1/faf [448446,93678] 0 2024-04-26T23:01:38.722 INFO:tasks.workunit.client.0.smithi027.stdout:8/782: mknod d9/de/d2e/d110/dc2/c112 0 2024-04-26T23:01:38.723 INFO:tasks.workunit.client.0.smithi027.stdout:8/783: unlink d9/da/d33/fef 0 2024-04-26T23:01:38.725 INFO:tasks.workunit.client.0.smithi027.stdout:8/784: symlink d9/de/d2e/d110/d8d/l113 0 2024-04-26T23:01:38.726 INFO:tasks.workunit.client.0.smithi027.stdout:8/785: symlink d9/da/d2a/d5e/d2b/d52/l114 0 2024-04-26T23:01:38.726 INFO:tasks.workunit.client.0.smithi027.stdout:8/786: chown d9/de/fa5 169530062 1 2024-04-26T23:01:38.728 INFO:tasks.workunit.client.0.smithi027.stdout:8/787: mkdir d9/da/d2a/d5e/d2b/d52/d115 0 2024-04-26T23:01:38.730 INFO:tasks.workunit.client.0.smithi027.stdout:8/788: creat d9/de/d2e/d110/f116 x:0 0 0 2024-04-26T23:01:38.730 INFO:tasks.workunit.client.0.smithi027.stdout:8/789: stat d9/da/d2a/d5e/d46/f6f 0 2024-04-26T23:01:38.730 INFO:tasks.workunit.client.0.smithi027.stdout:8/790: chown d9/da/d2a/d5e/d46/cb3 8 1 2024-04-26T23:01:38.745 INFO:tasks.workunit.client.0.smithi027.stdout:7/843: write d0/f62 [2817821,760] 0 2024-04-26T23:01:38.747 INFO:tasks.workunit.client.0.smithi027.stdout:7/844: rename d0/d28/d66/fd3 to d0/d28/d2b/d3a/d48/f10f 0 2024-04-26T23:01:38.749 INFO:tasks.workunit.client.0.smithi027.stdout:7/845: creat d0/d28/d2b/d55/d59/f110 x:0 0 0 2024-04-26T23:01:38.751 INFO:tasks.workunit.client.0.smithi027.stdout:5/833: dwrite d4/d1a/dcb/ff1 [0,4194304] 0 2024-04-26T23:01:38.751 INFO:tasks.workunit.client.0.smithi027.stdout:5/834: chown d4/d24/d5f/dac/d40/c8f 27 1 2024-04-26T23:01:38.751 INFO:tasks.workunit.client.0.smithi027.stdout:5/835: write d4/d24/d22/d29/d39/fa8 [244972,39000] 0 2024-04-26T23:01:38.751 INFO:tasks.workunit.client.0.smithi027.stdout:7/846: mkdir d0/d28/d2b/d3a/d90/dcb/d111 0 2024-04-26T23:01:38.753 INFO:tasks.workunit.client.0.smithi027.stdout:5/836: creat d4/d1a/d2b/f110 x:0 0 0 2024-04-26T23:01:38.756 INFO:tasks.workunit.client.0.smithi027.stdout:7/847: creat d0/d28/d2b/d55/d59/d6d/f112 x:0 0 0 2024-04-26T23:01:38.756 INFO:tasks.workunit.client.0.smithi027.stdout:7/848: chown d0/d28/c9d 2871958 1 2024-04-26T23:01:38.756 INFO:tasks.workunit.client.0.smithi027.stdout:7/849: chown d0/d28/d2b/d3a/d48/d56/db6 100 1 2024-04-26T23:01:38.764 INFO:tasks.workunit.client.0.smithi027.stdout:7/850: mkdir d0/d2f/d113 0 2024-04-26T23:01:38.765 INFO:tasks.workunit.client.0.smithi027.stdout:5/837: dread d4/d24/d5f/dac/d40/fc8 [0,4194304] 0 2024-04-26T23:01:38.767 INFO:tasks.workunit.client.0.smithi027.stdout:5/838: rename d4/d24/d5f/dac/d40/d54/c8e to d4/d1a/d2b/daa/d10c/c111 0 2024-04-26T23:01:38.773 INFO:tasks.workunit.client.0.smithi027.stdout:0/794: dwrite d0/df/d3f/d64/f5e [0,4194304] 0 2024-04-26T23:01:38.775 INFO:tasks.workunit.client.0.smithi027.stdout:0/795: mknod d0/d4/da/d1a/cf5 0 2024-04-26T23:01:38.780 INFO:tasks.workunit.client.0.smithi027.stdout:2/744: dwrite d2/d5/db/de/d21/d8f/fbf [0,4194304] 0 2024-04-26T23:01:38.784 INFO:tasks.workunit.client.0.smithi027.stdout:2/745: creat d2/d5/db/de/d21/d8f/d88/fe9 x:0 0 0 2024-04-26T23:01:38.786 INFO:tasks.workunit.client.0.smithi027.stdout:8/791: dwrite d9/da/f13 [0,4194304] 0 2024-04-26T23:01:38.792 INFO:tasks.workunit.client.0.smithi027.stdout:0/796: dread d0/d4/da/d15/f76 [0,4194304] 0 2024-04-26T23:01:38.793 INFO:tasks.workunit.client.0.smithi027.stdout:0/797: mkdir d0/d4/da/d1a/dc3/df6 0 2024-04-26T23:01:38.793 INFO:tasks.workunit.client.0.smithi027.stdout:0/798: chown d0/df/d3f 52931 1 2024-04-26T23:01:38.794 INFO:tasks.workunit.client.0.smithi027.stdout:0/799: chown d0/d4/da/d1a/l5f 56862 1 2024-04-26T23:01:38.795 INFO:tasks.workunit.client.0.smithi027.stdout:7/851: read d0/d28/d2b/d55/f7e [3952543,31530] 0 2024-04-26T23:01:38.796 INFO:tasks.workunit.client.0.smithi027.stdout:5/839: write d4/d24/d22/dab/f23 [1028462,32877] 0 2024-04-26T23:01:38.796 INFO:tasks.workunit.client.0.smithi027.stdout:5/840: readlink d4/d1a/d2b/l10e 0 2024-04-26T23:01:38.797 INFO:tasks.workunit.client.0.smithi027.stdout:5/841: write d4/d24/d22/dab/d16/d4b/d81/fe3 [466837,119259] 0 2024-04-26T23:01:38.797 INFO:tasks.workunit.client.0.smithi027.stdout:5/842: truncate d4/d24/d22/dab/d16/d4b/d81/fe3 1487759 0 2024-04-26T23:01:38.798 INFO:tasks.workunit.client.0.smithi027.stdout:7/852: symlink d0/d28/d2b/d55/df9/l114 0 2024-04-26T23:01:38.802 INFO:tasks.workunit.client.0.smithi027.stdout:3/900: dwrite d6/d5e/ff2 [0,4194304] 0 2024-04-26T23:01:38.802 INFO:tasks.workunit.client.0.smithi027.stdout:3/901: stat d6/dd3/l3b 0 2024-04-26T23:01:38.802 INFO:tasks.workunit.client.0.smithi027.stdout:5/843: mknod d4/d1a/dcb/dd1/dd9/c112 0 2024-04-26T23:01:38.813 INFO:tasks.workunit.client.0.smithi027.stdout:8/792: dread d9/da/d2a/d5e/d86/f8c [0,4194304] 0 2024-04-26T23:01:38.813 INFO:tasks.workunit.client.0.smithi027.stdout:5/844: symlink d4/d1a/dcb/dd1/l113 0 2024-04-26T23:01:38.815 INFO:tasks.workunit.client.0.smithi027.stdout:3/902: creat d6/dd3/d34/d2e/d9d/da7/dac/f12f x:0 0 0 2024-04-26T23:01:38.815 INFO:tasks.workunit.client.0.smithi027.stdout:4/869: dwrite da/dd/de/d14/d21/d41/d59/d74/faa [0,4194304] 0 2024-04-26T23:01:38.815 INFO:tasks.workunit.client.0.smithi027.stdout:4/870: write da/dd/de/d14/d17/d80/fa9 [1160276,82579] 0 2024-04-26T23:01:38.822 INFO:tasks.workunit.client.0.smithi027.stdout:5/845: truncate d4/d24/d22/dab/d16/d4b/f77 2511151 0 2024-04-26T23:01:38.822 INFO:tasks.workunit.client.0.smithi027.stdout:5/846: chown d4/d24/d5f/dac/d40/ff3 4612051 1 2024-04-26T23:01:38.822 INFO:tasks.workunit.client.0.smithi027.stdout:7/853: write d0/d6c/f8c [5754870,87934] 0 2024-04-26T23:01:38.822 INFO:tasks.workunit.client.0.smithi027.stdout:8/793: dread d9/f74 [0,4194304] 0 2024-04-26T23:01:38.825 INFO:tasks.workunit.client.0.smithi027.stdout:5/847: creat d4/d24/d22/d29/f114 x:0 0 0 2024-04-26T23:01:38.827 INFO:tasks.workunit.client.0.smithi027.stdout:8/794: creat d9/da6/f117 x:0 0 0 2024-04-26T23:01:38.828 INFO:tasks.workunit.client.0.smithi027.stdout:8/795: truncate d9/de/d2e/d110/da7/fde 1030717 0 2024-04-26T23:01:38.828 INFO:tasks.workunit.client.0.smithi027.stdout:8/796: chown d9/da/d2a/d5e/d72/d84/d5d/d6c/ce7 424949647 1 2024-04-26T23:01:38.835 INFO:tasks.workunit.client.0.smithi027.stdout:8/797: creat d9/de/d2e/d110/df0/f118 x:0 0 0 2024-04-26T23:01:38.840 INFO:tasks.workunit.client.0.smithi027.stdout:5/848: mknod d4/d9f/dc0/c115 0 2024-04-26T23:01:38.840 INFO:tasks.workunit.client.0.smithi027.stdout:5/849: chown d4/d24/d22/dab/d16/d4b/d81/fe3 1 1 2024-04-26T23:01:38.840 INFO:tasks.workunit.client.0.smithi027.stdout:7/854: mkdir d0/d28/d2b/d55/d59/d6d/d115 0 2024-04-26T23:01:38.841 INFO:tasks.workunit.client.0.smithi027.stdout:7/855: chown d0/d28/fed 8 1 2024-04-26T23:01:38.841 INFO:tasks.workunit.client.0.smithi027.stdout:7/856: truncate d0/fc2 217649 0 2024-04-26T23:01:38.841 INFO:tasks.workunit.client.0.smithi027.stdout:0/800: dwrite d0/df/d3f/d64/f70 [0,4194304] 0 2024-04-26T23:01:38.841 INFO:tasks.workunit.client.0.smithi027.stdout:7/857: mknod d0/d6c/c116 0 2024-04-26T23:01:38.841 INFO:tasks.workunit.client.0.smithi027.stdout:0/801: fsync d0/f7e 0 2024-04-26T23:01:38.841 INFO:tasks.workunit.client.0.smithi027.stdout:8/798: mkdir d9/da/d119 0 2024-04-26T23:01:38.841 INFO:tasks.workunit.client.0.smithi027.stdout:7/858: rename d0/c9e to d0/d28/d2b/d55/d69/d75/c117 0 2024-04-26T23:01:38.842 INFO:tasks.workunit.client.0.smithi027.stdout:7/859: rmdir d0/d28/d2b/d3a/d48/d56/d88/d73/dd6 39 2024-04-26T23:01:38.842 INFO:tasks.workunit.client.0.smithi027.stdout:7/860: chown d0/d28/d2b/d55/d69/c97 22221 1 2024-04-26T23:01:38.843 INFO:tasks.workunit.client.0.smithi027.stdout:7/861: creat d0/d28/d2b/f118 x:0 0 0 2024-04-26T23:01:38.844 INFO:tasks.workunit.client.0.smithi027.stdout:0/802: unlink d0/d4/l36 0 2024-04-26T23:01:38.845 INFO:tasks.workunit.client.0.smithi027.stdout:0/803: read d0/d4/da/d1a/d59/f9c [259564,13043] 0 2024-04-26T23:01:38.847 INFO:tasks.workunit.client.0.smithi027.stdout:0/804: symlink d0/df/d9d/de5/lf7 0 2024-04-26T23:01:38.847 INFO:tasks.workunit.client.0.smithi027.stdout:0/805: read d0/f37 [1857582,80191] 0 2024-04-26T23:01:38.847 INFO:tasks.workunit.client.0.smithi027.stdout:0/806: fsync d0/d4/faf 0 2024-04-26T23:01:38.849 INFO:tasks.workunit.client.0.smithi027.stdout:0/807: creat d0/df/d9d/ff8 x:0 0 0 2024-04-26T23:01:38.858 INFO:tasks.workunit.client.0.smithi027.stdout:8/799: dread d9/da/d33/d71/fc1 [0,4194304] 0 2024-04-26T23:01:38.860 INFO:tasks.workunit.client.0.smithi027.stdout:8/800: link d9/de/d22/c55 d9/da/d2a/d5e/d72/d84/d4f/c11a 0 2024-04-26T23:01:38.862 INFO:tasks.workunit.client.0.smithi027.stdout:8/801: mkdir d9/de/d106/d11b 0 2024-04-26T23:01:38.866 INFO:tasks.workunit.client.0.smithi027.stdout:8/802: truncate f3 1629445 0 2024-04-26T23:01:38.866 INFO:tasks.workunit.client.0.smithi027.stdout:8/803: stat d9/da/d2a/d5e/d72/d84/db8/c82 0 2024-04-26T23:01:38.866 INFO:tasks.workunit.client.0.smithi027.stdout:3/903: dwrite d6/dd3/d34/d78/fb6 [0,4194304] 0 2024-04-26T23:01:38.867 INFO:tasks.workunit.client.0.smithi027.stdout:3/904: fdatasync d6/dd3/d34/d2e/d9d/da7/dac/dca/ff7 0 2024-04-26T23:01:38.869 INFO:tasks.workunit.client.0.smithi027.stdout:8/804: mknod d9/da/d2a/d5e/d2b/d52/d115/c11c 0 2024-04-26T23:01:38.874 INFO:tasks.workunit.client.0.smithi027.stdout:3/905: creat d6/dd3/d34/d2e/d5f/d90/db9/f130 x:0 0 0 2024-04-26T23:01:38.874 INFO:tasks.workunit.client.0.smithi027.stdout:4/871: dwrite da/dd/de/d79/f8e [0,4194304] 0 2024-04-26T23:01:38.877 INFO:tasks.workunit.client.0.smithi027.stdout:3/906: truncate d6/dd3/d34/d2e/d9d/da7/dac/d74/f87 2121149 0 2024-04-26T23:01:38.877 INFO:tasks.workunit.client.0.smithi027.stdout:3/907: write d6/dd3/f21 [3403414,100320] 0 2024-04-26T23:01:38.877 INFO:tasks.workunit.client.0.smithi027.stdout:0/808: dread d0/d4/f7 [0,4194304] 0 2024-04-26T23:01:38.880 INFO:tasks.workunit.client.0.smithi027.stdout:3/908: write d6/dd3/d34/d2e/d50/fb2 [4989997,124569] 0 2024-04-26T23:01:38.887 INFO:tasks.workunit.client.0.smithi027.stdout:4/872: dread da/dd/d5f/f8b [0,4194304] 0 2024-04-26T23:01:38.905 INFO:tasks.workunit.client.0.smithi027.stdout:0/809: write d0/d4/faf [1234855,115336] 0 2024-04-26T23:01:38.905 INFO:tasks.workunit.client.0.smithi027.stdout:0/810: write d0/f12 [4458740,33965] 0 2024-04-26T23:01:38.906 INFO:tasks.workunit.client.0.smithi027.stdout:0/811: chown d0/df/d3f/d64/la0 536 1 2024-04-26T23:01:38.906 INFO:tasks.workunit.client.0.smithi027.stdout:0/812: chown d0/d4/da/d15/d55/lbd 34487520 1 2024-04-26T23:01:38.907 INFO:tasks.workunit.client.0.smithi027.stdout:0/813: link d0/d4/da/d15/d26/l9a d0/d4/da/d1a/dc3/df6/lf9 0 2024-04-26T23:01:38.908 INFO:tasks.workunit.client.0.smithi027.stdout:0/814: chown d0/df/d3f/d64/da8/ldf 124 1 2024-04-26T23:01:38.909 INFO:tasks.workunit.client.0.smithi027.stdout:0/815: mkdir d0/dfa 0 2024-04-26T23:01:38.910 INFO:tasks.workunit.client.0.smithi027.stdout:0/816: creat d0/ffb x:0 0 0 2024-04-26T23:01:38.912 INFO:tasks.workunit.client.0.smithi027.stdout:7/862: dwrite d0/d28/d2b/d3a/d48/d56/d88/d73/dd6/fea [0,4194304] 0 2024-04-26T23:01:38.913 INFO:tasks.workunit.client.0.smithi027.stdout:8/805: dread d9/da/d2a/d5e/d92/fb9 [0,4194304] 0 2024-04-26T23:01:38.915 INFO:tasks.workunit.client.0.smithi027.stdout:0/817: rename d0/df/d3f/d64/f38 to d0/df/d3f/d4a/ffc 0 2024-04-26T23:01:38.915 INFO:tasks.workunit.client.0.smithi027.stdout:7/863: mkdir d0/d28/d2b/d3a/d48/d56/d88/d73/d119 0 2024-04-26T23:01:38.918 INFO:tasks.workunit.client.0.smithi027.stdout:3/909: dwrite d6/dd3/d34/d2e/d70/d71/ff4 [0,4194304] 0 2024-04-26T23:01:38.921 INFO:tasks.workunit.client.0.smithi027.stdout:7/864: symlink d0/d3d/dba/l11a 0 2024-04-26T23:01:38.924 INFO:tasks.workunit.client.0.smithi027.stdout:7/865: truncate d0/d28/d2b/d3a/d48/fb2 3348903 0 2024-04-26T23:01:38.926 INFO:tasks.workunit.client.0.smithi027.stdout:5/850: dwrite d4/d24/d22/d29/d39/f9c [0,4194304] 0 2024-04-26T23:01:38.926 INFO:tasks.workunit.client.0.smithi027.stdout:0/818: dread d0/d4/da/d1a/d59/f9c [0,4194304] 0 2024-04-26T23:01:38.926 INFO:tasks.workunit.client.0.smithi027.stdout:0/819: stat d0/df/f72 0 2024-04-26T23:01:38.928 INFO:tasks.workunit.client.0.smithi027.stdout:5/851: rmdir d4/d1a/d2b/dd3/dfa 0 2024-04-26T23:01:38.928 INFO:tasks.workunit.client.0.smithi027.stdout:5/852: chown d4/d24/d55 2 1 2024-04-26T23:01:38.929 INFO:tasks.workunit.client.0.smithi027.stdout:0/820: getdents d0/d4/da/d15 0 2024-04-26T23:01:38.934 INFO:tasks.workunit.client.0.smithi027.stdout:0/821: symlink d0/df/d3f/d4a/de9/lfd 0 2024-04-26T23:01:38.934 INFO:tasks.workunit.client.0.smithi027.stdout:5/853: creat d4/d24/d5f/dac/f116 x:0 0 0 2024-04-26T23:01:38.934 INFO:tasks.workunit.client.0.smithi027.stdout:0/822: chown d0/d4/da/d15/d55 130347 1 2024-04-26T23:01:38.936 INFO:tasks.workunit.client.0.smithi027.stdout:8/806: dread d9/de/d22/f4d [4194304,4194304] 0 2024-04-26T23:01:38.936 INFO:tasks.workunit.client.0.smithi027.stdout:8/807: stat d9/de/d2e/f38 0 2024-04-26T23:01:38.936 INFO:tasks.workunit.client.0.smithi027.stdout:8/808: dread - d9/da/d2a/d5e/f10f zero size 2024-04-26T23:01:38.936 INFO:tasks.workunit.client.0.smithi027.stdout:8/809: chown d9/da/d2a/d5e/d92/fae 57715 1 2024-04-26T23:01:38.936 INFO:tasks.workunit.client.0.smithi027.stdout:5/854: symlink d4/d9f/dc0/l117 0 2024-04-26T23:01:38.940 INFO:tasks.workunit.client.0.smithi027.stdout:0/823: mknod d0/df/d3f/d4a/de9/cfe 0 2024-04-26T23:01:38.946 INFO:tasks.workunit.client.0.smithi027.stdout:9/758: sync 2024-04-26T23:01:38.947 INFO:tasks.workunit.client.0.smithi027.stdout:4/873: dwrite da/f48 [0,4194304] 0 2024-04-26T23:01:38.947 INFO:tasks.workunit.client.0.smithi027.stdout:6/676: sync 2024-04-26T23:01:38.947 INFO:tasks.workunit.client.0.smithi027.stdout:1/930: sync 2024-04-26T23:01:38.948 INFO:tasks.workunit.client.0.smithi027.stdout:6/677: chown d1/df/d43/d56/d6b/c8f 3929314 1 2024-04-26T23:01:38.955 INFO:tasks.workunit.client.0.smithi027.stdout:5/855: write d4/d24/d5f/dac/d40/ff3 [3204733,73543] 0 2024-04-26T23:01:38.956 INFO:tasks.workunit.client.0.smithi027.stdout:4/874: mkdir da/dd/d92/d115 0 2024-04-26T23:01:38.956 INFO:tasks.workunit.client.0.smithi027.stdout:4/875: readlink da/dd/d5f/l6e 0 2024-04-26T23:01:38.956 INFO:tasks.workunit.client.0.smithi027.stdout:6/678: mknod d1/df/d43/d56/d6b/dc4/cd8 0 2024-04-26T23:01:38.956 INFO:tasks.workunit.client.0.smithi027.stdout:1/931: creat d1/d26/d66/d73/dae/f131 x:0 0 0 2024-04-26T23:01:38.956 INFO:tasks.workunit.client.0.smithi027.stdout:6/679: readlink d1/d30/d6c/lb7 0 2024-04-26T23:01:38.956 INFO:tasks.workunit.client.0.smithi027.stdout:9/759: mkdir d4/d17/d2e/d10e 0 2024-04-26T23:01:38.956 INFO:tasks.workunit.client.0.smithi027.stdout:6/680: fsync d1/df/f2e 0 2024-04-26T23:01:38.956 INFO:tasks.workunit.client.0.smithi027.stdout:9/760: write d4/d17/d2e/d46/dd1/ffe [189126,71150] 0 2024-04-26T23:01:38.956 INFO:tasks.workunit.client.0.smithi027.stdout:7/866: write d0/f13 [1174231,47950] 0 2024-04-26T23:01:38.960 INFO:tasks.workunit.client.0.smithi027.stdout:4/876: symlink da/dd/de/d79/dcd/l116 0 2024-04-26T23:01:38.960 INFO:tasks.workunit.client.0.smithi027.stdout:5/856: creat d4/d1a/d2b/dd2/f118 x:0 0 0 2024-04-26T23:01:38.960 INFO:tasks.workunit.client.0.smithi027.stdout:4/877: dread - da/dd/de/d14/db3/fbd zero size 2024-04-26T23:01:38.960 INFO:tasks.workunit.client.0.smithi027.stdout:1/932: getdents d1/d26/d6b/d8e 0 2024-04-26T23:01:38.961 INFO:tasks.workunit.client.0.smithi027.stdout:1/933: mkdir d1/d26/d66/dab/de0/dfe/d132 0 2024-04-26T23:01:38.961 INFO:tasks.workunit.client.0.smithi027.stdout:1/934: readlink d1/d26/d66/dab/de0/lb3 0 2024-04-26T23:01:38.984 INFO:tasks.workunit.client.0.smithi027.stdout:7/867: dread d0/d28/fed [0,4194304] 0 2024-04-26T23:01:38.984 INFO:tasks.workunit.client.0.smithi027.stdout:3/910: dwrite d6/dd3/d34/d2e/d9d/da7/dac/f12f [0,4194304] 0 2024-04-26T23:01:38.989 INFO:tasks.workunit.client.0.smithi027.stdout:7/868: symlink d0/d28/d2b/d3a/d48/d56/d8d/dc5/df5/l11b 0 2024-04-26T23:01:38.989 INFO:tasks.workunit.client.0.smithi027.stdout:7/869: chown d0/d28/d2b/d3a/d48/d56/db6/fdc 24 1 2024-04-26T23:01:38.989 INFO:tasks.workunit.client.0.smithi027.stdout:0/824: dread d0/df/d3f/d64/f6a [0,4194304] 0 2024-04-26T23:01:38.992 INFO:tasks.workunit.client.0.smithi027.stdout:7/870: mkdir d0/d3d/dba/df8/d10b/d11c 0 2024-04-26T23:01:38.994 INFO:tasks.workunit.client.0.smithi027.stdout:4/878: dread f2 [0,4194304] 0 2024-04-26T23:01:38.997 INFO:tasks.workunit.client.0.smithi027.stdout:4/879: rename da/dd/de/d14/d21/d67/d8a/fa2 to da/d40/dd7/f117 0 2024-04-26T23:01:38.998 INFO:tasks.workunit.client.0.smithi027.stdout:6/681: dwrite d1/fc6 [0,4194304] 0 2024-04-26T23:01:38.998 INFO:tasks.workunit.client.0.smithi027.stdout:6/682: write d1/df/d43/faa [2976160,14145] 0 2024-04-26T23:01:38.998 INFO:tasks.workunit.client.0.smithi027.stdout:6/683: read - d1/d30/d6c/fbd zero size 2024-04-26T23:01:38.998 INFO:tasks.workunit.client.0.smithi027.stdout:6/684: chown d1/df/d43/d56/l5a 3236815 1 2024-04-26T23:01:39.004 INFO:tasks.workunit.client.0.smithi027.stdout:6/685: creat d1/d30/fd9 x:0 0 0 2024-04-26T23:01:39.004 INFO:tasks.workunit.client.0.smithi027.stdout:6/686: write d1/d30/d6c/db3/fcc [261720,108947] 0 2024-04-26T23:01:39.009 INFO:tasks.workunit.client.0.smithi027.stdout:0/825: write d0/df/d3f/f61 [261870,13414] 0 2024-04-26T23:01:39.009 INFO:tasks.workunit.client.0.smithi027.stdout:0/826: fsync d0/df/d3f/d64/f6a 0 2024-04-26T23:01:39.010 INFO:tasks.workunit.client.0.smithi027.stdout:6/687: creat d1/d30/d6c/d6f/dd3/fda x:0 0 0 2024-04-26T23:01:39.010 INFO:tasks.workunit.client.0.smithi027.stdout:3/911: write d6/dd3/d34/d78/da1/fa8 [3891251,14094] 0 2024-04-26T23:01:39.010 INFO:tasks.workunit.client.0.smithi027.stdout:3/912: write d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d40/f8b [1378039,76825] 0 2024-04-26T23:01:39.010 INFO:tasks.workunit.client.0.smithi027.stdout:3/913: read - d6/dd3/d34/d47/d117/f11a zero size 2024-04-26T23:01:39.010 INFO:tasks.workunit.client.0.smithi027.stdout:6/688: creat d1/fdb x:0 0 0 2024-04-26T23:01:39.011 INFO:tasks.workunit.client.0.smithi027.stdout:8/810: dwrite d9/da/d2a/d6b/fab [0,4194304] 0 2024-04-26T23:01:39.012 INFO:tasks.workunit.client.0.smithi027.stdout:8/811: chown d9/de/d2e/d110/d8d/f97 1999077 1 2024-04-26T23:01:39.017 INFO:tasks.workunit.client.0.smithi027.stdout:5/857: dwrite d4/d24/d5f/dac/d40/d54/f75 [4194304,4194304] 0 2024-04-26T23:01:39.019 INFO:tasks.workunit.client.0.smithi027.stdout:5/858: getdents d4/d24/d22/dab 0 2024-04-26T23:01:39.019 INFO:tasks.workunit.client.0.smithi027.stdout:5/859: truncate d4/d24/d22/fa3 4759224 0 2024-04-26T23:01:39.019 INFO:tasks.workunit.client.0.smithi027.stdout:5/860: readlink d4/d24/d22/dab/l79 0 2024-04-26T23:01:39.019 INFO:tasks.workunit.client.0.smithi027.stdout:3/914: creat d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/f131 x:0 0 0 2024-04-26T23:01:39.027 INFO:tasks.workunit.client.0.smithi027.stdout:8/812: getdents d9/de 0 2024-04-26T23:01:39.030 INFO:tasks.workunit.client.0.smithi027.stdout:8/813: link d9/de/d2e/d110/d8d/da3/fc3 d9/f11d 0 2024-04-26T23:01:39.045 INFO:tasks.workunit.client.0.smithi027.stdout:0/827: dread d0/d4/da/d1a/d2a/fcc [0,4194304] 0 2024-04-26T23:01:39.049 INFO:tasks.workunit.client.0.smithi027.stdout:0/828: creat d0/df/d3f/d64/fff x:0 0 0 2024-04-26T23:01:39.049 INFO:tasks.workunit.client.0.smithi027.stdout:0/829: write d0/d4/da/d1a/d1e/fb0 [1290327,70765] 0 2024-04-26T23:01:39.049 INFO:tasks.workunit.client.0.smithi027.stdout:0/830: dread - d0/d4/fbc zero size 2024-04-26T23:01:39.049 INFO:tasks.workunit.client.0.smithi027.stdout:0/831: write d0/d4/da/d15/f6f [4786561,122804] 0 2024-04-26T23:01:39.053 INFO:tasks.workunit.client.0.smithi027.stdout:0/832: mkdir d0/d4/dd3/d100 0 2024-04-26T23:01:39.055 INFO:tasks.workunit.client.0.smithi027.stdout:6/689: dwrite d1/df/d43/d56/d6b/f76 [0,4194304] 0 2024-04-26T23:01:39.069 INFO:tasks.workunit.client.0.smithi027.stdout:2/746: sync 2024-04-26T23:01:39.071 INFO:tasks.workunit.client.0.smithi027.stdout:1/935: dwrite d1/d26/d66/dab/de0/f44 [0,4194304] 0 2024-04-26T23:01:39.073 INFO:tasks.workunit.client.0.smithi027.stdout:2/747: creat d2/d5/db/d1b/d1c/d25/d19/fea x:0 0 0 2024-04-26T23:01:39.075 INFO:tasks.workunit.client.0.smithi027.stdout:1/936: unlink d1/d26/d6b/d8e/cc2 0 2024-04-26T23:01:39.077 INFO:tasks.workunit.client.0.smithi027.stdout:2/748: unlink d2/d5/db/d1b/d1c/d25/d19/d44/d7e/l9c 0 2024-04-26T23:01:39.080 INFO:tasks.workunit.client.0.smithi027.stdout:7/871: dwrite d0/d6c/f71 [0,4194304] 0 2024-04-26T23:01:39.080 INFO:tasks.workunit.client.0.smithi027.stdout:2/749: mkdir d2/d5/db/de/d21/d8f/deb 0 2024-04-26T23:01:39.080 INFO:tasks.workunit.client.0.smithi027.stdout:2/750: write d2/d5/db/d1b/d1c/d25/d19/d23/f47 [3697420,12250] 0 2024-04-26T23:01:39.081 INFO:tasks.workunit.client.0.smithi027.stdout:7/872: mkdir d0/d28/d66/d67/d11d 0 2024-04-26T23:01:39.083 INFO:tasks.workunit.client.0.smithi027.stdout:7/873: rename d0/d28/d2b/d3a/d48/d56/d88/lca to d0/d3d/dba/df8/d10b/d11c/l11e 0 2024-04-26T23:01:39.084 INFO:tasks.workunit.client.0.smithi027.stdout:2/751: rmdir d2 39 2024-04-26T23:01:39.085 INFO:tasks.workunit.client.0.smithi027.stdout:7/874: creat d0/d3d/dba/f11f x:0 0 0 2024-04-26T23:01:39.085 INFO:tasks.workunit.client.0.smithi027.stdout:2/752: stat d2/d5/db/de/l2a 0 2024-04-26T23:01:39.085 INFO:tasks.workunit.client.0.smithi027.stdout:2/753: stat d2/d5/db/de/d21/dc4/f43 0 2024-04-26T23:01:39.086 INFO:tasks.workunit.client.0.smithi027.stdout:7/875: mknod d0/d28/d2b/d3a/d48/dc0/c120 0 2024-04-26T23:01:39.086 INFO:tasks.workunit.client.0.smithi027.stdout:7/876: dread - d0/d28/d2b/d55/fe7 zero size 2024-04-26T23:01:39.087 INFO:tasks.workunit.client.0.smithi027.stdout:6/690: dread d1/df/d43/f6e [0,4194304] 0 2024-04-26T23:01:39.087 INFO:tasks.workunit.client.0.smithi027.stdout:3/915: dwrite d6/dd3/d34/d78/da1/f7c [0,4194304] 0 2024-04-26T23:01:39.087 INFO:tasks.workunit.client.0.smithi027.stdout:3/916: chown d6/dd3/d34/db3/ff0 3391327 1 2024-04-26T23:01:39.087 INFO:tasks.workunit.client.0.smithi027.stdout:3/917: chown d6/dd3/d34/d2e/d9d/fad 16732896 1 2024-04-26T23:01:39.088 INFO:tasks.workunit.client.0.smithi027.stdout:3/918: readlink d6/dd3/d2f/d7e/l109 0 2024-04-26T23:01:39.088 INFO:tasks.workunit.client.0.smithi027.stdout:3/919: stat d6/dd3/d34/d2e/d5f/d90 0 2024-04-26T23:01:39.088 INFO:tasks.workunit.client.0.smithi027.stdout:7/877: truncate d0/d28/d2b/d55/d59/d6d/fb5 928660 0 2024-04-26T23:01:39.088 INFO:tasks.workunit.client.0.smithi027.stdout:7/878: chown d0/d3d/dba/df8/d10b/d11c 313 1 2024-04-26T23:01:39.088 INFO:tasks.workunit.client.0.smithi027.stdout:7/879: write d0/d3d/dba/df8/f102 [69865,5518] 0 2024-04-26T23:01:39.090 INFO:tasks.workunit.client.0.smithi027.stdout:6/691: rename d1/de/d57/d8e/dd0/lc0 to d1/d6a/ldc 0 2024-04-26T23:01:39.092 INFO:tasks.workunit.client.0.smithi027.stdout:7/880: mknod d0/d28/d66/d67/d11d/c121 0 2024-04-26T23:01:39.092 INFO:tasks.workunit.client.0.smithi027.stdout:7/881: chown d0/d28/d2b/d3a/d48/ccc 4071 1 2024-04-26T23:01:39.092 INFO:tasks.workunit.client.0.smithi027.stdout:3/920: write d6/dd3/d34/d2e/d5f/d90/db9/f120 [574056,125987] 0 2024-04-26T23:01:39.092 INFO:tasks.workunit.client.0.smithi027.stdout:8/814: dwrite d9/da/d2a/d5e/d72/f8a [0,4194304] 0 2024-04-26T23:01:39.095 INFO:tasks.workunit.client.0.smithi027.stdout:6/692: unlink d1/c32 0 2024-04-26T23:01:39.095 INFO:tasks.workunit.client.0.smithi027.stdout:6/693: fsync d1/d30/d6c/fbd 0 2024-04-26T23:01:39.095 INFO:tasks.workunit.client.0.smithi027.stdout:6/694: read d1/d30/f36 [689101,37338] 0 2024-04-26T23:01:39.096 INFO:tasks.workunit.client.0.smithi027.stdout:6/695: write d1/df/d43/f5d [186387,111292] 0 2024-04-26T23:01:39.096 INFO:tasks.workunit.client.0.smithi027.stdout:7/882: creat d0/d28/d2b/d3a/d48/d56/d88/d73/dd6/f122 x:0 0 0 2024-04-26T23:01:39.096 INFO:tasks.workunit.client.0.smithi027.stdout:7/883: truncate d0/d28/d2b/d3a/d48/d56/d88/f91 272843 0 2024-04-26T23:01:39.096 INFO:tasks.workunit.client.0.smithi027.stdout:7/884: write d0/d2f/f65 [3421561,46215] 0 2024-04-26T23:01:39.099 INFO:tasks.workunit.client.0.smithi027.stdout:4/880: dwrite da/dd/de/d14/d21/f46 [0,4194304] 0 2024-04-26T23:01:39.100 INFO:tasks.workunit.client.0.smithi027.stdout:3/921: symlink d6/dd3/d34/d2e/d9d/da7/dac/dca/l132 0 2024-04-26T23:01:39.102 INFO:tasks.workunit.client.0.smithi027.stdout:4/881: mknod da/dd/de/c118 0 2024-04-26T23:01:39.102 INFO:tasks.workunit.client.0.smithi027.stdout:4/882: fsync da/dd/de/d14/d21/f23 0 2024-04-26T23:01:39.110 INFO:tasks.workunit.client.0.smithi027.stdout:1/937: dread d1/d26/d6b/d42/d79/f11b [0,4194304] 0 2024-04-26T23:01:39.110 INFO:tasks.workunit.client.0.smithi027.stdout:0/833: dwrite d0/d4/da/f11 [4194304,4194304] 0 2024-04-26T23:01:39.110 INFO:tasks.workunit.client.0.smithi027.stdout:0/834: stat d0/d4/da/d15/lf4 0 2024-04-26T23:01:39.115 INFO:tasks.workunit.client.0.smithi027.stdout:4/883: symlink da/dd/de/d14/d21/d41/df7/l119 0 2024-04-26T23:01:39.115 INFO:tasks.workunit.client.0.smithi027.stdout:4/884: write da/dd/de/fd2 [334145,98649] 0 2024-04-26T23:01:39.115 INFO:tasks.workunit.client.0.smithi027.stdout:4/885: chown da/dd/de/d14/d17/d80/dab/ccf 2980927 1 2024-04-26T23:01:39.115 INFO:tasks.workunit.client.0.smithi027.stdout:7/885: dread d0/d3d/dba/fe5 [0,4194304] 0 2024-04-26T23:01:39.119 INFO:tasks.workunit.client.0.smithi027.stdout:6/696: read d1/df/d43/da8/da1/fb6 [3861750,64728] 0 2024-04-26T23:01:39.126 INFO:tasks.workunit.client.0.smithi027.stdout:0/835: read d0/d4/da/d1a/d1e/f7f [783515,91609] 0 2024-04-26T23:01:39.126 INFO:tasks.workunit.client.0.smithi027.stdout:8/815: write d9/de/d2e/d110/f99 [1871342,116940] 0 2024-04-26T23:01:39.126 INFO:tasks.workunit.client.0.smithi027.stdout:1/938: getdents d1/d26/d66 0 2024-04-26T23:01:39.127 INFO:tasks.workunit.client.0.smithi027.stdout:4/886: creat da/dd/de/d14/d21/d41/d59/d74/f11a x:0 0 0 2024-04-26T23:01:39.127 INFO:tasks.workunit.client.0.smithi027.stdout:4/887: readlink da/dd/de/l32 0 2024-04-26T23:01:39.127 INFO:tasks.workunit.client.0.smithi027.stdout:1/939: write d1/d2a/ff7 [504989,78791] 0 2024-04-26T23:01:39.127 INFO:tasks.workunit.client.0.smithi027.stdout:3/922: read d6/f18 [1584346,46858] 0 2024-04-26T23:01:39.127 INFO:tasks.workunit.client.0.smithi027.stdout:3/923: write d6/f12c [749324,39859] 0 2024-04-26T23:01:39.127 INFO:tasks.workunit.client.0.smithi027.stdout:7/886: mknod d0/d28/d2b/d3a/d48/d56/d8d/c123 0 2024-04-26T23:01:39.130 INFO:tasks.workunit.client.0.smithi027.stdout:8/816: rename d9/da/d2a/d5e/d72/f8a to d9/da/d119/f11e 0 2024-04-26T23:01:39.131 INFO:tasks.workunit.client.0.smithi027.stdout:3/924: mknod d6/dd3/d34/d2e/d100/c133 0 2024-04-26T23:01:39.132 INFO:tasks.workunit.client.0.smithi027.stdout:6/697: unlink d1/d6a/cc5 0 2024-04-26T23:01:39.137 INFO:tasks.workunit.client.0.smithi027.stdout:1/940: mknod d1/d26/d6b/d42/c133 0 2024-04-26T23:01:39.137 INFO:tasks.workunit.client.0.smithi027.stdout:3/925: link d6/dd3/d34/d2e/d9d/da7/dac/dca/ld2 d6/dd3/d34/d2e/d9d/da7/dac/l134 0 2024-04-26T23:01:39.137 INFO:tasks.workunit.client.0.smithi027.stdout:0/836: dread d0/d4/da/d15/d26/f8a [0,4194304] 0 2024-04-26T23:01:39.137 INFO:tasks.workunit.client.0.smithi027.stdout:0/837: fdatasync d0/d4/f7 0 2024-04-26T23:01:39.137 INFO:tasks.workunit.client.0.smithi027.stdout:3/926: link d6/dd3/d34/d78/da1/f107 d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d108/f135 0 2024-04-26T23:01:39.138 INFO:tasks.workunit.client.0.smithi027.stdout:7/887: getdents d0/d28/d2b/d3a/d48/d56/d88/d73/dd6 0 2024-04-26T23:01:39.139 INFO:tasks.workunit.client.0.smithi027.stdout:0/838: write d0/df/d3f/d64/f31 [4536042,80570] 0 2024-04-26T23:01:39.139 INFO:tasks.workunit.client.0.smithi027.stdout:3/927: mkdir d6/dd3/d34/d2e/d5f/d90/d136 0 2024-04-26T23:01:39.139 INFO:tasks.workunit.client.0.smithi027.stdout:3/928: chown d6/dd3/d2f/f114 58721739 1 2024-04-26T23:01:39.139 INFO:tasks.workunit.client.0.smithi027.stdout:7/888: read d0/d28/d66/d67/ffd [16369,101920] 0 2024-04-26T23:01:39.140 INFO:tasks.workunit.client.0.smithi027.stdout:4/888: rename da/dd/d5f/dac/cb9 to da/d40/daf/c11b 0 2024-04-26T23:01:39.140 INFO:tasks.workunit.client.0.smithi027.stdout:6/698: chown d1/de/d57/d8e/dd0 21643 1 2024-04-26T23:01:39.141 INFO:tasks.workunit.client.0.smithi027.stdout:8/817: rename d9/de/d106 to d9/de/d2e/d110/df0/d11f 0 2024-04-26T23:01:39.142 INFO:tasks.workunit.client.0.smithi027.stdout:8/818: mknod d9/de/d2e/c120 0 2024-04-26T23:01:39.144 INFO:tasks.workunit.client.0.smithi027.stdout:7/889: mkdir d0/d28/d66/da3/d124 0 2024-04-26T23:01:39.145 INFO:tasks.workunit.client.0.smithi027.stdout:0/839: symlink d0/df/d3f/d64/da8/l101 0 2024-04-26T23:01:39.145 INFO:tasks.workunit.client.0.smithi027.stdout:0/840: write d0/fc2 [579713,41680] 0 2024-04-26T23:01:39.145 INFO:tasks.workunit.client.0.smithi027.stdout:0/841: readlink d0/df/d3f/d64/lca 0 2024-04-26T23:01:39.145 INFO:tasks.workunit.client.0.smithi027.stdout:0/842: fdatasync d0/df/d3f/d64/fff 0 2024-04-26T23:01:39.145 INFO:tasks.workunit.client.0.smithi027.stdout:3/929: rmdir d6/dd3/d34/db3/d125 0 2024-04-26T23:01:39.145 INFO:tasks.workunit.client.0.smithi027.stdout:3/930: stat d6/dd3/d34/d2e/d70/fc3 0 2024-04-26T23:01:39.145 INFO:tasks.workunit.client.0.smithi027.stdout:0/843: write d0/df/d3f/fd5 [693146,34628] 0 2024-04-26T23:01:39.148 INFO:tasks.workunit.client.0.smithi027.stdout:8/819: link d9/da/d33/f41 d9/da/d2a/d5e/f121 0 2024-04-26T23:01:39.148 INFO:tasks.workunit.client.0.smithi027.stdout:0/844: rmdir d0/d4/da/d15/d26/dc4/dc6 0 2024-04-26T23:01:39.149 INFO:tasks.workunit.client.0.smithi027.stdout:0/845: truncate d0/d4/da/d1a/d1e/f7f 1245906 0 2024-04-26T23:01:39.150 INFO:tasks.workunit.client.0.smithi027.stdout:3/931: symlink d6/d5e/l137 0 2024-04-26T23:01:39.155 INFO:tasks.workunit.client.0.smithi027.stdout:3/932: creat d6/dd3/d34/d2e/d70/d71/f138 x:0 0 0 2024-04-26T23:01:39.157 INFO:tasks.workunit.client.0.smithi027.stdout:3/933: getdents d6/dd3/d34/d2e/d70 0 2024-04-26T23:01:39.157 INFO:tasks.workunit.client.0.smithi027.stdout:3/934: fdatasync d6/dd3/d34/d2e/d9d/da7/dac/d74/f87 0 2024-04-26T23:01:39.157 INFO:tasks.workunit.client.0.smithi027.stdout:3/935: chown d6/dd3/d34/d2e/d9d 1478876 1 2024-04-26T23:01:39.160 INFO:tasks.workunit.client.0.smithi027.stdout:3/936: rename d6/dd3/d34/d2e/c122 to d6/dd3/d34/db3/c139 0 2024-04-26T23:01:39.163 INFO:tasks.workunit.client.0.smithi027.stdout:3/937: symlink d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/d5a/dcd/l13a 0 2024-04-26T23:01:39.173 INFO:tasks.workunit.client.0.smithi027.stdout:8/820: dread d9/da/d2a/d5e/d72/d84/d5d/f7d [0,4194304] 0 2024-04-26T23:01:39.183 INFO:tasks.workunit.client.0.smithi027.stdout:6/699: dwrite d1/df/d43/d56/d6b/f76 [0,4194304] 0 2024-04-26T23:01:39.185 INFO:tasks.workunit.client.0.smithi027.stdout:8/821: write d9/da/d33/d71/fc1 [1632457,105313] 0 2024-04-26T23:01:39.201 INFO:tasks.workunit.client.0.smithi027.stdout:1/941: dwrite d1/d26/d66/d73/db5/f114 [0,4194304] 0 2024-04-26T23:01:39.201 INFO:tasks.workunit.client.0.smithi027.stdout:1/942: write d1/d26/f29 [517424,77300] 0 2024-04-26T23:01:39.203 INFO:tasks.workunit.client.0.smithi027.stdout:1/943: mkdir d1/d26/d66/d73/d9e/d134 0 2024-04-26T23:01:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:4/889: dwrite da/dd/de/d14/d21/d41/d59/d74/faa [4194304,4194304] 0 2024-04-26T23:01:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:4/890: chown da/dd/de/d14/db3/lf3 44323101 1 2024-04-26T23:01:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:7/890: dwrite d0/d6c/f71 [0,4194304] 0 2024-04-26T23:01:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:7/891: chown d0/d6c/c116 11600 1 2024-04-26T23:01:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:8/822: dread d9/f17 [0,4194304] 0 2024-04-26T23:01:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:8/823: stat d9/da/d2a/d5e/d72/d84/db8/c45 0 2024-04-26T23:01:39.215 INFO:tasks.workunit.client.0.smithi027.stdout:6/700: write d1/df/d43/da8/d84/fb8 [2470836,72863] 0 2024-04-26T23:01:39.215 INFO:tasks.workunit.client.0.smithi027.stdout:4/891: mknod da/dd/de/d14/d17/d80/dab/dd6/c11c 0 2024-04-26T23:01:39.217 INFO:tasks.workunit.client.0.smithi027.stdout:7/892: rename d0/d28/d2b/d55/d69/d75/cf4 to d0/d28/d66/da3/c125 0 2024-04-26T23:01:39.218 INFO:tasks.workunit.client.0.smithi027.stdout:6/701: mknod d1/de/d57/d8e/cdd 0 2024-04-26T23:01:39.220 INFO:tasks.workunit.client.0.smithi027.stdout:7/893: unlink d0/f13 0 2024-04-26T23:01:39.220 INFO:tasks.workunit.client.0.smithi027.stdout:7/894: stat d0/d3d/dba 0 2024-04-26T23:01:39.223 INFO:tasks.workunit.client.0.smithi027.stdout:6/702: truncate d1/f69 295788 0 2024-04-26T23:01:39.223 INFO:tasks.workunit.client.0.smithi027.stdout:6/703: fsync d1/d30/d6c/d6f/d7d/f80 0 2024-04-26T23:01:39.224 INFO:tasks.workunit.client.0.smithi027.stdout:6/704: creat d1/fde x:0 0 0 2024-04-26T23:01:39.240 INFO:tasks.workunit.client.0.smithi027.stdout:6/705: dread d1/df/d49/f4f [0,4194304] 0 2024-04-26T23:01:39.244 INFO:tasks.workunit.client.0.smithi027.stdout:8/824: dread d9/da/d33/f60 [0,4194304] 0 2024-04-26T23:01:39.257 INFO:tasks.workunit.client.0.smithi027.stdout:7/895: dwrite d0/d28/d2b/d3a/d48/d56/d88/d73/f104 [0,4194304] 0 2024-04-26T23:01:39.260 INFO:tasks.workunit.client.0.smithi027.stdout:7/896: creat d0/d28/d66/da3/d124/f126 x:0 0 0 2024-04-26T23:01:39.264 INFO:tasks.workunit.client.0.smithi027.stdout:7/897: creat d0/d28/d2b/d3a/d48/d56/d88/d73/dd6/f127 x:0 0 0 2024-04-26T23:01:39.271 INFO:tasks.workunit.client.0.smithi027.stdout:6/706: write d1/de/f42 [2891004,101861] 0 2024-04-26T23:01:39.271 INFO:tasks.workunit.client.0.smithi027.stdout:1/944: dwrite d1/d26/d66/dab/de0/f27 [0,4194304] 0 2024-04-26T23:01:39.274 INFO:tasks.workunit.client.0.smithi027.stdout:6/707: mkdir d1/d30/d6c/d6f/dd3/ddf 0 2024-04-26T23:01:39.274 INFO:tasks.workunit.client.0.smithi027.stdout:6/708: chown d1/d30/d6c/c78 477 1 2024-04-26T23:01:39.274 INFO:tasks.workunit.client.0.smithi027.stdout:6/709: readlink d1/l95 0 2024-04-26T23:01:39.284 INFO:tasks.workunit.client.0.smithi027.stdout:1/945: link d1/d26/d86/l8f d1/d2a/deb/l135 0 2024-04-26T23:01:39.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:39 smithi094 ceph-mon[32889]: pgmap v94: 129 pgs: 129 active+clean; 1.5 GiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 196 MiB/s rd, 228 MiB/s wr, 628 op/s 2024-04-26T23:01:39.286 INFO:tasks.workunit.client.0.smithi027.stdout:6/710: mkdir d1/de/d57/d8e/dd0/de0 0 2024-04-26T23:01:39.288 INFO:tasks.workunit.client.0.smithi027.stdout:1/946: mknod d1/d26/d66/d73/db5/d105/c136 0 2024-04-26T23:01:39.288 INFO:tasks.workunit.client.0.smithi027.stdout:6/711: write d1/df/d43/f6e [2031625,88856] 0 2024-04-26T23:01:39.288 INFO:tasks.workunit.client.0.smithi027.stdout:6/712: dread - d1/df/d3c/fba zero size 2024-04-26T23:01:39.288 INFO:tasks.workunit.client.0.smithi027.stdout:6/713: write d1/df/d43/da8/d84/fcb [219248,107381] 0 2024-04-26T23:01:39.294 INFO:tasks.workunit.client.0.smithi027.stdout:6/714: rename d1/de/f1e to d1/de/fe1 0 2024-04-26T23:01:39.294 INFO:tasks.workunit.client.0.smithi027.stdout:6/715: write d1/df/d43/d56/d6b/f93 [135354,64187] 0 2024-04-26T23:01:39.295 INFO:tasks.workunit.client.0.smithi027.stdout:6/716: creat d1/d30/d6c/d6f/d7d/fe2 x:0 0 0 2024-04-26T23:01:39.295 INFO:tasks.workunit.client.0.smithi027.stdout:6/717: fsync d1/d30/f36 0 2024-04-26T23:01:39.296 INFO:tasks.workunit.client.0.smithi027.stdout:8/825: dwrite d9/da/d2a/d5e/f64 [0,4194304] 0 2024-04-26T23:01:39.296 INFO:tasks.workunit.client.0.smithi027.stdout:8/826: chown d9/de/d2e/l95 4705 1 2024-04-26T23:01:39.297 INFO:tasks.workunit.client.0.smithi027.stdout:8/827: readlink d9/da/d2a/d5e/d2b/lf4 0 2024-04-26T23:01:39.298 INFO:tasks.workunit.client.0.smithi027.stdout:8/828: symlink d9/da/d2a/d5e/d92/l122 0 2024-04-26T23:01:39.298 INFO:tasks.workunit.client.0.smithi027.stdout:8/829: getdents d9/da/d2a 0 2024-04-26T23:01:39.300 INFO:tasks.workunit.client.0.smithi027.stdout:8/830: unlink d9/de/d2e/d110/da7/fde 0 2024-04-26T23:01:39.301 INFO:tasks.workunit.client.0.smithi027.stdout:8/831: truncate d9/de/f14 1415071 0 2024-04-26T23:01:39.302 INFO:tasks.workunit.client.0.smithi027.stdout:7/898: dwrite d0/d28/d2b/d55/f7e [0,4194304] 0 2024-04-26T23:01:39.302 INFO:tasks.workunit.client.0.smithi027.stdout:7/899: chown d0/d2f/d113 484 1 2024-04-26T23:01:39.303 INFO:tasks.workunit.client.0.smithi027.stdout:8/832: creat d9/de/f123 x:0 0 0 2024-04-26T23:01:39.305 INFO:tasks.workunit.client.0.smithi027.stdout:6/718: mkdir d1/de/d57/de3 0 2024-04-26T23:01:39.305 INFO:tasks.workunit.client.0.smithi027.stdout:6/719: fdatasync d1/d30/d6c/f97 0 2024-04-26T23:01:39.305 INFO:tasks.workunit.client.0.smithi027.stdout:8/833: symlink d9/de/d2e/d110/d8d/da3/l124 0 2024-04-26T23:01:39.306 INFO:tasks.workunit.client.0.smithi027.stdout:7/900: mkdir d0/d6c/d128 0 2024-04-26T23:01:39.307 INFO:tasks.workunit.client.0.smithi027.stdout:6/720: symlink d1/le4 0 2024-04-26T23:01:39.308 INFO:tasks.workunit.client.0.smithi027.stdout:7/901: mkdir d0/d28/d9f/db8/d129 0 2024-04-26T23:01:39.308 INFO:tasks.workunit.client.0.smithi027.stdout:7/902: stat d0/d28/d2b/f34 0 2024-04-26T23:01:39.308 INFO:tasks.workunit.client.0.smithi027.stdout:7/903: fdatasync d0/d28/d2b/d3a/d48/d56/d88/d73/dd6/fea 0 2024-04-26T23:01:39.309 INFO:tasks.workunit.client.0.smithi027.stdout:7/904: readlink d0/d2f/l45 0 2024-04-26T23:01:39.309 INFO:tasks.workunit.client.0.smithi027.stdout:6/721: symlink d1/d6a/le5 0 2024-04-26T23:01:39.324 INFO:tasks.workunit.client.0.smithi027.stdout:1/947: dwrite d1/d26/d86/d5c/fdf [0,4194304] 0 2024-04-26T23:01:39.326 INFO:tasks.workunit.client.0.smithi027.stdout:1/948: creat d1/d26/d66/d73/d9e/d134/f137 x:0 0 0 2024-04-26T23:01:39.327 INFO:tasks.workunit.client.0.smithi027.stdout:1/949: fsync d1/d26/d6b/fa1 0 2024-04-26T23:01:39.327 INFO:tasks.workunit.client.0.smithi027.stdout:1/950: read d1/d5/f2f [549966,90541] 0 2024-04-26T23:01:39.329 INFO:tasks.workunit.client.0.smithi027.stdout:1/951: rmdir d1/d26/d66/dab/de0 39 2024-04-26T23:01:39.331 INFO:tasks.workunit.client.0.smithi027.stdout:1/952: mkdir d1/d26/d66/d73/d9e/db6/d138 0 2024-04-26T23:01:39.331 INFO:tasks.workunit.client.0.smithi027.stdout:1/953: fdatasync d1/d26/d6b/f7f 0 2024-04-26T23:01:39.332 INFO:tasks.workunit.client.0.smithi027.stdout:1/954: truncate d1/d26/d6b/d42/d79/f11b 694015 0 2024-04-26T23:01:39.338 INFO:tasks.workunit.client.0.smithi027.stdout:8/834: dread d9/da/d2a/d5e/d46/ddd/fc6 [0,4194304] 0 2024-04-26T23:01:39.338 INFO:tasks.workunit.client.0.smithi027.stdout:8/835: getdents d9/de/d2e/d110/dea 0 2024-04-26T23:01:39.344 INFO:tasks.workunit.client.0.smithi027.stdout:7/905: dwrite d0/d28/d2b/d3a/d48/f72 [4194304,4194304] 0 2024-04-26T23:01:39.346 INFO:tasks.workunit.client.0.smithi027.stdout:9/761: sync 2024-04-26T23:01:39.356 INFO:tasks.workunit.client.0.smithi027.stdout:9/762: write d4/d10/d40/f64 [28572,46056] 0 2024-04-26T23:01:39.360 INFO:tasks.workunit.client.0.smithi027.stdout:7/906: dread d0/d28/f18 [4194304,4194304] 0 2024-04-26T23:01:39.360 INFO:tasks.workunit.client.0.smithi027.stdout:7/907: stat d0/d28/d2b/d55/d59/c83 0 2024-04-26T23:01:39.360 INFO:tasks.workunit.client.0.smithi027.stdout:9/763: read d4/f52 [610469,24443] 0 2024-04-26T23:01:39.360 INFO:tasks.workunit.client.0.smithi027.stdout:9/764: dread - d4/d17/d2e/fcc zero size 2024-04-26T23:01:39.361 INFO:tasks.workunit.client.0.smithi027.stdout:7/908: creat d0/d28/d2b/d3a/d48/dd2/f12a x:0 0 0 2024-04-26T23:01:39.361 INFO:tasks.workunit.client.0.smithi027.stdout:7/909: stat d0/d28/d2b/d3a/d48/d56/d88/d73/dd6/l10e 0 2024-04-26T23:01:39.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:39 smithi027 ceph-mon[28034]: pgmap v94: 129 pgs: 129 active+clean; 1.5 GiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 196 MiB/s rd, 228 MiB/s wr, 628 op/s 2024-04-26T23:01:39.371 INFO:tasks.workunit.client.0.smithi027.stdout:6/722: dwrite d1/f69 [0,4194304] 0 2024-04-26T23:01:39.374 INFO:tasks.workunit.client.0.smithi027.stdout:6/723: truncate d1/d30/d6c/d6f/d7d/fbb 1235928 0 2024-04-26T23:01:39.377 INFO:tasks.workunit.client.0.smithi027.stdout:1/955: dwrite d1/d26/d7d/f76 [0,4194304] 0 2024-04-26T23:01:39.378 INFO:tasks.workunit.client.0.smithi027.stdout:6/724: symlink d1/de/d57/d8e/dd0/dc9/le6 0 2024-04-26T23:01:39.379 INFO:tasks.workunit.client.0.smithi027.stdout:7/910: dread d0/d28/d2b/f34 [0,4194304] 0 2024-04-26T23:01:39.379 INFO:tasks.workunit.client.0.smithi027.stdout:7/911: readlink d0/l33 0 2024-04-26T23:01:39.380 INFO:tasks.workunit.client.0.smithi027.stdout:9/765: write d4/d17/f93 [495699,34645] 0 2024-04-26T23:01:39.380 INFO:tasks.workunit.client.0.smithi027.stdout:9/766: readlink d4/d10/d67/le3 0 2024-04-26T23:01:39.381 INFO:tasks.workunit.client.0.smithi027.stdout:1/956: creat d1/f139 x:0 0 0 2024-04-26T23:01:39.381 INFO:tasks.workunit.client.0.smithi027.stdout:1/957: write d1/d26/d86/d5c/f104 [1161042,86737] 0 2024-04-26T23:01:39.381 INFO:tasks.workunit.client.0.smithi027.stdout:1/958: chown d1/dc/c41 147602 1 2024-04-26T23:01:39.381 INFO:tasks.workunit.client.0.smithi027.stdout:6/725: getdents d1/df/d3c/d9a 0 2024-04-26T23:01:39.383 INFO:tasks.workunit.client.0.smithi027.stdout:7/912: creat d0/d28/d2b/d55/df9/f12b x:0 0 0 2024-04-26T23:01:39.386 INFO:tasks.workunit.client.0.smithi027.stdout:8/836: dwrite d9/de/d2e/d110/d8d/f97 [0,4194304] 0 2024-04-26T23:01:39.389 INFO:tasks.workunit.client.0.smithi027.stdout:9/767: rename d4/d10/d40/f9a to d4/d10/d40/d58/d75/d84/d102/f10f 0 2024-04-26T23:01:39.389 INFO:tasks.workunit.client.0.smithi027.stdout:9/768: write d4/d17/d2e/d46/fd3 [4426544,128766] 0 2024-04-26T23:01:39.390 INFO:tasks.workunit.client.0.smithi027.stdout:1/959: creat d1/d26/d86/d5c/dd7/f13a x:0 0 0 2024-04-26T23:01:39.390 INFO:tasks.workunit.client.0.smithi027.stdout:1/960: chown d1/d26/d36/c3b 547445 1 2024-04-26T23:01:39.394 INFO:tasks.workunit.client.0.smithi027.stdout:9/769: symlink d4/d10/d25/dc9/l110 0 2024-04-26T23:01:39.394 INFO:tasks.workunit.client.0.smithi027.stdout:1/961: mknod d1/d26/d66/d73/d9e/d134/c13b 0 2024-04-26T23:01:39.397 INFO:tasks.workunit.client.0.smithi027.stdout:9/770: rename d4/d10/d40/fab to d4/d17/d2e/d10e/f111 0 2024-04-26T23:01:39.397 INFO:tasks.workunit.client.0.smithi027.stdout:1/962: getdents d1/d26/d7d 0 2024-04-26T23:01:39.399 INFO:tasks.workunit.client.0.smithi027.stdout:9/771: rename d4/d17/d2e/d46/cba to d4/d17/d2e/d46/da0/c112 0 2024-04-26T23:01:39.403 INFO:tasks.workunit.client.0.smithi027.stdout:2/754: sync 2024-04-26T23:01:39.403 INFO:tasks.workunit.client.0.smithi027.stdout:4/892: sync 2024-04-26T23:01:39.403 INFO:tasks.workunit.client.0.smithi027.stdout:2/755: readlink d2/d5/db/d1b/d1c/d25/d19/d23/l50 0 2024-04-26T23:01:39.404 INFO:tasks.workunit.client.0.smithi027.stdout:3/938: sync 2024-04-26T23:01:39.404 INFO:tasks.workunit.client.0.smithi027.stdout:3/939: fsync d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/dd6/ff3 0 2024-04-26T23:01:39.404 INFO:tasks.workunit.client.0.smithi027.stdout:3/940: readlink d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/l101 0 2024-04-26T23:01:39.404 INFO:tasks.workunit.client.0.smithi027.stdout:3/941: fdatasync d6/dd3/f11e 0 2024-04-26T23:01:39.404 INFO:tasks.workunit.client.0.smithi027.stdout:4/893: mkdir da/dd/de/d14/db3/d114/d11d 0 2024-04-26T23:01:39.404 INFO:tasks.workunit.client.0.smithi027.stdout:4/894: write da/dd/de/d14/d21/d41/df7/ff9 [923890,112670] 0 2024-04-26T23:01:39.406 INFO:tasks.workunit.client.0.smithi027.stdout:4/895: creat da/dd/de/f11e x:0 0 0 2024-04-26T23:01:39.406 INFO:tasks.workunit.client.0.smithi027.stdout:4/896: stat da/dd/de/d14/d17/d80/dab/f3a 0 2024-04-26T23:01:39.406 INFO:tasks.workunit.client.0.smithi027.stdout:4/897: fsync da/dd/de/d14/d17/f45 0 2024-04-26T23:01:39.407 INFO:tasks.workunit.client.0.smithi027.stdout:4/898: unlink da/dd/de/d14/d21/d67/c10c 0 2024-04-26T23:01:39.420 INFO:tasks.workunit.client.0.smithi027.stdout:1/963: dread d1/d5/f46 [0,4194304] 0 2024-04-26T23:01:39.420 INFO:tasks.workunit.client.0.smithi027.stdout:1/964: fdatasync d1/dc/f37 0 2024-04-26T23:01:39.421 INFO:tasks.workunit.client.0.smithi027.stdout:1/965: readlink d1/d26/d66/le1 0 2024-04-26T23:01:39.421 INFO:tasks.workunit.client.0.smithi027.stdout:1/966: chown d1/d26/d7d/c11a 1012839970 1 2024-04-26T23:01:39.421 INFO:tasks.workunit.client.0.smithi027.stdout:1/967: truncate d1/f94 300629 0 2024-04-26T23:01:39.436 INFO:tasks.workunit.client.0.smithi027.stdout:9/772: write d4/d77/f7a [3799502,65014] 0 2024-04-26T23:01:39.436 INFO:tasks.workunit.client.0.smithi027.stdout:9/773: dread - d4/d17/d2e/d81/da7/db8/feb zero size 2024-04-26T23:01:39.436 INFO:tasks.workunit.client.0.smithi027.stdout:8/837: dwrite d9/fcc [0,4194304] 0 2024-04-26T23:01:39.436 INFO:tasks.workunit.client.0.smithi027.stdout:8/838: write d9/da/d2a/d5e/d46/ddd/fc6 [4835229,38601] 0 2024-04-26T23:01:39.438 INFO:tasks.workunit.client.0.smithi027.stdout:9/774: link d4/d17/d2e/d46/da0/d8d/fa8 d4/d10/d40/f113 0 2024-04-26T23:01:39.440 INFO:tasks.workunit.client.0.smithi027.stdout:9/775: mkdir d4/d17/d2e/d46/da0/d8d/df2/d114 0 2024-04-26T23:01:39.440 INFO:tasks.workunit.client.0.smithi027.stdout:9/776: chown d4/d17/d2e/d81/c88 83 1 2024-04-26T23:01:39.448 INFO:tasks.workunit.client.0.smithi027.stdout:2/756: dwrite d2/d5/db/d1b/d1c/f77 [0,4194304] 0 2024-04-26T23:01:39.448 INFO:tasks.workunit.client.0.smithi027.stdout:2/757: write d2/d5/db/d1b/d1c/d25/d19/d44/fb1 [368968,120272] 0 2024-04-26T23:01:39.448 INFO:tasks.workunit.client.0.smithi027.stdout:2/758: write d2/d5/db/de/f30 [2722904,50212] 0 2024-04-26T23:01:39.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/759: dread d2/d5/db/d1b/d1c/d25/d19/f45 [0,4194304] 0 2024-04-26T23:01:39.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/760: write d2/d5/db/de/d21/fc2 [593554,70224] 0 2024-04-26T23:01:39.468 INFO:tasks.workunit.client.0.smithi027.stdout:2/761: creat d2/d5/db/de/d21/d8f/deb/fec x:0 0 0 2024-04-26T23:01:39.468 INFO:tasks.workunit.client.0.smithi027.stdout:2/762: readlink d2/d5/lc1 0 2024-04-26T23:01:39.468 INFO:tasks.workunit.client.0.smithi027.stdout:2/763: write d2/d5/db/de/d21/fd7 [87679,58976] 0 2024-04-26T23:01:39.470 INFO:tasks.workunit.client.0.smithi027.stdout:2/764: mkdir d2/d5/db/d1b/d1c/d25/d19/d23/ded 0 2024-04-26T23:01:39.470 INFO:tasks.workunit.client.0.smithi027.stdout:2/765: write d2/d5/db/d1b/d1c/d25/d19/d23/f47 [4575327,83480] 0 2024-04-26T23:01:39.473 INFO:tasks.workunit.client.0.smithi027.stdout:2/766: mkdir d2/d5/db/d1b/d1c/d25/d19/d44/dee 0 2024-04-26T23:01:39.473 INFO:tasks.workunit.client.0.smithi027.stdout:2/767: chown d2/d5/db/c87 924 1 2024-04-26T23:01:39.473 INFO:tasks.workunit.client.0.smithi027.stdout:2/768: chown d2/d5/db/de/d21/d8f/l9f 28404 1 2024-04-26T23:01:39.474 INFO:tasks.workunit.client.0.smithi027.stdout:1/968: dwrite d1/d26/d66/f12f [0,4194304] 0 2024-04-26T23:01:39.474 INFO:tasks.workunit.client.0.smithi027.stdout:8/839: dwrite d9/de/d2e/d110/df0/f118 [0,4194304] 0 2024-04-26T23:01:39.474 INFO:tasks.workunit.client.0.smithi027.stdout:8/840: truncate d9/da/d2a/fa2 1582131 0 2024-04-26T23:01:39.475 INFO:tasks.workunit.client.0.smithi027.stdout:8/841: fdatasync d9/da/d2a/f89 0 2024-04-26T23:01:39.475 INFO:tasks.workunit.client.0.smithi027.stdout:2/769: unlink d2/d5/db/de/d21/d8f/d88/ld9 0 2024-04-26T23:01:39.477 INFO:tasks.workunit.client.0.smithi027.stdout:8/842: link d9/da/d2a/d5e/d72/d84/d4f/dd3/cd7 d9/da/d2a/d5e/d9e/c125 0 2024-04-26T23:01:39.478 INFO:tasks.workunit.client.0.smithi027.stdout:8/843: readlink d9/da/d2a/d6b/l98 0 2024-04-26T23:01:39.478 INFO:tasks.workunit.client.0.smithi027.stdout:1/969: rename d1/d26/d66/d73/dae/dd0/f115 to d1/d26/dc8/f13c 0 2024-04-26T23:01:39.478 INFO:tasks.workunit.client.0.smithi027.stdout:0/846: sync 2024-04-26T23:01:39.478 INFO:tasks.workunit.client.0.smithi027.stdout:5/861: sync 2024-04-26T23:01:39.478 INFO:tasks.workunit.client.0.smithi027.stdout:1/970: write d1/d26/d66/d91/f109 [884980,125188] 0 2024-04-26T23:01:39.478 INFO:tasks.workunit.client.0.smithi027.stdout:5/862: fdatasync d4/d24/d22/d29/d39/f7f 0 2024-04-26T23:01:39.478 INFO:tasks.workunit.client.0.smithi027.stdout:5/863: readlink d4/d24/d22/d29/d39/l52 0 2024-04-26T23:01:39.478 INFO:tasks.workunit.client.0.smithi027.stdout:8/844: mkdir d9/de/d126 0 2024-04-26T23:01:39.479 INFO:tasks.workunit.client.0.smithi027.stdout:8/845: creat d9/da/d2a/d6b/f127 x:0 0 0 2024-04-26T23:01:39.480 INFO:tasks.workunit.client.0.smithi027.stdout:2/770: rename d2/d5/db/d1b/d1c/d25/d6a/f4c to d2/d5/db/d1b/d1c/d25/d19/d23/d7c/fef 0 2024-04-26T23:01:39.482 INFO:tasks.workunit.client.0.smithi027.stdout:5/864: symlink d4/d24/d22/d29/d39/d84/l119 0 2024-04-26T23:01:39.482 INFO:tasks.workunit.client.0.smithi027.stdout:5/865: truncate d4/f10a 284111 0 2024-04-26T23:01:39.482 INFO:tasks.workunit.client.0.smithi027.stdout:5/866: chown d4/d9f/dc0/c115 1302 1 2024-04-26T23:01:39.483 INFO:tasks.workunit.client.0.smithi027.stdout:8/846: truncate d9/de/d2e/d110/df0/f118 1048000 0 2024-04-26T23:01:39.483 INFO:tasks.workunit.client.0.smithi027.stdout:8/847: readlink d9/de/d2e/d110/df0/l107 0 2024-04-26T23:01:39.483 INFO:tasks.workunit.client.0.smithi027.stdout:8/848: dread - d9/da/d2a/fb6 zero size 2024-04-26T23:01:39.484 INFO:tasks.workunit.client.0.smithi027.stdout:1/971: creat d1/d2a/f13d x:0 0 0 2024-04-26T23:01:39.486 INFO:tasks.workunit.client.0.smithi027.stdout:1/972: mkdir d1/d26/d6b/d8e/d13e 0 2024-04-26T23:01:39.486 INFO:tasks.workunit.client.0.smithi027.stdout:1/973: fsync d1/d2a/deb/ffc 0 2024-04-26T23:01:39.486 INFO:tasks.workunit.client.0.smithi027.stdout:1/974: read - d1/d26/d6b/d42/fda zero size 2024-04-26T23:01:39.486 INFO:tasks.workunit.client.0.smithi027.stdout:1/975: write d1/d2a/ff7 [1542914,35230] 0 2024-04-26T23:01:39.487 INFO:tasks.workunit.client.0.smithi027.stdout:2/771: mkdir d2/d5/db/d1b/d1c/df0 0 2024-04-26T23:01:39.488 INFO:tasks.workunit.client.0.smithi027.stdout:8/849: rmdir d9/da/d2a/d5e/d2b/d52/d108 0 2024-04-26T23:01:39.489 INFO:tasks.workunit.client.0.smithi027.stdout:5/867: dread d4/d24/d5f/dac/f5a [0,4194304] 0 2024-04-26T23:01:39.489 INFO:tasks.workunit.client.0.smithi027.stdout:5/868: truncate d4/d1a/d2b/db8/fd5 1165391 0 2024-04-26T23:01:39.493 INFO:tasks.workunit.client.0.smithi027.stdout:1/976: rename d1/d26/d6b/d8e/dc5/fe7 to d1/d26/d6b/f13f 0 2024-04-26T23:01:39.493 INFO:tasks.workunit.client.0.smithi027.stdout:8/850: rename d9 to d9/da/d2a/d5e/d72/d84/d5d/d6c/d128 22 2024-04-26T23:01:39.493 INFO:tasks.workunit.client.0.smithi027.stdout:8/851: chown d9/da/d2a/d5e/d92/lfd 6896962 1 2024-04-26T23:01:39.493 INFO:tasks.workunit.client.0.smithi027.stdout:1/977: truncate d1/d26/d86/fa2 593228 0 2024-04-26T23:01:39.493 INFO:tasks.workunit.client.0.smithi027.stdout:1/978: write d1/d26/d66/d91/fed [4071845,46034] 0 2024-04-26T23:01:39.493 INFO:tasks.workunit.client.0.smithi027.stdout:8/852: write d9/de/d2e/d110/d8d/da3/f10a [246542,17710] 0 2024-04-26T23:01:39.495 INFO:tasks.workunit.client.0.smithi027.stdout:5/869: rename d4/f64 to d4/d1a/d2b/daa/d10c/f11a 0 2024-04-26T23:01:39.519 INFO:tasks.workunit.client.0.smithi027.stdout:5/870: dread f1 [4194304,4194304] 0 2024-04-26T23:01:39.519 INFO:tasks.workunit.client.0.smithi027.stdout:5/871: chown d4/d24/d22/d29/d2a/d3b/fa2 367 1 2024-04-26T23:01:39.519 INFO:tasks.workunit.client.0.smithi027.stdout:5/872: chown d4/d1a/l109 30 1 2024-04-26T23:01:39.519 INFO:tasks.workunit.client.0.smithi027.stdout:5/873: dread - d4/d24/d22/d29/d2a/fe1 zero size 2024-04-26T23:01:39.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:39 smithi080 ceph-mon[32324]: pgmap v94: 129 pgs: 129 active+clean; 1.5 GiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 196 MiB/s rd, 228 MiB/s wr, 628 op/s 2024-04-26T23:01:39.540 INFO:tasks.workunit.client.0.smithi027.stdout:2/772: dwrite d2/d5/db/de/d21/fd7 [0,4194304] 0 2024-04-26T23:01:39.541 INFO:tasks.workunit.client.0.smithi027.stdout:1/979: dwrite d1/d26/d66/f103 [0,4194304] 0 2024-04-26T23:01:39.547 INFO:tasks.workunit.client.0.smithi027.stdout:2/773: rmdir d2/d5/db/de/d21/d8f/d88 39 2024-04-26T23:01:39.548 INFO:tasks.workunit.client.0.smithi027.stdout:1/980: mknod d1/d26/d36/d56/df1/df2/c140 0 2024-04-26T23:01:39.548 INFO:tasks.workunit.client.0.smithi027.stdout:1/981: readlink d1/d26/d86/d5c/lc9 0 2024-04-26T23:01:39.552 INFO:tasks.workunit.client.0.smithi027.stdout:0/847: dwrite d0/df/d9d/fb5 [0,4194304] 0 2024-04-26T23:01:39.554 INFO:tasks.workunit.client.0.smithi027.stdout:2/774: dread d2/d5/db/d1b/d1c/d25/d19/d23/fab [0,4194304] 0 2024-04-26T23:01:39.555 INFO:tasks.workunit.client.0.smithi027.stdout:0/848: creat d0/df/d9d/d9f/dbb/f102 x:0 0 0 2024-04-26T23:01:39.563 INFO:tasks.workunit.client.0.smithi027.stdout:8/853: dwrite d9/f16 [0,4194304] 0 2024-04-26T23:01:39.567 INFO:tasks.workunit.client.0.smithi027.stdout:2/775: dread d2/d5/db/d1b/d1c/d25/d19/d23/fab [0,4194304] 0 2024-04-26T23:01:39.571 INFO:tasks.workunit.client.0.smithi027.stdout:2/776: unlink d2/d5/db/de/d21/dc4/d39/laf 0 2024-04-26T23:01:39.571 INFO:tasks.workunit.client.0.smithi027.stdout:2/777: write d2/d5/db/d1b/d1c/d25/fa4 [447914,104524] 0 2024-04-26T23:01:39.574 INFO:tasks.workunit.client.0.smithi027.stdout:2/778: mkdir d2/d5/db/d1b/d1c/d25/dc0/df1 0 2024-04-26T23:01:39.576 INFO:tasks.workunit.client.0.smithi027.stdout:2/779: mknod d2/d5/db/de/d21/dc4/cf2 0 2024-04-26T23:01:39.602 INFO:tasks.workunit.client.0.smithi027.stdout:5/874: dwrite d4/d24/d22/d29/d39/f6d [0,4194304] 0 2024-04-26T23:01:39.606 INFO:tasks.workunit.client.0.smithi027.stdout:5/875: rename d4/d24/d5f/dac/d40/d54/f75 to d4/d24/d22/d29/d39/dc7/d104/f11b 0 2024-04-26T23:01:39.608 INFO:tasks.workunit.client.0.smithi027.stdout:5/876: creat d4/d1a/d2b/f11c x:0 0 0 2024-04-26T23:01:39.608 INFO:tasks.workunit.client.0.smithi027.stdout:5/877: chown d4/d1a/dcb/dd1/dd9/cda 535311770 1 2024-04-26T23:01:39.610 INFO:tasks.workunit.client.0.smithi027.stdout:5/878: rename d4/d1a/dcb/lec to d4/d24/d22/d29/d39/df6/l11d 0 2024-04-26T23:01:39.611 INFO:tasks.workunit.client.0.smithi027.stdout:0/849: dwrite d0/d4/da/d1a/f90 [0,4194304] 0 2024-04-26T23:01:39.628 INFO:tasks.workunit.client.0.smithi027.stdout:6/726: sync 2024-04-26T23:01:39.628 INFO:tasks.workunit.client.0.smithi027.stdout:7/913: sync 2024-04-26T23:01:39.628 INFO:tasks.workunit.client.0.smithi027.stdout:7/914: stat d0/d28/d2b/d3a/d48/dd2 0 2024-04-26T23:01:39.629 INFO:tasks.workunit.client.0.smithi027.stdout:6/727: creat d1/de/d57/d8e/dd0/dc9/fe7 x:0 0 0 2024-04-26T23:01:39.630 INFO:tasks.workunit.client.0.smithi027.stdout:7/915: creat d0/d28/d2b/d3a/d48/dc0/f12c x:0 0 0 2024-04-26T23:01:39.631 INFO:tasks.workunit.client.0.smithi027.stdout:6/728: creat d1/de/d57/de3/fe8 x:0 0 0 2024-04-26T23:01:39.632 INFO:tasks.workunit.client.0.smithi027.stdout:7/916: getdents d0/d28/d66 0 2024-04-26T23:01:39.632 INFO:tasks.workunit.client.0.smithi027.stdout:7/917: truncate d0/d28/d2b/d3a/f3f 5618484 0 2024-04-26T23:01:39.634 INFO:tasks.workunit.client.0.smithi027.stdout:5/879: read d4/d24/d22/dab/ffb [4498540,121777] 0 2024-04-26T23:01:39.634 INFO:tasks.workunit.client.0.smithi027.stdout:2/780: dwrite d2/d5/f7b [0,4194304] 0 2024-04-26T23:01:39.639 INFO:tasks.workunit.client.0.smithi027.stdout:5/880: creat d4/d24/d22/d29/d2a/d3b/d50/f11e x:0 0 0 2024-04-26T23:01:39.639 INFO:tasks.workunit.client.0.smithi027.stdout:6/729: dread d1/d30/d6c/f97 [0,4194304] 0 2024-04-26T23:01:39.640 INFO:tasks.workunit.client.0.smithi027.stdout:2/781: symlink d2/d5/db/de/d21/dc4/d39/lf3 0 2024-04-26T23:01:39.643 INFO:tasks.workunit.client.0.smithi027.stdout:5/881: write d4/d24/d22/d29/f58 [217231,94226] 0 2024-04-26T23:01:39.645 INFO:tasks.workunit.client.0.smithi027.stdout:6/730: getdents d1 0 2024-04-26T23:01:39.645 INFO:tasks.workunit.client.0.smithi027.stdout:2/782: link d2/d5/db/de/d21/d8f/deb/fec d2/d5/db/d1b/d1c/d25/d19/d23/d82/ff4 0 2024-04-26T23:01:39.645 INFO:tasks.workunit.client.0.smithi027.stdout:2/783: chown d2/d5/f7b 2758 1 2024-04-26T23:01:39.645 INFO:tasks.workunit.client.0.smithi027.stdout:2/784: stat d2/d5/db/de/f42 0 2024-04-26T23:01:39.646 INFO:tasks.workunit.client.0.smithi027.stdout:2/785: write d2/d5/db/de/d21/dc4/f43 [4255885,107346] 0 2024-04-26T23:01:39.649 INFO:tasks.workunit.client.0.smithi027.stdout:5/882: creat d4/d24/d22/d29/d39/df6/f11f x:0 0 0 2024-04-26T23:01:39.650 INFO:tasks.workunit.client.0.smithi027.stdout:0/850: dwrite d0/d4/da/d15/d26/f8a [0,4194304] 0 2024-04-26T23:01:39.651 INFO:tasks.workunit.client.0.smithi027.stdout:6/731: rename d1/de/d57/d8e/dd0/dc9/fca to d1/d6a/fe9 0 2024-04-26T23:01:39.653 INFO:tasks.workunit.client.0.smithi027.stdout:0/851: mkdir d0/d4/da/d15/d26/dc4/d103 0 2024-04-26T23:01:39.653 INFO:tasks.workunit.client.0.smithi027.stdout:0/852: write d0/df/d3f/d64/f71 [739312,78373] 0 2024-04-26T23:01:39.654 INFO:tasks.workunit.client.0.smithi027.stdout:6/732: getdents d1/d30/d6c/d6f/dd3 0 2024-04-26T23:01:39.654 INFO:tasks.workunit.client.0.smithi027.stdout:2/786: dread d2/d5/db/d1b/d1c/d25/d19/d44/fb1 [0,4194304] 0 2024-04-26T23:01:39.655 INFO:tasks.workunit.client.0.smithi027.stdout:0/853: creat d0/d4/da/d1a/d2a/f104 x:0 0 0 2024-04-26T23:01:39.655 INFO:tasks.workunit.client.0.smithi027.stdout:0/854: chown d0/df/d9d/de5/lf7 178472 1 2024-04-26T23:01:39.659 INFO:tasks.workunit.client.0.smithi027.stdout:6/733: rmdir d1/de/d57/d8e/dd0/de0 0 2024-04-26T23:01:39.659 INFO:tasks.workunit.client.0.smithi027.stdout:6/734: truncate d1/df/d3c/d9a/fbc 22477 0 2024-04-26T23:01:39.659 INFO:tasks.workunit.client.0.smithi027.stdout:6/735: readlink d1/d30/l41 0 2024-04-26T23:01:39.662 INFO:tasks.workunit.client.0.smithi027.stdout:0/855: unlink d0/df/d3f/d64/la0 0 2024-04-26T23:01:39.662 INFO:tasks.workunit.client.0.smithi027.stdout:0/856: chown d0/d4/da/d1a/d2a/fd1 309568 1 2024-04-26T23:01:39.663 INFO:tasks.workunit.client.0.smithi027.stdout:2/787: rename d2/d5/f7b to d2/d5/db/ff5 0 2024-04-26T23:01:39.664 INFO:tasks.workunit.client.0.smithi027.stdout:0/857: truncate d0/d4/da/d1a/d2a/fd1 3982062 0 2024-04-26T23:01:39.668 INFO:tasks.workunit.client.0.smithi027.stdout:2/788: link d2/d5/db/de/d21/dc4/d39/lf3 d2/d5/db/de/d21/d8f/d88/de7/lf6 0 2024-04-26T23:01:39.668 INFO:tasks.workunit.client.0.smithi027.stdout:2/789: read - d2/d5/db/d1b/d1c/d25/d19/d23/d82/fd5 zero size 2024-04-26T23:01:39.668 INFO:tasks.workunit.client.0.smithi027.stdout:2/790: chown d2/d5/db/d1b/d1c/d25/d19/f45 1870636069 1 2024-04-26T23:01:39.669 INFO:tasks.workunit.client.0.smithi027.stdout:2/791: mknod d2/db9/cf7 0 2024-04-26T23:01:39.669 INFO:tasks.workunit.client.0.smithi027.stdout:2/792: chown d2/d5/db/de/d21/d8f/d88/fe9 672225 1 2024-04-26T23:01:39.670 INFO:tasks.workunit.client.0.smithi027.stdout:2/793: symlink d2/d5/db/d1b/d1c/d25/d19/lf8 0 2024-04-26T23:01:39.671 INFO:tasks.workunit.client.0.smithi027.stdout:2/794: rename d2/d5/db/f6c to d2/d5/db/d1b/d1c/d25/d6a/ff9 0 2024-04-26T23:01:39.672 INFO:tasks.workunit.client.0.smithi027.stdout:2/795: creat d2/d5/db/d1b/d1c/d25/d19/d23/d7c/ffa x:0 0 0 2024-04-26T23:01:39.673 INFO:tasks.workunit.client.0.smithi027.stdout:2/796: truncate d2/d5/fd0 424592 0 2024-04-26T23:01:39.675 INFO:tasks.workunit.client.0.smithi027.stdout:2/797: rename d2/d5/db/de/l84 to d2/d5/db/d1b/d1c/d25/d19/d23/ded/lfb 0 2024-04-26T23:01:39.688 INFO:tasks.workunit.client.0.smithi027.stdout:5/883: dwrite d4/d1a/f94 [4194304,4194304] 0 2024-04-26T23:01:39.691 INFO:tasks.workunit.client.0.smithi027.stdout:5/884: symlink d4/d1a/l120 0 2024-04-26T23:01:39.693 INFO:tasks.workunit.client.0.smithi027.stdout:5/885: truncate d4/d24/d22/d29/d39/d61/fde 54167 0 2024-04-26T23:01:39.695 INFO:tasks.workunit.client.0.smithi027.stdout:5/886: creat d4/d1a/dcb/dd1/f121 x:0 0 0 2024-04-26T23:01:39.696 INFO:tasks.workunit.client.0.smithi027.stdout:5/887: write d4/d24/d22/dab/d16/d4b/d81/fe3 [740985,90841] 0 2024-04-26T23:01:39.696 INFO:tasks.workunit.client.0.smithi027.stdout:5/888: stat d4/d9f/dc5/lce 0 2024-04-26T23:01:39.702 INFO:tasks.workunit.client.0.smithi027.stdout:6/736: dwrite d1/f2 [4194304,4194304] 0 2024-04-26T23:01:39.702 INFO:tasks.workunit.client.0.smithi027.stdout:6/737: dread - d1/de/d57/de3/fe8 zero size 2024-04-26T23:01:39.702 INFO:tasks.workunit.client.0.smithi027.stdout:0/858: dread d0/df/d3f/d4a/ffc [0,4194304] 0 2024-04-26T23:01:39.707 INFO:tasks.workunit.client.0.smithi027.stdout:0/859: mknod d0/d4/da/d1a/dc3/c105 0 2024-04-26T23:01:39.715 INFO:tasks.workunit.client.0.smithi027.stdout:2/798: dwrite d2/d5/db/de/d21/d8f/f95 [0,4194304] 0 2024-04-26T23:01:39.717 INFO:tasks.workunit.client.0.smithi027.stdout:2/799: unlink d2/d5/db/d1b/d1c/d25/d19/d23/l50 0 2024-04-26T23:01:39.717 INFO:tasks.workunit.client.0.smithi027.stdout:6/738: dread d1/df/d43/f45 [0,4194304] 0 2024-04-26T23:01:39.719 INFO:tasks.workunit.client.0.smithi027.stdout:2/800: link d2/d5/db/d1b/d1c/d25/d19/d23/d7c/ffa d2/d5/db/d1b/d1c/d25/d19/d23/d82/ffc 0 2024-04-26T23:01:39.720 INFO:tasks.workunit.client.0.smithi027.stdout:2/801: chown d2/d5/db/d1b/d1c/d25/d19/d23/fd8 57988 1 2024-04-26T23:01:39.720 INFO:tasks.workunit.client.0.smithi027.stdout:2/802: read - d2/d5/db/d1b/d1c/d25/d19/d23/fd3 zero size 2024-04-26T23:01:39.720 INFO:tasks.workunit.client.0.smithi027.stdout:2/803: chown d2/d5/db/d1b/d1c/d25/dc0 1085922 1 2024-04-26T23:01:39.721 INFO:tasks.workunit.client.0.smithi027.stdout:6/739: rename d1/df/d43/d56/d6b/l9c to d1/df/dc2/lea 0 2024-04-26T23:01:39.724 INFO:tasks.workunit.client.0.smithi027.stdout:2/804: link d2/d5/db/d1b/f38 d2/d5/db/d1b/d1c/d25/dc0/df1/ffd 0 2024-04-26T23:01:39.724 INFO:tasks.workunit.client.0.smithi027.stdout:2/805: write d2/d5/db/d1b/fb8 [112705,20892] 0 2024-04-26T23:01:39.724 INFO:tasks.workunit.client.0.smithi027.stdout:6/740: link d1/d30/l38 d1/df/dc2/leb 0 2024-04-26T23:01:39.725 INFO:tasks.workunit.client.0.smithi027.stdout:6/741: truncate d1/f4e 2285886 0 2024-04-26T23:01:39.727 INFO:tasks.workunit.client.0.smithi027.stdout:6/742: link d1/df/d43/d56/d6b/f8b d1/d30/d6c/d6f/dd3/ddf/fec 0 2024-04-26T23:01:39.727 INFO:tasks.workunit.client.0.smithi027.stdout:6/743: chown d1/df/d43/f5d 4 1 2024-04-26T23:01:39.741 INFO:tasks.workunit.client.0.smithi027.stdout:5/889: dwrite d4/d24/f98 [0,4194304] 0 2024-04-26T23:01:39.755 INFO:tasks.workunit.client.0.smithi027.stdout:0/860: dread d0/d4/da/f11 [0,4194304] 0 2024-04-26T23:01:39.762 INFO:tasks.workunit.client.0.smithi027.stdout:0/861: rename d0/d4/da/d15/d55/la1 to d0/df/d3f/d64/da8/l106 0 2024-04-26T23:01:39.763 INFO:tasks.workunit.client.0.smithi027.stdout:2/806: dwrite d2/d5/db/d1b/d1c/d25/d19/d23/fd8 [0,4194304] 0 2024-04-26T23:01:39.770 INFO:tasks.workunit.client.0.smithi027.stdout:2/807: creat d2/d5/db/de/d21/d8f/ffe x:0 0 0 2024-04-26T23:01:39.770 INFO:tasks.workunit.client.0.smithi027.stdout:6/744: dwrite d1/de/f13 [4194304,4194304] 0 2024-04-26T23:01:39.771 INFO:tasks.workunit.client.0.smithi027.stdout:6/745: symlink d1/de/led 0 2024-04-26T23:01:39.781 INFO:tasks.workunit.client.0.smithi027.stdout:2/808: dread d2/d5/db/d1b/d1c/d25/f59 [0,4194304] 0 2024-04-26T23:01:39.783 INFO:tasks.workunit.client.0.smithi027.stdout:2/809: mknod d2/d5/db/de/d21/cff 0 2024-04-26T23:01:39.784 INFO:tasks.workunit.client.0.smithi027.stdout:2/810: truncate d2/d5/f6e 1749658 0 2024-04-26T23:01:39.784 INFO:tasks.workunit.client.0.smithi027.stdout:5/890: dwrite d4/d1a/d2b/db8/ff4 [0,4194304] 0 2024-04-26T23:01:39.786 INFO:tasks.workunit.client.0.smithi027.stdout:5/891: symlink d4/d1a/d2b/dd3/l122 0 2024-04-26T23:01:39.786 INFO:tasks.workunit.client.0.smithi027.stdout:5/892: chown d4/d24/d22/d29/d39/d84/l119 1 1 2024-04-26T23:01:39.787 INFO:tasks.workunit.client.0.smithi027.stdout:5/893: creat d4/d24/d22/dab/d16/d4b/f123 x:0 0 0 2024-04-26T23:01:39.790 INFO:tasks.workunit.client.0.smithi027.stdout:5/894: rmdir d4/d24/d22/dab/d16/d4b 39 2024-04-26T23:01:39.790 INFO:tasks.workunit.client.0.smithi027.stdout:5/895: getdents d4/d24/d22/dab 0 2024-04-26T23:01:39.791 INFO:tasks.workunit.client.0.smithi027.stdout:5/896: mkdir d4/d24/d22/d29/d2a/dee/d124 0 2024-04-26T23:01:39.791 INFO:tasks.workunit.client.0.smithi027.stdout:5/897: dread - d4/d24/d22/d29/d2a/fa1 zero size 2024-04-26T23:01:39.791 INFO:tasks.workunit.client.0.smithi027.stdout:5/898: getdents d4/d24/d55 0 2024-04-26T23:01:39.792 INFO:tasks.workunit.client.0.smithi027.stdout:5/899: creat d4/d1a/d2b/dd2/de9/f125 x:0 0 0 2024-04-26T23:01:39.793 INFO:tasks.workunit.client.0.smithi027.stdout:5/900: truncate d4/d1a/dcb/ff1 215139 0 2024-04-26T23:01:39.798 INFO:tasks.workunit.client.0.smithi027.stdout:2/811: write d2/d5/db/de/d21/d8f/f95 [1699844,80397] 0 2024-04-26T23:01:39.799 INFO:tasks.workunit.client.0.smithi027.stdout:2/812: fsync d2/d5/db/d1b/d1c/d25/f59 0 2024-04-26T23:01:39.800 INFO:tasks.workunit.client.0.smithi027.stdout:2/813: mknod d2/d5/db/de/d21/dc4/d39/c100 0 2024-04-26T23:01:39.800 INFO:tasks.workunit.client.0.smithi027.stdout:2/814: readlink d2/d5/db/d1b/d1c/d25/dc3/ld4 0 2024-04-26T23:01:39.800 INFO:tasks.workunit.client.0.smithi027.stdout:2/815: readlink d2/d5/db/de/l2a 0 2024-04-26T23:01:39.801 INFO:tasks.workunit.client.0.smithi027.stdout:2/816: link d2/d5/db/de/d21/dc4/d39/lf3 d2/d5/l101 0 2024-04-26T23:01:39.802 INFO:tasks.workunit.client.0.smithi027.stdout:2/817: mkdir d2/d5/db/d1b/d1c/d25/d19/db0/d102 0 2024-04-26T23:01:39.803 INFO:tasks.workunit.client.0.smithi027.stdout:2/818: chown d2/d5/db/de/d21/d8f/deb/fec 18 1 2024-04-26T23:01:39.803 INFO:tasks.workunit.client.0.smithi027.stdout:2/819: chown d2/d5/db/de/d21/d8f/l9f 1381 1 2024-04-26T23:01:39.804 INFO:tasks.workunit.client.0.smithi027.stdout:2/820: creat d2/d5/db/d1b/d1c/d25/d19/d23/f103 x:0 0 0 2024-04-26T23:01:39.807 INFO:tasks.workunit.client.0.smithi027.stdout:5/901: write d4/d1a/f94 [2355738,61534] 0 2024-04-26T23:01:39.807 INFO:tasks.workunit.client.0.smithi027.stdout:5/902: write d4/d24/d22/d29/d2a/fa1 [872092,76056] 0 2024-04-26T23:01:39.807 INFO:tasks.workunit.client.0.smithi027.stdout:5/903: stat d4/d1a/dcb/dd1/dd9 0 2024-04-26T23:01:39.809 INFO:tasks.workunit.client.0.smithi027.stdout:5/904: link d4/d24/d22/d29/d39/d84/lc4 d4/d24/d22/l126 0 2024-04-26T23:01:39.810 INFO:tasks.workunit.client.0.smithi027.stdout:5/905: creat d4/d9f/d105/f127 x:0 0 0 2024-04-26T23:01:39.810 INFO:tasks.workunit.client.0.smithi027.stdout:5/906: write d4/d24/d22/dab/d16/d4b/f123 [668068,62282] 0 2024-04-26T23:01:39.810 INFO:tasks.workunit.client.0.smithi027.stdout:5/907: stat d4/d24/d5f/dac/d40/d54/l8c 0 2024-04-26T23:01:39.812 INFO:tasks.workunit.client.0.smithi027.stdout:5/908: creat d4/d1a/f128 x:0 0 0 2024-04-26T23:01:39.812 INFO:tasks.workunit.client.0.smithi027.stdout:5/909: getdents d4/d9f/dc5 0 2024-04-26T23:01:39.813 INFO:tasks.workunit.client.0.smithi027.stdout:5/910: creat d4/d24/f129 x:0 0 0 2024-04-26T23:01:39.814 INFO:tasks.workunit.client.0.smithi027.stdout:5/911: getdents d4/d24/d22/d29/d2a/dee/d124 0 2024-04-26T23:01:39.814 INFO:tasks.workunit.client.0.smithi027.stdout:5/912: mkdir d4/d24/d22/d29/d2a/dee/d124/d12a 0 2024-04-26T23:01:39.816 INFO:tasks.workunit.client.0.smithi027.stdout:5/913: rename d4/d1a/d2b/d101 to d4/d9f/dc5/d12b 0 2024-04-26T23:01:39.816 INFO:tasks.workunit.client.0.smithi027.stdout:5/914: readlink d4/d24/d22/d29/lc3 0 2024-04-26T23:01:39.837 INFO:tasks.workunit.client.0.smithi027.stdout:2/821: dwrite d2/d5/db/d1b/d1c/d25/d19/d23/d7c/fef [0,4194304] 0 2024-04-26T23:01:39.839 INFO:tasks.workunit.client.0.smithi027.stdout:2/822: link d2/d5/db/d1b/d1c/d25/d19/d44/l5a d2/d5/db/d1b/d1c/d25/dc0/df1/l104 0 2024-04-26T23:01:39.839 INFO:tasks.workunit.client.0.smithi027.stdout:2/823: write d2/d5/db/de/d21/dc4/fa0 [1706531,123832] 0 2024-04-26T23:01:39.843 INFO:tasks.workunit.client.0.smithi027.stdout:2/824: creat d2/d5/db/d1b/d1c/d25/d6a/d8a/d8d/f105 x:0 0 0 2024-04-26T23:01:39.854 INFO:tasks.workunit.client.0.smithi027.stdout:5/915: dwrite d4/d24/f7e [0,4194304] 0 2024-04-26T23:01:39.854 INFO:tasks.workunit.client.0.smithi027.stdout:2/825: write d2/d5/f6e [1552210,62628] 0 2024-04-26T23:01:39.855 INFO:tasks.workunit.client.0.smithi027.stdout:5/916: truncate d4/d24/d22/d29/d2a/d3b/fb7 217398 0 2024-04-26T23:01:39.856 INFO:tasks.workunit.client.0.smithi027.stdout:5/917: read d4/d24/d22/dab/d16/d4b/d81/fe3 [1141876,64886] 0 2024-04-26T23:01:39.857 INFO:tasks.workunit.client.0.smithi027.stdout:2/826: creat d2/d5/db/d1b/d1c/d25/d19/f106 x:0 0 0 2024-04-26T23:01:39.863 INFO:tasks.workunit.client.0.smithi027.stdout:5/918: creat d4/d24/d22/d29/d2a/f12c x:0 0 0 2024-04-26T23:01:39.864 INFO:tasks.workunit.client.0.smithi027.stdout:5/919: stat d4/d24/d22/dab/f23 0 2024-04-26T23:01:39.864 INFO:tasks.workunit.client.0.smithi027.stdout:5/920: readlink d4/d1a/dcb/dd1/l113 0 2024-04-26T23:01:39.865 INFO:tasks.workunit.client.0.smithi027.stdout:2/827: creat d2/d5/db/d1b/d1c/f107 x:0 0 0 2024-04-26T23:01:39.866 INFO:tasks.workunit.client.0.smithi027.stdout:2/828: creat d2/d5/db/d1b/d1c/d25/d19/d44/dee/f108 x:0 0 0 2024-04-26T23:01:39.872 INFO:tasks.workunit.client.0.smithi027.stdout:5/921: dread d4/d24/d22/dab/d16/f9b [0,4194304] 0 2024-04-26T23:01:39.875 INFO:tasks.workunit.client.0.smithi027.stdout:5/922: link d4/d24/d5f/ce0 d4/d9f/d105/c12d 0 2024-04-26T23:01:39.900 INFO:tasks.workunit.client.0.smithi027.stdout:2/829: dwrite d2/d5/db/d1b/d1c/d25/d19/d23/d82/ff4 [0,4194304] 0 2024-04-26T23:01:39.904 INFO:tasks.workunit.client.0.smithi027.stdout:2/830: creat d2/d5/db/de/d21/d8f/d88/f109 x:0 0 0 2024-04-26T23:01:39.904 INFO:tasks.workunit.client.0.smithi027.stdout:2/831: chown d2/d5/db/d1b/d1c/l48 26188 1 2024-04-26T23:01:39.904 INFO:tasks.workunit.client.0.smithi027.stdout:2/832: read d2/d5/db/d1b/d1c/d8b/fa9 [599096,28623] 0 2024-04-26T23:01:39.906 INFO:tasks.workunit.client.0.smithi027.stdout:2/833: creat d2/d5/db/d1b/d1c/d25/d19/db0/f10a x:0 0 0 2024-04-26T23:01:39.906 INFO:tasks.workunit.client.0.smithi027.stdout:2/834: write d2/d5/db/d1b/d1c/d8b/fe5 [1295740,52017] 0 2024-04-26T23:01:39.906 INFO:tasks.workunit.client.0.smithi027.stdout:2/835: stat d2/d5/db/de/d21/dc4/d39/f3f 0 2024-04-26T23:01:39.906 INFO:tasks.workunit.client.0.smithi027.stdout:2/836: fdatasync d2/d5/db/de/d21/d8f/d88/fe6 0 2024-04-26T23:01:39.908 INFO:tasks.workunit.client.0.smithi027.stdout:2/837: rename d2/d5/db/d1b/d1c/d25/d19/db0/d102 to d2/d5/db/d1b/d10b 0 2024-04-26T23:01:39.911 INFO:tasks.workunit.client.0.smithi027.stdout:5/923: dwrite d4/d1a/f128 [0,4194304] 0 2024-04-26T23:01:39.917 INFO:tasks.workunit.client.0.smithi027.stdout:2/838: dread d2/d5/db/d1b/d1c/d25/fa4 [0,4194304] 0 2024-04-26T23:01:39.917 INFO:tasks.workunit.client.0.smithi027.stdout:2/839: write d2/d5/db/de/d21/d8f/d88/fe9 [653641,11447] 0 2024-04-26T23:01:39.919 INFO:tasks.workunit.client.0.smithi027.stdout:2/840: creat d2/d5/db/d1b/d1c/d25/dc0/df1/f10c x:0 0 0 2024-04-26T23:01:39.919 INFO:tasks.workunit.client.0.smithi027.stdout:2/841: readlink d2/d5/db/de/d21/d8f/l71 0 2024-04-26T23:01:39.920 INFO:tasks.workunit.client.0.smithi027.stdout:2/842: rename d2/d5/db/d1b/d1c/d25/d6a/ccf to d2/d5/db/d1b/d1c/d25/d6a/c10d 0 2024-04-26T23:01:39.921 INFO:tasks.workunit.client.0.smithi027.stdout:2/843: link d2/d5/db/d1b/d1c/d25/dc0/df1/l104 d2/d5/db/d1b/d1c/d25/d19/d44/d7e/l10e 0 2024-04-26T23:01:39.923 INFO:tasks.workunit.client.0.smithi027.stdout:2/844: unlink d2/d5/db/de/d21/dc4/f43 0 2024-04-26T23:01:39.930 INFO:tasks.workunit.client.0.smithi027.stdout:9/777: sync 2024-04-26T23:01:39.930 INFO:tasks.workunit.client.0.smithi027.stdout:1/982: sync 2024-04-26T23:01:39.930 INFO:tasks.workunit.client.0.smithi027.stdout:3/942: sync 2024-04-26T23:01:39.930 INFO:tasks.workunit.client.0.smithi027.stdout:1/983: chown d1/d26/d66/f77 12141 1 2024-04-26T23:01:39.930 INFO:tasks.workunit.client.0.smithi027.stdout:8/854: sync 2024-04-26T23:01:39.932 INFO:tasks.workunit.client.0.smithi027.stdout:4/899: sync 2024-04-26T23:01:39.932 INFO:tasks.workunit.client.0.smithi027.stdout:7/918: sync 2024-04-26T23:01:39.932 INFO:tasks.workunit.client.0.smithi027.stdout:7/919: truncate d0/d28/d66/d67/ffd 740877 0 2024-04-26T23:01:39.933 INFO:tasks.workunit.client.0.smithi027.stdout:7/920: fsync d0/d28/f2c 0 2024-04-26T23:01:39.935 INFO:tasks.workunit.client.0.smithi027.stdout:0/862: sync 2024-04-26T23:01:39.935 INFO:tasks.workunit.client.0.smithi027.stdout:8/855: symlink d9/de/d2e/d110/l129 0 2024-04-26T23:01:39.935 INFO:tasks.workunit.client.0.smithi027.stdout:8/856: readlink d9/de/d22/l51 0 2024-04-26T23:01:39.935 INFO:tasks.workunit.client.0.smithi027.stdout:0/863: readlink d0/df/d9d/de5/lf7 0 2024-04-26T23:01:39.935 INFO:tasks.workunit.client.0.smithi027.stdout:8/857: chown d9/da/d2a/d6b 37305478 1 2024-04-26T23:01:39.936 INFO:tasks.workunit.client.0.smithi027.stdout:0/864: dread - d0/d4/dd3/fe7 zero size 2024-04-26T23:01:39.936 INFO:tasks.workunit.client.0.smithi027.stdout:9/778: rename d4/l6 to d4/d17/d2e/d46/dd1/l115 0 2024-04-26T23:01:39.936 INFO:tasks.workunit.client.0.smithi027.stdout:9/779: fdatasync d4/f52 0 2024-04-26T23:01:39.936 INFO:tasks.workunit.client.0.smithi027.stdout:9/780: fsync d4/d10/d25/dc9/f104 0 2024-04-26T23:01:39.936 INFO:tasks.workunit.client.0.smithi027.stdout:9/781: chown d4/d17/d2e/f95 132601 1 2024-04-26T23:01:39.938 INFO:tasks.workunit.client.0.smithi027.stdout:0/865: mkdir d0/d4/da/d1a/d59/d92/d107 0 2024-04-26T23:01:39.938 INFO:tasks.workunit.client.0.smithi027.stdout:0/866: dread - d0/df/d3f/d4a/f60 zero size 2024-04-26T23:01:39.939 INFO:tasks.workunit.client.0.smithi027.stdout:3/943: symlink d6/dd3/d34/d2e/d5f/df5/l13b 0 2024-04-26T23:01:39.939 INFO:tasks.workunit.client.0.smithi027.stdout:0/867: creat d0/df/d3f/f108 x:0 0 0 2024-04-26T23:01:39.943 INFO:tasks.workunit.client.0.smithi027.stdout:3/944: getdents d6/dd3/d34/d47 0 2024-04-26T23:01:39.943 INFO:tasks.workunit.client.0.smithi027.stdout:3/945: write d6/f7 [1483656,120470] 0 2024-04-26T23:01:39.947 INFO:tasks.workunit.client.0.smithi027.stdout:8/858: mkdir d9/da/d2a/d5e/d72/d84/d12a 0 2024-04-26T23:01:39.948 INFO:tasks.workunit.client.0.smithi027.stdout:3/946: mknod d6/dd3/d34/d78/c13c 0 2024-04-26T23:01:39.953 INFO:tasks.workunit.client.0.smithi027.stdout:8/859: creat d9/da/d2a/d5e/d72/d84/db8/f12b x:0 0 0 2024-04-26T23:01:39.953 INFO:tasks.workunit.client.0.smithi027.stdout:8/860: fdatasync d9/de/fa5 0 2024-04-26T23:01:39.953 INFO:tasks.workunit.client.0.smithi027.stdout:8/861: write d9/da/d2a/d5e/d9e/dba/ffa [987493,102743] 0 2024-04-26T23:01:39.953 INFO:tasks.workunit.client.0.smithi027.stdout:8/862: chown d9/de/d2e/d110/f99 7951461 1 2024-04-26T23:01:39.953 INFO:tasks.workunit.client.0.smithi027.stdout:8/863: read d9/da/d2a/d5e/f121 [525250,125840] 0 2024-04-26T23:01:39.953 INFO:tasks.workunit.client.0.smithi027.stdout:3/947: symlink d6/dd3/d34/d2e/d9d/da7/dac/l13d 0 2024-04-26T23:01:39.963 INFO:tasks.workunit.client.0.smithi027.stdout:3/948: creat d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/d5a/dcd/f13e x:0 0 0 2024-04-26T23:01:39.965 INFO:tasks.workunit.client.0.smithi027.stdout:3/949: mknod d6/dd3/d34/d2e/d5f/d90/c13f 0 2024-04-26T23:01:39.969 INFO:tasks.workunit.client.0.smithi027.stdout:3/950: unlink d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/f86 0 2024-04-26T23:01:39.969 INFO:tasks.workunit.client.0.smithi027.stdout:3/951: fsync d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/f8d 0 2024-04-26T23:01:39.969 INFO:tasks.workunit.client.0.smithi027.stdout:3/952: truncate d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d40/f4d 893815 0 2024-04-26T23:01:39.971 INFO:tasks.workunit.client.0.smithi027.stdout:3/953: symlink d6/dd3/d34/d2e/d5f/d90/d136/l140 0 2024-04-26T23:01:39.973 INFO:tasks.workunit.client.0.smithi027.stdout:3/954: creat d6/dd3/d34/d2e/d5f/d90/ddb/dce/f141 x:0 0 0 2024-04-26T23:01:39.974 INFO:tasks.workunit.client.0.smithi027.stdout:4/900: dread da/dd/de/d14/d21/d41/f49 [0,4194304] 0 2024-04-26T23:01:39.977 INFO:tasks.workunit.client.0.smithi027.stdout:5/924: dwrite f1 [0,4194304] 0 2024-04-26T23:01:39.977 INFO:tasks.workunit.client.0.smithi027.stdout:7/921: write d0/d28/d2b/d3a/d90/fbd [579573,53296] 0 2024-04-26T23:01:39.979 INFO:tasks.workunit.client.0.smithi027.stdout:5/925: unlink d4/d24/d22/d29/d2a/la0 0 2024-04-26T23:01:39.979 INFO:tasks.workunit.client.0.smithi027.stdout:5/926: fdatasync d4/d1a/d4a/f76 0 2024-04-26T23:01:39.979 INFO:tasks.workunit.client.0.smithi027.stdout:5/927: write d4/d24/d22/d29/d2a/d3b/d50/f11e [885840,123648] 0 2024-04-26T23:01:39.979 INFO:tasks.workunit.client.0.smithi027.stdout:5/928: fsync d4/d24/f5d 0 2024-04-26T23:01:39.979 INFO:tasks.workunit.client.0.smithi027.stdout:5/929: readlink d4/d24/d55/l88 0 2024-04-26T23:01:39.981 INFO:tasks.workunit.client.0.smithi027.stdout:5/930: link d4/d24/d5f/dac/d40/d54/fa6 d4/d24/d22/d29/d39/dc7/d104/f12e 0 2024-04-26T23:01:39.990 INFO:tasks.workunit.client.0.smithi027.stdout:3/955: dread d6/dd3/d34/d2e/f45 [4194304,4194304] 0 2024-04-26T23:01:39.992 INFO:tasks.workunit.client.0.smithi027.stdout:5/931: dread d4/d24/d5f/dac/d40/d54/fa6 [0,4194304] 0 2024-04-26T23:01:39.993 INFO:tasks.workunit.client.0.smithi027.stdout:5/932: truncate d4/d24/f98 2333015 0 2024-04-26T23:01:39.994 INFO:tasks.workunit.client.0.smithi027.stdout:5/933: mkdir d4/d24/d22/d29/d2a/dee/d124/d12a/d12f 0 2024-04-26T23:01:39.997 INFO:tasks.workunit.client.0.smithi027.stdout:4/901: write da/dd/f5d [2250312,19602] 0 2024-04-26T23:01:39.997 INFO:tasks.workunit.client.0.smithi027.stdout:4/902: write da/dd/de/fd2 [965130,26368] 0 2024-04-26T23:01:40.001 INFO:tasks.workunit.client.0.smithi027.stdout:4/903: creat da/dd/de/d14/d17/d111/f11f x:0 0 0 2024-04-26T23:01:40.001 INFO:tasks.workunit.client.0.smithi027.stdout:4/904: readlink da/l63 0 2024-04-26T23:01:40.002 INFO:tasks.workunit.client.0.smithi027.stdout:4/905: creat da/d40/daf/d103/f120 x:0 0 0 2024-04-26T23:01:40.028 INFO:tasks.workunit.client.0.smithi027.stdout:8/864: dread d9/da/d2a/d5e/f57 [4194304,4194304] 0 2024-04-26T23:01:40.029 INFO:tasks.workunit.client.0.smithi027.stdout:8/865: fdatasync d9/da/d2a/fbd 0 2024-04-26T23:01:40.031 INFO:tasks.workunit.client.0.smithi027.stdout:7/922: dwrite d0/f42 [0,4194304] 0 2024-04-26T23:01:40.034 INFO:tasks.workunit.client.0.smithi027.stdout:1/984: dwrite d1/f1f [0,4194304] 0 2024-04-26T23:01:40.046 INFO:tasks.workunit.client.0.smithi027.stdout:5/934: dwrite d4/d24/d22/d29/d39/f7f [0,4194304] 0 2024-04-26T23:01:40.048 INFO:tasks.workunit.client.0.smithi027.stdout:5/935: rmdir d4/d24/d5f 39 2024-04-26T23:01:40.054 INFO:tasks.workunit.client.0.smithi027.stdout:1/985: dread d1/d26/d66/d73/f7c [4194304,4194304] 0 2024-04-26T23:01:40.056 INFO:tasks.workunit.client.0.smithi027.stdout:1/986: creat d1/d26/d36/d56/f141 x:0 0 0 2024-04-26T23:01:40.058 INFO:tasks.workunit.client.0.smithi027.stdout:1/987: symlink d1/d26/d66/d73/d9e/db6/d138/l142 0 2024-04-26T23:01:40.058 INFO:tasks.workunit.client.0.smithi027.stdout:1/988: write d1/d26/d7d/f126 [175707,49408] 0 2024-04-26T23:01:40.058 INFO:tasks.workunit.client.0.smithi027.stdout:1/989: write d1/dc/fa7 [559741,125114] 0 2024-04-26T23:01:40.059 INFO:tasks.workunit.client.0.smithi027.stdout:1/990: mknod d1/d26/d6b/d42/c143 0 2024-04-26T23:01:40.061 INFO:tasks.workunit.client.0.smithi027.stdout:2/845: dwrite d2/d5/db/de/d21/f51 [4194304,4194304] 0 2024-04-26T23:01:40.061 INFO:tasks.workunit.client.0.smithi027.stdout:7/923: dread d0/f42 [0,4194304] 0 2024-04-26T23:01:40.064 INFO:tasks.workunit.client.0.smithi027.stdout:5/936: write f1 [2630261,108469] 0 2024-04-26T23:01:40.067 INFO:tasks.workunit.client.0.smithi027.stdout:5/937: mknod d4/d1a/d2b/daa/d10c/c130 0 2024-04-26T23:01:40.068 INFO:tasks.workunit.client.0.smithi027.stdout:5/938: creat d4/d24/d22/dab/d16/f131 x:0 0 0 2024-04-26T23:01:40.070 INFO:tasks.workunit.client.0.smithi027.stdout:5/939: truncate d4/d24/f5d 3572430 0 2024-04-26T23:01:40.073 INFO:tasks.workunit.client.0.smithi027.stdout:5/940: mknod d4/d24/d22/dab/c132 0 2024-04-26T23:01:40.076 INFO:tasks.workunit.client.0.smithi027.stdout:2/846: truncate d2/d5/db/de/d21/f51 3871600 0 2024-04-26T23:01:40.077 INFO:tasks.workunit.client.0.smithi027.stdout:4/906: dwrite da/dd/f2b [0,4194304] 0 2024-04-26T23:01:40.078 INFO:tasks.workunit.client.0.smithi027.stdout:2/847: mkdir d2/d5/db/d1b/d10b/d10f 0 2024-04-26T23:01:40.078 INFO:tasks.workunit.client.0.smithi027.stdout:2/848: write d2/d5/db/d1b/d1c/d8b/fa9 [1692907,99225] 0 2024-04-26T23:01:40.079 INFO:tasks.workunit.client.0.smithi027.stdout:4/907: creat da/dd/de/d14/f121 x:0 0 0 2024-04-26T23:01:40.080 INFO:tasks.workunit.client.0.smithi027.stdout:4/908: symlink da/dd/de/d14/db3/d114/d11d/l122 0 2024-04-26T23:01:40.081 INFO:tasks.workunit.client.0.smithi027.stdout:4/909: write da/dd/de/db8/f10b [409571,77338] 0 2024-04-26T23:01:40.081 INFO:tasks.workunit.client.0.smithi027.stdout:4/910: truncate da/dd/de/f11e 740448 0 2024-04-26T23:01:40.081 INFO:tasks.workunit.client.0.smithi027.stdout:4/911: chown da/dd/de/d14/d21/d67/c88 4960793 1 2024-04-26T23:01:40.083 INFO:tasks.workunit.client.0.smithi027.stdout:4/912: rename da/dd/de/d14/d21/d41/d59/fe9 to da/dd/d5f/f123 0 2024-04-26T23:01:40.088 INFO:tasks.workunit.client.0.smithi027.stdout:7/924: dread d0/d28/d2b/d3a/d48/d56/db6/fdc [0,4194304] 0 2024-04-26T23:01:40.088 INFO:tasks.workunit.client.0.smithi027.stdout:6/746: sync 2024-04-26T23:01:40.090 INFO:tasks.workunit.client.0.smithi027.stdout:7/925: unlink d0/d28/d9f/db8/dbc/fd5 0 2024-04-26T23:01:40.092 INFO:tasks.workunit.client.0.smithi027.stdout:6/747: mkdir d1/de/d57/d8e/dd0/dc9/dee 0 2024-04-26T23:01:40.094 INFO:tasks.workunit.client.0.smithi027.stdout:8/866: dwrite d9/da/d2a/fbd [0,4194304] 0 2024-04-26T23:01:40.103 INFO:tasks.workunit.client.0.smithi027.stdout:7/926: rename d0/d28/d2b/f118 to d0/d28/d2b/d3a/d48/d56/d8d/dc5/f12d 0 2024-04-26T23:01:40.105 INFO:tasks.workunit.client.0.smithi027.stdout:7/927: mkdir d0/d28/d2b/d3a/d48/d56/dee/d100/d12e 0 2024-04-26T23:01:40.107 INFO:tasks.workunit.client.0.smithi027.stdout:7/928: link d0/d2f/f31 d0/d28/d2b/d3a/d48/d56/d88/d73/f12f 0 2024-04-26T23:01:40.107 INFO:tasks.workunit.client.0.smithi027.stdout:7/929: fsync d0/d28/d2b/d3a/d48/d56/d88/d73/f104 0 2024-04-26T23:01:40.107 INFO:tasks.workunit.client.0.smithi027.stdout:7/930: write d0/d28/d9f/db8/dbc/fe4 [666279,71047] 0 2024-04-26T23:01:40.108 INFO:tasks.workunit.client.0.smithi027.stdout:7/931: chown d0/d28/d2b/d3a/d48/d56 1 1 2024-04-26T23:01:40.108 INFO:tasks.workunit.client.0.smithi027.stdout:7/932: fsync d0/d28/d2b/d3a/d48/d56/d88/d73/dd6/f127 0 2024-04-26T23:01:40.108 INFO:tasks.workunit.client.0.smithi027.stdout:7/933: chown d0/d28/d2b/d3a/d48/f10f 768 1 2024-04-26T23:01:40.108 INFO:tasks.workunit.client.0.smithi027.stdout:7/934: fsync d0/fc2 0 2024-04-26T23:01:40.108 INFO:tasks.workunit.client.0.smithi027.stdout:7/935: write d0/d28/d2b/d55/fe7 [55012,124734] 0 2024-04-26T23:01:40.117 INFO:tasks.workunit.client.0.smithi027.stdout:6/748: read d1/df/d49/f4d [1491386,7501] 0 2024-04-26T23:01:40.117 INFO:tasks.workunit.client.0.smithi027.stdout:6/749: chown d1/df/d3c/d9a/c9d 539992 1 2024-04-26T23:01:40.118 INFO:tasks.workunit.client.0.smithi027.stdout:8/867: rename d9/da/d2a/d5e/d72/d84/d5d/d6c/dc0/fcf to d9/da/d2a/d5e/d2b/d52/d115/f12c 0 2024-04-26T23:01:40.118 INFO:tasks.workunit.client.0.smithi027.stdout:8/868: chown d9/de/d2e/d110/da7 18118 1 2024-04-26T23:01:40.119 INFO:tasks.workunit.client.0.smithi027.stdout:8/869: mkdir d9/da/d2a/d12d 0 2024-04-26T23:01:40.119 INFO:tasks.workunit.client.0.smithi027.stdout:6/750: creat d1/df/d43/d56/d6b/dc4/fef x:0 0 0 2024-04-26T23:01:40.121 INFO:tasks.workunit.client.0.smithi027.stdout:6/751: unlink d1/df/d49/c7e 0 2024-04-26T23:01:40.125 INFO:tasks.workunit.client.0.smithi027.stdout:6/752: creat d1/d30/d6c/db3/ff0 x:0 0 0 2024-04-26T23:01:40.125 INFO:tasks.workunit.client.0.smithi027.stdout:6/753: readlink d1/lb1 0 2024-04-26T23:01:40.125 INFO:tasks.workunit.client.0.smithi027.stdout:6/754: mkdir d1/df/d43/d56/d6b/dc4/df1 0 2024-04-26T23:01:40.127 INFO:tasks.workunit.client.0.smithi027.stdout:5/941: dwrite d4/d24/d22/d29/f36 [0,4194304] 0 2024-04-26T23:01:40.128 INFO:tasks.workunit.client.0.smithi027.stdout:5/942: creat d4/d9f/dc0/f133 x:0 0 0 2024-04-26T23:01:40.133 INFO:tasks.workunit.client.0.smithi027.stdout:6/755: write d1/f72 [896918,68614] 0 2024-04-26T23:01:40.134 INFO:tasks.workunit.client.0.smithi027.stdout:6/756: stat d1/f59 0 2024-04-26T23:01:40.140 INFO:tasks.workunit.client.0.smithi027.stdout:4/913: dwrite da/dd/de/d14/d17/d80/dab/f3e [0,4194304] 0 2024-04-26T23:01:40.149 INFO:tasks.workunit.client.0.smithi027.stdout:5/943: read d4/fc1 [10771471,75054] 0 2024-04-26T23:01:40.149 INFO:tasks.workunit.client.0.smithi027.stdout:5/944: chown d4/d24/d22/d29/d39/d84/f87 689250 1 2024-04-26T23:01:40.150 INFO:tasks.workunit.client.0.smithi027.stdout:6/757: dread d1/df/d43/f45 [0,4194304] 0 2024-04-26T23:01:40.151 INFO:tasks.workunit.client.0.smithi027.stdout:6/758: chown d1/de/f82 68701 1 2024-04-26T23:01:40.151 INFO:tasks.workunit.client.0.smithi027.stdout:6/759: chown d1/df/d43/da8/d84 1042162097 1 2024-04-26T23:01:40.163 INFO:tasks.workunit.client.0.smithi027.stdout:7/936: dwrite d0/d28/d2b/d55/f7e [0,4194304] 0 2024-04-26T23:01:40.166 INFO:tasks.workunit.client.0.smithi027.stdout:7/937: rename d0/d28/d2b/fce to d0/d3d/d10a/f130 0 2024-04-26T23:01:40.174 INFO:tasks.workunit.client.0.smithi027.stdout:6/760: dread d1/de/f1a [0,4194304] 0 2024-04-26T23:01:40.177 INFO:tasks.workunit.client.0.smithi027.stdout:4/914: dwrite da/dd/de/d14/d17/d80/dab/f16 [4194304,4194304] 0 2024-04-26T23:01:40.179 INFO:tasks.workunit.client.0.smithi027.stdout:4/915: mknod da/dd/de/d14/c124 0 2024-04-26T23:01:40.181 INFO:tasks.workunit.client.0.smithi027.stdout:4/916: mknod da/dd/de/db8/c125 0 2024-04-26T23:01:40.182 INFO:tasks.workunit.client.0.smithi027.stdout:6/761: link d1/de/c67 d1/de/cf2 0 2024-04-26T23:01:40.197 INFO:tasks.workunit.client.0.smithi027.stdout:4/917: dread f2 [0,4194304] 0 2024-04-26T23:01:40.197 INFO:tasks.workunit.client.0.smithi027.stdout:4/918: stat da/dd/de/d14/d21/d41/f49 0 2024-04-26T23:01:40.199 INFO:tasks.workunit.client.0.smithi027.stdout:4/919: unlink da/dd/de/d79/da8/fd3 0 2024-04-26T23:01:40.199 INFO:tasks.workunit.client.0.smithi027.stdout:4/920: chown da/dd/de/d14/d21/d4a/l4c 58023756 1 2024-04-26T23:01:40.201 INFO:tasks.workunit.client.0.smithi027.stdout:4/921: mknod da/dd/de/d79/dcd/c126 0 2024-04-26T23:01:40.201 INFO:tasks.workunit.client.0.smithi027.stdout:4/922: read - da/dd/de/d14/d21/dad/ff4 zero size 2024-04-26T23:01:40.201 INFO:tasks.workunit.client.0.smithi027.stdout:4/923: write da/dd/de/d14/f121 [823968,65101] 0 2024-04-26T23:01:40.211 INFO:tasks.workunit.client.0.smithi027.stdout:5/945: dwrite d4/d1a/d2b/daa/d10c/f11a [0,4194304] 0 2024-04-26T23:01:40.223 INFO:tasks.workunit.client.0.smithi027.stdout:5/946: truncate d4/d24/f1b 721377 0 2024-04-26T23:01:40.223 INFO:tasks.workunit.client.0.smithi027.stdout:9/782: sync 2024-04-26T23:01:40.227 INFO:tasks.workunit.client.0.smithi027.stdout:4/924: dread da/dd/f2b [0,4194304] 0 2024-04-26T23:01:40.227 INFO:tasks.workunit.client.0.smithi027.stdout:6/762: dwrite d1/df/d43/d56/fd4 [0,4194304] 0 2024-04-26T23:01:40.227 INFO:tasks.workunit.client.0.smithi027.stdout:5/947: unlink d4/d24/d22/dab/c85 0 2024-04-26T23:01:40.227 INFO:tasks.workunit.client.0.smithi027.stdout:9/783: getdents d4/d10 0 2024-04-26T23:01:40.228 INFO:tasks.workunit.client.0.smithi027.stdout:5/948: mkdir d4/d1a/d2b/d134 0 2024-04-26T23:01:40.228 INFO:tasks.workunit.client.0.smithi027.stdout:5/949: truncate d4/d24/d22/dab/d16/f131 551608 0 2024-04-26T23:01:40.230 INFO:tasks.workunit.client.0.smithi027.stdout:9/784: getdents d4/d10/d40/d58/d75/d84/d86/dc5 0 2024-04-26T23:01:40.234 INFO:tasks.workunit.client.0.smithi027.stdout:9/785: mkdir d4/d10/d40/d58/d75/d84/dca/d116 0 2024-04-26T23:01:40.234 INFO:tasks.workunit.client.0.smithi027.stdout:5/950: mknod d4/c135 0 2024-04-26T23:01:40.236 INFO:tasks.workunit.client.0.smithi027.stdout:9/786: mkdir d4/d10/d40/d58/d75/d117 0 2024-04-26T23:01:40.236 INFO:tasks.workunit.client.0.smithi027.stdout:9/787: write d4/d17/d2e/d46/da0/f41 [4644525,38735] 0 2024-04-26T23:01:40.239 INFO:tasks.workunit.client.0.smithi027.stdout:5/951: unlink d4/d24/d22/d29/d2a/d3b/fb7 0 2024-04-26T23:01:40.239 INFO:tasks.workunit.client.0.smithi027.stdout:6/763: rename d1/d30/d6c to d1/de/df3 0 2024-04-26T23:01:40.240 INFO:tasks.workunit.client.0.smithi027.stdout:6/764: creat d1/df/ff4 x:0 0 0 2024-04-26T23:01:40.242 INFO:tasks.workunit.client.0.smithi027.stdout:6/765: rename d1/d30/faf to d1/df/d43/da8/d84/ff5 0 2024-04-26T23:01:40.243 INFO:tasks.workunit.client.0.smithi027.stdout:6/766: creat d1/df/d3c/d9a/ff6 x:0 0 0 2024-04-26T23:01:40.245 INFO:tasks.workunit.client.0.smithi027.stdout:4/925: dread da/dd/de/d14/d17/d80/fa9 [0,4194304] 0 2024-04-26T23:01:40.252 INFO:tasks.workunit.client.0.smithi027.stdout:5/952: dread d4/d24/d22/d29/d2a/d3b/d50/f11e [0,4194304] 0 2024-04-26T23:01:40.285 INFO:tasks.workunit.client.0.smithi027.stdout:9/788: dwrite d4/d17/d2e/fb7 [0,4194304] 0 2024-04-26T23:01:40.286 INFO:tasks.workunit.client.0.smithi027.stdout:9/789: mknod d4/d10/d40/d58/d75/d84/dc4/de4/c118 0 2024-04-26T23:01:40.293 INFO:tasks.workunit.client.0.smithi027.stdout:6/767: dwrite d1/fde [0,4194304] 0 2024-04-26T23:01:40.293 INFO:tasks.workunit.client.0.smithi027.stdout:6/768: fsync d1/de/df3/d6f/dd3/fda 0 2024-04-26T23:01:40.295 INFO:tasks.workunit.client.0.smithi027.stdout:6/769: mknod d1/de/d57/d8e/cf7 0 2024-04-26T23:01:40.298 INFO:tasks.workunit.client.0.smithi027.stdout:6/770: unlink d1/df/d3c/d9a/fcf 0 2024-04-26T23:01:40.298 INFO:tasks.workunit.client.0.smithi027.stdout:6/771: write d1/df/d43/da8/d84/fb8 [4434651,35358] 0 2024-04-26T23:01:40.298 INFO:tasks.workunit.client.0.smithi027.stdout:6/772: stat d1/df/dc2 0 2024-04-26T23:01:40.298 INFO:tasks.workunit.client.0.smithi027.stdout:6/773: fsync d1/de/df3/d6f/f89 0 2024-04-26T23:01:40.298 INFO:tasks.workunit.client.0.smithi027.stdout:6/774: chown d1/df/d43/da8/d84/ff5 1487822 1 2024-04-26T23:01:40.299 INFO:tasks.workunit.client.0.smithi027.stdout:4/926: dwrite da/fda [0,4194304] 0 2024-04-26T23:01:40.300 INFO:tasks.workunit.client.0.smithi027.stdout:4/927: chown da/dd/de/fd2 9873 1 2024-04-26T23:01:40.302 INFO:tasks.workunit.client.0.smithi027.stdout:4/928: symlink da/dd/de/d14/db3/d114/d11d/l127 0 2024-04-26T23:01:40.311 INFO:tasks.workunit.client.0.smithi027.stdout:9/790: dread d4/fcd [0,4194304] 0 2024-04-26T23:01:40.319 INFO:tasks.workunit.client.0.smithi027.stdout:6/775: write d1/df/f3a [3514413,58385] 0 2024-04-26T23:01:40.319 INFO:tasks.workunit.client.0.smithi027.stdout:5/953: dwrite d4/d24/d22/dab/ffb [4194304,4194304] 0 2024-04-26T23:01:40.320 INFO:tasks.workunit.client.0.smithi027.stdout:9/791: dread d4/d77/da2/daf/fc8 [0,4194304] 0 2024-04-26T23:01:40.321 INFO:tasks.workunit.client.0.smithi027.stdout:6/776: link d1/de/df3/fbd d1/df/d43/d56/d6b/ff8 0 2024-04-26T23:01:40.326 INFO:tasks.workunit.client.0.smithi027.stdout:6/777: write d1/de/df3/d6f/d7d/fbb [559402,28739] 0 2024-04-26T23:01:40.329 INFO:tasks.workunit.client.0.smithi027.stdout:6/778: rename d1/df/f5f to d1/de/df3/db3/ff9 0 2024-04-26T23:01:40.329 INFO:tasks.workunit.client.0.smithi027.stdout:6/779: dread - d1/de/d57/d8e/dd0/dc9/fd6 zero size 2024-04-26T23:01:40.329 INFO:tasks.workunit.client.0.smithi027.stdout:6/780: write d1/de/f13 [3163281,17692] 0 2024-04-26T23:01:40.333 INFO:tasks.workunit.client.0.smithi027.stdout:5/954: write d4/d24/f42 [385002,100000] 0 2024-04-26T23:01:40.343 INFO:tasks.workunit.client.0.smithi027.stdout:9/792: dread d4/d10/d40/d58/f96 [0,4194304] 0 2024-04-26T23:01:40.343 INFO:tasks.workunit.client.0.smithi027.stdout:1/991: sync 2024-04-26T23:01:40.343 INFO:tasks.workunit.client.0.smithi027.stdout:0/868: sync 2024-04-26T23:01:40.343 INFO:tasks.workunit.client.0.smithi027.stdout:3/956: sync 2024-04-26T23:01:40.343 INFO:tasks.workunit.client.0.smithi027.stdout:3/957: readlink d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/lb1 0 2024-04-26T23:01:40.345 INFO:tasks.workunit.client.0.smithi027.stdout:0/869: creat d0/df/d9d/d9f/dbb/f109 x:0 0 0 2024-04-26T23:01:40.345 INFO:tasks.workunit.client.0.smithi027.stdout:0/870: fdatasync d0/df/d3f/d64/f5e 0 2024-04-26T23:01:40.345 INFO:tasks.workunit.client.0.smithi027.stdout:0/871: dread - d0/df/d9d/d9f/dbb/f102 zero size 2024-04-26T23:01:40.345 INFO:tasks.workunit.client.0.smithi027.stdout:4/929: dwrite da/dd/de/d14/d17/d80/fa9 [0,4194304] 0 2024-04-26T23:01:40.345 INFO:tasks.workunit.client.0.smithi027.stdout:0/872: truncate d0/d4/faf 5138040 0 2024-04-26T23:01:40.347 INFO:tasks.workunit.client.0.smithi027.stdout:0/873: mknod d0/df/d3f/d64/da8/c10a 0 2024-04-26T23:01:40.348 INFO:tasks.workunit.client.0.smithi027.stdout:3/958: creat d6/dd3/d34/d2e/d70/f142 x:0 0 0 2024-04-26T23:01:40.348 INFO:tasks.workunit.client.0.smithi027.stdout:0/874: chown d0/d4/da/d1a/d1e/de8 551648 1 2024-04-26T23:01:40.348 INFO:tasks.workunit.client.0.smithi027.stdout:3/959: write d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/dd6/f119 [369761,57910] 0 2024-04-26T23:01:40.350 INFO:tasks.workunit.client.0.smithi027.stdout:1/992: rmdir d1/d2a 39 2024-04-26T23:01:40.353 INFO:tasks.workunit.client.0.smithi027.stdout:6/781: dread d1/de/df3/d6f/f89 [0,4194304] 0 2024-04-26T23:01:40.353 INFO:tasks.workunit.client.0.smithi027.stdout:6/782: write d1/df/d43/d56/d6b/dc4/fef [828693,17873] 0 2024-04-26T23:01:40.354 INFO:tasks.workunit.client.0.smithi027.stdout:6/783: write d1/df/d43/da8/d84/ff5 [181571,43053] 0 2024-04-26T23:01:40.354 INFO:tasks.workunit.client.0.smithi027.stdout:6/784: stat d1/f72 0 2024-04-26T23:01:40.354 INFO:tasks.workunit.client.0.smithi027.stdout:6/785: readlink d1/d30/lb0 0 2024-04-26T23:01:40.354 INFO:tasks.workunit.client.0.smithi027.stdout:6/786: chown d1/df/dc2/leb 422 1 2024-04-26T23:01:40.354 INFO:tasks.workunit.client.0.smithi027.stdout:6/787: chown d1/df/d43/d56/cd5 0 1 2024-04-26T23:01:40.356 INFO:tasks.workunit.client.0.smithi027.stdout:0/875: dread d0/d4/da/d15/f48 [0,4194304] 0 2024-04-26T23:01:40.367 INFO:tasks.workunit.client.0.smithi027.stdout:4/930: rename da/d40/daf to da/dd/de/d14/d21/d41/d128 0 2024-04-26T23:01:40.367 INFO:tasks.workunit.client.0.smithi027.stdout:6/788: creat d1/de/df3/d6f/ffa x:0 0 0 2024-04-26T23:01:40.370 INFO:tasks.workunit.client.0.smithi027.stdout:6/789: unlink d1/de/d57/d98/fae 0 2024-04-26T23:01:40.370 INFO:tasks.workunit.client.0.smithi027.stdout:6/790: write d1/df/d43/f6e [1473975,14789] 0 2024-04-26T23:01:40.373 INFO:tasks.workunit.client.0.smithi027.stdout:7/938: sync 2024-04-26T23:01:40.373 INFO:tasks.workunit.client.0.smithi027.stdout:2/849: sync 2024-04-26T23:01:40.373 INFO:tasks.workunit.client.0.smithi027.stdout:8/870: sync 2024-04-26T23:01:40.373 INFO:tasks.workunit.client.0.smithi027.stdout:4/931: truncate da/dd/de/d14/d21/d41/d59/d74/fa1 1000748 0 2024-04-26T23:01:40.373 INFO:tasks.workunit.client.0.smithi027.stdout:4/932: chown da/dd/de/d14/d21/d41/df7 0 1 2024-04-26T23:01:40.374 INFO:tasks.workunit.client.0.smithi027.stdout:1/993: rename d1/d2a/deb/c12b to d1/d26/d66/dab/c144 0 2024-04-26T23:01:40.375 INFO:tasks.workunit.client.0.smithi027.stdout:6/791: mknod d1/de/d57/d8e/dd0/dc9/dee/cfb 0 2024-04-26T23:01:40.376 INFO:tasks.workunit.client.0.smithi027.stdout:8/871: link d9/da/d2a/d5e/d2b/d52/d115/c11c d9/da/c12e 0 2024-04-26T23:01:40.378 INFO:tasks.workunit.client.0.smithi027.stdout:8/872: symlink d9/da/d2a/d5e/d72/d84/d4f/l12f 0 2024-04-26T23:01:40.378 INFO:tasks.workunit.client.0.smithi027.stdout:9/793: write d4/d17/f9e [2249087,53891] 0 2024-04-26T23:01:40.379 INFO:tasks.workunit.client.0.smithi027.stdout:7/939: creat d0/d3d/f131 x:0 0 0 2024-04-26T23:01:40.379 INFO:tasks.workunit.client.0.smithi027.stdout:7/940: stat d0/d28/fed 0 2024-04-26T23:01:40.380 INFO:tasks.workunit.client.0.smithi027.stdout:2/850: rmdir d2/db9 39 2024-04-26T23:01:40.380 INFO:tasks.workunit.client.0.smithi027.stdout:2/851: chown d2/d5/db/d1b/d1c/d25/d19/d23/f9a 45132 1 2024-04-26T23:01:40.380 INFO:tasks.workunit.client.0.smithi027.stdout:2/852: fsync d2/d5/db/de/d21/fd7 0 2024-04-26T23:01:40.380 INFO:tasks.workunit.client.0.smithi027.stdout:4/933: mkdir da/dd/d5f/dac/d129 0 2024-04-26T23:01:40.381 INFO:tasks.workunit.client.0.smithi027.stdout:6/792: creat d1/de/d57/d8e/ffc x:0 0 0 2024-04-26T23:01:40.382 INFO:tasks.workunit.client.0.smithi027.stdout:8/873: rmdir d9/de/d2e/d110/df0 39 2024-04-26T23:01:40.382 INFO:tasks.workunit.client.0.smithi027.stdout:8/874: readlink d9/da/d2a/d5e/d72/d84/d5d/le6 0 2024-04-26T23:01:40.382 INFO:tasks.workunit.client.0.smithi027.stdout:8/875: readlink d9/da/d2a/lb1 0 2024-04-26T23:01:40.382 INFO:tasks.workunit.client.0.smithi027.stdout:9/794: mkdir d4/d10/d40/d58/d75/d84/dc4/d119 0 2024-04-26T23:01:40.384 INFO:tasks.workunit.client.0.smithi027.stdout:9/795: write d4/d17/d2e/d46/da0/f41 [5435848,60480] 0 2024-04-26T23:01:40.384 INFO:tasks.workunit.client.0.smithi027.stdout:2/853: mknod d2/d5/db/d1b/d1c/df0/c110 0 2024-04-26T23:01:40.385 INFO:tasks.workunit.client.0.smithi027.stdout:6/793: symlink d1/de/df3/d6f/dd3/lfd 0 2024-04-26T23:01:40.386 INFO:tasks.workunit.client.0.smithi027.stdout:9/796: mkdir d4/d10/d40/d58/d75/d11a 0 2024-04-26T23:01:40.386 INFO:tasks.workunit.client.0.smithi027.stdout:9/797: fsync d4/d17/d2e/d46/da0/d72/f90 0 2024-04-26T23:01:40.386 INFO:tasks.workunit.client.0.smithi027.stdout:9/798: chown d4/d10/d67 314 1 2024-04-26T23:01:40.387 INFO:tasks.workunit.client.0.smithi027.stdout:2/854: rmdir d2/d5/db/de/d21/d8f/d88 39 2024-04-26T23:01:40.387 INFO:tasks.workunit.client.0.smithi027.stdout:2/855: readlink d2/d5/db/de/d21/dc4/l69 0 2024-04-26T23:01:40.387 INFO:tasks.workunit.client.0.smithi027.stdout:2/856: dread - d2/fdc zero size 2024-04-26T23:01:40.387 INFO:tasks.workunit.client.0.smithi027.stdout:2/857: fsync d2/d5/db/d1b/d1c/d25/d19/d23/f9a 0 2024-04-26T23:01:40.388 INFO:tasks.workunit.client.0.smithi027.stdout:2/858: write d2/d5/db/d1b/d1c/d25/d19/d44/fc5 [906513,40107] 0 2024-04-26T23:01:40.388 INFO:tasks.workunit.client.0.smithi027.stdout:8/876: mkdir d9/da/d33/d71/d130 0 2024-04-26T23:01:40.388 INFO:tasks.workunit.client.0.smithi027.stdout:8/877: truncate d9/de/d22/fdb 1817044 0 2024-04-26T23:01:40.388 INFO:tasks.workunit.client.0.smithi027.stdout:8/878: fdatasync d9/da/d2a/d6b/fab 0 2024-04-26T23:01:40.390 INFO:tasks.workunit.client.0.smithi027.stdout:9/799: truncate d4/d77/f7a 1686118 0 2024-04-26T23:01:40.391 INFO:tasks.workunit.client.0.smithi027.stdout:8/879: symlink d9/da/d2a/d5e/d72/d84/d5d/d6c/l131 0 2024-04-26T23:01:40.392 INFO:tasks.workunit.client.0.smithi027.stdout:2/859: link d2/d5/db/d1b/d1c/d25/d19/lf8 d2/d5/db/de/d21/d8f/d88/de7/l111 0 2024-04-26T23:01:40.392 INFO:tasks.workunit.client.0.smithi027.stdout:3/960: dwrite d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d61/d113/f12b [0,4194304] 0 2024-04-26T23:01:40.392 INFO:tasks.workunit.client.0.smithi027.stdout:2/860: readlink d2/d5/db/de/d21/d8f/d88/de7/lf6 0 2024-04-26T23:01:40.392 INFO:tasks.workunit.client.0.smithi027.stdout:3/961: readlink d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/ld4 0 2024-04-26T23:01:40.392 INFO:tasks.workunit.client.0.smithi027.stdout:3/962: dread - d6/dd3/f11e zero size 2024-04-26T23:01:40.392 INFO:tasks.workunit.client.0.smithi027.stdout:3/963: write d6/dd3/d34/d2e/d9d/da7/dac/d74/f87 [587771,72932] 0 2024-04-26T23:01:40.393 INFO:tasks.workunit.client.0.smithi027.stdout:9/800: creat d4/d77/da2/daf/f11b x:0 0 0 2024-04-26T23:01:40.393 INFO:tasks.workunit.client.0.smithi027.stdout:9/801: chown d4/d17/d2e/c37 1021621 1 2024-04-26T23:01:40.394 INFO:tasks.workunit.client.0.smithi027.stdout:2/861: mkdir d2/d5/db/d1b/d10b/d10f/d112 0 2024-04-26T23:01:40.395 INFO:tasks.workunit.client.0.smithi027.stdout:8/880: mkdir d9/de/d2e/d132 0 2024-04-26T23:01:40.396 INFO:tasks.workunit.client.0.smithi027.stdout:2/862: symlink d2/d5/db/de/d21/dc4/d39/dd2/l113 0 2024-04-26T23:01:40.398 INFO:tasks.workunit.client.0.smithi027.stdout:3/964: mkdir d6/dd3/d34/d2e/d9d/da7/dac/dc0/d12e/d143 0 2024-04-26T23:01:40.399 INFO:tasks.workunit.client.0.smithi027.stdout:2/863: mkdir d2/db9/d114 0 2024-04-26T23:01:40.399 INFO:tasks.workunit.client.0.smithi027.stdout:2/864: read - d2/d5/db/de/d21/f65 zero size 2024-04-26T23:01:40.399 INFO:tasks.workunit.client.0.smithi027.stdout:2/865: chown d2/d5/db/d1b/d1c/d25/d19/d23/fd3 1751655 1 2024-04-26T23:01:40.400 INFO:tasks.workunit.client.0.smithi027.stdout:8/881: creat d9/da/d2a/d5e/d9e/dba/f133 x:0 0 0 2024-04-26T23:01:40.401 INFO:tasks.workunit.client.0.smithi027.stdout:3/965: mknod d6/dd3/d34/d2e/d9d/da7/dac/d74/c144 0 2024-04-26T23:01:40.403 INFO:tasks.workunit.client.0.smithi027.stdout:0/876: dwrite d0/df/d9d/d9f/dbb/f102 [0,4194304] 0 2024-04-26T23:01:40.403 INFO:tasks.workunit.client.0.smithi027.stdout:1/994: dread d1/f1f [0,4194304] 0 2024-04-26T23:01:40.403 INFO:tasks.workunit.client.0.smithi027.stdout:1/995: stat d1/d26/d66/dab/de0/f27 0 2024-04-26T23:01:40.406 INFO:tasks.workunit.client.0.smithi027.stdout:0/877: getdents d0/d4 0 2024-04-26T23:01:40.406 INFO:tasks.workunit.client.0.smithi027.stdout:8/882: rename d9/de/d2e/f1f to d9/da/d2a/d5e/d72/d84/db8/f134 0 2024-04-26T23:01:40.406 INFO:tasks.workunit.client.0.smithi027.stdout:8/883: read - d9/da/d2a/d5e/d72/d84/db8/f12b zero size 2024-04-26T23:01:40.409 INFO:tasks.workunit.client.0.smithi027.stdout:1/996: symlink d1/d26/d6b/d8e/d13e/l145 0 2024-04-26T23:01:40.410 INFO:tasks.workunit.client.0.smithi027.stdout:1/997: symlink d1/d26/d66/d91/l146 0 2024-04-26T23:01:40.417 INFO:tasks.workunit.client.0.smithi027.stdout:1/998: link d1/d26/d6b/c64 d1/d26/d66/d73/d9e/d134/c147 0 2024-04-26T23:01:40.417 INFO:tasks.workunit.client.0.smithi027.stdout:9/802: dread d4/d17/f93 [0,4194304] 0 2024-04-26T23:01:40.417 INFO:tasks.workunit.client.0.smithi027.stdout:9/803: mknod d4/d17/d2e/d70/c11c 0 2024-04-26T23:01:40.417 INFO:tasks.workunit.client.0.smithi027.stdout:9/804: chown d4/d10/d40/d58/d75/d84/dc4/de4/f108 1575687 1 2024-04-26T23:01:40.417 INFO:tasks.workunit.client.0.smithi027.stdout:9/805: dread - d4/d77/da2/daf/f11b zero size 2024-04-26T23:01:40.417 INFO:tasks.workunit.client.0.smithi027.stdout:9/806: truncate d4/d17/d2e/d70/ffd 253545 0 2024-04-26T23:01:40.417 INFO:tasks.workunit.client.0.smithi027.stdout:9/807: truncate d4/d17/f29 1717906 0 2024-04-26T23:01:40.417 INFO:tasks.workunit.client.0.smithi027.stdout:9/808: chown d4/d10/d40/d58/d6e 116924 1 2024-04-26T23:01:40.419 INFO:tasks.workunit.client.0.smithi027.stdout:9/809: truncate d4/d17/d2e/d46/da0/d8d/fa8 1595266 0 2024-04-26T23:01:40.420 INFO:tasks.workunit.client.0.smithi027.stdout:8/884: rename d9/da/d2a/d5e/d72/d84/db8/f12b to d9/da/d2a/d5e/d72/d84/d5d/d6c/dc0/f135 0 2024-04-26T23:01:40.420 INFO:tasks.workunit.client.0.smithi027.stdout:8/885: write d9/da/d2a/d5e/d72/d84/f3f [3505915,20224] 0 2024-04-26T23:01:40.421 INFO:tasks.workunit.client.0.smithi027.stdout:9/810: rename d4/d77/lc6 to d4/d10/l11d 0 2024-04-26T23:01:40.422 INFO:tasks.workunit.client.0.smithi027.stdout:9/811: symlink d4/d17/l11e 0 2024-04-26T23:01:40.426 INFO:tasks.workunit.client.0.smithi027.stdout:9/812: creat d4/d10/d25/dc9/f11f x:0 0 0 2024-04-26T23:01:40.427 INFO:tasks.workunit.client.0.smithi027.stdout:9/813: readlink d4/d10/d67/l68 0 2024-04-26T23:01:40.427 INFO:tasks.workunit.client.0.smithi027.stdout:8/886: rename d9/da/d2a/d5e/d92/fae to d9/de/d2e/d110/dea/f136 0 2024-04-26T23:01:40.427 INFO:tasks.workunit.client.0.smithi027.stdout:9/814: truncate d4/d17/d2e/fcc 296743 0 2024-04-26T23:01:40.427 INFO:tasks.workunit.client.0.smithi027.stdout:8/887: creat d9/da/d2a/d5e/d72/d84/d5d/d6c/f137 x:0 0 0 2024-04-26T23:01:40.427 INFO:tasks.workunit.client.0.smithi027.stdout:8/888: read d9/de/f2c [1662569,106050] 0 2024-04-26T23:01:40.428 INFO:tasks.workunit.client.0.smithi027.stdout:0/878: dread d0/df/d3f/d64/f5e [0,4194304] 0 2024-04-26T23:01:40.431 INFO:tasks.workunit.client.0.smithi027.stdout:0/879: mknod d0/df/d3f/d64/da8/c10b 0 2024-04-26T23:01:40.432 INFO:tasks.workunit.client.0.smithi027.stdout:0/880: mknod d0/df/d9d/c10c 0 2024-04-26T23:01:40.433 INFO:tasks.workunit.client.0.smithi027.stdout:0/881: getdents d0/d4/da/d1a/d59/d92/dcd 0 2024-04-26T23:01:40.444 INFO:tasks.workunit.client.0.smithi027.stdout:8/889: rename d9/da/d2a/d5e/d9e/dba to d9/da/d2a/d5e/d72/d84/d4f/dd3/d138 0 2024-04-26T23:01:40.446 INFO:tasks.workunit.client.0.smithi027.stdout:8/890: creat d9/da/d119/f139 x:0 0 0 2024-04-26T23:01:40.449 INFO:tasks.workunit.client.0.smithi027.stdout:8/891: mkdir d9/de/d2e/d110/dc2/dc7/d13a 0 2024-04-26T23:01:40.462 INFO:tasks.workunit.client.0.smithi027.stdout:1/999: dwrite d1/d26/d66/d73/dae/fe8 [0,4194304] 0 2024-04-26T23:01:40.477 INFO:tasks.workunit.client.0.smithi027.stdout:2/866: dwrite d2/d5/db/de/f30 [0,4194304] 0 2024-04-26T23:01:40.477 INFO:tasks.workunit.client.0.smithi027.stdout:2/867: truncate d2/d5/db/de/d21/d8f/f2d 4282972 0 2024-04-26T23:01:40.477 INFO:tasks.workunit.client.0.smithi027.stdout:2/868: readlink d2/d5/db/d1b/d1c/d25/d19/lb5 0 2024-04-26T23:01:40.477 INFO:tasks.workunit.client.0.smithi027.stdout:0/882: dwrite d0/d4/f24 [0,4194304] 0 2024-04-26T23:01:40.477 INFO:tasks.workunit.client.0.smithi027.stdout:0/883: stat d0/d4/da/d1a/d59/c5d 0 2024-04-26T23:01:40.477 INFO:tasks.workunit.client.0.smithi027.stdout:0/884: readlink d0/d4/da/d1a/dc3/df6/lf9 0 2024-04-26T23:01:40.478 INFO:tasks.workunit.client.0.smithi027.stdout:0/885: mknod d0/c10d 0 2024-04-26T23:01:40.478 INFO:tasks.workunit.client.0.smithi027.stdout:0/886: truncate d0/df/d9d/d9f/dbb/f109 861142 0 2024-04-26T23:01:40.513 INFO:tasks.workunit.client.0.smithi027.stdout:0/887: dwrite d0/d4/da/f19 [4194304,4194304] 0 2024-04-26T23:01:40.514 INFO:tasks.workunit.client.0.smithi027.stdout:2/869: dwrite d2/db9/fe1 [0,4194304] 0 2024-04-26T23:01:40.514 INFO:tasks.workunit.client.0.smithi027.stdout:9/815: dwrite d4/d10/d40/d58/d75/f78 [0,4194304] 0 2024-04-26T23:01:40.515 INFO:tasks.workunit.client.0.smithi027.stdout:9/816: stat d4/d17/d2e/d46/da0/d72/fd0 0 2024-04-26T23:01:40.517 INFO:tasks.workunit.client.0.smithi027.stdout:0/888: rename d0/fac to d0/d4/da/d1a/d59/d92/d107/f10e 0 2024-04-26T23:01:40.522 INFO:tasks.workunit.client.0.smithi027.stdout:9/817: creat d4/d17/d2e/d46/da0/d8d/df2/f120 x:0 0 0 2024-04-26T23:01:40.526 INFO:tasks.workunit.client.0.smithi027.stdout:0/889: rmdir d0/df/d9d/de5 39 2024-04-26T23:01:40.526 INFO:tasks.workunit.client.0.smithi027.stdout:0/890: mkdir d0/d4/da/d10f 0 2024-04-26T23:01:40.526 INFO:tasks.workunit.client.0.smithi027.stdout:8/892: dwrite d9/f17 [0,4194304] 0 2024-04-26T23:01:40.526 INFO:tasks.workunit.client.0.smithi027.stdout:8/893: stat d9/da/d2a/d5e/d72/d84/d4f/dd3/lf6 0 2024-04-26T23:01:40.527 INFO:tasks.workunit.client.0.smithi027.stdout:8/894: creat d9/da/f13b x:0 0 0 2024-04-26T23:01:40.528 INFO:tasks.workunit.client.0.smithi027.stdout:9/818: write d4/d77/da2/fa4 [5304195,59708] 0 2024-04-26T23:01:40.528 INFO:tasks.workunit.client.0.smithi027.stdout:9/819: dread - d4/d17/ff6 zero size 2024-04-26T23:01:40.529 INFO:tasks.workunit.client.0.smithi027.stdout:8/895: creat d9/da/d2a/d5e/d72/d84/d5d/f13c x:0 0 0 2024-04-26T23:01:40.530 INFO:tasks.workunit.client.0.smithi027.stdout:9/820: mknod d4/d17/d2e/d81/c121 0 2024-04-26T23:01:40.530 INFO:tasks.workunit.client.0.smithi027.stdout:8/896: creat d9/da6/f13d x:0 0 0 2024-04-26T23:01:40.533 INFO:tasks.workunit.client.0.smithi027.stdout:2/870: dread d2/d5/db/d1b/d1c/d8b/f56 [0,4194304] 0 2024-04-26T23:01:40.534 INFO:tasks.workunit.client.0.smithi027.stdout:9/821: rename d4/d10/d40/d58/f101 to d4/d17/d2e/d10e/f122 0 2024-04-26T23:01:40.534 INFO:tasks.workunit.client.0.smithi027.stdout:9/822: fdatasync d4/f52 0 2024-04-26T23:01:40.535 INFO:tasks.workunit.client.0.smithi027.stdout:2/871: unlink d2/d5/db/d1b/d1c/d25/d6a/ff9 0 2024-04-26T23:01:40.537 INFO:tasks.workunit.client.0.smithi027.stdout:2/872: link d2/d5/db/de/f42 d2/d5/db/d1b/d1c/d25/dc0/df1/f115 0 2024-04-26T23:01:40.537 INFO:tasks.workunit.client.0.smithi027.stdout:2/873: fdatasync d2/d5/db/d1b/d1c/d25/d19/d23/d7c/fb3 0 2024-04-26T23:01:40.537 INFO:tasks.workunit.client.0.smithi027.stdout:2/874: chown d2/db9/le3 77 1 2024-04-26T23:01:40.537 INFO:tasks.workunit.client.0.smithi027.stdout:2/875: dread - d2/d5/db/d1b/d1c/d25/d6a/d8a/d8d/f105 zero size 2024-04-26T23:01:40.538 INFO:tasks.workunit.client.0.smithi027.stdout:2/876: creat d2/d5/db/de/d21/dc4/d39/dd2/f116 x:0 0 0 2024-04-26T23:01:40.546 INFO:tasks.workunit.client.0.smithi027.stdout:2/877: dread d2/d5/fd0 [0,4194304] 0 2024-04-26T23:01:40.546 INFO:tasks.workunit.client.0.smithi027.stdout:2/878: fsync d2/d5/db/de/d21/d8f/d88/fe6 0 2024-04-26T23:01:40.546 INFO:tasks.workunit.client.0.smithi027.stdout:2/879: read - d2/d5/db/d1b/d1c/d25/d6a/d8a/d8d/f105 zero size 2024-04-26T23:01:40.549 INFO:tasks.workunit.client.0.smithi027.stdout:2/880: mknod d2/d5/db/de/d21/dc4/d39/dd2/c117 0 2024-04-26T23:01:40.566 INFO:tasks.workunit.client.0.smithi027.stdout:2/881: dread d2/d5/f6e [0,4194304] 0 2024-04-26T23:01:40.568 INFO:tasks.workunit.client.0.smithi027.stdout:9/823: dwrite d4/f43 [0,4194304] 0 2024-04-26T23:01:40.581 INFO:tasks.workunit.client.0.smithi027.stdout:5/955: sync 2024-04-26T23:01:40.581 INFO:tasks.workunit.client.0.smithi027.stdout:8/897: read d9/da/d2a/d5e/d72/d84/d5d/d8b/f9a [230340,127636] 0 2024-04-26T23:01:40.582 INFO:tasks.workunit.client.0.smithi027.stdout:5/956: write d4/d24/d22/dab/d16/f9b [1013509,96633] 0 2024-04-26T23:01:40.582 INFO:tasks.workunit.client.0.smithi027.stdout:5/957: write d4/d24/d22/dab/d16/d4b/fef [43326,41017] 0 2024-04-26T23:01:40.583 INFO:tasks.workunit.client.0.smithi027.stdout:8/898: creat d9/de/d2e/f13e x:0 0 0 2024-04-26T23:01:40.584 INFO:tasks.workunit.client.0.smithi027.stdout:9/824: dread d4/d10/d40/d58/f96 [0,4194304] 0 2024-04-26T23:01:40.585 INFO:tasks.workunit.client.0.smithi027.stdout:5/958: unlink d4/d24/d22/d29/d2a/f45 0 2024-04-26T23:01:40.585 INFO:tasks.workunit.client.0.smithi027.stdout:5/959: fsync d4/d24/d5f/dac/d40/f6c 0 2024-04-26T23:01:40.585 INFO:tasks.workunit.client.0.smithi027.stdout:8/899: creat d9/da6/f13f x:0 0 0 2024-04-26T23:01:40.585 INFO:tasks.workunit.client.0.smithi027.stdout:8/900: truncate d9/da/d33/f105 515599 0 2024-04-26T23:01:40.592 INFO:tasks.workunit.client.0.smithi027.stdout:9/825: unlink d4/d10/l11d 0 2024-04-26T23:01:40.592 INFO:tasks.workunit.client.0.smithi027.stdout:0/891: dwrite d0/d4/da/f2f [4194304,4194304] 0 2024-04-26T23:01:40.592 INFO:tasks.workunit.client.0.smithi027.stdout:9/826: truncate d4/d77/da2/daf/f11b 1048006 0 2024-04-26T23:01:40.592 INFO:tasks.workunit.client.0.smithi027.stdout:0/892: stat d0/d4/f3d 0 2024-04-26T23:01:40.592 INFO:tasks.workunit.client.0.smithi027.stdout:9/827: chown d4/d17/d2e/d46/da0/fc0 7315 1 2024-04-26T23:01:40.592 INFO:tasks.workunit.client.0.smithi027.stdout:0/893: chown d0/df/d3f/c63 14038 1 2024-04-26T23:01:40.592 INFO:tasks.workunit.client.0.smithi027.stdout:9/828: stat d4/d10/d40/d58/d75/d84/dc4/d119 0 2024-04-26T23:01:40.592 INFO:tasks.workunit.client.0.smithi027.stdout:0/894: creat d0/d4/da/d1a/dc3/df6/f110 x:0 0 0 2024-04-26T23:01:40.592 INFO:tasks.workunit.client.0.smithi027.stdout:0/895: write d0/df/d3f/f61 [49391,110009] 0 2024-04-26T23:01:40.593 INFO:tasks.workunit.client.0.smithi027.stdout:0/896: rename d0/d4/da/d1a/d1e/dba/cc1 to d0/d4/da/c111 0 2024-04-26T23:01:40.593 INFO:tasks.workunit.client.0.smithi027.stdout:0/897: truncate d0/df/d3f/d64/fff 407815 0 2024-04-26T23:01:40.596 INFO:tasks.workunit.client.0.smithi027.stdout:9/829: dread d4/d77/da2/daf/f11b [0,4194304] 0 2024-04-26T23:01:40.596 INFO:tasks.workunit.client.0.smithi027.stdout:8/901: dread d9/f74 [0,4194304] 0 2024-04-26T23:01:40.598 INFO:tasks.workunit.client.0.smithi027.stdout:9/830: creat d4/d17/d2e/d46/da0/d8d/f123 x:0 0 0 2024-04-26T23:01:40.598 INFO:tasks.workunit.client.0.smithi027.stdout:9/831: fdatasync d4/d17/d2e/d81/da7/db8/feb 0 2024-04-26T23:01:40.606 INFO:tasks.workunit.client.0.smithi027.stdout:8/902: read d9/da/d2a/d6b/f81 [26854,8555] 0 2024-04-26T23:01:40.607 INFO:tasks.workunit.client.0.smithi027.stdout:5/960: dread d4/d24/f5d [0,4194304] 0 2024-04-26T23:01:40.607 INFO:tasks.workunit.client.0.smithi027.stdout:5/961: readlink d4/d1a/l120 0 2024-04-26T23:01:40.607 INFO:tasks.workunit.client.0.smithi027.stdout:5/962: fdatasync d4/d9f/dc0/f133 0 2024-04-26T23:01:40.607 INFO:tasks.workunit.client.0.smithi027.stdout:5/963: chown d4/d1a/d2b/c96 167304925 1 2024-04-26T23:01:40.607 INFO:tasks.workunit.client.0.smithi027.stdout:5/964: readlink d4/d24/d22/d29/d2a/l72 0 2024-04-26T23:01:40.608 INFO:tasks.workunit.client.0.smithi027.stdout:8/903: link d9/da/d2a/d5e/d2b/lf4 d9/da/d2a/d5e/d2b/d52/l140 0 2024-04-26T23:01:40.608 INFO:tasks.workunit.client.0.smithi027.stdout:8/904: write d9/da/d2a/d5e/d72/d84/d5d/d6c/ffc [823922,128608] 0 2024-04-26T23:01:40.608 INFO:tasks.workunit.client.0.smithi027.stdout:8/905: chown d9/da/d2a/d5e/d72/d84/d5d/d6c/c9f 8654 1 2024-04-26T23:01:40.609 INFO:tasks.workunit.client.0.smithi027.stdout:8/906: readlink d9/da/d2a/d5e/d2b/d52/l5a 0 2024-04-26T23:01:40.610 INFO:tasks.workunit.client.0.smithi027.stdout:8/907: mknod d9/da/d2a/d5e/d46/c141 0 2024-04-26T23:01:40.630 INFO:tasks.workunit.client.0.smithi027.stdout:9/832: dwrite d4/d17/d2e/f53 [0,4194304] 0 2024-04-26T23:01:40.630 INFO:tasks.workunit.client.0.smithi027.stdout:8/908: dread d9/da/d2a/d5e/d2b/f3d [0,4194304] 0 2024-04-26T23:01:40.633 INFO:tasks.workunit.client.0.smithi027.stdout:9/833: rename d4/d10/d25/l2a to d4/d10/d40/d58/d75/d11a/l124 0 2024-04-26T23:01:40.633 INFO:tasks.workunit.client.0.smithi027.stdout:9/834: write d4/d17/d2e/d46/da0/d8d/fb5 [896675,31337] 0 2024-04-26T23:01:40.634 INFO:tasks.workunit.client.0.smithi027.stdout:9/835: link d4/d77/da2/daf/fc8 d4/d77/da2/daf/f125 0 2024-04-26T23:01:40.636 INFO:tasks.workunit.client.0.smithi027.stdout:9/836: mkdir d4/d10/d40/d126 0 2024-04-26T23:01:40.637 INFO:tasks.workunit.client.0.smithi027.stdout:9/837: creat d4/d17/d2e/f127 x:0 0 0 2024-04-26T23:01:40.638 INFO:tasks.workunit.client.0.smithi027.stdout:9/838: mkdir d4/d10/d40/d58/d75/d84/d86/d128 0 2024-04-26T23:01:40.638 INFO:tasks.workunit.client.0.smithi027.stdout:9/839: dread - d4/d10/d40/d58/d75/d84/dc4/de4/f108 zero size 2024-04-26T23:01:40.640 INFO:tasks.workunit.client.0.smithi027.stdout:9/840: symlink d4/l129 0 2024-04-26T23:01:40.640 INFO:tasks.workunit.client.0.smithi027.stdout:9/841: readlink d4/d17/d2e/d46/l60 0 2024-04-26T23:01:40.640 INFO:tasks.workunit.client.0.smithi027.stdout:9/842: dread - d4/d17/d2e/d10e/f122 zero size 2024-04-26T23:01:40.640 INFO:tasks.workunit.client.0.smithi027.stdout:9/843: truncate d4/d10/d40/d58/d75/f78 4900511 0 2024-04-26T23:01:40.640 INFO:tasks.workunit.client.0.smithi027.stdout:9/844: dread - d4/d77/da2/daf/fcb zero size 2024-04-26T23:01:40.641 INFO:tasks.workunit.client.0.smithi027.stdout:9/845: unlink d4/d77/da2/daf/fc8 0 2024-04-26T23:01:40.643 INFO:tasks.workunit.client.0.smithi027.stdout:9/846: symlink d4/d10/d40/d58/d75/d84/d86/dc5/l12a 0 2024-04-26T23:01:40.646 INFO:tasks.workunit.client.0.smithi027.stdout:9/847: rmdir d4/d10/d40/d58/d75 39 2024-04-26T23:01:40.689 INFO:tasks.workunit.client.0.smithi027.stdout:7/941: sync 2024-04-26T23:01:40.694 INFO:tasks.workunit.client.0.smithi027.stdout:7/942: link d0/d3d/f54 d0/d28/d2b/d3a/d90/dcb/d111/f132 0 2024-04-26T23:01:40.697 INFO:tasks.workunit.client.0.smithi027.stdout:7/943: rename d0/d28/d2b/d3a/d48/d56/d8d/c123 to d0/d28/d2b/d3a/d90/c133 0 2024-04-26T23:01:40.707 INFO:tasks.workunit.client.0.smithi027.stdout:9/848: write d4/d10/d67/fb6 [1292313,7531] 0 2024-04-26T23:01:40.707 INFO:tasks.workunit.client.0.smithi027.stdout:9/849: chown d4/d10/d40/d58/d75/d84/c105 16426535 1 2024-04-26T23:01:40.707 INFO:tasks.workunit.client.0.smithi027.stdout:8/909: dwrite d9/da/d2a/d5e/d92/fb9 [0,4194304] 0 2024-04-26T23:01:40.711 INFO:tasks.workunit.client.0.smithi027.stdout:9/850: mkdir d4/d17/d2e/d46/da0/d8d/df2/d114/d12b 0 2024-04-26T23:01:40.711 INFO:tasks.workunit.client.0.smithi027.stdout:9/851: chown d4/d10/d40/d58/d75/d84/d86/f94 28 1 2024-04-26T23:01:40.711 INFO:tasks.workunit.client.0.smithi027.stdout:9/852: truncate d4/f2b 4624323 0 2024-04-26T23:01:40.711 INFO:tasks.workunit.client.0.smithi027.stdout:9/853: fsync d4/d10/d40/f64 0 2024-04-26T23:01:40.711 INFO:tasks.workunit.client.0.smithi027.stdout:9/854: dread - d4/d10/d40/d58/d75/d84/dc4/de4/f10c zero size 2024-04-26T23:01:40.712 INFO:tasks.workunit.client.0.smithi027.stdout:9/855: chown d4/d10/d40/ce0 888015 1 2024-04-26T23:01:40.713 INFO:tasks.workunit.client.0.smithi027.stdout:9/856: symlink d4/d10/d40/d58/d75/d84/dc4/l12c 0 2024-04-26T23:01:40.713 INFO:tasks.workunit.client.0.smithi027.stdout:9/857: fdatasync d4/d10/d40/d58/d75/fe1 0 2024-04-26T23:01:40.714 INFO:tasks.workunit.client.0.smithi027.stdout:9/858: symlink d4/d17/d2e/d46/l12d 0 2024-04-26T23:01:40.716 INFO:tasks.workunit.client.0.smithi027.stdout:9/859: truncate d4/d10/f2f 1750995 0 2024-04-26T23:01:40.721 INFO:tasks.workunit.client.0.smithi027.stdout:7/944: write d0/d28/f2c [3379840,52949] 0 2024-04-26T23:01:40.721 INFO:tasks.workunit.client.0.smithi027.stdout:7/945: fdatasync d0/d3d/f131 0 2024-04-26T23:01:40.723 INFO:tasks.workunit.client.0.smithi027.stdout:7/946: read d0/fac [33652,19190] 0 2024-04-26T23:01:40.723 INFO:tasks.workunit.client.0.smithi027.stdout:7/947: chown d0/d28/d2b/d3a/d48/d56/d88 83 1 2024-04-26T23:01:40.723 INFO:tasks.workunit.client.0.smithi027.stdout:9/860: write d4/d10/d40/d58/d75/f47 [99857,121720] 0 2024-04-26T23:01:40.723 INFO:tasks.workunit.client.0.smithi027.stdout:8/910: dread d9/f11d [0,4194304] 0 2024-04-26T23:01:40.725 INFO:tasks.workunit.client.0.smithi027.stdout:7/948: truncate d0/d28/d2b/d55/d59/d6d/fff 736655 0 2024-04-26T23:01:40.729 INFO:tasks.workunit.client.0.smithi027.stdout:9/861: creat d4/d17/d2e/d46/da0/d8d/f12e x:0 0 0 2024-04-26T23:01:40.729 INFO:tasks.workunit.client.0.smithi027.stdout:9/862: truncate d4/d17/d2e/d46/dd1/ffe 1220652 0 2024-04-26T23:01:40.729 INFO:tasks.workunit.client.0.smithi027.stdout:8/911: mknod d9/da6/c142 0 2024-04-26T23:01:40.729 INFO:tasks.workunit.client.0.smithi027.stdout:7/949: truncate d0/d28/d2b/d3a/d48/d56/d88/d73/f104 1118130 0 2024-04-26T23:01:40.729 INFO:tasks.workunit.client.0.smithi027.stdout:7/950: read d0/d28/d2b/d55/d59/f7b [6102048,52675] 0 2024-04-26T23:01:40.729 INFO:tasks.workunit.client.0.smithi027.stdout:7/951: symlink d0/d28/d66/da3/d124/l134 0 2024-04-26T23:01:40.729 INFO:tasks.workunit.client.0.smithi027.stdout:7/952: stat d0/d28/d2b/d3a/ffb 0 2024-04-26T23:01:40.729 INFO:tasks.workunit.client.0.smithi027.stdout:7/953: write d0/d28/d2b/d3a/d90/dcb/d111/f132 [376164,42351] 0 2024-04-26T23:01:40.729 INFO:tasks.workunit.client.0.smithi027.stdout:7/954: write d0/d28/d2b/d3a/d90/dcb/fe9 [616295,52388] 0 2024-04-26T23:01:40.733 INFO:tasks.workunit.client.0.smithi027.stdout:7/955: mkdir d0/d3d/d135 0 2024-04-26T23:01:40.735 INFO:tasks.workunit.client.0.smithi027.stdout:7/956: truncate d0/f27 4859622 0 2024-04-26T23:01:40.736 INFO:tasks.workunit.client.0.smithi027.stdout:7/957: symlink d0/d28/d2b/d3a/d107/l136 0 2024-04-26T23:01:40.737 INFO:tasks.workunit.client.0.smithi027.stdout:7/958: write d0/d28/d2b/d55/d69/da7/fc6 [92182,119048] 0 2024-04-26T23:01:40.738 INFO:tasks.workunit.client.0.smithi027.stdout:7/959: getdents d0/d28/d9f/db8 0 2024-04-26T23:01:40.748 INFO:tasks.workunit.client.0.smithi027.stdout:8/912: write d9/da/d2a/fbd [249887,83563] 0 2024-04-26T23:01:40.748 INFO:tasks.workunit.client.0.smithi027.stdout:8/913: write d9/f74 [542953,68313] 0 2024-04-26T23:01:40.751 INFO:tasks.workunit.client.0.smithi027.stdout:7/960: mkdir d0/d3d/dba/df8/d137 0 2024-04-26T23:01:40.753 INFO:tasks.workunit.client.0.smithi027.stdout:8/914: mkdir d9/de/d2e/d110/d8d/d143 0 2024-04-26T23:01:40.753 INFO:tasks.workunit.client.0.smithi027.stdout:8/915: dread - d9/de/d22/fe3 zero size 2024-04-26T23:01:40.766 INFO:tasks.workunit.client.0.smithi027.stdout:9/863: dwrite d4/d10/d40/d58/d75/f63 [0,4194304] 0 2024-04-26T23:01:40.772 INFO:tasks.workunit.client.0.smithi027.stdout:8/916: dread d9/da/d33/f88 [0,4194304] 0 2024-04-26T23:01:40.775 INFO:tasks.workunit.client.0.smithi027.stdout:8/917: mkdir d9/da/d2a/d5e/d144 0 2024-04-26T23:01:40.776 INFO:tasks.workunit.client.0.smithi027.stdout:8/918: mkdir d9/da/d2a/d5e/d144/d145 0 2024-04-26T23:01:40.790 INFO:tasks.workunit.client.0.smithi027.stdout:7/961: dwrite d0/d28/d2b/d3a/d48/d56/dee/d100/ff1 [0,4194304] 0 2024-04-26T23:01:40.790 INFO:tasks.workunit.client.0.smithi027.stdout:7/962: chown d0/d28/d2b/d3a/d48/d56/d8d/dc5/ldf 1 1 2024-04-26T23:01:40.798 INFO:tasks.workunit.client.0.smithi027.stdout:7/963: dread d0/d28/d9f/f108 [0,4194304] 0 2024-04-26T23:01:40.798 INFO:tasks.workunit.client.0.smithi027.stdout:7/964: stat d0/d28/d2b/d3a/d48/f10f 0 2024-04-26T23:01:40.817 INFO:tasks.workunit.client.0.smithi027.stdout:9/864: dwrite d4/d17/d2e/f127 [0,4194304] 0 2024-04-26T23:01:40.817 INFO:tasks.workunit.client.0.smithi027.stdout:9/865: read - d4/d17/d2e/d81/da7/db8/fbd zero size 2024-04-26T23:01:40.817 INFO:tasks.workunit.client.0.smithi027.stdout:9/866: readlink d4/d17/d2e/l59 0 2024-04-26T23:01:40.820 INFO:tasks.workunit.client.0.smithi027.stdout:9/867: mkdir d4/d77/da2/daf/d12f 0 2024-04-26T23:01:40.823 INFO:tasks.workunit.client.0.smithi027.stdout:3/966: sync 2024-04-26T23:01:40.823 INFO:tasks.workunit.client.0.smithi027.stdout:2/882: sync 2024-04-26T23:01:40.823 INFO:tasks.workunit.client.0.smithi027.stdout:4/934: sync 2024-04-26T23:01:40.823 INFO:tasks.workunit.client.0.smithi027.stdout:6/794: sync 2024-04-26T23:01:40.823 INFO:tasks.workunit.client.0.smithi027.stdout:4/935: stat da/dd/de/f94 0 2024-04-26T23:01:40.824 INFO:tasks.workunit.client.0.smithi027.stdout:9/868: write d4/d77/da2/daf/f125 [262845,69654] 0 2024-04-26T23:01:40.824 INFO:tasks.workunit.client.0.smithi027.stdout:4/936: creat da/dd/de/d14/db3/d114/f12a x:0 0 0 2024-04-26T23:01:40.824 INFO:tasks.workunit.client.0.smithi027.stdout:4/937: write da/dd/de/d79/da8/dc5/de8/f104 [89741,75733] 0 2024-04-26T23:01:40.825 INFO:tasks.workunit.client.0.smithi027.stdout:2/883: getdents d2/d5/db/d1b/d1c/d25/d19/d44/d7e 0 2024-04-26T23:01:40.827 INFO:tasks.workunit.client.0.smithi027.stdout:9/869: rename d4/d17/d2e/d81/da7/db8 to d4/d77/da2/d130 0 2024-04-26T23:01:40.830 INFO:tasks.workunit.client.0.smithi027.stdout:6/795: dread d1/df/d43/da8/d84/fcb [0,4194304] 0 2024-04-26T23:01:40.831 INFO:tasks.workunit.client.0.smithi027.stdout:2/884: mkdir d2/d5/db/d1b/d1c/d25/d19/d44/dee/d118 0 2024-04-26T23:01:40.831 INFO:tasks.workunit.client.0.smithi027.stdout:6/796: write d1/df/d3c/f75 [1318550,34416] 0 2024-04-26T23:01:40.834 INFO:tasks.workunit.client.0.smithi027.stdout:7/965: dwrite d0/fc2 [0,4194304] 0 2024-04-26T23:01:40.836 INFO:tasks.workunit.client.0.smithi027.stdout:2/885: rename d2/d5/db/de/d21/d8f/d88 to d2/d5/db/d1b/d1c/d119 0 2024-04-26T23:01:40.836 INFO:tasks.workunit.client.0.smithi027.stdout:2/886: readlink d2/d5/db/d1b/d1c/d25/d6a/d8a/l9b 0 2024-04-26T23:01:40.836 INFO:tasks.workunit.client.0.smithi027.stdout:2/887: chown d2/d5/db/de/d21/ccd 12607852 1 2024-04-26T23:01:40.841 INFO:tasks.workunit.client.0.smithi027.stdout:7/966: read d0/d28/d2b/d3a/d48/d56/d88/d73/f104 [1033477,7341] 0 2024-04-26T23:01:40.842 INFO:tasks.workunit.client.0.smithi027.stdout:6/797: truncate d1/df/d3c/f75 579569 0 2024-04-26T23:01:40.843 INFO:tasks.workunit.client.0.smithi027.stdout:7/967: creat d0/d28/d2b/d55/d69/d75/f138 x:0 0 0 2024-04-26T23:01:40.843 INFO:tasks.workunit.client.0.smithi027.stdout:6/798: getdents d1/de 0 2024-04-26T23:01:40.844 INFO:tasks.workunit.client.0.smithi027.stdout:4/938: write da/dd/de/d14/d21/f46 [1143110,39827] 0 2024-04-26T23:01:40.844 INFO:tasks.workunit.client.0.smithi027.stdout:4/939: readlink da/dd/de/d14/l55 0 2024-04-26T23:01:40.849 INFO:tasks.workunit.client.0.smithi027.stdout:6/799: mkdir d1/df/dfe 0 2024-04-26T23:01:40.849 INFO:tasks.workunit.client.0.smithi027.stdout:7/968: mkdir d0/d28/d66/da3/d139 0 2024-04-26T23:01:40.850 INFO:tasks.workunit.client.0.smithi027.stdout:6/800: dread - d1/df/d3c/fba zero size 2024-04-26T23:01:40.850 INFO:tasks.workunit.client.0.smithi027.stdout:4/940: mkdir da/dd/de/d14/d17/d111/d12b 0 2024-04-26T23:01:40.850 INFO:tasks.workunit.client.0.smithi027.stdout:2/888: write d2/d5/db/d1b/d1c/d8b/fa9 [828438,114991] 0 2024-04-26T23:01:40.850 INFO:tasks.workunit.client.0.smithi027.stdout:4/941: fsync da/dd/de/d14/d21/d41/d59/f9c 0 2024-04-26T23:01:40.850 INFO:tasks.workunit.client.0.smithi027.stdout:9/870: write d4/d17/d2e/d46/f5f [752349,62743] 0 2024-04-26T23:01:40.852 INFO:tasks.workunit.client.0.smithi027.stdout:6/801: rmdir d1/de/d57 39 2024-04-26T23:01:40.853 INFO:tasks.workunit.client.0.smithi027.stdout:2/889: stat d2/d5/db/d1b/d1c/l6b 0 2024-04-26T23:01:40.853 INFO:tasks.workunit.client.0.smithi027.stdout:2/890: readlink d2/d5/db/de/d21/dc4/d39/dd2/lda 0 2024-04-26T23:01:40.854 INFO:tasks.workunit.client.0.smithi027.stdout:9/871: stat d4/d17/d2e/d46/lf1 0 2024-04-26T23:01:40.856 INFO:tasks.workunit.client.0.smithi027.stdout:6/802: rmdir d1/de/d57/d8e/dd0 39 2024-04-26T23:01:40.856 INFO:tasks.workunit.client.0.smithi027.stdout:6/803: fdatasync d1/de/f42 0 2024-04-26T23:01:40.859 INFO:tasks.workunit.client.0.smithi027.stdout:2/891: creat d2/f11a x:0 0 0 2024-04-26T23:01:40.862 INFO:tasks.workunit.client.0.smithi027.stdout:9/872: creat d4/d17/d2e/d10e/f131 x:0 0 0 2024-04-26T23:01:40.864 INFO:tasks.workunit.client.0.smithi027.stdout:2/892: rename d2/d5/db/c87 to d2/d5/db/de/d21/dc4/c11b 0 2024-04-26T23:01:40.865 INFO:tasks.workunit.client.0.smithi027.stdout:2/893: write d2/d5/db/d1b/d1c/d25/d19/f5b [162884,53399] 0 2024-04-26T23:01:40.865 INFO:tasks.workunit.client.0.smithi027.stdout:0/898: sync 2024-04-26T23:01:40.865 INFO:tasks.workunit.client.0.smithi027.stdout:5/965: sync 2024-04-26T23:01:40.865 INFO:tasks.workunit.client.0.smithi027.stdout:0/899: write d0/d4/da/d1a/d59/f9c [767327,34415] 0 2024-04-26T23:01:40.867 INFO:tasks.workunit.client.0.smithi027.stdout:2/894: mkdir d2/d5/db/d1b/d1c/d25/d19/db0/d11c 0 2024-04-26T23:01:40.870 INFO:tasks.workunit.client.0.smithi027.stdout:0/900: mknod d0/d4/da/d1a/d59/c112 0 2024-04-26T23:01:40.872 INFO:tasks.workunit.client.0.smithi027.stdout:0/901: link d0/d4/lc d0/d4/da/d1a/d6d/dc9/l113 0 2024-04-26T23:01:40.876 INFO:tasks.workunit.client.0.smithi027.stdout:0/902: rename d0/df/d3f/d4a/c98 to d0/df/c114 0 2024-04-26T23:01:40.876 INFO:tasks.workunit.client.0.smithi027.stdout:6/804: write d1/de/f1a [3548136,76107] 0 2024-04-26T23:01:40.877 INFO:tasks.workunit.client.0.smithi027.stdout:6/805: link d1/df/d43/da8/cb2 d1/cff 0 2024-04-26T23:01:40.878 INFO:tasks.workunit.client.0.smithi027.stdout:0/903: mknod d0/dfa/c115 0 2024-04-26T23:01:40.880 INFO:tasks.workunit.client.0.smithi027.stdout:4/942: dread da/dd/de/d14/d21/d41/d59/f8d [0,4194304] 0 2024-04-26T23:01:40.880 INFO:tasks.workunit.client.0.smithi027.stdout:2/895: dread d2/d5/db/d1b/d1c/d119/fe9 [0,4194304] 0 2024-04-26T23:01:40.880 INFO:tasks.workunit.client.0.smithi027.stdout:0/904: creat d0/d4/da/d1a/d1e/f116 x:0 0 0 2024-04-26T23:01:40.884 INFO:tasks.workunit.client.0.smithi027.stdout:9/873: dread d4/d17/d2e/d46/da0/fc0 [0,4194304] 0 2024-04-26T23:01:40.888 INFO:tasks.workunit.client.0.smithi027.stdout:0/905: unlink d0/l57 0 2024-04-26T23:01:40.889 INFO:tasks.workunit.client.0.smithi027.stdout:2/896: rename d2/d5/db/de/d21/d8f/fbf to d2/d5/db/d1b/d1c/d8b/f11d 0 2024-04-26T23:01:40.889 INFO:tasks.workunit.client.0.smithi027.stdout:2/897: chown d2/d5/db/de/d21/ccd 88051802 1 2024-04-26T23:01:40.889 INFO:tasks.workunit.client.0.smithi027.stdout:4/943: rmdir da/dd 39 2024-04-26T23:01:40.889 INFO:tasks.workunit.client.0.smithi027.stdout:9/874: unlink d4/d17/d2e/d81/da7/lac 0 2024-04-26T23:01:40.889 INFO:tasks.workunit.client.0.smithi027.stdout:9/875: readlink d4/l15 0 2024-04-26T23:01:40.889 INFO:tasks.workunit.client.0.smithi027.stdout:9/876: stat d4/d10/d25/la6 0 2024-04-26T23:01:40.889 INFO:tasks.workunit.client.0.smithi027.stdout:2/898: rename d2/d5/db/d1b/d1c/f20 to d2/d5/db/d1b/d1c/d25/d6a/d8a/d8d/f11e 0 2024-04-26T23:01:40.889 INFO:tasks.workunit.client.0.smithi027.stdout:2/899: creat d2/db9/dc6/f11f x:0 0 0 2024-04-26T23:01:40.890 INFO:tasks.workunit.client.0.smithi027.stdout:7/969: dwrite d0/d6c/f71 [0,4194304] 0 2024-04-26T23:01:40.892 INFO:tasks.workunit.client.0.smithi027.stdout:7/970: link d0/d28/d2b/d3a/d48/d56/d88/d73/d80/d85/lb0 d0/d28/d66/da3/d139/l13a 0 2024-04-26T23:01:40.923 INFO:tasks.workunit.client.0.smithi027.stdout:6/806: dwrite d1/df/d3c/d9a/ff6 [0,4194304] 0 2024-04-26T23:01:40.924 INFO:tasks.workunit.client.0.smithi027.stdout:9/877: dread d4/d17/d2e/d70/ffd [0,4194304] 0 2024-04-26T23:01:40.927 INFO:tasks.workunit.client.0.smithi027.stdout:9/878: getdents d4/d10/d40/d58 0 2024-04-26T23:01:40.933 INFO:tasks.workunit.client.0.smithi027.stdout:7/971: dwrite d0/d28/d2b/d3a/d48/d56/d88/f91 [0,4194304] 0 2024-04-26T23:01:40.938 INFO:tasks.workunit.client.0.smithi027.stdout:6/807: write d1/f62 [796928,38826] 0 2024-04-26T23:01:40.944 INFO:tasks.workunit.client.0.smithi027.stdout:5/966: dwrite d4/d24/f4c [0,4194304] 0 2024-04-26T23:01:40.944 INFO:tasks.workunit.client.0.smithi027.stdout:5/967: fdatasync d4/d1a/d2b/db8/ff4 0 2024-04-26T23:01:40.948 INFO:tasks.workunit.client.0.smithi027.stdout:6/808: symlink d1/de/df3/d6f/d7d/l100 0 2024-04-26T23:01:40.949 INFO:tasks.workunit.client.0.smithi027.stdout:6/809: chown d1/de/d57/d8e/ffc 6 1 2024-04-26T23:01:40.949 INFO:tasks.workunit.client.0.smithi027.stdout:6/810: write d1/df/d43/da8/f9f [178150,36762] 0 2024-04-26T23:01:40.950 INFO:tasks.workunit.client.0.smithi027.stdout:0/906: dwrite d0/d4/da/d1a/d59/d92/dcd/fd4 [0,4194304] 0 2024-04-26T23:01:40.950 INFO:tasks.workunit.client.0.smithi027.stdout:0/907: readlink d0/df/d9d/de5/lf7 0 2024-04-26T23:01:40.951 INFO:tasks.workunit.client.0.smithi027.stdout:5/968: creat d4/d24/d5f/dac/d40/d54/f136 x:0 0 0 2024-04-26T23:01:40.956 INFO:tasks.workunit.client.0.smithi027.stdout:0/908: creat d0/d4/da/d1a/dc3/df6/f117 x:0 0 0 2024-04-26T23:01:40.957 INFO:tasks.workunit.client.0.smithi027.stdout:5/969: unlink d4/d24/d55/d63/fdd 0 2024-04-26T23:01:40.959 INFO:tasks.workunit.client.0.smithi027.stdout:5/970: mkdir d4/d1a/d2b/db8/d137 0 2024-04-26T23:01:40.966 INFO:tasks.workunit.client.0.smithi027.stdout:7/972: dwrite d0/d28/f18 [4194304,4194304] 0 2024-04-26T23:01:40.966 INFO:tasks.workunit.client.0.smithi027.stdout:7/973: write d0/d28/d2b/d3a/d48/dd2/f12a [620021,101696] 0 2024-04-26T23:01:40.968 INFO:tasks.workunit.client.0.smithi027.stdout:7/974: truncate d0/d6c/f8c 5464194 0 2024-04-26T23:01:40.978 INFO:tasks.workunit.client.0.smithi027.stdout:5/971: write d4/d1a/d4a/f76 [2801074,89968] 0 2024-04-26T23:01:40.980 INFO:tasks.workunit.client.0.smithi027.stdout:5/972: creat d4/d24/d22/f138 x:0 0 0 2024-04-26T23:01:40.980 INFO:tasks.workunit.client.0.smithi027.stdout:7/975: write d0/d28/d2b/d55/d59/d6d/fb5 [779828,121932] 0 2024-04-26T23:01:40.981 INFO:tasks.workunit.client.0.smithi027.stdout:7/976: chown d0/d6c/c116 75134726 1 2024-04-26T23:01:40.981 INFO:tasks.workunit.client.0.smithi027.stdout:6/811: fsync d1/f62 0 2024-04-26T23:01:40.981 INFO:tasks.workunit.client.0.smithi027.stdout:5/973: symlink d4/d1a/d2b/daa/d10c/l139 0 2024-04-26T23:01:40.984 INFO:tasks.workunit.client.0.smithi027.stdout:7/977: rename d0/d28/d66/da3/d139/l13a to d0/d28/d66/d67/l13b 0 2024-04-26T23:01:40.984 INFO:tasks.workunit.client.0.smithi027.stdout:7/978: chown d0/d28/d66/d67/ccf 2 1 2024-04-26T23:01:40.984 INFO:tasks.workunit.client.0.smithi027.stdout:7/979: write d0/d28/f37 [4299526,97220] 0 2024-04-26T23:01:40.984 INFO:tasks.workunit.client.0.smithi027.stdout:5/974: mkdir d4/d9f/d105/d13a 0 2024-04-26T23:01:40.985 INFO:tasks.workunit.client.0.smithi027.stdout:6/812: mkdir d1/df/d43/d56/d101 0 2024-04-26T23:01:40.985 INFO:tasks.workunit.client.0.smithi027.stdout:9/879: dwrite d4/d10/d40/d58/f96 [0,4194304] 0 2024-04-26T23:01:40.986 INFO:tasks.workunit.client.0.smithi027.stdout:7/980: creat d0/d28/d66/da3/d124/f13c x:0 0 0 2024-04-26T23:01:40.989 INFO:tasks.workunit.client.0.smithi027.stdout:5/975: mknod d4/d24/d22/d29/d2a/dee/d124/c13b 0 2024-04-26T23:01:40.990 INFO:tasks.workunit.client.0.smithi027.stdout:9/880: symlink d4/d10/d40/d58/d75/d117/l132 0 2024-04-26T23:01:40.990 INFO:tasks.workunit.client.0.smithi027.stdout:7/981: rmdir d0/d28/d66/d67/d11d 39 2024-04-26T23:01:40.992 INFO:tasks.workunit.client.0.smithi027.stdout:9/881: symlink d4/d17/d2e/d46/da0/d8d/df2/l133 0 2024-04-26T23:01:40.992 INFO:tasks.workunit.client.0.smithi027.stdout:9/882: truncate d4/d77/da2/daf/f125 655572 0 2024-04-26T23:01:40.993 INFO:tasks.workunit.client.0.smithi027.stdout:7/982: getdents d0/d28/d2b/d55/d69/da7 0 2024-04-26T23:01:41.000 INFO:tasks.workunit.client.0.smithi027.stdout:5/976: read d4/d24/f98 [1378970,83451] 0 2024-04-26T23:01:41.000 INFO:tasks.workunit.client.0.smithi027.stdout:5/977: chown d4/d1a/l109 0 1 2024-04-26T23:01:41.000 INFO:tasks.workunit.client.0.smithi027.stdout:7/983: symlink d0/d28/d2b/d55/d59/d6d/l13d 0 2024-04-26T23:01:41.000 INFO:tasks.workunit.client.0.smithi027.stdout:9/883: mkdir d4/d17/d2e/d134 0 2024-04-26T23:01:41.000 INFO:tasks.workunit.client.0.smithi027.stdout:9/884: write d4/d17/d2e/d10e/f111 [760067,75560] 0 2024-04-26T23:01:41.000 INFO:tasks.workunit.client.0.smithi027.stdout:7/984: mkdir d0/d28/d2b/d3a/d107/d13e 0 2024-04-26T23:01:41.001 INFO:tasks.workunit.client.0.smithi027.stdout:9/885: symlink d4/d10/d40/d58/d75/d117/l135 0 2024-04-26T23:01:41.002 INFO:tasks.workunit.client.0.smithi027.stdout:7/985: mknod d0/d28/d2b/d55/d59/d6d/c13f 0 2024-04-26T23:01:41.003 INFO:tasks.workunit.client.0.smithi027.stdout:9/886: rename d4/d10/d40/d58/d75/d11a/l124 to d4/d17/d2e/dcf/l136 0 2024-04-26T23:01:41.004 INFO:tasks.workunit.client.0.smithi027.stdout:9/887: mkdir d4/d17/d2e/d137 0 2024-04-26T23:01:41.006 INFO:tasks.workunit.client.0.smithi027.stdout:9/888: creat d4/f138 x:0 0 0 2024-04-26T23:01:41.006 INFO:tasks.workunit.client.0.smithi027.stdout:9/889: stat d4/d17/d2e/dcf 0 2024-04-26T23:01:41.007 INFO:tasks.workunit.client.0.smithi027.stdout:9/890: mkdir d4/d10/d40/d58/d75/d84/d86/dc5/d139 0 2024-04-26T23:01:41.008 INFO:tasks.workunit.client.0.smithi027.stdout:9/891: unlink d4/d17/d2e/d46/c69 0 2024-04-26T23:01:41.009 INFO:tasks.workunit.client.0.smithi027.stdout:9/892: getdents d4/d17/d2e/d46/da0/d8d/df2/d114/d12b 0 2024-04-26T23:01:41.009 INFO:tasks.workunit.client.0.smithi027.stdout:9/893: fdatasync d4/d17/d2e/d46/da0/d72/ffb 0 2024-04-26T23:01:41.010 INFO:tasks.workunit.client.0.smithi027.stdout:7/986: dread d0/d28/d2b/d3a/d90/dcb/fe9 [0,4194304] 0 2024-04-26T23:01:41.010 INFO:tasks.workunit.client.0.smithi027.stdout:7/987: chown d0/d28/d2b/d3a/d90/dcb/d111/f132 14729 1 2024-04-26T23:01:41.010 INFO:tasks.workunit.client.0.smithi027.stdout:7/988: truncate d0/d28/d2b/d55/d59/d6d/fff 1259598 0 2024-04-26T23:01:41.012 INFO:tasks.workunit.client.0.smithi027.stdout:7/989: link d0/d28/d2b/d55/d59/d6d/c13f d0/d28/d2b/d3a/d90/dcb/d111/c140 0 2024-04-26T23:01:41.013 INFO:tasks.workunit.client.0.smithi027.stdout:7/990: creat d0/d28/d2b/d55/f141 x:0 0 0 2024-04-26T23:01:41.013 INFO:tasks.workunit.client.0.smithi027.stdout:7/991: write d0/d28/d2b/d55/f141 [738345,96985] 0 2024-04-26T23:01:41.014 INFO:tasks.workunit.client.0.smithi027.stdout:7/992: mknod d0/d28/d2b/d3a/d48/d56/d88/c142 0 2024-04-26T23:01:41.025 INFO:tasks.workunit.client.0.smithi027.stdout:9/894: dread d4/d10/f2f [0,4194304] 0 2024-04-26T23:01:41.025 INFO:tasks.workunit.client.0.smithi027.stdout:9/895: stat d4/d10/d40/d58/d75/d84/dca 0 2024-04-26T23:01:41.027 INFO:tasks.workunit.client.0.smithi027.stdout:9/896: rmdir d4/d77/da2/daf 39 2024-04-26T23:01:41.028 INFO:tasks.workunit.client.0.smithi027.stdout:7/993: write d0/d28/d2b/d55/f7e [1534730,55735] 0 2024-04-26T23:01:41.030 INFO:tasks.workunit.client.0.smithi027.stdout:9/897: rename d4/d10/d40/d58/ce6 to d4/d17/d2e/d46/da0/d8d/df2/c13a 0 2024-04-26T23:01:41.030 INFO:tasks.workunit.client.0.smithi027.stdout:9/898: write d4/d17/d2e/d46/fd3 [5163971,41659] 0 2024-04-26T23:01:41.030 INFO:tasks.workunit.client.0.smithi027.stdout:9/899: fdatasync d4/d10/d40/d58/d75/f63 0 2024-04-26T23:01:41.031 INFO:tasks.workunit.client.0.smithi027.stdout:9/900: chown d4/d17/d2e/d46/da0/d8d/fb5 1 1 2024-04-26T23:01:41.032 INFO:tasks.workunit.client.0.smithi027.stdout:5/978: dwrite d4/d24/d22/dab/d16/d4b/d81/fe3 [0,4194304] 0 2024-04-26T23:01:41.032 INFO:tasks.workunit.client.0.smithi027.stdout:9/901: creat d4/d10/d40/d58/d75/d84/dc4/ded/f13b x:0 0 0 2024-04-26T23:01:41.035 INFO:tasks.workunit.client.0.smithi027.stdout:9/902: symlink d4/d10/d40/d58/d75/d84/l13c 0 2024-04-26T23:01:41.036 INFO:tasks.workunit.client.0.smithi027.stdout:5/979: mknod d4/c13c 0 2024-04-26T23:01:41.036 INFO:tasks.workunit.client.0.smithi027.stdout:9/903: creat d4/d10/d40/d58/d75/d84/dca/f13d x:0 0 0 2024-04-26T23:01:41.038 INFO:tasks.workunit.client.0.smithi027.stdout:6/813: dwrite d1/df/f39 [0,4194304] 0 2024-04-26T23:01:41.038 INFO:tasks.workunit.client.0.smithi027.stdout:6/814: dread - d1/df/d43/da8/f92 zero size 2024-04-26T23:01:41.038 INFO:tasks.workunit.client.0.smithi027.stdout:6/815: write d1/de/df3/db3/ff9 [1888068,7849] 0 2024-04-26T23:01:41.039 INFO:tasks.workunit.client.0.smithi027.stdout:5/980: getdents d4/d24/d22/d29/d2a/dee/d124 0 2024-04-26T23:01:41.039 INFO:tasks.workunit.client.0.smithi027.stdout:9/904: creat d4/d10/d67/f13e x:0 0 0 2024-04-26T23:01:41.040 INFO:tasks.workunit.client.0.smithi027.stdout:9/905: truncate d4/d10/d67/f13e 535703 0 2024-04-26T23:01:41.043 INFO:tasks.workunit.client.0.smithi027.stdout:7/994: dread d0/d2f/f84 [0,4194304] 0 2024-04-26T23:01:41.043 INFO:tasks.workunit.client.0.smithi027.stdout:7/995: chown d0/d28/d2b/d3a/f52 42168637 1 2024-04-26T23:01:41.043 INFO:tasks.workunit.client.0.smithi027.stdout:7/996: truncate d0/d28/d2b/d3a/ffb 915243 0 2024-04-26T23:01:41.055 INFO:tasks.workunit.client.0.smithi027.stdout:6/816: mkdir d1/de/d57/d8e/d102 0 2024-04-26T23:01:41.056 INFO:tasks.workunit.client.0.smithi027.stdout:5/981: unlink d4/d24/c41 0 2024-04-26T23:01:41.058 INFO:tasks.workunit.client.0.smithi027.stdout:6/817: link d1/de/fe1 d1/de/d57/d8e/dd0/f103 0 2024-04-26T23:01:41.058 INFO:tasks.workunit.client.0.smithi027.stdout:9/906: dread d4/d10/d40/d58/d75/d84/d86/fa5 [0,4194304] 0 2024-04-26T23:01:41.059 INFO:tasks.workunit.client.0.smithi027.stdout:6/818: link d1/df/d43/d56/d6b/f8b d1/de/df3/d6f/d7d/f104 0 2024-04-26T23:01:41.061 INFO:tasks.workunit.client.0.smithi027.stdout:7/997: write d0/d28/d2b/d55/d59/f6b [1230711,82106] 0 2024-04-26T23:01:41.061 INFO:tasks.workunit.client.0.smithi027.stdout:7/998: fdatasync d0/d28/d2b/d3a/d48/d56/d88/d73/f104 0 2024-04-26T23:01:41.062 INFO:tasks.workunit.client.0.smithi027.stdout:6/819: creat d1/de/df3/d6f/dd3/ddf/f105 x:0 0 0 2024-04-26T23:01:41.062 INFO:tasks.workunit.client.0.smithi027.stdout:6/820: dread - d1/d6a/fe9 zero size 2024-04-26T23:01:41.064 INFO:tasks.workunit.client.0.smithi027.stdout:3/967: sync 2024-04-26T23:01:41.064 INFO:tasks.workunit.client.0.smithi027.stdout:4/944: sync 2024-04-26T23:01:41.064 INFO:tasks.workunit.client.0.smithi027.stdout:0/909: sync 2024-04-26T23:01:41.065 INFO:tasks.workunit.client.0.smithi027.stdout:2/900: sync 2024-04-26T23:01:41.065 INFO:tasks.workunit.client.0.smithi027.stdout:8/919: sync 2024-04-26T23:01:41.065 INFO:tasks.workunit.client.0.smithi027.stdout:7/999: truncate d0/d28/d2b/d3a/f52 1496676 0 2024-04-26T23:01:41.068 INFO:tasks.workunit.client.0.smithi027.stdout:4/945: symlink da/dd/de/d14/d21/d41/d128/l12c 0 2024-04-26T23:01:41.069 INFO:tasks.workunit.client.0.smithi027.stdout:0/910: creat d0/df/d3f/d64/da8/f118 x:0 0 0 2024-04-26T23:01:41.070 INFO:tasks.workunit.client.0.smithi027.stdout:4/946: rename da/dd/d5e/ca3 to da/dd/de/d79/c12d 0 2024-04-26T23:01:41.075 INFO:tasks.workunit.client.0.smithi027.stdout:3/968: write d6/dd3/d34/d2e/d5f/d90/ddb/d98/fc7 [126214,100805] 0 2024-04-26T23:01:41.075 INFO:tasks.workunit.client.0.smithi027.stdout:4/947: mknod da/dd/de/d14/d21/d4a/c12e 0 2024-04-26T23:01:41.075 INFO:tasks.workunit.client.0.smithi027.stdout:4/948: truncate da/dd/f101 55479 0 2024-04-26T23:01:41.075 INFO:tasks.workunit.client.0.smithi027.stdout:4/949: fsync da/d40/ff6 0 2024-04-26T23:01:41.075 INFO:tasks.workunit.client.0.smithi027.stdout:2/901: write d2/f49 [567456,54787] 0 2024-04-26T23:01:41.075 INFO:tasks.workunit.client.0.smithi027.stdout:2/902: write d2/d5/db/d1b/d1c/d25/d19/d23/d82/fd5 [599503,58891] 0 2024-04-26T23:01:41.075 INFO:tasks.workunit.client.0.smithi027.stdout:2/903: truncate d2/d5/fd0 810458 0 2024-04-26T23:01:41.075 INFO:tasks.workunit.client.0.smithi027.stdout:4/950: creat da/dd/de/d14/db3/d114/d11d/f12f x:0 0 0 2024-04-26T23:01:41.078 INFO:tasks.workunit.client.0.smithi027.stdout:2/904: creat d2/d5/db/de/d21/dc4/d39/dd2/f120 x:0 0 0 2024-04-26T23:01:41.078 INFO:tasks.workunit.client.0.smithi027.stdout:2/905: dread - d2/d5/db/d1b/d1c/f107 zero size 2024-04-26T23:01:41.078 INFO:tasks.workunit.client.0.smithi027.stdout:2/906: dread - d2/d5/db/d1b/d1c/d25/d19/d23/d7c/fb3 zero size 2024-04-26T23:01:41.080 INFO:tasks.workunit.client.0.smithi027.stdout:2/907: mkdir d2/d5/db/d1b/d1c/d119/de7/d121 0 2024-04-26T23:01:41.082 INFO:tasks.workunit.client.0.smithi027.stdout:2/908: creat d2/d5/db/d1b/d1c/d25/d19/d44/dee/d118/f122 x:0 0 0 2024-04-26T23:01:41.082 INFO:tasks.workunit.client.0.smithi027.stdout:2/909: write d2/d5/db/d1b/d1c/d25/d19/d44/dee/d118/f122 [415366,13118] 0 2024-04-26T23:01:41.082 INFO:tasks.workunit.client.0.smithi027.stdout:2/910: write d2/d5/db/de/f42 [4231163,60008] 0 2024-04-26T23:01:41.082 INFO:tasks.workunit.client.0.smithi027.stdout:2/911: chown d2/d5/fa 151753714 1 2024-04-26T23:01:41.084 INFO:tasks.workunit.client.0.smithi027.stdout:2/912: mkdir d2/d5/db/de/d21/d123 0 2024-04-26T23:01:41.094 INFO:tasks.workunit.client.0.smithi027.stdout:0/911: write d0/df/d3f/d4a/f4d [6539440,13145] 0 2024-04-26T23:01:41.095 INFO:tasks.workunit.client.0.smithi027.stdout:0/912: creat d0/d4/da/d15/ddc/f119 x:0 0 0 2024-04-26T23:01:41.116 INFO:tasks.workunit.client.0.smithi027.stdout:3/969: dwrite d6/dd3/d34/d2e/d50/fb2 [4194304,4194304] 0 2024-04-26T23:01:41.118 INFO:tasks.workunit.client.0.smithi027.stdout:3/970: mkdir d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d59/d108/d145 0 2024-04-26T23:01:41.125 INFO:tasks.workunit.client.0.smithi027.stdout:4/951: dwrite da/dd/de/d14/d21/d67/d8a/f8c [4194304,4194304] 0 2024-04-26T23:01:41.128 INFO:tasks.workunit.client.0.smithi027.stdout:4/952: creat da/dd/de/d14/d17/d80/dab/dd6/f130 x:0 0 0 2024-04-26T23:01:41.128 INFO:tasks.workunit.client.0.smithi027.stdout:0/913: dread d0/d4/da/d1a/d1e/f84 [0,4194304] 0 2024-04-26T23:01:41.129 INFO:tasks.workunit.client.0.smithi027.stdout:0/914: fdatasync d0/df/d3f/d4a/f6c 0 2024-04-26T23:01:41.129 INFO:tasks.workunit.client.0.smithi027.stdout:5/982: dwrite d4/d1a/d2b/db8/fd5 [0,4194304] 0 2024-04-26T23:01:41.129 INFO:tasks.workunit.client.0.smithi027.stdout:4/953: mknod da/dd/d5f/dac/c131 0 2024-04-26T23:01:41.129 INFO:tasks.workunit.client.0.smithi027.stdout:0/915: fsync d0/d4/da/d15/f48 0 2024-04-26T23:01:41.129 INFO:tasks.workunit.client.0.smithi027.stdout:0/916: dread - d0/d4/da/d1a/d2a/f104 zero size 2024-04-26T23:01:41.129 INFO:tasks.workunit.client.0.smithi027.stdout:2/913: dwrite d2/d5/db/d1b/d1c/d25/d19/f5b [0,4194304] 0 2024-04-26T23:01:41.131 INFO:tasks.workunit.client.0.smithi027.stdout:5/983: mkdir d4/d24/d13d 0 2024-04-26T23:01:41.131 INFO:tasks.workunit.client.0.smithi027.stdout:4/954: mkdir da/dd/de/d14/d21/d41/d132 0 2024-04-26T23:01:41.132 INFO:tasks.workunit.client.0.smithi027.stdout:4/955: fdatasync da/dd/de/d14/d17/d80/dab/f3a 0 2024-04-26T23:01:41.132 INFO:tasks.workunit.client.0.smithi027.stdout:0/917: mknod d0/df/d3f/d64/c11a 0 2024-04-26T23:01:41.133 INFO:tasks.workunit.client.0.smithi027.stdout:4/956: rename da/dd/c73 to da/dd/de/d14/d21/d67/c133 0 2024-04-26T23:01:41.134 INFO:tasks.workunit.client.0.smithi027.stdout:0/918: creat d0/df/d3f/f11b x:0 0 0 2024-04-26T23:01:41.137 INFO:tasks.workunit.client.0.smithi027.stdout:4/957: symlink da/dd/de/d14/d21/d67/d8a/l134 0 2024-04-26T23:01:41.137 INFO:tasks.workunit.client.0.smithi027.stdout:4/958: write da/dd/de/d14/d17/f28 [4716658,46991] 0 2024-04-26T23:01:41.137 INFO:tasks.workunit.client.0.smithi027.stdout:4/959: dread - da/dd/d5e/fe5 zero size 2024-04-26T23:01:41.138 INFO:tasks.workunit.client.0.smithi027.stdout:0/919: creat d0/df/d9d/de5/f11c x:0 0 0 2024-04-26T23:01:41.138 INFO:tasks.workunit.client.0.smithi027.stdout:0/920: truncate d0/df/d9d/d9f/dbb/fef 599250 0 2024-04-26T23:01:41.138 INFO:tasks.workunit.client.0.smithi027.stdout:4/960: creat da/dd/de/d14/d21/f135 x:0 0 0 2024-04-26T23:01:41.138 INFO:tasks.workunit.client.0.smithi027.stdout:4/961: chown da/dd/de/d79/f85 12554 1 2024-04-26T23:01:41.139 INFO:tasks.workunit.client.0.smithi027.stdout:0/921: link d0/df/c73 d0/df/d3f/d4a/de9/c11d 0 2024-04-26T23:01:41.140 INFO:tasks.workunit.client.0.smithi027.stdout:0/922: creat d0/d4/da/d1a/d59/d92/f11e x:0 0 0 2024-04-26T23:01:41.140 INFO:tasks.workunit.client.0.smithi027.stdout:0/923: chown d0/fc2 24 1 2024-04-26T23:01:41.141 INFO:tasks.workunit.client.0.smithi027.stdout:0/924: symlink d0/d4/da/d1a/l11f 0 2024-04-26T23:01:41.142 INFO:tasks.workunit.client.0.smithi027.stdout:0/925: unlink d0/d4/da/d15/l29 0 2024-04-26T23:01:41.147 INFO:tasks.workunit.client.0.smithi027.stdout:2/914: dread d2/d5/db/d1b/f38 [4194304,4194304] 0 2024-04-26T23:01:41.170 INFO:tasks.workunit.client.0.smithi027.stdout:4/962: dwrite da/dd/d5f/dac/f96 [0,4194304] 0 2024-04-26T23:01:41.171 INFO:tasks.workunit.client.0.smithi027.stdout:4/963: unlink da/dd/de/d14/d21/d67/da6/l108 0 2024-04-26T23:01:41.181 INFO:tasks.workunit.client.0.smithi027.stdout:5/984: dwrite d4/d24/f5d [0,4194304] 0 2024-04-26T23:01:41.183 INFO:tasks.workunit.client.0.smithi027.stdout:2/915: dwrite d2/d5/db/ff5 [0,4194304] 0 2024-04-26T23:01:41.183 INFO:tasks.workunit.client.0.smithi027.stdout:2/916: chown d2/d5/db/de/d21/d8f/f95 28 1 2024-04-26T23:01:41.187 INFO:tasks.workunit.client.0.smithi027.stdout:4/964: rename da/dd/de/d79 to da/dd/d136 0 2024-04-26T23:01:41.189 INFO:tasks.workunit.client.0.smithi027.stdout:2/917: creat d2/d5/db/de/d21/d8f/f124 x:0 0 0 2024-04-26T23:01:41.190 INFO:tasks.workunit.client.0.smithi027.stdout:4/965: symlink da/dd/de/d14/d17/d80/dab/dd6/l137 0 2024-04-26T23:01:41.190 INFO:tasks.workunit.client.0.smithi027.stdout:4/966: dread - da/dd/de/d14/d21/d41/d59/f77 zero size 2024-04-26T23:01:41.199 INFO:tasks.workunit.client.0.smithi027.stdout:5/985: write d4/d24/d22/d29/f36 [879719,48159] 0 2024-04-26T23:01:41.230 INFO:tasks.workunit.client.0.smithi027.stdout:4/967: dwrite da/dd/de/d14/db3/d114/d11d/f12f [0,4194304] 0 2024-04-26T23:01:41.230 INFO:tasks.workunit.client.0.smithi027.stdout:4/968: truncate da/dd/d5f/f7a 650787 0 2024-04-26T23:01:41.230 INFO:tasks.workunit.client.0.smithi027.stdout:4/969: dread - da/dd/de/d14/d21/dad/ffb zero size 2024-04-26T23:01:41.231 INFO:tasks.workunit.client.0.smithi027.stdout:4/970: fdatasync da/dd/de/d14/d21/d67/da6/f109 0 2024-04-26T23:01:41.231 INFO:tasks.workunit.client.0.smithi027.stdout:2/918: dwrite d2/d5/db/de/d21/d8f/f95 [0,4194304] 0 2024-04-26T23:01:41.232 INFO:tasks.workunit.client.0.smithi027.stdout:4/971: creat da/dd/d136/dcd/f138 x:0 0 0 2024-04-26T23:01:41.235 INFO:tasks.workunit.client.0.smithi027.stdout:5/986: dwrite d4/d24/d22/d29/d2a/d3b/d50/fbb [0,4194304] 0 2024-04-26T23:01:41.235 INFO:tasks.workunit.client.0.smithi027.stdout:5/987: readlink d4/d1a/dcb/dd1/dd9/ldb 0 2024-04-26T23:01:41.250 INFO:tasks.workunit.client.0.smithi027.stdout:5/988: write d4/d1a/d2b/db8/fd5 [953394,115897] 0 2024-04-26T23:01:41.251 INFO:tasks.workunit.client.0.smithi027.stdout:5/989: creat d4/d24/d5f/dac/d40/f13e x:0 0 0 2024-04-26T23:01:41.253 INFO:tasks.workunit.client.0.smithi027.stdout:5/990: symlink d4/d24/d22/d29/d39/dc7/l13f 0 2024-04-26T23:01:41.263 INFO:tasks.workunit.client.0.smithi027.stdout:2/919: dread d2/d5/db/d1b/d1c/d119/fe9 [0,4194304] 0 2024-04-26T23:01:41.263 INFO:tasks.workunit.client.0.smithi027.stdout:2/920: write d2/d5/db/d1b/d1c/d25/d19/f106 [177160,114316] 0 2024-04-26T23:01:41.268 INFO:tasks.workunit.client.0.smithi027.stdout:2/921: write d2/d5/db/d1b/d1c/d25/d19/d44/fc5 [143780,61285] 0 2024-04-26T23:01:41.268 INFO:tasks.workunit.client.0.smithi027.stdout:2/922: fdatasync d2/d5/f34 0 2024-04-26T23:01:41.269 INFO:tasks.workunit.client.0.smithi027.stdout:2/923: symlink d2/d5/db/de/d21/d8f/deb/l125 0 2024-04-26T23:01:41.270 INFO:tasks.workunit.client.0.smithi027.stdout:2/924: symlink d2/d5/db/de/d21/dc4/d39/dd2/l126 0 2024-04-26T23:01:41.272 INFO:tasks.workunit.client.0.smithi027.stdout:2/925: mkdir d2/d5/db/d1b/d1c/d25/dc3/d127 0 2024-04-26T23:01:41.272 INFO:tasks.workunit.client.0.smithi027.stdout:2/926: dread - d2/d5/db/d1b/d1c/d119/f109 zero size 2024-04-26T23:01:41.283 INFO:tasks.workunit.client.0.smithi027.stdout:2/927: dread d2/d5/f6e [0,4194304] 0 2024-04-26T23:01:41.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:41 smithi094 ceph-mon[32889]: pgmap v95: 129 pgs: 129 active+clean; 1.5 GiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 133 MiB/s rd, 148 MiB/s wr, 528 op/s 2024-04-26T23:01:41.285 INFO:tasks.workunit.client.0.smithi027.stdout:4/972: dwrite da/dd/de/d14/d21/d41/df7/ff9 [0,4194304] 0 2024-04-26T23:01:41.287 INFO:tasks.workunit.client.0.smithi027.stdout:4/973: mknod da/dd/de/d14/d17/d80/dab/d37/c139 0 2024-04-26T23:01:41.299 INFO:tasks.workunit.client.0.smithi027.stdout:5/991: dwrite d4/d24/d22/d29/d39/f7f [4194304,4194304] 0 2024-04-26T23:01:41.305 INFO:tasks.workunit.client.0.smithi027.stdout:2/928: write d2/d5/db/d1b/d1c/d25/d19/d23/f64 [4317014,70107] 0 2024-04-26T23:01:41.307 INFO:tasks.workunit.client.0.smithi027.stdout:2/929: symlink d2/d5/db/d1b/d1c/d25/dc0/l128 0 2024-04-26T23:01:41.307 INFO:tasks.workunit.client.0.smithi027.stdout:2/930: chown d2/d5/db/de/d21/d8f/deb/l125 2 1 2024-04-26T23:01:41.314 INFO:tasks.workunit.client.0.smithi027.stdout:5/992: write d4/d24/d22/d29/f33 [3465670,103617] 0 2024-04-26T23:01:41.315 INFO:tasks.workunit.client.0.smithi027.stdout:5/993: readlink d4/d24/d22/d29/d39/dc7/d104/l108 0 2024-04-26T23:01:41.317 INFO:tasks.workunit.client.0.smithi027.stdout:5/994: symlink d4/d1a/d2b/db8/d137/l140 0 2024-04-26T23:01:41.318 INFO:tasks.workunit.client.0.smithi027.stdout:5/995: mkdir d4/d1a/d2b/d141 0 2024-04-26T23:01:41.319 INFO:tasks.workunit.client.0.smithi027.stdout:5/996: dread - d4/d9f/dc0/ff9 zero size 2024-04-26T23:01:41.325 INFO:tasks.workunit.client.0.smithi027.stdout:5/997: dread d4/d1a/dcb/ff1 [0,4194304] 0 2024-04-26T23:01:41.326 INFO:tasks.workunit.client.0.smithi027.stdout:5/998: read d4/d24/d22/dab/d16/d4b/fef [4218,73396] 0 2024-04-26T23:01:41.329 INFO:tasks.workunit.client.0.smithi027.stdout:5/999: creat d4/d24/d22/f142 x:0 0 0 2024-04-26T23:01:41.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:41 smithi027 ceph-mon[28034]: pgmap v95: 129 pgs: 129 active+clean; 1.5 GiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 133 MiB/s rd, 148 MiB/s wr, 528 op/s 2024-04-26T23:01:41.381 INFO:tasks.workunit.client.0.smithi027.stdout:6/821: sync 2024-04-26T23:01:41.382 INFO:tasks.workunit.client.0.smithi027.stdout:6/822: stat d1/de/df3/lb9 0 2024-04-26T23:01:41.382 INFO:tasks.workunit.client.0.smithi027.stdout:8/920: sync 2024-04-26T23:01:41.382 INFO:tasks.workunit.client.0.smithi027.stdout:9/907: sync 2024-04-26T23:01:41.384 INFO:tasks.workunit.client.0.smithi027.stdout:6/823: unlink d1/df/d43/da8/d84/ff5 0 2024-04-26T23:01:41.385 INFO:tasks.workunit.client.0.smithi027.stdout:6/824: mknod d1/de/d57/d8e/dd0/c106 0 2024-04-26T23:01:41.386 INFO:tasks.workunit.client.0.smithi027.stdout:6/825: creat d1/de/f107 x:0 0 0 2024-04-26T23:01:41.389 INFO:tasks.workunit.client.0.smithi027.stdout:8/921: unlink d9/de/d2e/d110/da7/ld8 0 2024-04-26T23:01:41.392 INFO:tasks.workunit.client.0.smithi027.stdout:8/922: getdents d9/da/d2a/d5e/d46/ddd 0 2024-04-26T23:01:41.392 INFO:tasks.workunit.client.0.smithi027.stdout:8/923: stat d9/de/d2e/d110/df0/d11f 0 2024-04-26T23:01:41.392 INFO:tasks.workunit.client.0.smithi027.stdout:8/924: read d9/da/d2a/d5e/d2b/d52/f7a [134456,77744] 0 2024-04-26T23:01:41.396 INFO:tasks.workunit.client.0.smithi027.stdout:8/925: rename d9/de/d2e/d110/d8d/da3/dfe/l10e to d9/da/d2a/d6b/l146 0 2024-04-26T23:01:41.397 INFO:tasks.workunit.client.0.smithi027.stdout:8/926: write d9/f17 [5797777,89559] 0 2024-04-26T23:01:41.397 INFO:tasks.workunit.client.0.smithi027.stdout:8/927: chown d9/da/d2a/d5e/d2b/c7b 27044 1 2024-04-26T23:01:41.397 INFO:tasks.workunit.client.0.smithi027.stdout:8/928: chown d9/de 483 1 2024-04-26T23:01:41.433 INFO:tasks.workunit.client.0.smithi027.stdout:6/826: dwrite d1/de/df3/d6f/f89 [0,4194304] 0 2024-04-26T23:01:41.433 INFO:tasks.workunit.client.0.smithi027.stdout:6/827: write d1/de/f107 [633294,24547] 0 2024-04-26T23:01:41.433 INFO:tasks.workunit.client.0.smithi027.stdout:6/828: truncate d1/df/d43/da8/f9f 1226027 0 2024-04-26T23:01:41.436 INFO:tasks.workunit.client.0.smithi027.stdout:6/829: creat d1/de/d57/d8e/dd0/f108 x:0 0 0 2024-04-26T23:01:41.444 INFO:tasks.workunit.client.0.smithi027.stdout:9/908: dwrite d4/d10/d40/d58/f96 [0,4194304] 0 2024-04-26T23:01:41.455 INFO:tasks.workunit.client.0.smithi027.stdout:6/830: dread d1/fc6 [0,4194304] 0 2024-04-26T23:01:41.460 INFO:tasks.workunit.client.0.smithi027.stdout:6/831: getdents d1/de/d57/d8e/dd0 0 2024-04-26T23:01:41.462 INFO:tasks.workunit.client.0.smithi027.stdout:6/832: symlink d1/de/df3/db3/l109 0 2024-04-26T23:01:41.462 INFO:tasks.workunit.client.0.smithi027.stdout:6/833: stat d1/de/df3/d6f/d7d/fe2 0 2024-04-26T23:01:41.462 INFO:tasks.workunit.client.0.smithi027.stdout:6/834: mknod d1/d30/d40/c10a 0 2024-04-26T23:01:41.477 INFO:tasks.workunit.client.0.smithi027.stdout:8/929: dwrite d9/da/d33/f105 [0,4194304] 0 2024-04-26T23:01:41.480 INFO:tasks.workunit.client.0.smithi027.stdout:8/930: read d9/da/d2a/d5e/d72/d84/db8/ff3 [166271,118336] 0 2024-04-26T23:01:41.481 INFO:tasks.workunit.client.0.smithi027.stdout:8/931: dread - d9/da/da1/f101 zero size 2024-04-26T23:01:41.482 INFO:tasks.workunit.client.0.smithi027.stdout:8/932: mkdir d9/de/d2e/d147 0 2024-04-26T23:01:41.483 INFO:tasks.workunit.client.0.smithi027.stdout:3/971: sync 2024-04-26T23:01:41.483 INFO:tasks.workunit.client.0.smithi027.stdout:0/926: sync 2024-04-26T23:01:41.484 INFO:tasks.workunit.client.0.smithi027.stdout:0/927: write d0/d4/da/d1a/d1e/f91 [4934325,29263] 0 2024-04-26T23:01:41.485 INFO:tasks.workunit.client.0.smithi027.stdout:0/928: creat d0/df/d3f/f120 x:0 0 0 2024-04-26T23:01:41.487 INFO:tasks.workunit.client.0.smithi027.stdout:3/972: link d6/dd3/d34/d2e/d9d/da7/dac/l134 d6/dd3/l146 0 2024-04-26T23:01:41.489 INFO:tasks.workunit.client.0.smithi027.stdout:0/929: rename d0/d4/da/d1a/d2a/lb7 to d0/d4/da/d15/d55/l121 0 2024-04-26T23:01:41.489 INFO:tasks.workunit.client.0.smithi027.stdout:0/930: dread - d0/d4/da/d15/ddc/f119 zero size 2024-04-26T23:01:41.504 INFO:tasks.workunit.client.0.smithi027.stdout:8/933: dread d9/da/d2a/d6b/fab [0,4194304] 0 2024-04-26T23:01:41.514 INFO:tasks.workunit.client.0.smithi027.stdout:0/931: dread d0/d4/f20 [0,4194304] 0 2024-04-26T23:01:41.515 INFO:tasks.workunit.client.0.smithi027.stdout:0/932: truncate d0/f37 5973729 0 2024-04-26T23:01:41.516 INFO:tasks.workunit.client.0.smithi027.stdout:0/933: truncate d0/d4/da/d15/f8e 1015230 0 2024-04-26T23:01:41.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:41 smithi080 ceph-mon[32324]: pgmap v95: 129 pgs: 129 active+clean; 1.5 GiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 133 MiB/s rd, 148 MiB/s wr, 528 op/s 2024-04-26T23:01:41.521 INFO:tasks.workunit.client.0.smithi027.stdout:0/934: unlink d0/d4/da/d1a/d6d/dc9/fce 0 2024-04-26T23:01:41.523 INFO:tasks.workunit.client.0.smithi027.stdout:0/935: link d0/df/d9d/d9f/dbb/f102 d0/df/d3f/d4a/de9/f122 0 2024-04-26T23:01:41.525 INFO:tasks.workunit.client.0.smithi027.stdout:0/936: getdents d0/d4/da/d1a 0 2024-04-26T23:01:41.526 INFO:tasks.workunit.client.0.smithi027.stdout:0/937: symlink d0/l123 0 2024-04-26T23:01:41.537 INFO:tasks.workunit.client.0.smithi027.stdout:3/973: dwrite d6/dd3/d34/d47/ffd [0,4194304] 0 2024-04-26T23:01:41.537 INFO:tasks.workunit.client.0.smithi027.stdout:3/974: chown d6/dd3/d34/d2e/d50/f8a 15955485 1 2024-04-26T23:01:41.537 INFO:tasks.workunit.client.0.smithi027.stdout:3/975: chown d6/d5e/c10a 167688 1 2024-04-26T23:01:41.567 INFO:tasks.workunit.client.0.smithi027.stdout:8/934: mknod d9/da/d2a/d5e/c148 0 2024-04-26T23:01:41.572 INFO:tasks.workunit.client.0.smithi027.stdout:3/976: read d6/d5e/fa0 [4007109,91551] 0 2024-04-26T23:01:41.573 INFO:tasks.workunit.client.0.smithi027.stdout:3/977: symlink d6/dd3/d34/d2e/d9d/da7/dac/dc0/l147 0 2024-04-26T23:01:41.575 INFO:tasks.workunit.client.0.smithi027.stdout:3/978: mknod d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d11d/c148 0 2024-04-26T23:01:41.575 INFO:tasks.workunit.client.0.smithi027.stdout:3/979: truncate d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/f131 795391 0 2024-04-26T23:01:41.578 INFO:tasks.workunit.client.0.smithi027.stdout:3/980: creat d6/dd3/d34/d2e/d70/f149 x:0 0 0 2024-04-26T23:01:41.579 INFO:tasks.workunit.client.0.smithi027.stdout:3/981: getdents d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52 0 2024-04-26T23:01:41.585 INFO:tasks.workunit.client.0.smithi027.stdout:3/982: getdents d6/dd3/d34/d2e/d70 0 2024-04-26T23:01:41.599 INFO:tasks.workunit.client.0.smithi027.stdout:3/983: dread d6/dd3/d34/db3/fc1 [0,4194304] 0 2024-04-26T23:01:41.599 INFO:tasks.workunit.client.0.smithi027.stdout:3/984: fdatasync d6/dd3/d34/d47/ffd 0 2024-04-26T23:01:41.630 INFO:tasks.workunit.client.0.smithi027.stdout:3/985: dread d6/f2d [0,4194304] 0 2024-04-26T23:01:41.631 INFO:tasks.workunit.client.0.smithi027.stdout:3/986: write d6/dd3/f25 [3956189,19092] 0 2024-04-26T23:01:41.635 INFO:tasks.workunit.client.0.smithi027.stdout:3/987: rename d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d40/f11b to d6/dd3/d34/d2e/d5f/d90/d136/f14a 0 2024-04-26T23:01:41.637 INFO:tasks.workunit.client.0.smithi027.stdout:3/988: rmdir d6/dd3/d34/d2e/d70 39 2024-04-26T23:01:41.637 INFO:tasks.workunit.client.0.smithi027.stdout:3/989: fdatasync d6/f12c 0 2024-04-26T23:01:41.639 INFO:tasks.workunit.client.0.smithi027.stdout:3/990: symlink d6/dd3/d34/d2e/d10b/l14b 0 2024-04-26T23:01:41.639 INFO:tasks.workunit.client.0.smithi027.stdout:3/991: write d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d52/dd6/f119 [1408416,59263] 0 2024-04-26T23:01:41.652 INFO:tasks.workunit.client.0.smithi027.stdout:0/938: dwrite d0/df/d9d/fb5 [0,4194304] 0 2024-04-26T23:01:41.652 INFO:tasks.workunit.client.0.smithi027.stdout:0/939: readlink d0/df/d3f/la4 0 2024-04-26T23:01:41.658 INFO:tasks.workunit.client.0.smithi027.stdout:2/931: sync 2024-04-26T23:01:41.658 INFO:tasks.workunit.client.0.smithi027.stdout:4/974: sync 2024-04-26T23:01:41.658 INFO:tasks.workunit.client.0.smithi027.stdout:4/975: read - da/dd/de/d14/db3/fbd zero size 2024-04-26T23:01:41.659 INFO:tasks.workunit.client.0.smithi027.stdout:4/976: rmdir da/dd/de/d14/d17/d80/dab/dd6 39 2024-04-26T23:01:41.660 INFO:tasks.workunit.client.0.smithi027.stdout:8/935: dwrite d9/da/d2a/f89 [0,4194304] 0 2024-04-26T23:01:41.660 INFO:tasks.workunit.client.0.smithi027.stdout:2/932: mkdir d2/d129 0 2024-04-26T23:01:41.663 INFO:tasks.workunit.client.0.smithi027.stdout:2/933: truncate d2/d5/fa 714930 0 2024-04-26T23:01:41.665 INFO:tasks.workunit.client.0.smithi027.stdout:8/936: link d9/da/d2a/d6b/f81 d9/da/f149 0 2024-04-26T23:01:41.667 INFO:tasks.workunit.client.0.smithi027.stdout:2/934: mkdir d2/d5/db/d1b/d1c/d12a 0 2024-04-26T23:01:41.668 INFO:tasks.workunit.client.0.smithi027.stdout:2/935: creat d2/f12b x:0 0 0 2024-04-26T23:01:41.668 INFO:tasks.workunit.client.0.smithi027.stdout:2/936: dread - d2/d5/db/d1b/d1c/d119/f109 zero size 2024-04-26T23:01:41.670 INFO:tasks.workunit.client.0.smithi027.stdout:2/937: mkdir d2/d5/db/de/d12c 0 2024-04-26T23:01:41.670 INFO:tasks.workunit.client.0.smithi027.stdout:2/938: write d2/fdc [332303,85915] 0 2024-04-26T23:01:41.671 INFO:tasks.workunit.client.0.smithi027.stdout:2/939: mkdir d2/d5/db/d1b/d1c/d25/dc3/d127/d12d 0 2024-04-26T23:01:41.673 INFO:tasks.workunit.client.0.smithi027.stdout:2/940: mknod d2/d5/db/de/d21/c12e 0 2024-04-26T23:01:41.673 INFO:tasks.workunit.client.0.smithi027.stdout:2/941: truncate d2/d5/db/d1b/d1c/d25/d6a/d8a/fad 631765 0 2024-04-26T23:01:41.673 INFO:tasks.workunit.client.0.smithi027.stdout:2/942: stat d2/d5/db/d1b/d1c/d25/d19/db0/lb7 0 2024-04-26T23:01:41.676 INFO:tasks.workunit.client.0.smithi027.stdout:2/943: creat d2/d5/db/d1b/d1c/d25/d19/d23/d7c/f12f x:0 0 0 2024-04-26T23:01:41.676 INFO:tasks.workunit.client.0.smithi027.stdout:2/944: readlink d2/d5/db/d1b/d1c/l6b 0 2024-04-26T23:01:41.676 INFO:tasks.workunit.client.0.smithi027.stdout:2/945: readlink d2/d5/db/de/d21/d8f/l9f 0 2024-04-26T23:01:41.695 INFO:tasks.workunit.client.0.smithi027.stdout:0/940: dwrite d0/d4/da/d1a/d6d/fa7 [0,4194304] 0 2024-04-26T23:01:41.695 INFO:tasks.workunit.client.0.smithi027.stdout:0/941: write d0/df/d3f/d64/f71 [1333802,22692] 0 2024-04-26T23:01:41.697 INFO:tasks.workunit.client.0.smithi027.stdout:0/942: mkdir d0/df/d9d/db4/d124 0 2024-04-26T23:01:41.699 INFO:tasks.workunit.client.0.smithi027.stdout:0/943: link d0/df/c73 d0/d4/da/c125 0 2024-04-26T23:01:41.701 INFO:tasks.workunit.client.0.smithi027.stdout:0/944: mkdir d0/d4/da/d15/d26/dc4/d103/d126 0 2024-04-26T23:01:41.703 INFO:tasks.workunit.client.0.smithi027.stdout:0/945: mknod d0/d4/da/d1a/d59/d92/dcd/c127 0 2024-04-26T23:01:41.704 INFO:tasks.workunit.client.0.smithi027.stdout:0/946: symlink d0/d4/da/d1a/d1e/dba/l128 0 2024-04-26T23:01:41.710 INFO:tasks.workunit.client.0.smithi027.stdout:0/947: link d0/d4/da/d15/f5a d0/df/d9d/db4/d124/f129 0 2024-04-26T23:01:41.721 INFO:tasks.workunit.client.0.smithi027.stdout:8/937: dwrite d9/da/d2a/d5e/d72/d84/d5d/fcb [0,4194304] 0 2024-04-26T23:01:41.721 INFO:tasks.workunit.client.0.smithi027.stdout:8/938: readlink d9/de/d22/l27 0 2024-04-26T23:01:41.724 INFO:tasks.workunit.client.0.smithi027.stdout:8/939: rename d9/da/d119/f11e to d9/de/d2e/d110/d8d/f14a 0 2024-04-26T23:01:41.727 INFO:tasks.workunit.client.0.smithi027.stdout:8/940: rmdir d9/de/d2e/d110 39 2024-04-26T23:01:41.731 INFO:tasks.workunit.client.0.smithi027.stdout:8/941: creat d9/da/d2a/d5e/d46/ddd/f14b x:0 0 0 2024-04-26T23:01:41.737 INFO:tasks.workunit.client.0.smithi027.stdout:2/946: dwrite d2/d5/db/de/d21/fc2 [0,4194304] 0 2024-04-26T23:01:41.750 INFO:tasks.workunit.client.0.smithi027.stdout:4/977: dwrite da/f1f [0,4194304] 0 2024-04-26T23:01:41.751 INFO:tasks.workunit.client.0.smithi027.stdout:4/978: creat da/dd/de/d14/d21/d41/d132/f13a x:0 0 0 2024-04-26T23:01:41.753 INFO:tasks.workunit.client.0.smithi027.stdout:8/942: dread d9/de/d2e/d110/f56 [0,4194304] 0 2024-04-26T23:01:41.755 INFO:tasks.workunit.client.0.smithi027.stdout:8/943: creat d9/da/d2a/d5e/d2b/d52/d115/f14c x:0 0 0 2024-04-26T23:01:41.755 INFO:tasks.workunit.client.0.smithi027.stdout:8/944: write d9/da/d2a/d5e/d2b/f3d [69251,31899] 0 2024-04-26T23:01:41.755 INFO:tasks.workunit.client.0.smithi027.stdout:8/945: write d9/da/f149 [466264,15996] 0 2024-04-26T23:01:41.757 INFO:tasks.workunit.client.0.smithi027.stdout:8/946: mknod d9/da/d2a/d5e/d2b/d52/d115/c14d 0 2024-04-26T23:01:41.767 INFO:tasks.workunit.client.0.smithi027.stdout:0/948: dwrite d0/f12 [0,4194304] 0 2024-04-26T23:01:41.767 INFO:tasks.workunit.client.0.smithi027.stdout:2/947: write d2/f89 [2566800,112901] 0 2024-04-26T23:01:41.768 INFO:tasks.workunit.client.0.smithi027.stdout:0/949: rmdir d0/df/d9d/de5 39 2024-04-26T23:01:41.769 INFO:tasks.workunit.client.0.smithi027.stdout:0/950: mknod d0/df/d9d/de5/c12a 0 2024-04-26T23:01:41.802 INFO:tasks.workunit.client.0.smithi027.stdout:0/951: dwrite d0/d4/faf [0,4194304] 0 2024-04-26T23:01:41.803 INFO:tasks.workunit.client.0.smithi027.stdout:0/952: mkdir d0/df/d9d/db4/deb/d12b 0 2024-04-26T23:01:41.807 INFO:tasks.workunit.client.0.smithi027.stdout:0/953: link d0/df/d3f/f108 d0/df/d3f/f12c 0 2024-04-26T23:01:41.818 INFO:tasks.workunit.client.0.smithi027.stdout:2/948: dwrite d2/d5/db/de/d21/dc4/d39/f3f [4194304,4194304] 0 2024-04-26T23:01:41.821 INFO:tasks.workunit.client.0.smithi027.stdout:2/949: symlink d2/d5/db/d1b/d1c/d119/de7/d121/l130 0 2024-04-26T23:01:41.821 INFO:tasks.workunit.client.0.smithi027.stdout:2/950: stat d2/d5/db/de/d21/dc4/d39 0 2024-04-26T23:01:41.823 INFO:tasks.workunit.client.0.smithi027.stdout:2/951: creat d2/d5/db/d1b/d1c/d25/d19/d44/d7e/f131 x:0 0 0 2024-04-26T23:01:41.825 INFO:tasks.workunit.client.0.smithi027.stdout:2/952: link d2/d5/db/d1b/fb8 d2/d5/db/d1b/d1c/d25/d19/d23/ded/f132 0 2024-04-26T23:01:41.826 INFO:tasks.workunit.client.0.smithi027.stdout:6/835: sync 2024-04-26T23:01:41.826 INFO:tasks.workunit.client.0.smithi027.stdout:9/909: sync 2024-04-26T23:01:41.826 INFO:tasks.workunit.client.0.smithi027.stdout:9/910: write d4/d10/d67/fb6 [4033726,61770] 0 2024-04-26T23:01:41.830 INFO:tasks.workunit.client.0.smithi027.stdout:6/836: unlink d1/d30/c60 0 2024-04-26T23:01:41.831 INFO:tasks.workunit.client.0.smithi027.stdout:2/953: mknod d2/d5/db/d1b/d1c/d25/d19/d44/d7e/c133 0 2024-04-26T23:01:41.838 INFO:tasks.workunit.client.0.smithi027.stdout:8/947: dwrite d9/de/d2e/d110/f99 [0,4194304] 0 2024-04-26T23:01:41.839 INFO:tasks.workunit.client.0.smithi027.stdout:6/837: dread d1/de/df3/db3/fcc [0,4194304] 0 2024-04-26T23:01:41.840 INFO:tasks.workunit.client.0.smithi027.stdout:6/838: chown d1/df/f29 6974 1 2024-04-26T23:01:41.840 INFO:tasks.workunit.client.0.smithi027.stdout:6/839: chown d1/f72 6 1 2024-04-26T23:01:41.840 INFO:tasks.workunit.client.0.smithi027.stdout:6/840: chown d1/de/d57/d8e/dd0/ld2 554 1 2024-04-26T23:01:41.844 INFO:tasks.workunit.client.0.smithi027.stdout:8/948: getdents d9/de/d2e/d110/d8d/da3 0 2024-04-26T23:01:41.844 INFO:tasks.workunit.client.0.smithi027.stdout:6/841: truncate d1/f4e 736789 0 2024-04-26T23:01:41.853 INFO:tasks.workunit.client.0.smithi027.stdout:2/954: dread d2/d5/db/d1b/d1c/d25/d19/f45 [0,4194304] 0 2024-04-26T23:01:41.853 INFO:tasks.workunit.client.0.smithi027.stdout:2/955: read - d2/d5/db/d1b/fcb zero size 2024-04-26T23:01:41.854 INFO:tasks.workunit.client.0.smithi027.stdout:0/954: dwrite d0/df/d3f/d64/da8/f118 [0,4194304] 0 2024-04-26T23:01:41.855 INFO:tasks.workunit.client.0.smithi027.stdout:0/955: creat d0/dfa/f12d x:0 0 0 2024-04-26T23:01:41.857 INFO:tasks.workunit.client.0.smithi027.stdout:0/956: link d0/df/d9d/d9f/dbb/ce0 d0/d4/dd3/c12e 0 2024-04-26T23:01:41.876 INFO:tasks.workunit.client.0.smithi027.stdout:0/957: write d0/f12 [3688652,26365] 0 2024-04-26T23:01:41.893 INFO:tasks.workunit.client.0.smithi027.stdout:2/956: dwrite d2/d5/db/d1b/d1c/f107 [0,4194304] 0 2024-04-26T23:01:41.893 INFO:tasks.workunit.client.0.smithi027.stdout:6/842: dwrite d1/de/d57/d8e/dd0/dc9/fe7 [0,4194304] 0 2024-04-26T23:01:41.893 INFO:tasks.workunit.client.0.smithi027.stdout:6/843: readlink d1/de/df3/la3 0 2024-04-26T23:01:41.895 INFO:tasks.workunit.client.0.smithi027.stdout:8/949: dwrite d9/da/d2a/d5e/d9e/fc8 [4194304,4194304] 0 2024-04-26T23:01:41.896 INFO:tasks.workunit.client.0.smithi027.stdout:2/957: symlink d2/db9/dc6/l134 0 2024-04-26T23:01:41.899 INFO:tasks.workunit.client.0.smithi027.stdout:2/958: link d2/db9/fe1 d2/d5/db/d1b/d1c/d25/d19/db0/f135 0 2024-04-26T23:01:41.899 INFO:tasks.workunit.client.0.smithi027.stdout:2/959: dread - d2/d5/db/d1b/d1c/d25/d19/d23/d7c/fb3 zero size 2024-04-26T23:01:41.902 INFO:tasks.workunit.client.0.smithi027.stdout:2/960: truncate d2/d5/db/d1b/d1c/d25/d19/d44/dee/d118/f122 254418 0 2024-04-26T23:01:41.903 INFO:tasks.workunit.client.0.smithi027.stdout:0/958: dread d0/d4/da/d15/f5a [0,4194304] 0 2024-04-26T23:01:41.903 INFO:tasks.workunit.client.0.smithi027.stdout:0/959: readlink d0/df/d3f/d64/da8/ldf 0 2024-04-26T23:01:41.906 INFO:tasks.workunit.client.0.smithi027.stdout:0/960: getdents d0/d4/da/d15/d26 0 2024-04-26T23:01:41.906 INFO:tasks.workunit.client.0.smithi027.stdout:0/961: readlink d0/d4/da/d1a/dc3/df6/lf9 0 2024-04-26T23:01:41.906 INFO:tasks.workunit.client.0.smithi027.stdout:0/962: truncate d0/d4/da/d1a/d1e/fb0 1640828 0 2024-04-26T23:01:41.907 INFO:tasks.workunit.client.0.smithi027.stdout:0/963: write d0/d4/da/f11 [6385707,127482] 0 2024-04-26T23:01:41.909 INFO:tasks.workunit.client.0.smithi027.stdout:3/992: sync 2024-04-26T23:01:41.911 INFO:tasks.workunit.client.0.smithi027.stdout:3/993: symlink d6/dd3/d34/d2e/d105/l14c 0 2024-04-26T23:01:41.911 INFO:tasks.workunit.client.0.smithi027.stdout:3/994: dread - d6/dd3/d34/d2e/d9d/da7/dac/d74/da6/d40/fee zero size 2024-04-26T23:01:41.915 INFO:tasks.workunit.client.0.smithi027.stdout:6/844: dread d1/df/d3c/d9a/ff6 [0,4194304] 0 2024-04-26T23:01:41.915 INFO:tasks.workunit.client.0.smithi027.stdout:6/845: write d1/f62 [2360753,23414] 0 2024-04-26T23:01:41.929 INFO:tasks.workunit.client.0.smithi027.stdout:0/964: dread d0/d4/da/d1a/dc3/fcf [0,4194304] 0 2024-04-26T23:01:41.930 INFO:tasks.workunit.client.0.smithi027.stdout:2/961: write d2/d5/db/d1b/d1c/f77 [2136333,108698] 0 2024-04-26T23:01:41.930 INFO:tasks.workunit.client.0.smithi027.stdout:2/962: chown d2/d5/db/d1b/d1c/df0 6566090 1 2024-04-26T23:01:41.931 INFO:tasks.workunit.client.0.smithi027.stdout:0/965: creat d0/d4/da/d1a/dc3/df6/f12f x:0 0 0 2024-04-26T23:01:41.935 INFO:tasks.workunit.client.0.smithi027.stdout:2/963: mkdir d2/d5/db/d1b/d1c/d25/d19/d44/d7e/d136 0 2024-04-26T23:01:41.935 INFO:tasks.workunit.client.0.smithi027.stdout:3/995: write d6/dd3/d34/d47/ffd [2625776,45478] 0 2024-04-26T23:01:41.935 INFO:tasks.workunit.client.0.smithi027.stdout:2/964: dread - d2/d5/db/de/d21/f65 zero size 2024-04-26T23:01:41.935 INFO:tasks.workunit.client.0.smithi027.stdout:0/966: creat d0/d4/da/d1a/d1e/de8/f130 x:0 0 0 2024-04-26T23:01:41.960 INFO:tasks.workunit.client.0.smithi027.stdout:0/967: read d0/d4/da/d15/d55/f97 [787798,17985] 0 2024-04-26T23:01:41.960 INFO:tasks.workunit.client.0.smithi027.stdout:0/968: write d0/d4/da/d1a/d1e/dba/fd6 [92536,6660] 0 2024-04-26T23:01:41.964 INFO:tasks.workunit.client.0.smithi027.stdout:8/950: dwrite d9/da/d2a/d5e/d72/d84/f29 [0,4194304] 0 2024-04-26T23:01:41.964 INFO:tasks.workunit.client.0.smithi027.stdout:8/951: chown d9/de/d2e/d110/dc2/c112 21515595 1 2024-04-26T23:01:41.964 INFO:tasks.workunit.client.0.smithi027.stdout:8/952: fdatasync d9/de/d2e/f53 0 2024-04-26T23:01:41.975 INFO:tasks.workunit.client.0.smithi027.stdout:8/953: write d9/da/d2a/d5e/d72/d84/f3f [4371293,6704] 0 2024-04-26T23:01:41.975 INFO:tasks.workunit.client.0.smithi027.stdout:8/954: dread - d9/da/d2a/d6b/f127 zero size 2024-04-26T23:01:41.979 INFO:tasks.workunit.client.0.smithi027.stdout:8/955: mkdir d9/de/d2e/d14e 0 2024-04-26T23:01:41.981 INFO:tasks.workunit.client.0.smithi027.stdout:2/965: dwrite d2/d5/db/d1b/d1c/d25/d19/f5b [0,4194304] 0 2024-04-26T23:01:41.991 INFO:tasks.workunit.client.0.smithi027.stdout:0/969: write d0/d4/da/f2f [4762023,118526] 0 2024-04-26T23:01:41.992 INFO:tasks.workunit.client.0.smithi027.stdout:6/846: dwrite d1/df/d43/da8/d84/fb8 [0,4194304] 0 2024-04-26T23:01:41.993 INFO:tasks.workunit.client.0.smithi027.stdout:0/970: getdents d0/d4/da/d1a/d1e/dba 0 2024-04-26T23:01:41.994 INFO:tasks.workunit.client.0.smithi027.stdout:6/847: mknod d1/de/d57/d8e/dd0/c10b 0 2024-04-26T23:01:41.994 INFO:tasks.workunit.client.0.smithi027.stdout:3/996: dwrite d6/dd3/d34/f58 [4194304,4194304] 0 2024-04-26T23:01:41.994 INFO:tasks.workunit.client.0.smithi027.stdout:3/997: chown d6/dd3/d34/d78/da1/f107 400 1 2024-04-26T23:01:41.996 INFO:tasks.workunit.client.0.smithi027.stdout:6/848: mknod d1/de/df3/db3/c10c 0 2024-04-26T23:01:41.996 INFO:tasks.workunit.client.0.smithi027.stdout:6/849: chown d1/df/dc2 109822486 1 2024-04-26T23:01:41.997 INFO:tasks.workunit.client.0.smithi027.stdout:3/998: mknod d6/dd3/d34/db3/db4/c14d 0 2024-04-26T23:01:41.998 INFO:tasks.workunit.client.0.smithi027.stdout:6/850: rmdir d1/df 39 2024-04-26T23:01:41.998 INFO:tasks.workunit.client.0.smithi027.stdout:6/851: readlink d1/de/df3/db3/l109 0 2024-04-26T23:01:41.998 INFO:tasks.workunit.client.0.smithi027.stdout:6/852: dread - d1/df/ff4 zero size 2024-04-26T23:01:41.999 INFO:tasks.workunit.client.0.smithi027.stdout:8/956: dread d9/da/d33/f88 [0,4194304] 0 2024-04-26T23:01:41.999 INFO:tasks.workunit.client.0.smithi027.stdout:6/853: creat d1/de/d57/f10d x:0 0 0 2024-04-26T23:01:42.000 INFO:tasks.workunit.client.0.smithi027.stdout:4/979: sync 2024-04-26T23:01:42.001 INFO:tasks.workunit.client.0.smithi027.stdout:6/854: symlink d1/de/d57/d8e/dd0/dc9/l10e 0 2024-04-26T23:01:42.002 INFO:tasks.workunit.client.0.smithi027.stdout:6/855: creat d1/df/d43/da8/f10f x:0 0 0 2024-04-26T23:01:42.003 INFO:tasks.workunit.client.0.smithi027.stdout:6/856: getdents d1/de/d57/d98 0 2024-04-26T23:01:42.004 INFO:tasks.workunit.client.0.smithi027.stdout:8/957: unlink d9/da/d119/f139 0 2024-04-26T23:01:42.004 INFO:tasks.workunit.client.0.smithi027.stdout:8/958: write d9/da/d2a/d5e/d46/feb [1020011,25902] 0 2024-04-26T23:01:42.004 INFO:tasks.workunit.client.0.smithi027.stdout:6/857: symlink d1/de/df3/db3/l110 0 2024-04-26T23:01:42.004 INFO:tasks.workunit.client.0.smithi027.stdout:6/858: chown d1/df/d43/c4c 36 1 2024-04-26T23:01:42.007 INFO:tasks.workunit.client.0.smithi027.stdout:8/959: rename d9/de/d22/c36 to d9/de/d2e/d110/dc2/c14f 0 2024-04-26T23:01:42.007 INFO:tasks.workunit.client.0.smithi027.stdout:8/960: fsync d9/da/f13 0 2024-04-26T23:01:42.007 INFO:tasks.workunit.client.0.smithi027.stdout:8/961: getdents d9/da/d2a/d12d 0 2024-04-26T23:01:42.016 INFO:tasks.workunit.client.0.smithi027.stdout:6/859: creat d1/de/df3/f111 x:0 0 0 2024-04-26T23:01:42.030 INFO:tasks.workunit.client.0.smithi027.stdout:8/962: write d9/de/d2e/d110/d8d/f97 [2761413,52901] 0 2024-04-26T23:01:42.045 INFO:tasks.workunit.client.0.smithi027.stdout:0/971: dwrite d0/d4/f7 [4194304,4194304] 0 2024-04-26T23:01:42.046 INFO:tasks.workunit.client.0.smithi027.stdout:3/999: dwrite d6/dd3/d34/d47/d117/f11a [0,4194304] 0 2024-04-26T23:01:42.047 INFO:tasks.workunit.client.0.smithi027.stdout:0/972: creat d0/d4/da/d1a/d2a/f131 x:0 0 0 2024-04-26T23:01:42.049 INFO:tasks.workunit.client.0.smithi027.stdout:0/973: creat d0/d4/da/d1a/d59/d92/d107/f132 x:0 0 0 2024-04-26T23:01:42.050 INFO:tasks.workunit.client.0.smithi027.stdout:4/980: dwrite da/dd/de/d14/d21/d41/d59/d74/f11a [0,4194304] 0 2024-04-26T23:01:42.050 INFO:tasks.workunit.client.0.smithi027.stdout:0/974: creat d0/df/d3f/d4a/f133 x:0 0 0 2024-04-26T23:01:42.053 INFO:tasks.workunit.client.0.smithi027.stdout:0/975: mkdir d0/d4/da/d15/ddc/d134 0 2024-04-26T23:01:42.058 INFO:tasks.workunit.client.0.smithi027.stdout:0/976: rmdir d0/d4/da/d10f 0 2024-04-26T23:01:42.059 INFO:tasks.workunit.client.0.smithi027.stdout:8/963: dread d9/de/d2e/d110/d8d/f97 [0,4194304] 0 2024-04-26T23:01:42.059 INFO:tasks.workunit.client.0.smithi027.stdout:8/964: truncate d9/da6/f13d 201452 0 2024-04-26T23:01:42.059 INFO:tasks.workunit.client.0.smithi027.stdout:0/977: mkdir d0/df/d9d/db4/deb/d135 0 2024-04-26T23:01:42.060 INFO:tasks.workunit.client.0.smithi027.stdout:0/978: symlink d0/dfa/l136 0 2024-04-26T23:01:42.060 INFO:tasks.workunit.client.0.smithi027.stdout:8/965: unlink d9/da/l91 0 2024-04-26T23:01:42.078 INFO:tasks.workunit.client.0.smithi027.stdout:8/966: write d9/da/d33/f60 [597178,71795] 0 2024-04-26T23:01:42.078 INFO:tasks.workunit.client.0.smithi027.stdout:8/967: write d9/da/d2a/d5e/d46/feb [1124847,114866] 0 2024-04-26T23:01:42.080 INFO:tasks.workunit.client.0.smithi027.stdout:8/968: unlink d9/de/f2c 0 2024-04-26T23:01:42.080 INFO:tasks.workunit.client.0.smithi027.stdout:8/969: fdatasync d9/de/f14 0 2024-04-26T23:01:42.105 INFO:tasks.workunit.client.0.smithi027.stdout:0/979: dwrite d0/d4/da/d15/f6f [0,4194304] 0 2024-04-26T23:01:42.105 INFO:tasks.workunit.client.0.smithi027.stdout:0/980: stat d0/d4/da/d1a/d59/d92/dcd/c127 0 2024-04-26T23:01:42.105 INFO:tasks.workunit.client.0.smithi027.stdout:0/981: readlink d0/d4/da/d1a/d1e/le2 0 2024-04-26T23:01:42.107 INFO:tasks.workunit.client.0.smithi027.stdout:0/982: creat d0/d4/da/d1a/f137 x:0 0 0 2024-04-26T23:01:42.109 INFO:tasks.workunit.client.0.smithi027.stdout:0/983: unlink d0/d4/f53 0 2024-04-26T23:01:42.109 INFO:tasks.workunit.client.0.smithi027.stdout:0/984: fsync d0/d4/da/d1a/d2a/f74 0 2024-04-26T23:01:42.131 INFO:tasks.workunit.client.0.smithi027.stdout:8/970: dwrite d9/da/d2a/d5e/d92/fce [0,4194304] 0 2024-04-26T23:01:42.133 INFO:tasks.workunit.client.0.smithi027.stdout:8/971: rmdir d9/da/d33 39 2024-04-26T23:01:42.135 INFO:tasks.workunit.client.0.smithi027.stdout:8/972: creat d9/da/d2a/d5e/d144/d145/f150 x:0 0 0 2024-04-26T23:01:42.136 INFO:tasks.workunit.client.0.smithi027.stdout:8/973: mknod d9/de/d2e/d110/dc2/c151 0 2024-04-26T23:01:42.138 INFO:tasks.workunit.client.0.smithi027.stdout:8/974: mknod d9/da/d2a/c152 0 2024-04-26T23:01:42.141 INFO:tasks.workunit.client.0.smithi027.stdout:8/975: rmdir d9/de/d2e/d110/da7 0 2024-04-26T23:01:42.143 INFO:tasks.workunit.client.0.smithi027.stdout:8/976: mkdir d9/de/d2e/d132/d153 0 2024-04-26T23:01:42.143 INFO:tasks.workunit.client.0.smithi027.stdout:0/985: dwrite d0/df/d9d/d9f/dbb/fef [0,4194304] 0 2024-04-26T23:01:42.161 INFO:tasks.workunit.client.0.smithi027.stdout:0/986: read d0/d4/da/d15/d26/f8a [1324145,14321] 0 2024-04-26T23:01:42.163 INFO:tasks.workunit.client.0.smithi027.stdout:9/911: sync 2024-04-26T23:01:42.164 INFO:tasks.workunit.client.0.smithi027.stdout:0/987: symlink d0/df/d3f/l138 0 2024-04-26T23:01:42.166 INFO:tasks.workunit.client.0.smithi027.stdout:0/988: mkdir d0/d4/da/d139 0 2024-04-26T23:01:42.166 INFO:tasks.workunit.client.0.smithi027.stdout:0/989: stat d0/d4/da/d15/d26/dc4/cf3 0 2024-04-26T23:01:42.183 INFO:tasks.workunit.client.0.smithi027.stdout:8/977: dwrite d9/de/d22/fdb [0,4194304] 0 2024-04-26T23:01:42.186 INFO:tasks.workunit.client.0.smithi027.stdout:8/978: unlink d9/da/d2a/d5e/c148 0 2024-04-26T23:01:42.189 INFO:tasks.workunit.client.0.smithi027.stdout:8/979: creat d9/de/d2e/d110/f154 x:0 0 0 2024-04-26T23:01:42.190 INFO:tasks.workunit.client.0.smithi027.stdout:8/980: mkdir d9/de/d2e/d14e/d155 0 2024-04-26T23:01:42.190 INFO:tasks.workunit.client.0.smithi027.stdout:8/981: stat d9/de/d2e/d132/d153 0 2024-04-26T23:01:42.191 INFO:tasks.workunit.client.0.smithi027.stdout:9/912: dread d4/d10/d40/d58/d75/f63 [0,4194304] 0 2024-04-26T23:01:42.193 INFO:tasks.workunit.client.0.smithi027.stdout:9/913: creat d4/d10/d40/d58/d75/d84/dca/f13f x:0 0 0 2024-04-26T23:01:42.195 INFO:tasks.workunit.client.0.smithi027.stdout:9/914: mkdir d4/d10/d25/d140 0 2024-04-26T23:01:42.207 INFO:tasks.workunit.client.0.smithi027.stdout:0/990: dwrite d0/df/d9d/fb5 [0,4194304] 0 2024-04-26T23:01:42.207 INFO:tasks.workunit.client.0.smithi027.stdout:0/991: truncate d0/d4/da/d1a/d2a/f131 471358 0 2024-04-26T23:01:42.207 INFO:tasks.workunit.client.0.smithi027.stdout:0/992: dread - d0/df/d3f/f11b zero size 2024-04-26T23:01:42.208 INFO:tasks.workunit.client.0.smithi027.stdout:0/993: mknod d0/d4/c13a 0 2024-04-26T23:01:42.209 INFO:tasks.workunit.client.0.smithi027.stdout:0/994: rename d0/ce4 to d0/d4/da/d1a/d6d/c13b 0 2024-04-26T23:01:42.213 INFO:tasks.workunit.client.0.smithi027.stdout:0/995: creat d0/df/f13c x:0 0 0 2024-04-26T23:01:42.214 INFO:tasks.workunit.client.0.smithi027.stdout:9/915: write d4/d17/d2e/f95 [2391850,109576] 0 2024-04-26T23:01:42.214 INFO:tasks.workunit.client.0.smithi027.stdout:9/916: write d4/d17/f1d [5153120,44877] 0 2024-04-26T23:01:42.215 INFO:tasks.workunit.client.0.smithi027.stdout:0/996: unlink d0/d4/da/d1a/d2a/fcc 0 2024-04-26T23:01:42.217 INFO:tasks.workunit.client.0.smithi027.stdout:9/917: symlink d4/d10/d25/l141 0 2024-04-26T23:01:42.218 INFO:tasks.workunit.client.0.smithi027.stdout:9/918: mknod d4/d17/d2e/d81/c142 0 2024-04-26T23:01:42.240 INFO:tasks.workunit.client.0.smithi027.stdout:9/919: write d4/d17/d2e/d46/da0/fc3 [3560192,96329] 0 2024-04-26T23:01:42.246 INFO:tasks.workunit.client.0.smithi027.stdout:0/997: dread d0/d4/f20 [0,4194304] 0 2024-04-26T23:01:42.246 INFO:tasks.workunit.client.0.smithi027.stdout:0/998: chown d0/df/d3f/d64/da8 1 1 2024-04-26T23:01:42.248 INFO:tasks.workunit.client.0.smithi027.stdout:0/999: mkdir d0/d4/da/d1a/d6d/d13d 0 2024-04-26T23:01:42.252 INFO:tasks.workunit.client.0.smithi027.stdout:9/920: dread d4/d10/d40/d58/d75/ff9 [0,4194304] 0 2024-04-26T23:01:42.267 INFO:tasks.workunit.client.0.smithi027.stdout:2/966: sync 2024-04-26T23:01:42.267 INFO:tasks.workunit.client.0.smithi027.stdout:6/860: sync 2024-04-26T23:01:42.268 INFO:tasks.workunit.client.0.smithi027.stdout:9/921: dread d4/d10/d40/f64 [0,4194304] 0 2024-04-26T23:01:42.269 INFO:tasks.workunit.client.0.smithi027.stdout:6/861: rename d1/df/d43/da8/d84/fcb to d1/df/d3c/d9a/f112 0 2024-04-26T23:01:42.269 INFO:tasks.workunit.client.0.smithi027.stdout:6/862: chown d1/df/d43/da8/f9f 48365033 1 2024-04-26T23:01:42.272 INFO:tasks.workunit.client.0.smithi027.stdout:9/922: rename d4/d10/d67/l8a to d4/d77/da2/l143 0 2024-04-26T23:01:42.272 INFO:tasks.workunit.client.0.smithi027.stdout:9/923: write d4/d10/d40/d58/d75/d84/dc4/fd7 [219139,30232] 0 2024-04-26T23:01:42.272 INFO:tasks.workunit.client.0.smithi027.stdout:9/924: chown d4/d10/d40/d58/d75/d84/d86/dc5/d139 0 1 2024-04-26T23:01:42.273 INFO:tasks.workunit.client.0.smithi027.stdout:6/863: mknod d1/d30/c113 0 2024-04-26T23:01:42.274 INFO:tasks.workunit.client.0.smithi027.stdout:9/925: rename d4/d17/d2e/d46/l60 to d4/d10/d25/dc9/l144 0 2024-04-26T23:01:42.275 INFO:tasks.workunit.client.0.smithi027.stdout:6/864: mkdir d1/df/d49/d114 0 2024-04-26T23:01:42.277 INFO:tasks.workunit.client.0.smithi027.stdout:9/926: getdents d4/d17/d2e 0 2024-04-26T23:01:42.277 INFO:tasks.workunit.client.0.smithi027.stdout:9/927: truncate d4/d17/d2e/d10e/f131 372267 0 2024-04-26T23:01:42.278 INFO:tasks.workunit.client.0.smithi027.stdout:6/865: link d1/l21 d1/df/d43/da8/da1/l115 0 2024-04-26T23:01:42.279 INFO:tasks.workunit.client.0.smithi027.stdout:9/928: creat d4/d10/d67/f145 x:0 0 0 2024-04-26T23:01:42.279 INFO:tasks.workunit.client.0.smithi027.stdout:6/866: symlink d1/df/dfe/l116 0 2024-04-26T23:01:42.281 INFO:tasks.workunit.client.0.smithi027.stdout:9/929: creat d4/d17/d2e/f146 x:0 0 0 2024-04-26T23:01:42.281 INFO:tasks.workunit.client.0.smithi027.stdout:6/867: creat d1/df/dc2/f117 x:0 0 0 2024-04-26T23:01:42.281 INFO:tasks.workunit.client.0.smithi027.stdout:6/868: truncate d1/df/d43/d56/d6b/ff8 550188 0 2024-04-26T23:01:42.282 INFO:tasks.workunit.client.0.smithi027.stdout:6/869: readlink d1/de/df3/db3/l110 0 2024-04-26T23:01:42.282 INFO:tasks.workunit.client.0.smithi027.stdout:9/930: rmdir d4/d17/d2e/d137 0 2024-04-26T23:01:42.284 INFO:tasks.workunit.client.0.smithi027.stdout:6/870: unlink d1/de/d57/d8e/dd0/dc9/fd6 0 2024-04-26T23:01:42.284 INFO:tasks.workunit.client.0.smithi027.stdout:6/871: chown d1/de/df3/d6f/f89 856043 1 2024-04-26T23:01:42.284 INFO:tasks.workunit.client.0.smithi027.stdout:6/872: stat d1/de/d57/d8e/dd0/f103 0 2024-04-26T23:01:42.284 INFO:tasks.workunit.client.0.smithi027.stdout:6/873: chown d1/df/d43/da8/cad 656 1 2024-04-26T23:01:42.284 INFO:tasks.workunit.client.0.smithi027.stdout:9/931: symlink d4/d77/da2/d130/l147 0 2024-04-26T23:01:42.286 INFO:tasks.workunit.client.0.smithi027.stdout:9/932: creat d4/d10/d40/d58/d75/d117/f148 x:0 0 0 2024-04-26T23:01:42.329 INFO:tasks.workunit.client.0.smithi027.stdout:4/981: sync 2024-04-26T23:01:42.331 INFO:tasks.workunit.client.0.smithi027.stdout:4/982: symlink da/dd/de/d14/d21/d41/df7/l13b 0 2024-04-26T23:01:42.331 INFO:tasks.workunit.client.0.smithi027.stdout:4/983: write da/dd/de/db8/f10b [547371,112250] 0 2024-04-26T23:01:42.334 INFO:tasks.workunit.client.0.smithi027.stdout:4/984: rename da/dd/de/d14/d17/d80/dab/l7f to da/dd/de/d14/d21/d67/d8a/l13c 0 2024-04-26T23:01:42.375 INFO:tasks.workunit.client.0.smithi027.stdout:4/985: dread da/dd/de/d14/d21/d4a/f6a [0,4194304] 0 2024-04-26T23:01:42.377 INFO:tasks.workunit.client.0.smithi027.stdout:4/986: mkdir da/dd/de/d14/d21/d67/da6/d13d 0 2024-04-26T23:01:42.378 INFO:tasks.workunit.client.0.smithi027.stdout:4/987: mknod da/dd/de/d14/d21/d4a/c13e 0 2024-04-26T23:01:42.380 INFO:tasks.workunit.client.0.smithi027.stdout:4/988: creat da/dd/de/d14/f13f x:0 0 0 2024-04-26T23:01:42.386 INFO:tasks.workunit.client.0.smithi027.stdout:4/989: write da/dd/d5f/f7a [1193859,3712] 0 2024-04-26T23:01:42.387 INFO:tasks.workunit.client.0.smithi027.stdout:4/990: creat da/dd/de/d14/d21/dad/f140 x:0 0 0 2024-04-26T23:01:42.401 INFO:tasks.workunit.client.0.smithi027.stdout:8/982: sync 2024-04-26T23:01:42.407 INFO:tasks.workunit.client.0.smithi027.stdout:8/983: rename d9/da/d2a/d5e/d72/d84/c90 to d9/da/d33/c156 0 2024-04-26T23:01:42.407 INFO:tasks.workunit.client.0.smithi027.stdout:8/984: rename d9 to d9/de/d2e/d110/dc2/dc7/d13a/d157 22 2024-04-26T23:01:42.412 INFO:tasks.workunit.client.0.smithi027.stdout:8/985: link d9/da/d2a/d5e/f57 d9/de/d2e/d110/df0/d11f/f158 0 2024-04-26T23:01:42.415 INFO:tasks.workunit.client.0.smithi027.stdout:8/986: creat d9/de/d2e/d110/d8d/d143/f159 x:0 0 0 2024-04-26T23:01:42.418 INFO:tasks.workunit.client.0.smithi027.stdout:8/987: mknod d9/de/d22/c15a 0 2024-04-26T23:01:42.421 INFO:tasks.workunit.client.0.smithi027.stdout:8/988: mkdir d9/de/d2e/d14e/d15b 0 2024-04-26T23:01:42.426 INFO:tasks.workunit.client.0.smithi027.stdout:8/989: rename d9/fcc to d9/da/d2a/d5e/d2b/d52/f15c 0 2024-04-26T23:01:42.427 INFO:tasks.workunit.client.0.smithi027.stdout:8/990: dread - d9/da/d2a/d5e/d72/d84/d5d/d6c/fc4 zero size 2024-04-26T23:01:42.430 INFO:tasks.workunit.client.0.smithi027.stdout:8/991: creat d9/da/d2a/d5e/d46/f15d x:0 0 0 2024-04-26T23:01:42.432 INFO:tasks.workunit.client.0.smithi027.stdout:8/992: getdents d9/da/d2a/d5e/d72/d84/d4f/dd3/d138 0 2024-04-26T23:01:42.437 INFO:tasks.workunit.client.0.smithi027.stdout:8/993: creat d9/da/d2a/d5e/d2b/d52/d115/f15e x:0 0 0 2024-04-26T23:01:42.440 INFO:tasks.workunit.client.0.smithi027.stdout:8/994: mkdir d9/da/d2a/d5e/d72/d15f 0 2024-04-26T23:01:42.440 INFO:tasks.workunit.client.0.smithi027.stdout:8/995: read - d9/da/d2a/d5e/d72/d84/d4f/ff1 zero size 2024-04-26T23:01:42.461 INFO:tasks.workunit.client.0.smithi027.stdout:4/991: dwrite da/dd/de/d14/d21/dad/f140 [0,4194304] 0 2024-04-26T23:01:42.463 INFO:tasks.workunit.client.0.smithi027.stdout:4/992: symlink da/dd/de/d14/d21/d67/d8a/ded/l141 0 2024-04-26T23:01:42.463 INFO:tasks.workunit.client.0.smithi027.stdout:4/993: write da/dd/de/d14/d21/d41/d59/f78 [1136605,90459] 0 2024-04-26T23:01:42.486 INFO:tasks.workunit.client.0.smithi027.stdout:8/996: dread d9/da/d33/f60 [0,4194304] 0 2024-04-26T23:01:42.490 INFO:tasks.workunit.client.0.smithi027.stdout:8/997: link d9/da/d2a/d5e/d72/f7e d9/da/d2a/d5e/d72/d15f/f160 0 2024-04-26T23:01:42.490 INFO:tasks.workunit.client.0.smithi027.stdout:8/998: write d9/da/d33/f41 [2091237,101570] 0 2024-04-26T23:01:42.495 INFO:tasks.workunit.client.0.smithi027.stdout:8/999: mkdir d9/de/d2e/d110/d8d/d161 0 2024-04-26T23:01:42.499 INFO:tasks.workunit.client.0.smithi027.stdout:9/933: sync 2024-04-26T23:01:42.499 INFO:tasks.workunit.client.0.smithi027.stdout:2/967: sync 2024-04-26T23:01:42.499 INFO:tasks.workunit.client.0.smithi027.stdout:6/874: sync 2024-04-26T23:01:42.499 INFO:tasks.workunit.client.0.smithi027.stdout:9/934: write d4/d10/d40/d58/d6e/f9f [658909,50589] 0 2024-04-26T23:01:42.502 INFO:tasks.workunit.client.0.smithi027.stdout:2/968: rename d2/d5/db/d1b/d1c/d25/cb6 to d2/d5/db/d1b/d1c/d25/d19/d44/dee/c137 0 2024-04-26T23:01:42.502 INFO:tasks.workunit.client.0.smithi027.stdout:2/969: truncate d2/d5/db/d1b/d1c/d25/d19/fea 656659 0 2024-04-26T23:01:42.502 INFO:tasks.workunit.client.0.smithi027.stdout:6/875: read d1/de/df3/f97 [9694,50999] 0 2024-04-26T23:01:42.502 INFO:tasks.workunit.client.0.smithi027.stdout:6/876: chown d1/df/d49/lc3 119161 1 2024-04-26T23:01:42.505 INFO:tasks.workunit.client.0.smithi027.stdout:9/935: rename d4/d10/d67 to d4/d10/d40/d58/d75/d84/d86/dc5/d139/d149 0 2024-04-26T23:01:42.505 INFO:tasks.workunit.client.0.smithi027.stdout:9/936: fsync d4/d17/d2e/d46/da0/d8d/fdc 0 2024-04-26T23:01:42.506 INFO:tasks.workunit.client.0.smithi027.stdout:2/970: creat d2/d5/db/d1b/d1c/d25/d19/d44/f138 x:0 0 0 2024-04-26T23:01:42.506 INFO:tasks.workunit.client.0.smithi027.stdout:2/971: fsync d2/d5/db/d1b/d1c/d25/d19/db0/fe2 0 2024-04-26T23:01:42.506 INFO:tasks.workunit.client.0.smithi027.stdout:9/937: creat d4/d10/d40/d58/d75/d84/dc4/f14a x:0 0 0 2024-04-26T23:01:42.507 INFO:tasks.workunit.client.0.smithi027.stdout:2/972: truncate d2/d5/db/d1b/fb8 59394 0 2024-04-26T23:01:42.508 INFO:tasks.workunit.client.0.smithi027.stdout:9/938: symlink d4/d10/d40/d58/d75/d84/dc4/l14b 0 2024-04-26T23:01:42.509 INFO:tasks.workunit.client.0.smithi027.stdout:2/973: link d2/d5/db/d1b/d1c/d25/d19/d44/d7e/l10e d2/d5/db/d1b/d1c/d119/de7/d121/l139 0 2024-04-26T23:01:42.510 INFO:tasks.workunit.client.0.smithi027.stdout:2/974: write d2/d5/db/d1b/d1c/d25/d19/d23/d82/ffc [482361,85127] 0 2024-04-26T23:01:42.511 INFO:tasks.workunit.client.0.smithi027.stdout:2/975: fsync d2/d5/db/d1b/d1c/d25/d19/db0/f10a 0 2024-04-26T23:01:42.511 INFO:tasks.workunit.client.0.smithi027.stdout:2/976: chown d2/d5/db/de/d21/dc4/c11b 3 1 2024-04-26T23:01:42.511 INFO:tasks.workunit.client.0.smithi027.stdout:9/939: rename d4/d17/d2e/c65 to d4/d10/d40/d58/d75/d117/c14c 0 2024-04-26T23:01:42.512 INFO:tasks.workunit.client.0.smithi027.stdout:2/977: symlink d2/d5/db/d1b/d10b/l13a 0 2024-04-26T23:01:42.513 INFO:tasks.workunit.client.0.smithi027.stdout:2/978: truncate d2/f12b 860577 0 2024-04-26T23:01:42.514 INFO:tasks.workunit.client.0.smithi027.stdout:9/940: mknod d4/d10/d40/d58/d75/d84/d86/d128/c14d 0 2024-04-26T23:01:42.516 INFO:tasks.workunit.client.0.smithi027.stdout:2/979: mkdir d2/d5/db/d1b/d1c/d25/d19/d23/d7c/d13b 0 2024-04-26T23:01:42.518 INFO:tasks.workunit.client.0.smithi027.stdout:2/980: creat d2/d5/db/d1b/d1c/d25/dc3/d127/f13c x:0 0 0 2024-04-26T23:01:42.533 INFO:tasks.workunit.client.0.smithi027.stdout:9/941: dread d4/d10/d40/d58/f9c [0,4194304] 0 2024-04-26T23:01:42.536 INFO:tasks.workunit.client.0.smithi027.stdout:9/942: creat d4/d17/d2e/d46/da0/d8d/df2/f14e x:0 0 0 2024-04-26T23:01:42.537 INFO:tasks.workunit.client.0.smithi027.stdout:9/943: creat d4/d10/d25/f14f x:0 0 0 2024-04-26T23:01:42.537 INFO:tasks.workunit.client.0.smithi027.stdout:9/944: chown d4/d10/d40/d58/d75/d84 2878 1 2024-04-26T23:01:42.539 INFO:tasks.workunit.client.0.smithi027.stdout:9/945: getdents d4/d10/d40/d58/d6e 0 2024-04-26T23:01:42.540 INFO:tasks.workunit.client.0.smithi027.stdout:9/946: rename d4/fcd to d4/d77/da2/d130/f150 0 2024-04-26T23:01:42.544 INFO:tasks.workunit.client.0.smithi027.stdout:9/947: mknod d4/d17/d2e/c151 0 2024-04-26T23:01:42.546 INFO:tasks.workunit.client.0.smithi027.stdout:9/948: getdents d4/d10/d40/d58/d75/d84/d86 0 2024-04-26T23:01:42.547 INFO:tasks.workunit.client.0.smithi027.stdout:9/949: getdents d4/d17/d2e/dcf 0 2024-04-26T23:01:42.547 INFO:tasks.workunit.client.0.smithi027.stdout:9/950: read d4/d10/d40/d58/d6e/f9f [664422,65822] 0 2024-04-26T23:01:42.549 INFO:tasks.workunit.client.0.smithi027.stdout:9/951: creat d4/d10/d25/dc9/f152 x:0 0 0 2024-04-26T23:01:42.550 INFO:tasks.workunit.client.0.smithi027.stdout:9/952: mkdir d4/d10/d40/d58/d75/d84/dca/d153 0 2024-04-26T23:01:42.552 INFO:tasks.workunit.client.0.smithi027.stdout:9/953: creat d4/d17/d2e/d10e/f154 x:0 0 0 2024-04-26T23:01:42.553 INFO:tasks.workunit.client.0.smithi027.stdout:9/954: fdatasync d4/d10/d40/f113 0 2024-04-26T23:01:42.553 INFO:tasks.workunit.client.0.smithi027.stdout:9/955: chown d4/d17/lb9 1 1 2024-04-26T23:01:42.553 INFO:tasks.workunit.client.0.smithi027.stdout:9/956: chown d4/d10/d40/d58/d6e 1005 1 2024-04-26T23:01:42.553 INFO:tasks.workunit.client.0.smithi027.stdout:9/957: dread - d4/d10/d40/d58/d75/fe1 zero size 2024-04-26T23:01:42.554 INFO:tasks.workunit.client.0.smithi027.stdout:9/958: mknod d4/d77/da2/d130/c155 0 2024-04-26T23:01:42.558 INFO:tasks.workunit.client.0.smithi027.stdout:9/959: rename d4/d77/da2/daf/cd9 to d4/d10/d40/d58/d75/d84/c156 0 2024-04-26T23:01:42.593 INFO:tasks.workunit.client.0.smithi027.stdout:2/981: dwrite d2/f67 [0,4194304] 0 2024-04-26T23:01:42.607 INFO:tasks.workunit.client.0.smithi027.stdout:2/982: dread d2/d5/db/d1b/d1c/d25/d19/f79 [0,4194304] 0 2024-04-26T23:01:42.610 INFO:tasks.workunit.client.0.smithi027.stdout:2/983: creat d2/db9/dc6/f13d x:0 0 0 2024-04-26T23:01:42.610 INFO:tasks.workunit.client.0.smithi027.stdout:2/984: write d2/d5/db/de/d21/dc4/d39/dd2/f116 [331510,109973] 0 2024-04-26T23:01:42.612 INFO:tasks.workunit.client.0.smithi027.stdout:6/877: dwrite d1/df/d3c/f75 [0,4194304] 0 2024-04-26T23:01:42.615 INFO:tasks.workunit.client.0.smithi027.stdout:2/985: creat d2/d5/db/d1b/d10b/d10f/f13e x:0 0 0 2024-04-26T23:01:42.615 INFO:tasks.workunit.client.0.smithi027.stdout:2/986: write d2/f12b [1539304,58002] 0 2024-04-26T23:01:42.617 INFO:tasks.workunit.client.0.smithi027.stdout:2/987: creat d2/d5/db/d1b/d1c/d25/d19/d23/f13f x:0 0 0 2024-04-26T23:01:42.618 INFO:tasks.workunit.client.0.smithi027.stdout:2/988: rmdir d2/db9/dc6 39 2024-04-26T23:01:42.624 INFO:tasks.workunit.client.0.smithi027.stdout:4/994: sync 2024-04-26T23:01:42.627 INFO:tasks.workunit.client.0.smithi027.stdout:9/960: dwrite d4/d10/d40/d58/d6e/f7c [0,4194304] 0 2024-04-26T23:01:42.629 INFO:tasks.workunit.client.0.smithi027.stdout:9/961: mkdir d4/d17/d2e/d134/d157 0 2024-04-26T23:01:42.630 INFO:tasks.workunit.client.0.smithi027.stdout:9/962: truncate d4/d77/da2/daf/fcb 635975 0 2024-04-26T23:01:42.630 INFO:tasks.workunit.client.0.smithi027.stdout:9/963: stat d4/d17/d2e/d46/dd1/ffe 0 2024-04-26T23:01:42.631 INFO:tasks.workunit.client.0.smithi027.stdout:9/964: mknod d4/d10/d40/d58/d75/d117/c158 0 2024-04-26T23:01:42.631 INFO:tasks.workunit.client.0.smithi027.stdout:9/965: write d4/d10/d40/d58/d75/d84/dc4/de4/f10c [777772,126509] 0 2024-04-26T23:01:42.634 INFO:tasks.workunit.client.0.smithi027.stdout:4/995: write da/dd/de/d14/d21/d41/d59/f9c [928444,82309] 0 2024-04-26T23:01:42.635 INFO:tasks.workunit.client.0.smithi027.stdout:4/996: mknod da/dd/de/d14/d21/d41/d59/c142 0 2024-04-26T23:01:42.670 INFO:tasks.workunit.client.0.smithi027.stdout:4/997: write da/dd/d136/da8/fba [163067,59349] 0 2024-04-26T23:01:42.671 INFO:tasks.workunit.client.0.smithi027.stdout:6/878: dread d1/df/f9b [0,4194304] 0 2024-04-26T23:01:42.691 INFO:tasks.workunit.client.0.smithi027.stdout:6/879: dread d1/df/d43/da8/f9f [0,4194304] 0 2024-04-26T23:01:42.711 INFO:tasks.workunit.client.0.smithi027.stdout:4/998: dread da/dd/de/d14/d21/f75 [0,4194304] 0 2024-04-26T23:01:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:2/989: dwrite d2/d5/db/d1b/f80 [0,4194304] 0 2024-04-26T23:01:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:2/990: write d2/d5/f6e [201899,130691] 0 2024-04-26T23:01:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:2/991: fdatasync d2/d5/db/d1b/d1c/d25/d19/f79 0 2024-04-26T23:01:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:2/992: rmdir d2/d5 39 2024-04-26T23:01:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:2/993: stat d2/d5/db/de/d21/d8f/deb/fec 0 2024-04-26T23:01:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:2/994: creat d2/d5/db/d1b/f140 x:0 0 0 2024-04-26T23:01:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:2/995: dread - d2/d5/db/d1b/d1c/f99 zero size 2024-04-26T23:01:42.737 INFO:tasks.workunit.client.0.smithi027.stdout:2/996: creat d2/d5/db/de/d21/dc4/d39/dd2/f141 x:0 0 0 2024-04-26T23:01:42.744 INFO:tasks.workunit.client.0.smithi027.stdout:9/966: dwrite d4/d10/d40/d58/d6e/f9f [0,4194304] 0 2024-04-26T23:01:42.749 INFO:tasks.workunit.client.0.smithi027.stdout:6/880: dwrite d1/df/f3a [4194304,4194304] 0 2024-04-26T23:01:42.750 INFO:tasks.workunit.client.0.smithi027.stdout:2/997: rename d2/d5/db/de/d21/dc4/d39 to d2/d5/db/d1b/d1c/d25/dc0/d142 0 2024-04-26T23:01:42.753 INFO:tasks.workunit.client.0.smithi027.stdout:9/967: rename d4/cdb to d4/d10/d40/d58/d75/d84/dca/d153/c159 0 2024-04-26T23:01:42.757 INFO:tasks.workunit.client.0.smithi027.stdout:6/881: rmdir d1/de/d57/d98 0 2024-04-26T23:01:42.777 INFO:tasks.workunit.client.0.smithi027.stdout:6/882: write d1/df/d43/d56/d6b/f76 [1891050,17514] 0 2024-04-26T23:01:42.777 INFO:tasks.workunit.client.0.smithi027.stdout:6/883: write d1/de/f42 [42276,58536] 0 2024-04-26T23:01:42.779 INFO:tasks.workunit.client.0.smithi027.stdout:6/884: creat d1/de/d57/d8e/dd0/dc9/f118 x:0 0 0 2024-04-26T23:01:42.781 INFO:tasks.workunit.client.0.smithi027.stdout:6/885: rename d1/df/d43/da8/f92 to d1/de/d57/d8e/f119 0 2024-04-26T23:01:42.781 INFO:tasks.workunit.client.0.smithi027.stdout:6/886: write d1/df/fd [262166,34619] 0 2024-04-26T23:01:42.782 INFO:tasks.workunit.client.0.smithi027.stdout:6/887: truncate d1/de/d57/d8e/dd0/f8a 584533 0 2024-04-26T23:01:42.783 INFO:tasks.workunit.client.0.smithi027.stdout:6/888: creat d1/df/d49/f11a x:0 0 0 2024-04-26T23:01:42.784 INFO:tasks.workunit.client.0.smithi027.stdout:6/889: creat d1/d6a/f11b x:0 0 0 2024-04-26T23:01:42.787 INFO:tasks.workunit.client.0.smithi027.stdout:2/998: write d2/db9/fe1 [3594640,102769] 0 2024-04-26T23:01:42.804 INFO:tasks.workunit.client.0.smithi027.stdout:9/968: dwrite d4/d17/d2e/d46/f55 [0,4194304] 0 2024-04-26T23:01:42.807 INFO:tasks.workunit.client.0.smithi027.stdout:9/969: creat d4/d17/d2e/d46/da0/d8d/f15a x:0 0 0 2024-04-26T23:01:42.821 INFO:tasks.workunit.client.0.smithi027.stdout:4/999: dwrite da/dd/de/d14/d17/d80/dab/dd6/feb [4194304,4194304] 0 2024-04-26T23:01:42.823 INFO:tasks.workunit.client.0.smithi027.stdout:2/999: dread d2/d5/db/d1b/d1c/d25/d19/db0/f135 [0,4194304] 0 2024-04-26T23:01:42.856 INFO:tasks.workunit.client.0.smithi027.stdout:6/890: dwrite d1/df/f39 [0,4194304] 0 2024-04-26T23:01:42.858 INFO:tasks.workunit.client.0.smithi027.stdout:6/891: unlink d1/df/d43/d56/l71 0 2024-04-26T23:01:42.858 INFO:tasks.workunit.client.0.smithi027.stdout:6/892: chown d1/df/fa5 1 1 2024-04-26T23:01:42.858 INFO:tasks.workunit.client.0.smithi027.stdout:6/893: dread - d1/df/d49/f11a zero size 2024-04-26T23:01:42.858 INFO:tasks.workunit.client.0.smithi027.stdout:6/894: stat d1/de/f82 0 2024-04-26T23:01:42.859 INFO:tasks.workunit.client.0.smithi027.stdout:6/895: creat d1/de/df3/d6f/dd3/f11c x:0 0 0 2024-04-26T23:01:42.859 INFO:tasks.workunit.client.0.smithi027.stdout:6/896: chown d1/de/d57/de3/fe8 127 1 2024-04-26T23:01:42.859 INFO:tasks.workunit.client.0.smithi027.stdout:9/970: dwrite d4/d10/d40/d58/f5a [0,4194304] 0 2024-04-26T23:01:42.862 INFO:tasks.workunit.client.0.smithi027.stdout:9/971: creat d4/d77/da2/daf/d12f/f15b x:0 0 0 2024-04-26T23:01:42.862 INFO:tasks.workunit.client.0.smithi027.stdout:9/972: fdatasync d4/d17/f1d 0 2024-04-26T23:01:42.863 INFO:tasks.workunit.client.0.smithi027.stdout:9/973: link d4/f138 d4/d10/d40/d58/d75/d84/d102/f15c 0 2024-04-26T23:01:42.863 INFO:tasks.workunit.client.0.smithi027.stdout:6/897: fsync d1/df/fd 0 2024-04-26T23:01:42.864 INFO:tasks.workunit.client.0.smithi027.stdout:9/974: symlink d4/d17/d2e/d81/l15d 0 2024-04-26T23:01:42.866 INFO:tasks.workunit.client.0.smithi027.stdout:6/898: link d1/df/dfe/l116 d1/de/df3/d6f/l11d 0 2024-04-26T23:01:42.866 INFO:tasks.workunit.client.0.smithi027.stdout:6/899: fsync d1/df/d49/f4d 0 2024-04-26T23:01:42.867 INFO:tasks.workunit.client.0.smithi027.stdout:9/975: unlink d4/d17/d2e/dcf/c100 0 2024-04-26T23:01:42.868 INFO:tasks.workunit.client.0.smithi027.stdout:6/900: creat d1/df/d43/f11e x:0 0 0 2024-04-26T23:01:42.872 INFO:tasks.workunit.client.0.smithi027.stdout:9/976: mkdir d4/d10/d40/d58/d75/d11a/d15e 0 2024-04-26T23:01:42.872 INFO:tasks.workunit.client.0.smithi027.stdout:9/977: write d4/d10/d40/f5d [3191702,21450] 0 2024-04-26T23:01:42.872 INFO:tasks.workunit.client.0.smithi027.stdout:9/978: dread - d4/d10/d25/f14f zero size 2024-04-26T23:01:42.872 INFO:tasks.workunit.client.0.smithi027.stdout:9/979: dread - d4/d17/d2e/d10e/f154 zero size 2024-04-26T23:01:42.873 INFO:tasks.workunit.client.0.smithi027.stdout:9/980: creat d4/d17/d2e/d46/da0/f15f x:0 0 0 2024-04-26T23:01:42.873 INFO:tasks.workunit.client.0.smithi027.stdout:9/981: fsync d4/d17/d2e/d46/f62 0 2024-04-26T23:01:42.874 INFO:tasks.workunit.client.0.smithi027.stdout:9/982: fsync d4/d17/d2e/fcc 0 2024-04-26T23:01:42.876 INFO:tasks.workunit.client.0.smithi027.stdout:9/983: creat d4/d17/d2e/d46/da0/f160 x:0 0 0 2024-04-26T23:01:42.877 INFO:tasks.workunit.client.0.smithi027.stdout:9/984: write d4/d17/d2e/fcc [212934,78874] 0 2024-04-26T23:01:42.877 INFO:tasks.workunit.client.0.smithi027.stdout:9/985: fdatasync d4/d17/d2e/d46/da0/d8d/f12e 0 2024-04-26T23:01:42.877 INFO:tasks.workunit.client.0.smithi027.stdout:9/986: chown d4/d10/d40/d58/d75/d109 0 1 2024-04-26T23:01:42.877 INFO:tasks.workunit.client.0.smithi027.stdout:9/987: chown d4/d10/d40/d58/d75/d84/d86/d128/c14d 353830 1 2024-04-26T23:01:42.878 INFO:tasks.workunit.client.0.smithi027.stdout:9/988: symlink d4/d17/d2e/dcf/l161 0 2024-04-26T23:01:42.878 INFO:tasks.workunit.client.0.smithi027.stdout:9/989: chown d4/d17/l4e 0 1 2024-04-26T23:01:42.879 INFO:tasks.workunit.client.0.smithi027.stdout:9/990: fsync d4/d77/da2/daf/d12f/f15b 0 2024-04-26T23:01:42.890 INFO:tasks.workunit.client.0.smithi027.stdout:9/991: getdents d4/d17/d2e/d70 0 2024-04-26T23:01:42.891 INFO:tasks.workunit.client.0.smithi027.stdout:9/992: creat d4/d10/d40/d58/d75/d84/d102/f162 x:0 0 0 2024-04-26T23:01:42.928 INFO:tasks.workunit.client.0.smithi027.stdout:6/901: dwrite d1/de/df3/f97 [0,4194304] 0 2024-04-26T23:01:42.931 INFO:tasks.workunit.client.0.smithi027.stdout:6/902: creat d1/de/df3/d6f/dd3/f11f x:0 0 0 2024-04-26T23:01:42.936 INFO:tasks.workunit.client.0.smithi027.stdout:6/903: creat d1/d6a/f120 x:0 0 0 2024-04-26T23:01:42.954 INFO:tasks.workunit.client.0.smithi027.stdout:9/993: dwrite d4/d17/f9e [0,4194304] 0 2024-04-26T23:01:42.955 INFO:tasks.workunit.client.0.smithi027.stdout:9/994: truncate d4/d77/da2/d130/f10a 932536 0 2024-04-26T23:01:42.956 INFO:tasks.workunit.client.0.smithi027.stdout:9/995: link d4/d10/f19 d4/d10/d40/d58/d75/d84/d86/d128/f163 0 2024-04-26T23:01:42.958 INFO:tasks.workunit.client.0.smithi027.stdout:9/996: creat d4/d17/d2e/d81/da7/f164 x:0 0 0 2024-04-26T23:01:42.958 INFO:tasks.workunit.client.0.smithi027.stdout:9/997: readlink d4/d77/da2/daf/ld8 0 2024-04-26T23:01:42.959 INFO:tasks.workunit.client.0.smithi027.stdout:9/998: stat d4/d17/d2e/dcf/lf8 0 2024-04-26T23:01:42.964 INFO:tasks.workunit.client.0.smithi027.stdout:9/999: fsync d4/d10/d40/f5d 0 2024-04-26T23:01:42.985 INFO:tasks.workunit.client.0.smithi027.stdout:6/904: dwrite d1/de/d57/d8e/dd0/f103 [0,4194304] 0 2024-04-26T23:01:42.994 INFO:tasks.workunit.client.0.smithi027.stdout:6/905: dread d1/df/d43/f6e [4194304,4194304] 0 2024-04-26T23:01:42.996 INFO:tasks.workunit.client.0.smithi027.stdout:6/906: mknod d1/df/d43/c121 0 2024-04-26T23:01:42.996 INFO:tasks.workunit.client.0.smithi027.stdout:6/907: chown d1/de/d57/d8e/dd0/dc9/le6 28083 1 2024-04-26T23:01:43.044 INFO:tasks.workunit.client.0.smithi027.stdout:6/908: dwrite d1/de/df3/d6f/dd3/f11c [0,4194304] 0 2024-04-26T23:01:43.044 INFO:tasks.workunit.client.0.smithi027.stdout:6/909: write d1/d30/f36 [503639,59683] 0 2024-04-26T23:01:43.047 INFO:tasks.workunit.client.0.smithi027.stdout:6/910: rename d1/df/dc2 to d1/de/df3/d122 0 2024-04-26T23:01:43.048 INFO:tasks.workunit.client.0.smithi027.stdout:6/911: stat d1/de/fe1 0 2024-04-26T23:01:43.050 INFO:tasks.workunit.client.0.smithi027.stdout:6/912: read d1/df/d43/d56/d6b/f8b [4201134,24215] 0 2024-04-26T23:01:43.055 INFO:tasks.workunit.client.0.smithi027.stdout:6/913: truncate d1/df/d43/faa 3480700 0 2024-04-26T23:01:43.055 INFO:tasks.workunit.client.0.smithi027.stdout:6/914: readlink d1/df/d49/lc3 0 2024-04-26T23:01:43.110 INFO:tasks.workunit.client.0.smithi027.stdout:6/915: dwrite d1/fdb [0,4194304] 0 2024-04-26T23:01:43.110 INFO:tasks.workunit.client.0.smithi027.stdout:6/916: dread - d1/de/df3/f111 zero size 2024-04-26T23:01:43.110 INFO:tasks.workunit.client.0.smithi027.stdout:6/917: dread - d1/de/d57/d8e/f119 zero size 2024-04-26T23:01:43.110 INFO:tasks.workunit.client.0.smithi027.stdout:6/918: write d1/f72 [1153317,5524] 0 2024-04-26T23:01:43.111 INFO:tasks.workunit.client.0.smithi027.stdout:6/919: truncate d1/df/d3c/d9a/f112 1333224 0 2024-04-26T23:01:43.111 INFO:tasks.workunit.client.0.smithi027.stdout:6/920: chown d1/de/d57/d8e/dd0 10 1 2024-04-26T23:01:43.111 INFO:tasks.workunit.client.0.smithi027.stdout:6/921: truncate d1/de/f82 852238 0 2024-04-26T23:01:43.113 INFO:tasks.workunit.client.0.smithi027.stdout:6/922: mknod d1/de/df3/d122/c123 0 2024-04-26T23:01:43.167 INFO:tasks.workunit.client.0.smithi027.stdout:6/923: dwrite d1/f4e [0,4194304] 0 2024-04-26T23:01:43.224 INFO:tasks.workunit.client.0.smithi027.stdout:6/924: dwrite d1/f72 [0,4194304] 0 2024-04-26T23:01:43.254 INFO:tasks.workunit.client.0.smithi027.stdout:6/925: dread d1/de/df3/f99 [0,4194304] 0 2024-04-26T23:01:43.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:43 smithi094 ceph-mon[32889]: pgmap v96: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 198 MiB/s rd, 223 MiB/s wr, 647 op/s 2024-04-26T23:01:43.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:43 smithi027 ceph-mon[28034]: pgmap v96: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 198 MiB/s rd, 223 MiB/s wr, 647 op/s 2024-04-26T23:01:43.381 INFO:tasks.workunit.client.0.smithi027.stdout:6/926: sync 2024-04-26T23:01:43.381 INFO:tasks.workunit.client.0.smithi027.stdout:6/927: chown d1/df/d43/d56/d6b/ff8 6 1 2024-04-26T23:01:43.387 INFO:tasks.workunit.client.0.smithi027.stdout:6/928: unlink d1/de/df3/d6f/l11d 0 2024-04-26T23:01:43.415 INFO:tasks.workunit.client.0.smithi027.stdout:6/929: dread d1/de/df3/d6f/dd3/ddf/fec [0,4194304] 0 2024-04-26T23:01:43.417 INFO:tasks.workunit.client.0.smithi027.stdout:6/930: mknod d1/df/d49/c124 0 2024-04-26T23:01:43.418 INFO:tasks.workunit.client.0.smithi027.stdout:6/931: mkdir d1/de/df3/d6f/d7d/d125 0 2024-04-26T23:01:43.496 INFO:tasks.workunit.client.0.smithi027.stdout:6/932: dwrite d1/fc6 [0,4194304] 0 2024-04-26T23:01:43.497 INFO:tasks.workunit.client.0.smithi027.stdout:6/933: dread - d1/de/d57/d8e/dd0/f108 zero size 2024-04-26T23:01:43.498 INFO:tasks.workunit.client.0.smithi027.stdout:6/934: mknod d1/df/d43/da8/da1/c126 0 2024-04-26T23:01:43.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:43 smithi080 ceph-mon[32324]: pgmap v96: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 198 MiB/s rd, 223 MiB/s wr, 647 op/s 2024-04-26T23:01:43.530 INFO:tasks.workunit.client.0.smithi027.stdout:6/935: read d1/de/f42 [3511728,125673] 0 2024-04-26T23:01:43.530 INFO:tasks.workunit.client.0.smithi027.stdout:6/936: chown d1/de/df3/f99 20 1 2024-04-26T23:01:43.530 INFO:tasks.workunit.client.0.smithi027.stdout:6/937: stat d1/lb1 0 2024-04-26T23:01:43.532 INFO:tasks.workunit.client.0.smithi027.stdout:6/938: symlink d1/df/d3c/l127 0 2024-04-26T23:01:43.534 INFO:tasks.workunit.client.0.smithi027.stdout:6/939: creat d1/de/d57/d8e/f128 x:0 0 0 2024-04-26T23:01:43.536 INFO:tasks.workunit.client.0.smithi027.stdout:6/940: rmdir d1/df/d43/d56/d6b/dc4/df1 0 2024-04-26T23:01:43.627 INFO:tasks.workunit.client.0.smithi027.stdout:6/941: rename d1/f2b to d1/df/d43/da8/f129 0 2024-04-26T23:01:43.669 INFO:tasks.workunit.client.0.smithi027.stdout:6/942: dwrite d1/de/df3/fbd [0,4194304] 0 2024-04-26T23:01:43.774 INFO:tasks.workunit.client.0.smithi027.stdout:6/943: dwrite d1/df/d49/f4d [4194304,4194304] 0 2024-04-26T23:01:43.774 INFO:tasks.workunit.client.0.smithi027.stdout:6/944: truncate d1/df/d43/d56/fd4 4275410 0 2024-04-26T23:01:43.774 INFO:tasks.workunit.client.0.smithi027.stdout:6/945: dread - d1/df/ff4 zero size 2024-04-26T23:01:43.774 INFO:tasks.workunit.client.0.smithi027.stdout:6/946: truncate d1/df/d43/f74 1538218 0 2024-04-26T23:01:43.808 INFO:tasks.workunit.client.0.smithi027.stdout:6/947: rename d1/df/d43/d56 to d1/de/df3/d6f/d7d/d12a 0 2024-04-26T23:01:43.808 INFO:tasks.workunit.client.0.smithi027.stdout:6/948: fdatasync d1/f72 0 2024-04-26T23:01:43.809 INFO:tasks.workunit.client.0.smithi027.stdout:6/949: getdents d1/de/d57/d8e/d102 0 2024-04-26T23:01:45.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:45 smithi027 ceph-mon[28034]: pgmap v97: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 140 MiB/s wr, 528 op/s 2024-04-26T23:01:45.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:45 smithi080 ceph-mon[32324]: pgmap v97: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 140 MiB/s wr, 528 op/s 2024-04-26T23:01:45.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:45 smithi094 ceph-mon[32889]: pgmap v97: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 140 MiB/s wr, 528 op/s 2024-04-26T23:01:46.679 INFO:tasks.workunit.client.0.smithi027.stdout:6/950: link d1/de/df3/db3/l110 d1/de/df3/d6f/d7d/d12a/l12b 0 2024-04-26T23:01:46.700 INFO:tasks.workunit.client.0.smithi027.stdout:6/951: rename d1/df/d43 to d1/d30/d12c 0 2024-04-26T23:01:47.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:47 smithi027 ceph-mon[28034]: pgmap v98: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 140 MiB/s wr, 528 op/s 2024-04-26T23:01:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.501+0000 7f4a86c00640 1 -- 172.21.15.27:0/4132404350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a80107650 msgr2=0x7f4a80107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.501+0000 7f4a86c00640 1 --2- 172.21.15.27:0/4132404350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a80107650 0x7f4a80107a30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009860 tx=0x7f4a7402a340 comp rx=0 tx=0).stop 2024-04-26T23:01:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.502+0000 7f4a86c00640 1 -- 172.21.15.27:0/4132404350 shutdown_connections 2024-04-26T23:01:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.502+0000 7f4a86c00640 1 --2- 172.21.15.27:0/4132404350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a80102000 0x7f4a8010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.502+0000 7f4a86c00640 1 --2- 172.21.15.27:0/4132404350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a80101640 0x7f4a80101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.502+0000 7f4a86c00640 1 --2- 172.21.15.27:0/4132404350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a80107650 0x7f4a80107a30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.502+0000 7f4a86c00640 1 -- 172.21.15.27:0/4132404350 >> 172.21.15.27:0/4132404350 conn(0x7f4a800fd110 msgr2=0x7f4a800ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.502+0000 7f4a86c00640 1 -- 172.21.15.27:0/4132404350 shutdown_connections 2024-04-26T23:01:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.502+0000 7f4a86c00640 1 -- 172.21.15.27:0/4132404350 wait complete. 2024-04-26T23:01:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.503+0000 7f4a86c00640 1 Processor -- start 2024-04-26T23:01:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.503+0000 7f4a86c00640 1 -- start start 2024-04-26T23:01:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.503+0000 7f4a86c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a80101640 0x7f4a8010efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.503+0000 7f4a86c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a80102000 0x7f4a8010f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.503+0000 7f4a86c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a80107650 0x7f4a8010fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.503+0000 7f4a86c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4a80112a10 con 0x7f4a80107650 2024-04-26T23:01:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.503+0000 7f4a86c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4a80112890 con 0x7f4a80101640 2024-04-26T23:01:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.503+0000 7f4a86c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a80112b90 con 0x7f4a80102000 2024-04-26T23:01:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.504+0000 7f4a86200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a80107650 0x7f4a8010fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.504+0000 7f4a84e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a80102000 0x7f4a8010f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.504+0000 7f4a86200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a80107650 0x7f4a8010fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59264/0 (socket says 172.21.15.27:59264) 2024-04-26T23:01:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.504+0000 7f4a86200640 1 -- 172.21.15.27:0/118306790 learned_addr learned my addr 172.21.15.27:0/118306790 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.504+0000 7f4a85800640 1 --2- 172.21.15.27:0/118306790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a80101640 0x7f4a8010efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.504+0000 7f4a86200640 1 -- 172.21.15.27:0/118306790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a80102000 msgr2=0x7f4a8010f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.504+0000 7f4a86200640 1 --2- 172.21.15.27:0/118306790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a80102000 0x7f4a8010f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.504+0000 7f4a86200640 1 -- 172.21.15.27:0/118306790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a80101640 msgr2=0x7f4a8010efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.504+0000 7f4a86200640 1 --2- 172.21.15.27:0/118306790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a80101640 0x7f4a8010efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.504+0000 7f4a86200640 1 -- 172.21.15.27:0/118306790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a801a7430 con 0x7f4a80107650 2024-04-26T23:01:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.504+0000 7f4a84e00640 1 --2- 172.21.15.27:0/118306790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a80102000 0x7f4a8010f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:01:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.504+0000 7f4a86200640 1 --2- 172.21.15.27:0/118306790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a80107650 0x7f4a8010fa20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800efc0 tx=0x7f4a7800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.505+0000 7f4a7e000640 1 -- 172.21.15.27:0/118306790 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78003970 con 0x7f4a80107650 2024-04-26T23:01:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.505+0000 7f4a86c00640 1 -- 172.21.15.27:0/118306790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a801a7720 con 0x7f4a80107650 2024-04-26T23:01:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.505+0000 7f4a7e000640 1 -- 172.21.15.27:0/118306790 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a78004020 con 0x7f4a80107650 2024-04-26T23:01:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.505+0000 7f4a7e000640 1 -- 172.21.15.27:0/118306790 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7800b940 con 0x7f4a80107650 2024-04-26T23:01:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.505+0000 7f4a86c00640 1 -- 172.21.15.27:0/118306790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a801a7c20 con 0x7f4a80107650 2024-04-26T23:01:47.525 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:47 smithi080 ceph-mon[32324]: pgmap v98: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 140 MiB/s wr, 528 op/s 2024-04-26T23:01:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.508+0000 7f4a8afe5740 1 Processor -- start 2024-04-26T23:01:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.508+0000 7f4a7e000640 1 -- 172.21.15.27:0/118306790 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4a7801d020 con 0x7f4a80107650 2024-04-26T23:01:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.509+0000 7f4a8afe5740 1 -- start start 2024-04-26T23:01:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.509+0000 7f4a8afe5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df8653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.509+0000 7f4a8afe5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df865900 0x55e4df86fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.509+0000 7f4a8afe5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df8701c0 0x55e4df8725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.509+0000 7f4a8afe5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e4df7ca380 con 0x55e4df865900 2024-04-26T23:01:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.509+0000 7f4a8afe5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55e4df766c30 con 0x55e4df8701c0 2024-04-26T23:01:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.509+0000 7f4a8afe5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e4df7c6340 con 0x55e4df864fe0 2024-04-26T23:01:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.509+0000 7f4a85800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df865900 0x55e4df86fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.509+0000 7f4a84e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df8653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.510+0000 7f4a85800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df865900 0x55e4df86fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59278/0 (socket says 172.21.15.27:59278) 2024-04-26T23:01:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.510+0000 7f4a85800640 1 -- 172.21.15.27:0/2130427813 learned_addr learned my addr 172.21.15.27:0/2130427813 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.510+0000 7f4a84e00640 1 --2- 172.21.15.27:0/2130427813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df8701c0 0x55e4df8725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.510+0000 7f4a85800640 1 -- 172.21.15.27:0/2130427813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 msgr2=0x55e4df8653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.510+0000 7f4a85800640 1 --2- 172.21.15.27:0/2130427813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df8653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.510+0000 7f4a85800640 1 -- 172.21.15.27:0/2130427813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df8701c0 msgr2=0x55e4df8725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.510+0000 7f4a85800640 1 --2- 172.21.15.27:0/2130427813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df8701c0 0x55e4df8725b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.510+0000 7f4a85800640 1 -- 172.21.15.27:0/2130427813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4df788940 con 0x55e4df865900 2024-04-26T23:01:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.510+0000 7f4a7e000640 1 --2- 172.21.15.27:0/118306790 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f4a680a3d30 0x7f4a680a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.510+0000 7f4a7e000640 1 -- 172.21.15.27:0/118306790 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f4a780d0a70 con 0x7f4a80107650 2024-04-26T23:01:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.510+0000 7f4a85800640 1 --2- 172.21.15.27:0/2130427813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df865900 0x55e4df86fc80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009780 tx=0x7f4a74005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e695df735c6f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.510+0000 7f4a84e00640 1 --2- 172.21.15.27:0/118306790 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f4a680a3d30 0x7f4a680a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.511+0000 7f4a72a00640 1 -- 172.21.15.27:0/2130427813 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74041020 con 0x55e4df865900 2024-04-26T23:01:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.511+0000 7f4a72a00640 1 -- 172.21.15.27:0/2130427813 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a74030e80 con 0x55e4df865900 2024-04-26T23:01:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.511+0000 7f4a72a00640 1 -- 172.21.15.27:0/2130427813 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74037040 con 0x55e4df865900 2024-04-26T23:01:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.511+0000 7f4a8afe5740 1 -- 172.21.15.27:0/2130427813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df865900 msgr2=0x55e4df86fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.511+0000 7f4a84e00640 1 --2- 172.21.15.27:0/118306790 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f4a680a3d30 0x7f4a680a61f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x55e4df862910 tx=0x7f4a6c005f90 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.511+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/2130427813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df865900 0x55e4df86fc80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009780 tx=0x7f4a74005d50 comp rx=0 tx=0).stop 2024-04-26T23:01:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.512+0000 7f4a8afe5740 1 -- 172.21.15.27:0/2130427813 shutdown_connections 2024-04-26T23:01:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.512+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/2130427813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df8701c0 0x55e4df8725b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.512+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/2130427813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df865900 0x55e4df86fc80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.512+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/2130427813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df8653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.512+0000 7f4a8afe5740 1 -- 172.21.15.27:0/2130427813 >> 172.21.15.27:0/2130427813 conn(0x55e4df8614f0 msgr2=0x55e4df8618d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.512+0000 7f4a8afe5740 1 -- 172.21.15.27:0/2130427813 shutdown_connections 2024-04-26T23:01:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.512+0000 7f4a8afe5740 1 -- 172.21.15.27:0/2130427813 wait complete. 2024-04-26T23:01:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.512+0000 7f4a8afe5740 1 -- start start 2024-04-26T23:01:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.512+0000 7f4a8afe5740 1 Processor -- start 2024-04-26T23:01:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.513+0000 7f4a8afe5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df86d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.513+0000 7f4a8afe5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 0x55e4df86d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.513+0000 7f4a8afe5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df8701c0 0x55e4df86db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.513+0000 7f4a8afe5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e4df872d90 con 0x55e4df8701c0 2024-04-26T23:01:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.513+0000 7f4a8afe5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55e4df751c60 con 0x55e4df865900 2024-04-26T23:01:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.513+0000 7f4a8afe5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e4df872f10 con 0x55e4df864fe0 2024-04-26T23:01:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.513+0000 7f4a8afe5740 10 client.0 _ll_register_callbacks cb 0x55e4df77de50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:01:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.513+0000 7f4a84e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 0x55e4df86d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.513+0000 7f4a86200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df8701c0 0x55e4df86db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.513+0000 7f4a84e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 0x55e4df86d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:49112/0 (socket says 172.21.15.27:49112) 2024-04-26T23:01:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.513+0000 7f4a84e00640 1 -- 172.21.15.27:0/1168047193 learned_addr learned my addr 172.21.15.27:0/1168047193 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.513+0000 7f4a85800640 1 --2- 172.21.15.27:0/1168047193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df86d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.514+0000 7f4a85800640 1 -- 172.21.15.27:0/1168047193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 msgr2=0x55e4df86d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.514+0000 7f4a85800640 1 --2- 172.21.15.27:0/1168047193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 0x55e4df86d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.514+0000 7f4a85800640 1 -- 172.21.15.27:0/1168047193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df8701c0 msgr2=0x55e4df86db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.514+0000 7f4a85800640 1 --2- 172.21.15.27:0/1168047193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df8701c0 0x55e4df86db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.514+0000 7f4a85800640 1 -- 172.21.15.27:0/1168047193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4df760510 con 0x55e4df864fe0 2024-04-26T23:01:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.514+0000 7f4a85800640 1 --2- 172.21.15.27:0/1168047193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df86d0a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4a74035040 tx=0x7f4a7402a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=797edd7209ff1dd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.514+0000 7f4a67400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T23:01:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.514+0000 7f4a67400640 1 -- 172.21.15.27:0/1168047193 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74037050 con 0x55e4df864fe0 2024-04-26T23:01:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.514+0000 7f4a8afe5740 1 -- 172.21.15.27:0/1168047193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a74030e80 con 0x55e4df864fe0 2024-04-26T23:01:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.514+0000 7f4a67400640 1 -- 172.21.15.27:0/1168047193 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f4a74030690 con 0x55e4df864fe0 2024-04-26T23:01:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.515+0000 7f4a67400640 1 -- 172.21.15.27:0/1168047193 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a74059050 con 0x55e4df864fe0 2024-04-26T23:01:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.515+0000 7f4a8afe5740 10 client.25747 fetch_fsmap learned FSMap version 28 2024-04-26T23:01:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.515+0000 7f4a8afe5740 1 -- 172.21.15.27:0/1168047193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4df4dc210 con 0x55e4df864fe0 2024-04-26T23:01:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.515+0000 7f4a67400640 1 -- 172.21.15.27:0/1168047193 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a74030e80 con 0x55e4df864fe0 2024-04-26T23:01:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.515+0000 7f4a67400640 1 -- 172.21.15.27:0/1168047193 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a7403d480 con 0x55e4df864fe0 2024-04-26T23:01:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.515+0000 7f4a8afe5740 10 client.25747 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:01:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.515+0000 7f4a8afe5740 10 client.25747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:01:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.515+0000 7f4a8afe5740 20 client.25747 populate_metadata read hostname 'smithi027' 2024-04-26T23:01:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.515+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/1168047193 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e4df87fcd0 0x55e4df8820c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.515+0000 7f4a8afe5740 4 client.25747 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:01:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.515+0000 7f4a8afe5740 1 -- 172.21.15.27:0/1168047193 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4df7d7f90 con 0x55e4df87fcd0 2024-04-26T23:01:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.516+0000 7f4a84e00640 1 --2- 172.21.15.27:0/1168047193 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e4df87fcd0 0x55e4df8820c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.516+0000 7f4a84e00640 1 --2- 172.21.15.27:0/1168047193 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e4df87fcd0 0x55e4df8820c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126988db538a367d server_cookie=6591f4928c595a47 in_seq=0 out_seq=0 2024-04-26T23:01:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.516+0000 7f4a67400640 10 client.25747 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:01:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.517+0000 7f4a67400640 1 -- 172.21.15.27:0/1168047193 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4df7d7f90 con 0x55e4df87fcd0 2024-04-26T23:01:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.517+0000 7f4a67400640 10 client.25747 handle_command_reply: tid=0 2024-04-26T23:01:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.517+0000 7f4a8afe5740 1 client.25747 shutdown 2024-04-26T23:01:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.518+0000 7f4a8afe5740 1 -- 172.21.15.27:0/1168047193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 msgr2=0x55e4df86d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.518+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/1168047193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df86d0a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4a74035040 tx=0x7f4a7402a340 comp rx=0 tx=0).stop 2024-04-26T23:01:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.518+0000 7f4a8afe5740 1 -- 172.21.15.27:0/1168047193 shutdown_connections 2024-04-26T23:01:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.518+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/1168047193 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e4df87fcd0 0x55e4df8820c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.518+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/1168047193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 0x55e4df86d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.518+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/1168047193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df8701c0 0x55e4df86db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.518+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/1168047193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df86d0a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.518+0000 7f4a8afe5740 1 -- 172.21.15.27:0/1168047193 >> 172.21.15.27:0/1168047193 conn(0x55e4df8614f0 msgr2=0x55e4df867220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.518+0000 7f4a67400640 0 client.25747 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:01:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.518+0000 7f4a8afe5740 1 -- 172.21.15.27:0/1168047193 shutdown_connections 2024-04-26T23:01:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.518+0000 7f4a8afe5740 1 -- 172.21.15.27:0/1168047193 wait complete. 2024-04-26T23:01:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.518+0000 7f4a8afe5740 20 client.25747 trim_cache size 0 max 16384 2024-04-26T23:01:47.543 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:47 smithi094 ceph-mon[32889]: pgmap v98: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 140 MiB/s wr, 528 op/s 2024-04-26T23:01:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.535+0000 7f4a8afe5740 1 Processor -- start 2024-04-26T23:01:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.535+0000 7f4a8afe5740 1 -- start start 2024-04-26T23:01:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.535+0000 7f4a8afe5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df87bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.535+0000 7f4a8afe5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 0x55e4df8679a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.535+0000 7f4a8afe5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df8701c0 0x55e4df887780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.535+0000 7f4a8afe5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4a801b4840 con 0x55e4df8701c0 2024-04-26T23:01:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.535+0000 7f4a8afe5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4a801b20b0 con 0x55e4df865900 2024-04-26T23:01:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.535+0000 7f4a8afe5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a801aff00 con 0x55e4df864fe0 2024-04-26T23:01:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.536+0000 7f4a85800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df87bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.536+0000 7f4a85800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df87bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:45828/0 (socket says 172.21.15.27:45828) 2024-04-26T23:01:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.536+0000 7f4a85800640 1 -- 172.21.15.27:0/3699915735 learned_addr learned my addr 172.21.15.27:0/3699915735 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.536+0000 7f4a86200640 1 --2- 172.21.15.27:0/3699915735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df8701c0 0x55e4df887780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.536+0000 7f4a84e00640 1 --2- 172.21.15.27:0/3699915735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 0x55e4df8679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.536+0000 7f4a85800640 1 -- 172.21.15.27:0/3699915735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 msgr2=0x55e4df8679a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.536+0000 7f4a85800640 1 --2- 172.21.15.27:0/3699915735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 0x55e4df8679a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.536+0000 7f4a85800640 1 -- 172.21.15.27:0/3699915735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df8701c0 msgr2=0x55e4df887780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.536+0000 7f4a85800640 1 --2- 172.21.15.27:0/3699915735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df8701c0 0x55e4df887780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.536+0000 7f4a85800640 1 -- 172.21.15.27:0/3699915735 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4df5a8620 con 0x55e4df864fe0 2024-04-26T23:01:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.536+0000 7f4a85800640 1 --2- 172.21.15.27:0/3699915735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df87bc50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a7402a860 tx=0x7f4a740303e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e500e8cfa971ef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.536+0000 7f4a72000640 1 -- 172.21.15.27:0/3699915735 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74041070 con 0x55e4df864fe0 2024-04-26T23:01:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.537+0000 7f4a72000640 1 -- 172.21.15.27:0/3699915735 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a74009450 con 0x55e4df864fe0 2024-04-26T23:01:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.537+0000 7f4a72000640 1 -- 172.21.15.27:0/3699915735 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74037040 con 0x55e4df864fe0 2024-04-26T23:01:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.537+0000 7f4a8afe5740 1 -- 172.21.15.27:0/3699915735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 msgr2=0x55e4df87bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.537+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/3699915735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df87bc50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a7402a860 tx=0x7f4a740303e0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.537+0000 7f4a8afe5740 1 -- 172.21.15.27:0/3699915735 shutdown_connections 2024-04-26T23:01:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.537+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/3699915735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 0x55e4df8679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.537+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/3699915735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df8701c0 0x55e4df887780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.537+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/3699915735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df87bc50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.537+0000 7f4a8afe5740 1 -- 172.21.15.27:0/3699915735 >> 172.21.15.27:0/3699915735 conn(0x55e4df8614f0 msgr2=0x55e4df860650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.537+0000 7f4a8afe5740 1 -- 172.21.15.27:0/3699915735 shutdown_connections 2024-04-26T23:01:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.537+0000 7f4a8afe5740 1 -- 172.21.15.27:0/3699915735 wait complete. 2024-04-26T23:01:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.538+0000 7f4a8afe5740 1 -- start start 2024-04-26T23:01:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.538+0000 7f4a8afe5740 1 Processor -- start 2024-04-26T23:01:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.538+0000 7f4a8afe5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df869db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.538+0000 7f4a8afe5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 0x55e4df86a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.538+0000 7f4a8afe5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df8701c0 0x55e4df86a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.538+0000 7f4a8afe5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4a801b0570 con 0x55e4df8701c0 2024-04-26T23:01:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.538+0000 7f4a8afe5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4a801b19e0 con 0x55e4df865900 2024-04-26T23:01:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.538+0000 7f4a8afe5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a801b0c40 con 0x55e4df864fe0 2024-04-26T23:01:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.538+0000 7f4a8afe5740 10 client.0 _ll_register_callbacks cb 0x7f4a6c011610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:01:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.539+0000 7f4a85800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df869db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.539+0000 7f4a85800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df869db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:45834/0 (socket says 172.21.15.27:45834) 2024-04-26T23:01:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.539+0000 7f4a85800640 1 -- 172.21.15.27:0/3136772826 learned_addr learned my addr 172.21.15.27:0/3136772826 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:01:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.539+0000 7f4a86200640 1 --2- 172.21.15.27:0/3136772826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df8701c0 0x55e4df86a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.539+0000 7f4a84e00640 1 --2- 172.21.15.27:0/3136772826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 0x55e4df86a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.539+0000 7f4a85800640 1 -- 172.21.15.27:0/3136772826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 msgr2=0x55e4df86a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.539+0000 7f4a85800640 1 --2- 172.21.15.27:0/3136772826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 0x55e4df86a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.539+0000 7f4a85800640 1 -- 172.21.15.27:0/3136772826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df8701c0 msgr2=0x55e4df86a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:01:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.539+0000 7f4a85800640 1 --2- 172.21.15.27:0/3136772826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df8701c0 0x55e4df86a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.539+0000 7f4a85800640 1 -- 172.21.15.27:0/3136772826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4df784cb0 con 0x55e4df864fe0 2024-04-26T23:01:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.539+0000 7f4a84e00640 1 --2- 172.21.15.27:0/3136772826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 0x55e4df86a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:01:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.539+0000 7f4a85800640 1 --2- 172.21.15.27:0/3136772826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df869db0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4a74005ac0 tx=0x7f4a7403d730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e92461170270ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:01:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.539+0000 7f4a67400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T23:01:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.540+0000 7f4a67400640 1 -- 172.21.15.27:0/3136772826 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74041070 con 0x55e4df864fe0 2024-04-26T23:01:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.540+0000 7f4a8afe5740 1 -- 172.21.15.27:0/3136772826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a74009450 con 0x55e4df864fe0 2024-04-26T23:01:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.540+0000 7f4a67400640 1 -- 172.21.15.27:0/3136772826 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f4a7403da40 con 0x55e4df864fe0 2024-04-26T23:01:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.540+0000 7f4a67400640 1 -- 172.21.15.27:0/3136772826 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a7406b050 con 0x55e4df864fe0 2024-04-26T23:01:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.540+0000 7f4a8afe5740 10 client.25753 fetch_fsmap learned FSMap version 28 2024-04-26T23:01:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.540+0000 7f4a8afe5740 1 -- 172.21.15.27:0/3136772826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4df8e6180 con 0x55e4df864fe0 2024-04-26T23:01:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.540+0000 7f4a67400640 1 -- 172.21.15.27:0/3136772826 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a74009450 con 0x55e4df864fe0 2024-04-26T23:01:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.540+0000 7f4a67400640 1 -- 172.21.15.27:0/3136772826 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a740047c0 con 0x55e4df864fe0 2024-04-26T23:01:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.541+0000 7f4a8afe5740 10 client.25753 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:01:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.541+0000 7f4a8afe5740 10 client.25753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:01:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.541+0000 7f4a8afe5740 20 client.25753 populate_metadata read hostname 'smithi027' 2024-04-26T23:01:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.541+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/3136772826 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e4df8e6810 0x55e4df8e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:01:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.541+0000 7f4a8afe5740 4 client.25753 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:01:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.541+0000 7f4a8afe5740 1 -- 172.21.15.27:0/3136772826 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e4df7d1e00 con 0x55e4df8e6810 2024-04-26T23:01:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.541+0000 7f4a84e00640 1 --2- 172.21.15.27:0/3136772826 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e4df8e6810 0x55e4df8e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:01:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.541+0000 7f4a84e00640 1 --2- 172.21.15.27:0/3136772826 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e4df8e6810 0x55e4df8e8c30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbd16073aee4af80 server_cookie=2c4ea5900d753c5f in_seq=0 out_seq=0 2024-04-26T23:01:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.541+0000 7f4a67400640 10 client.25753 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:01:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.712+0000 7f4a67400640 1 -- 172.21.15.27:0/3136772826 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e4df7d1e00 con 0x55e4df8e6810 2024-04-26T23:01:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.712+0000 7f4a67400640 10 client.25753 handle_command_reply: tid=0 2024-04-26T23:01:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.712+0000 7f4a8afe5740 1 client.25753 shutdown 2024-04-26T23:01:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.713+0000 7f4a8afe5740 1 -- 172.21.15.27:0/3136772826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 msgr2=0x55e4df869db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.713+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/3136772826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df869db0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4a74005ac0 tx=0x7f4a7403d730 comp rx=0 tx=0).stop 2024-04-26T23:01:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.713+0000 7f4a8afe5740 1 -- 172.21.15.27:0/3136772826 shutdown_connections 2024-04-26T23:01:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.713+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/3136772826 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55e4df8e6810 0x55e4df8e8c30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.713+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/3136772826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e4df865900 0x55e4df86a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.713+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/3136772826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e4df8701c0 0x55e4df86a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.713+0000 7f4a8afe5740 1 --2- 172.21.15.27:0/3136772826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4df864fe0 0x55e4df869db0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.713+0000 7f4a67400640 0 client.25753 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:01:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.713+0000 7f4a8afe5740 1 -- 172.21.15.27:0/3136772826 >> 172.21.15.27:0/3136772826 conn(0x55e4df8614f0 msgr2=0x55e4df86d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.713+0000 7f4a8afe5740 1 -- 172.21.15.27:0/3136772826 shutdown_connections 2024-04-26T23:01:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.713+0000 7f4a8afe5740 1 -- 172.21.15.27:0/3136772826 wait complete. 2024-04-26T23:01:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.713+0000 7f4a8afe5740 20 client.25753 trim_cache size 0 max 16384 2024-04-26T23:01:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T23:01:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T23:01:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T23:01:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T23:01:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.714+0000 7f4a86c00640 1 -- 172.21.15.27:0/118306790 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f4a680a3d30 msgr2=0x7f4a680a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.714+0000 7f4a86c00640 1 --2- 172.21.15.27:0/118306790 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f4a680a3d30 0x7f4a680a61f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x55e4df862910 tx=0x7f4a6c005f90 comp rx=0 tx=0).stop 2024-04-26T23:01:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.714+0000 7f4a86c00640 1 -- 172.21.15.27:0/118306790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a80107650 msgr2=0x7f4a8010fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:01:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.714+0000 7f4a86c00640 1 --2- 172.21.15.27:0/118306790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a80107650 0x7f4a8010fa20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800efc0 tx=0x7f4a7800c800 comp rx=0 tx=0).stop 2024-04-26T23:01:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.714+0000 7f4a86c00640 1 -- 172.21.15.27:0/118306790 shutdown_connections 2024-04-26T23:01:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.714+0000 7f4a86c00640 1 --2- 172.21.15.27:0/118306790 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f4a680a3d30 0x7f4a680a61f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.714+0000 7f4a86c00640 1 --2- 172.21.15.27:0/118306790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a80107650 0x7f4a8010fa20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.714+0000 7f4a86c00640 1 --2- 172.21.15.27:0/118306790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a80102000 0x7f4a8010f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.714+0000 7f4a86c00640 1 --2- 172.21.15.27:0/118306790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a80101640 0x7f4a8010efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:01:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.714+0000 7f4a86c00640 1 -- 172.21.15.27:0/118306790 >> 172.21.15.27:0/118306790 conn(0x7f4a800fd110 msgr2=0x7f4a800fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:01:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.714+0000 7f4a86c00640 1 -- 172.21.15.27:0/118306790 shutdown_connections 2024-04-26T23:01:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:01:47.714+0000 7f4a86c00640 1 -- 172.21.15.27:0/118306790 wait complete. 2024-04-26T23:01:49.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:49 smithi027 ceph-mon[28034]: pgmap v99: 129 pgs: 129 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 133 MiB/s rd, 156 MiB/s wr, 572 op/s 2024-04-26T23:01:49.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:49 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:01:49.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:49 smithi080 ceph-mon[32324]: pgmap v99: 129 pgs: 129 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 133 MiB/s rd, 156 MiB/s wr, 572 op/s 2024-04-26T23:01:49.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:49 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:01:49.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:49 smithi094 ceph-mon[32889]: pgmap v99: 129 pgs: 129 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 133 MiB/s rd, 156 MiB/s wr, 572 op/s 2024-04-26T23:01:49.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:49 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:01:50.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:50 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:01:50.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:50 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:01:50.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:50 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:01:50.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:50 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:01:50.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:50 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:01:50.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:50 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:01:50.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:50 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:01:50.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:50 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:01:50.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:50 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:01:51.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:51 smithi027 ceph-mon[28034]: pgmap v100: 129 pgs: 129 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s rd, 91 MiB/s wr, 162 op/s 2024-04-26T23:01:51.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:51 smithi080 ceph-mon[32324]: pgmap v100: 129 pgs: 129 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s rd, 91 MiB/s wr, 162 op/s 2024-04-26T23:01:51.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:51 smithi094 ceph-mon[32889]: pgmap v100: 129 pgs: 129 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s rd, 91 MiB/s wr, 162 op/s 2024-04-26T23:01:51.666 INFO:tasks.workunit.client.0.smithi027.stdout:6/952: rmdir d1/d30/d12c/da8/da1 39 2024-04-26T23:01:51.667 INFO:tasks.workunit.client.0.smithi027.stdout:6/953: write d1/df/f31 [2790059,120794] 0 2024-04-26T23:01:51.668 INFO:tasks.workunit.client.0.smithi027.stdout:6/954: symlink d1/de/d57/de3/l12d 0 2024-04-26T23:01:51.670 INFO:tasks.workunit.client.0.smithi027.stdout:6/955: truncate d1/df/d49/f86 320605 0 2024-04-26T23:01:51.670 INFO:tasks.workunit.client.0.smithi027.stdout:6/956: stat d1/df/f31 0 2024-04-26T23:01:52.255 INFO:tasks.workunit.client.0.smithi027.stdout:6/957: sync 2024-04-26T23:01:52.305 INFO:tasks.workunit.client.0.smithi027.stdout:6/958: dwrite d1/de/d57/d8e/dd0/f8d [0,4194304] 0 2024-04-26T23:01:52.343 INFO:tasks.workunit.client.0.smithi027.stdout:6/959: dread d1/f2 [0,4194304] 0 2024-04-26T23:01:52.351 INFO:tasks.workunit.client.0.smithi027.stdout:6/960: creat d1/d30/d40/f12e x:0 0 0 2024-04-26T23:01:52.390 INFO:tasks.workunit.client.0.smithi027.stdout:6/961: dread d1/de/df3/d6f/d7d/f80 [0,4194304] 0 2024-04-26T23:01:52.390 INFO:tasks.workunit.client.0.smithi027.stdout:6/962: stat d1/d30/d12c/faa 0 2024-04-26T23:01:52.390 INFO:tasks.workunit.client.0.smithi027.stdout:6/963: read - d1/de/d57/d8e/f128 zero size 2024-04-26T23:01:52.390 INFO:tasks.workunit.client.0.smithi027.stdout:6/964: stat d1/de/d57/d8e/cdd 0 2024-04-26T23:01:52.390 INFO:tasks.workunit.client.0.smithi027.stdout:6/965: chown d1/cff 24 1 2024-04-26T23:01:52.393 INFO:tasks.workunit.client.0.smithi027.stdout:6/966: symlink d1/de/d57/de3/l12f 0 2024-04-26T23:01:52.396 INFO:tasks.workunit.client.0.smithi027.stdout:6/967: mkdir d1/d6a/d130 0 2024-04-26T23:01:52.437 INFO:tasks.workunit.client.0.smithi027.stdout:6/968: dread d1/df/d3c/f75 [0,4194304] 0 2024-04-26T23:01:52.470 INFO:tasks.workunit.client.0.smithi027.stdout:6/969: write d1/f72 [3039384,63671] 0 2024-04-26T23:01:52.487 INFO:tasks.workunit.client.0.smithi027.stdout:6/970: dread d1/de/df3/d6f/d7d/d12a/d6b/f93 [0,4194304] 0 2024-04-26T23:01:52.487 INFO:tasks.workunit.client.0.smithi027.stdout:6/971: chown d1/de/df3/d6f/d7d/d12a/d6b/f8b 93901352 1 2024-04-26T23:01:52.496 INFO:tasks.workunit.client.0.smithi027.stdout:6/972: creat d1/df/d49/f131 x:0 0 0 2024-04-26T23:01:52.502 INFO:tasks.workunit.client.0.smithi027.stdout:6/973: mknod d1/de/d57/c132 0 2024-04-26T23:01:52.502 INFO:tasks.workunit.client.0.smithi027.stdout:6/974: stat d1/de/f48 0 2024-04-26T23:01:52.507 INFO:tasks.workunit.client.0.smithi027.stdout:6/975: truncate d1/d30/d12c/f5d 198211 0 2024-04-26T23:01:52.507 INFO:tasks.workunit.client.0.smithi027.stdout:6/976: stat d1/de/d57/d8e 0 2024-04-26T23:01:52.507 INFO:tasks.workunit.client.0.smithi027.stdout:6/977: dread - d1/de/df3/d122/f117 zero size 2024-04-26T23:01:52.512 INFO:tasks.workunit.client.0.smithi027.stdout:6/978: rename d1/c90 to d1/de/c133 0 2024-04-26T23:01:52.512 INFO:tasks.workunit.client.0.smithi027.stdout:6/979: readlink d1/df/dfe/l116 0 2024-04-26T23:01:52.512 INFO:tasks.workunit.client.0.smithi027.stdout:6/980: fdatasync d1/de/d57/d8e/dd0/f8a 0 2024-04-26T23:01:52.545 INFO:tasks.workunit.client.0.smithi027.stdout:6/981: dread d1/f59 [0,4194304] 0 2024-04-26T23:01:52.550 INFO:tasks.workunit.client.0.smithi027.stdout:6/982: truncate d1/d30/d12c/f55 1143291 0 2024-04-26T23:01:52.553 INFO:tasks.workunit.client.0.smithi027.stdout:6/983: creat d1/de/df3/d6f/d7d/d12a/d101/f134 x:0 0 0 2024-04-26T23:01:52.600 INFO:tasks.workunit.client.0.smithi027.stdout:6/984: dread d1/de/d57/d8e/dd0/f103 [0,4194304] 0 2024-04-26T23:01:52.603 INFO:tasks.workunit.client.0.smithi027.stdout:6/985: creat d1/df/d49/f135 x:0 0 0 2024-04-26T23:01:52.640 INFO:tasks.workunit.client.0.smithi027.stdout:6/986: dread d1/df/f39 [0,4194304] 0 2024-04-26T23:01:52.645 INFO:tasks.workunit.client.0.smithi027.stdout:6/987: rename d1/cff to d1/de/d57/d8e/d102/c136 0 2024-04-26T23:01:52.645 INFO:tasks.workunit.client.0.smithi027.stdout:6/988: chown d1/df/dfe 6 1 2024-04-26T23:01:52.649 INFO:tasks.workunit.client.0.smithi027.stdout:6/989: rmdir d1/de/df3/d6f/d7d/d12a/d6b 39 2024-04-26T23:01:52.649 INFO:tasks.workunit.client.0.smithi027.stdout:6/990: fsync d1/de/f48 0 2024-04-26T23:01:52.652 INFO:tasks.workunit.client.0.smithi027.stdout:6/991: creat d1/d6a/f137 x:0 0 0 2024-04-26T23:01:52.652 INFO:tasks.workunit.client.0.smithi027.stdout:6/992: truncate d1/d30/d40/f12e 486758 0 2024-04-26T23:01:52.715 INFO:tasks.workunit.client.0.smithi027.stdout:6/993: dwrite d1/df/d3c/d9a/ff6 [4194304,4194304] 0 2024-04-26T23:01:52.780 INFO:tasks.workunit.client.0.smithi027.stdout:6/994: dwrite d1/de/fe1 [0,4194304] 0 2024-04-26T23:01:52.792 INFO:tasks.workunit.client.0.smithi027.stdout:6/995: write d1/de/f82 [821332,75757] 0 2024-04-26T23:01:52.879 INFO:tasks.workunit.client.0.smithi027.stdout:6/996: dwrite d1/de/d57/d8e/f128 [0,4194304] 0 2024-04-26T23:01:52.989 INFO:tasks.workunit.client.0.smithi027.stdout:6/997: dwrite d1/d30/d12c/f55 [0,4194304] 0 2024-04-26T23:01:52.989 INFO:tasks.workunit.client.0.smithi027.stdout:6/998: stat d1/f2 0 2024-04-26T23:01:52.994 INFO:tasks.workunit.client.0.smithi027.stdout:6/999: symlink d1/de/df3/d6f/d7d/d12a/d101/l138 0 2024-04-26T23:01:53.002 INFO:tasks.workunit.client.0.smithi027.stderr:+ rm -rf -- ./tmp.l3JEpWWxh6 2024-04-26T23:01:53.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:53 smithi027 ceph-mon[28034]: pgmap v101: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s rd, 91 MiB/s wr, 278 op/s 2024-04-26T23:01:53.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:53 smithi080 ceph-mon[32324]: pgmap v101: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s rd, 91 MiB/s wr, 278 op/s 2024-04-26T23:01:53.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:53 smithi094 ceph-mon[32889]: pgmap v101: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s rd, 91 MiB/s wr, 278 op/s 2024-04-26T23:01:54.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:54 smithi027 ceph-mon[28034]: pgmap v102: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 159 op/s 2024-04-26T23:01:54.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:54 smithi080 ceph-mon[32324]: pgmap v102: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 159 op/s 2024-04-26T23:01:54.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:54 smithi094 ceph-mon[32889]: pgmap v102: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 159 op/s 2024-04-26T23:01:57.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:56 smithi080 ceph-mon[32324]: pgmap v103: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 159 op/s 2024-04-26T23:01:57.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:56 smithi094 ceph-mon[32889]: pgmap v103: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 159 op/s 2024-04-26T23:01:57.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:56 smithi027 ceph-mon[28034]: pgmap v103: 129 pgs: 129 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 159 op/s 2024-04-26T23:01:59.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:01:58 smithi080 ceph-mon[32324]: pgmap v104: 129 pgs: 129 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 191 op/s 2024-04-26T23:01:59.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:01:58 smithi094 ceph-mon[32889]: pgmap v104: 129 pgs: 129 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 191 op/s 2024-04-26T23:01:59.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:01:58 smithi027 ceph-mon[28034]: pgmap v104: 129 pgs: 129 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 191 op/s 2024-04-26T23:02:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.870+0000 7f43d7e00640 1 -- 172.21.15.27:0/3764651069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d8103910 msgr2=0x7f43d8103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.870+0000 7f43d7e00640 1 --2- 172.21.15.27:0/3764651069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d8103910 0x7f43d8103cf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f43c8009900 tx=0x7f43c802a480 comp rx=0 tx=0).stop 2024-04-26T23:02:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.870+0000 7f43d7e00640 1 -- 172.21.15.27:0/3764651069 shutdown_connections 2024-04-26T23:02:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.870+0000 7f43d7e00640 1 --2- 172.21.15.27:0/3764651069 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43d8104230 0x7f43d810c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.870+0000 7f43d7e00640 1 --2- 172.21.15.27:0/3764651069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43d810cd30 0x7f43d810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.870+0000 7f43d7e00640 1 --2- 172.21.15.27:0/3764651069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d8103910 0x7f43d8103cf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.870+0000 7f43d7e00640 1 -- 172.21.15.27:0/3764651069 >> 172.21.15.27:0/3764651069 conn(0x7f43d80fd110 msgr2=0x7f43d80ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.870+0000 7f43d7e00640 1 -- 172.21.15.27:0/3764651069 shutdown_connections 2024-04-26T23:02:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.870+0000 7f43d7e00640 1 -- 172.21.15.27:0/3764651069 wait complete. 2024-04-26T23:02:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.871+0000 7f43d7e00640 1 Processor -- start 2024-04-26T23:02:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.872+0000 7f43d7e00640 1 -- start start 2024-04-26T23:02:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.872+0000 7f43d7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43d8103910 0x7f43d819b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.872+0000 7f43d7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43d8104230 0x7f43d819b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.872+0000 7f43d7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d810cd30 0x7f43d819fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.872+0000 7f43d7e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43d8112a30 con 0x7f43d810cd30 2024-04-26T23:02:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.872+0000 7f43d7e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f43d81128b0 con 0x7f43d8103910 2024-04-26T23:02:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.872+0000 7f43d7e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f43d8112bb0 con 0x7f43d8104230 2024-04-26T23:02:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.873+0000 7f43d6a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43d8103910 0x7f43d819b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.873+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43d8104230 0x7f43d819b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.873+0000 7f43d6a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43d8103910 0x7f43d819b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:46362/0 (socket says 172.21.15.27:46362) 2024-04-26T23:02:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.873+0000 7f43d6a00640 1 -- 172.21.15.27:0/1308980269 learned_addr learned my addr 172.21.15.27:0/1308980269 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.873+0000 7f43d7400640 1 --2- 172.21.15.27:0/1308980269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d810cd30 0x7f43d819fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.873+0000 7f43d6a00640 1 -- 172.21.15.27:0/1308980269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43d8104230 msgr2=0x7f43d819b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.873+0000 7f43d6a00640 1 --2- 172.21.15.27:0/1308980269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43d8104230 0x7f43d819b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.873+0000 7f43d6a00640 1 -- 172.21.15.27:0/1308980269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d810cd30 msgr2=0x7f43d819fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.873+0000 7f43d6a00640 1 --2- 172.21.15.27:0/1308980269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d810cd30 0x7f43d819fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.873+0000 7f43d6a00640 1 -- 172.21.15.27:0/1308980269 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d81a0130 con 0x7f43d8103910 2024-04-26T23:02:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.873+0000 7f43cfe00640 1 --2- 172.21.15.27:0/1308980269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43d8104230 0x7f43d819b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:02:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.873+0000 7f43d6a00640 1 --2- 172.21.15.27:0/1308980269 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43d8103910 0x7f43d819b190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f43c8005ab0 tx=0x7f43c8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.874+0000 7f43cf400640 1 -- 172.21.15.27:0/1308980269 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c803f070 con 0x7f43d8103910 2024-04-26T23:02:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.874+0000 7f43cf400640 1 -- 172.21.15.27:0/1308980269 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43c8031b60 con 0x7f43d8103910 2024-04-26T23:02:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.874+0000 7f43d7e00640 1 -- 172.21.15.27:0/1308980269 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d81a03c0 con 0x7f43d8103910 2024-04-26T23:02:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.874+0000 7f43cf400640 1 -- 172.21.15.27:0/1308980269 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8035030 con 0x7f43d8103910 2024-04-26T23:02:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.874+0000 7f43d7e00640 1 -- 172.21.15.27:0/1308980269 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d81a7cb0 con 0x7f43d8103910 2024-04-26T23:02:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.877+0000 7f43e0572740 1 Processor -- start 2024-04-26T23:02:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.877+0000 7f43cf400640 1 -- 172.21.15.27:0/1308980269 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f43c8042070 con 0x7f43d8103910 2024-04-26T23:02:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.877+0000 7f43e0572740 1 -- start start 2024-04-26T23:02:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.878+0000 7f43e0572740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32e6fe0 0x5595b32e73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.878+0000 7f43e0572740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 0x5595b32f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.878+0000 7f43e0572740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32f21c0 0x5595b32f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.878+0000 7f43e0572740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5595b324c380 con 0x5595b32e7900 2024-04-26T23:02:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.878+0000 7f43e0572740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5595b31e8c30 con 0x5595b32f21c0 2024-04-26T23:02:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.878+0000 7f43e0572740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5595b3248340 con 0x5595b32e6fe0 2024-04-26T23:02:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.878+0000 7f43d7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32e6fe0 0x5595b32e73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.878+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32f21c0 0x5595b32f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.878+0000 7f43d7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32e6fe0 0x5595b32e73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:34572/0 (socket says 172.21.15.27:34572) 2024-04-26T23:02:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.878+0000 7f43d7400640 1 -- 172.21.15.27:0/2444332283 learned_addr learned my addr 172.21.15.27:0/2444332283 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.878+0000 7f43d6a00640 1 --2- 172.21.15.27:0/2444332283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 0x5595b32f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.879+0000 7f43d7400640 1 -- 172.21.15.27:0/2444332283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32f21c0 msgr2=0x5595b32f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.879+0000 7f43d7400640 1 --2- 172.21.15.27:0/2444332283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32f21c0 0x5595b32f45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.879+0000 7f43d7400640 1 -- 172.21.15.27:0/2444332283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 msgr2=0x5595b32f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.879+0000 7f43d7400640 1 --2- 172.21.15.27:0/2444332283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 0x5595b32f1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.879+0000 7f43d7400640 1 -- 172.21.15.27:0/2444332283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595b320a940 con 0x5595b32e6fe0 2024-04-26T23:02:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.879+0000 7f43d6a00640 1 --2- 172.21.15.27:0/2444332283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 0x5595b32f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:02:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.879+0000 7f43d7400640 1 --2- 172.21.15.27:0/2444332283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32e6fe0 0x5595b32e73c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009f50 tx=0x7f43d00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca1d6ffe7cd4fae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.879+0000 7f43c7400640 1 -- 172.21.15.27:0/2444332283 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0004d20 con 0x5595b32e6fe0 2024-04-26T23:02:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.879+0000 7f43c7400640 1 -- 172.21.15.27:0/2444332283 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d0004050 con 0x5595b32e6fe0 2024-04-26T23:02:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.879+0000 7f43c7400640 1 -- 172.21.15.27:0/2444332283 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0005280 con 0x5595b32e6fe0 2024-04-26T23:02:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.879+0000 7f43cf400640 1 --2- 172.21.15.27:0/1308980269 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f43bc0a3d00 0x7f43bc0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.880+0000 7f43cf400640 1 -- 172.21.15.27:0/1308980269 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f43c80f1280 con 0x7f43d8103910 2024-04-26T23:02:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.880+0000 7f43d6a00640 1 --2- 172.21.15.27:0/1308980269 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f43bc0a3d00 0x7f43bc0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.880+0000 7f43e0572740 1 -- 172.21.15.27:0/2444332283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32e6fe0 msgr2=0x5595b32e73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.880+0000 7f43e0572740 1 --2- 172.21.15.27:0/2444332283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32e6fe0 0x5595b32e73c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009f50 tx=0x7f43d00118d0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.880+0000 7f43e0572740 1 -- 172.21.15.27:0/2444332283 shutdown_connections 2024-04-26T23:02:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.880+0000 7f43e0572740 1 --2- 172.21.15.27:0/2444332283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32f21c0 0x5595b32f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.880+0000 7f43e0572740 1 --2- 172.21.15.27:0/2444332283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 0x5595b32f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.880+0000 7f43e0572740 1 --2- 172.21.15.27:0/2444332283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32e6fe0 0x5595b32e73c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.880+0000 7f43e0572740 1 -- 172.21.15.27:0/2444332283 >> 172.21.15.27:0/2444332283 conn(0x5595b32e34f0 msgr2=0x5595b32e38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.881+0000 7f43e0572740 1 -- 172.21.15.27:0/2444332283 shutdown_connections 2024-04-26T23:02:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.881+0000 7f43e0572740 1 -- 172.21.15.27:0/2444332283 wait complete. 2024-04-26T23:02:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.881+0000 7f43d6a00640 1 --2- 172.21.15.27:0/1308980269 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f43bc0a3d00 0x7f43bc0a61c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f43c80ee640 tx=0x7f43c8009260 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.881+0000 7f43e0572740 1 -- start start 2024-04-26T23:02:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.882+0000 7f43e0572740 1 Processor -- start 2024-04-26T23:02:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.882+0000 7f43e0572740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32e6fe0 0x5595b32ef100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.882+0000 7f43e0572740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 0x5595b32ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.882+0000 7f43e0572740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32f21c0 0x5595b32efb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.882+0000 7f43e0572740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5595b32f4df0 con 0x5595b32e7900 2024-04-26T23:02:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.882+0000 7f43e0572740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5595b31d3c60 con 0x5595b32e6fe0 2024-04-26T23:02:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.882+0000 7f43e0572740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5595b32f4f70 con 0x5595b32f21c0 2024-04-26T23:02:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.882+0000 7f43e0572740 10 client.0 _ll_register_callbacks cb 0x5595b31ffe50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:02:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.882+0000 7f43d7400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32e6fe0 0x5595b32ef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.882+0000 7f43d7400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32e6fe0 0x5595b32ef100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:46378/0 (socket says 172.21.15.27:46378) 2024-04-26T23:02:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.882+0000 7f43d7400640 1 -- 172.21.15.27:0/1437657914 learned_addr learned my addr 172.21.15.27:0/1437657914 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.883+0000 7f43d7400640 1 --2- 172.21.15.27:0/1437657914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32f21c0 0x5595b32efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.883+0000 7f43cfe00640 1 --2- 172.21.15.27:0/1437657914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 0x5595b32ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.883+0000 7f43d7400640 1 -- 172.21.15.27:0/1437657914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32f21c0 msgr2=0x5595b32efb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.883+0000 7f43d7400640 1 --2- 172.21.15.27:0/1437657914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32f21c0 0x5595b32efb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.883+0000 7f43d7400640 1 -- 172.21.15.27:0/1437657914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 msgr2=0x5595b32ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.883+0000 7f43d7400640 1 --2- 172.21.15.27:0/1437657914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 0x5595b32ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.883+0000 7f43d7400640 1 -- 172.21.15.27:0/1437657914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595b31e2510 con 0x5595b32e6fe0 2024-04-26T23:02:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.883+0000 7f43cfe00640 1 --2- 172.21.15.27:0/1437657914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 0x5595b32ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:02:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.883+0000 7f43d7400640 1 --2- 172.21.15.27:0/1437657914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32e6fe0 0x5595b32ef100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f43d0005c40 tx=0x7f43d0005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=954c88f352369593 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.883+0000 7f43bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T23:02:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.884+0000 7f43bbe00640 1 -- 172.21.15.27:0/1437657914 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d002d060 con 0x5595b32e6fe0 2024-04-26T23:02:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.884+0000 7f43bbe00640 1 -- 172.21.15.27:0/1437657914 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f43d0004c40 con 0x5595b32e6fe0 2024-04-26T23:02:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.884+0000 7f43e0572740 1 -- 172.21.15.27:0/1437657914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43d0004050 con 0x5595b32e6fe0 2024-04-26T23:02:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.884+0000 7f43bbe00640 1 -- 172.21.15.27:0/1437657914 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f43d0028450 con 0x5595b32e6fe0 2024-04-26T23:02:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.884+0000 7f43e0572740 10 client.25718 fetch_fsmap learned FSMap version 28 2024-04-26T23:02:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.884+0000 7f43e0572740 1 -- 172.21.15.27:0/1437657914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595b2f5e210 con 0x5595b32e6fe0 2024-04-26T23:02:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.884+0000 7f43bbe00640 1 -- 172.21.15.27:0/1437657914 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d0004050 con 0x5595b32e6fe0 2024-04-26T23:02:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.884+0000 7f43bbe00640 1 -- 172.21.15.27:0/1437657914 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f43d000f680 con 0x5595b32e6fe0 2024-04-26T23:02:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.885+0000 7f43e0572740 10 client.25718 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:02:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.885+0000 7f43e0572740 10 client.25718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:02:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.885+0000 7f43e0572740 20 client.25718 populate_metadata read hostname 'smithi027' 2024-04-26T23:02:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.885+0000 7f43e0572740 1 --2- 172.21.15.27:0/1437657914 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5595b32f8df0 0x5595b32fb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.885+0000 7f43e0572740 4 client.25718 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:02:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.885+0000 7f43e0572740 1 -- 172.21.15.27:0/1437657914 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595b3259f90 con 0x5595b32f8df0 2024-04-26T23:02:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.885+0000 7f43cfe00640 1 --2- 172.21.15.27:0/1437657914 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5595b32f8df0 0x5595b32fb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.885+0000 7f43cfe00640 1 --2- 172.21.15.27:0/1437657914 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5595b32f8df0 0x5595b32fb250 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78f26054fd5d1be0 server_cookie=209545b5cdd370b in_seq=0 out_seq=0 2024-04-26T23:02:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.885+0000 7f43bbe00640 10 client.25718 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.886+0000 7f43bbe00640 1 -- 172.21.15.27:0/1437657914 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595b3259f90 con 0x5595b32f8df0 2024-04-26T23:02:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.886+0000 7f43bbe00640 10 client.25718 handle_command_reply: tid=0 2024-04-26T23:02:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.887+0000 7f43e0572740 1 client.25718 shutdown 2024-04-26T23:02:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.887+0000 7f43e0572740 1 -- 172.21.15.27:0/1437657914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32e6fe0 msgr2=0x5595b32ef100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.887+0000 7f43e0572740 1 --2- 172.21.15.27:0/1437657914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32e6fe0 0x5595b32ef100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f43d0005c40 tx=0x7f43d0005bd0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.887+0000 7f43e0572740 1 -- 172.21.15.27:0/1437657914 shutdown_connections 2024-04-26T23:02:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.887+0000 7f43e0572740 1 --2- 172.21.15.27:0/1437657914 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5595b32f8df0 0x5595b32fb250 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.887+0000 7f43e0572740 1 --2- 172.21.15.27:0/1437657914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32f21c0 0x5595b32efb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.887+0000 7f43e0572740 1 --2- 172.21.15.27:0/1437657914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 0x5595b32ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.887+0000 7f43e0572740 1 --2- 172.21.15.27:0/1437657914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32e6fe0 0x5595b32ef100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.887+0000 7f43e0572740 1 -- 172.21.15.27:0/1437657914 >> 172.21.15.27:0/1437657914 conn(0x5595b32e34f0 msgr2=0x5595b32e9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.887+0000 7f43bbe00640 0 client.25718 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.887+0000 7f43e0572740 1 -- 172.21.15.27:0/1437657914 shutdown_connections 2024-04-26T23:02:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.887+0000 7f43e0572740 1 -- 172.21.15.27:0/1437657914 wait complete. 2024-04-26T23:02:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.887+0000 7f43e0572740 20 client.25718 trim_cache size 0 max 16384 2024-04-26T23:02:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.905+0000 7f43e0572740 1 Processor -- start 2024-04-26T23:02:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.905+0000 7f43e0572740 1 -- start start 2024-04-26T23:02:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.905+0000 7f43e0572740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32e6fe0 0x5595b33064a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.905+0000 7f43e0572740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 0x5595b32ee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.905+0000 7f43e0572740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32f21c0 0x5595b331ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.905+0000 7f43e0572740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43d81b0490 con 0x5595b32e7900 2024-04-26T23:02:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.905+0000 7f43e0572740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f43d81add00 con 0x5595b32f21c0 2024-04-26T23:02:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.905+0000 7f43e0572740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f43d81abbb0 con 0x5595b32e6fe0 2024-04-26T23:02:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.906+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 0x5595b32ee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.906+0000 7f43d7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32e6fe0 0x5595b33064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.906+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 0x5595b32ee0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46990/0 (socket says 172.21.15.27:46990) 2024-04-26T23:02:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.906+0000 7f43cfe00640 1 -- 172.21.15.27:0/3708233184 learned_addr learned my addr 172.21.15.27:0/3708233184 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.906+0000 7f43d7400640 1 --2- 172.21.15.27:0/3708233184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32f21c0 0x5595b331ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.906+0000 7f43cfe00640 1 -- 172.21.15.27:0/3708233184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32e6fe0 msgr2=0x5595b33064a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.906+0000 7f43cfe00640 1 --2- 172.21.15.27:0/3708233184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32e6fe0 0x5595b33064a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.906+0000 7f43cfe00640 1 -- 172.21.15.27:0/3708233184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32f21c0 msgr2=0x5595b331ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.906+0000 7f43cfe00640 1 --2- 172.21.15.27:0/3708233184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32f21c0 0x5595b331ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.906+0000 7f43cfe00640 1 -- 172.21.15.27:0/3708233184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595b302a620 con 0x5595b32e7900 2024-04-26T23:02:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.906+0000 7f43d7400640 1 --2- 172.21.15.27:0/3708233184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32f21c0 0x5595b331ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:02:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.907+0000 7f43cfe00640 1 --2- 172.21.15.27:0/3708233184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 0x5595b32ee0e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x5595b32e4680 tx=0x7f43c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ee46811cf6c585 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.907+0000 7f43c6a00640 1 -- 172.21.15.27:0/3708233184 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c00122d0 con 0x5595b32e7900 2024-04-26T23:02:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.907+0000 7f43c6a00640 1 -- 172.21.15.27:0/3708233184 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43c000fda0 con 0x5595b32e7900 2024-04-26T23:02:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.907+0000 7f43c6a00640 1 -- 172.21.15.27:0/3708233184 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c00047a0 con 0x5595b32e7900 2024-04-26T23:02:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.908+0000 7f43e0572740 1 -- 172.21.15.27:0/3708233184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 msgr2=0x5595b32ee0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.908+0000 7f43e0572740 1 --2- 172.21.15.27:0/3708233184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 0x5595b32ee0e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x5595b32e4680 tx=0x7f43c00046c0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.908+0000 7f43e0572740 1 -- 172.21.15.27:0/3708233184 shutdown_connections 2024-04-26T23:02:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.908+0000 7f43e0572740 1 --2- 172.21.15.27:0/3708233184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32f21c0 0x5595b331ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.908+0000 7f43e0572740 1 --2- 172.21.15.27:0/3708233184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e7900 0x5595b32ee0e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.908+0000 7f43e0572740 1 --2- 172.21.15.27:0/3708233184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32e6fe0 0x5595b33064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.908+0000 7f43e0572740 1 -- 172.21.15.27:0/3708233184 >> 172.21.15.27:0/3708233184 conn(0x5595b32e34f0 msgr2=0x5595b32ed9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.908+0000 7f43e0572740 1 -- 172.21.15.27:0/3708233184 shutdown_connections 2024-04-26T23:02:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.908+0000 7f43e0572740 1 -- 172.21.15.27:0/3708233184 wait complete. 2024-04-26T23:02:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.908+0000 7f43e0572740 1 -- start start 2024-04-26T23:02:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.909+0000 7f43e0572740 1 Processor -- start 2024-04-26T23:02:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.909+0000 7f43e0572740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e6fe0 0x5595b32ece20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.909+0000 7f43e0572740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32e7900 0x5595b3300910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.909+0000 7f43e0572740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32f21c0 0x5595b32eb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.909+0000 7f43e0572740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43d81ac200 con 0x5595b32e6fe0 2024-04-26T23:02:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.909+0000 7f43e0572740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f43d81ad630 con 0x5595b32e7900 2024-04-26T23:02:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.909+0000 7f43e0572740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f43d81ac890 con 0x5595b32f21c0 2024-04-26T23:02:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.909+0000 7f43e0572740 10 client.0 _ll_register_callbacks cb 0x7f43c0009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:02:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.909+0000 7f43d7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e6fe0 0x5595b32ece20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.909+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32e7900 0x5595b3300910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.910+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32e7900 0x5595b3300910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:46398/0 (socket says 172.21.15.27:46398) 2024-04-26T23:02:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.910+0000 7f43cfe00640 1 -- 172.21.15.27:0/2656441581 learned_addr learned my addr 172.21.15.27:0/2656441581 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.910+0000 7f43d7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e6fe0 0x5595b32ece20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47002/0 (socket says 172.21.15.27:47002) 2024-04-26T23:02:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.910+0000 7f43d7400640 1 --2- 172.21.15.27:0/2656441581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32f21c0 0x5595b32eb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.910+0000 7f43d7400640 1 -- 172.21.15.27:0/2656441581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32f21c0 msgr2=0x5595b32eb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.910+0000 7f43d7400640 1 --2- 172.21.15.27:0/2656441581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32f21c0 0x5595b32eb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.910+0000 7f43d7400640 1 -- 172.21.15.27:0/2656441581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32e7900 msgr2=0x5595b3300910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.910+0000 7f43d7400640 1 --2- 172.21.15.27:0/2656441581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32e7900 0x5595b3300910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.910+0000 7f43d7400640 1 -- 172.21.15.27:0/2656441581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595b3206cb0 con 0x5595b32e6fe0 2024-04-26T23:02:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.910+0000 7f43cfe00640 1 --2- 172.21.15.27:0/2656441581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32e7900 0x5595b3300910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:02:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.910+0000 7f43d7400640 1 --2- 172.21.15.27:0/2656441581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e6fe0 0x5595b32ece20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f43d0012fd0 tx=0x7f43d0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4735bcb31651cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.910+0000 7f43bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T23:02:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.911+0000 7f43bbe00640 1 -- 172.21.15.27:0/2656441581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d002d040 con 0x5595b32e6fe0 2024-04-26T23:02:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.911+0000 7f43bbe00640 1 -- 172.21.15.27:0/2656441581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f43d000f680 con 0x5595b32e6fe0 2024-04-26T23:02:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.911+0000 7f43e0572740 1 -- 172.21.15.27:0/2656441581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43c000fda0 con 0x5595b32e6fe0 2024-04-26T23:02:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.911+0000 7f43bbe00640 1 -- 172.21.15.27:0/2656441581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f43d00048d0 con 0x5595b32e6fe0 2024-04-26T23:02:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.911+0000 7f43e0572740 10 client.15588 fetch_fsmap learned FSMap version 28 2024-04-26T23:02:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.911+0000 7f43e0572740 1 -- 172.21.15.27:0/2656441581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595b3378760 con 0x5595b32e6fe0 2024-04-26T23:02:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.911+0000 7f43bbe00640 1 -- 172.21.15.27:0/2656441581 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43c000fda0 con 0x5595b32e6fe0 2024-04-26T23:02:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.912+0000 7f43bbe00640 1 -- 172.21.15.27:0/2656441581 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f43d0028940 con 0x5595b32e6fe0 2024-04-26T23:02:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.912+0000 7f43e0572740 10 client.15588 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:02:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.912+0000 7f43e0572740 10 client.15588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:02:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.912+0000 7f43e0572740 20 client.15588 populate_metadata read hostname 'smithi027' 2024-04-26T23:02:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.912+0000 7f43e0572740 1 --2- 172.21.15.27:0/2656441581 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5595b3378ef0 0x5595b337b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.912+0000 7f43e0572740 4 client.15588 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:02:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.912+0000 7f43e0572740 1 -- 172.21.15.27:0/2656441581 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5595b3253e00 con 0x5595b3378ef0 2024-04-26T23:02:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.912+0000 7f43cfe00640 1 --2- 172.21.15.27:0/2656441581 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5595b3378ef0 0x5595b337b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.913+0000 7f43cfe00640 1 --2- 172.21.15.27:0/2656441581 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5595b3378ef0 0x5595b337b310 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef42dc82e02e8e6d server_cookie=1d8866e9d617dd96 in_seq=0 out_seq=0 2024-04-26T23:02:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.913+0000 7f43bbe00640 10 client.15588 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.995+0000 7f43bbe00640 1 -- 172.21.15.27:0/2656441581 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5595b3253e00 con 0x5595b3378ef0 2024-04-26T23:02:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.995+0000 7f43bbe00640 10 client.15588 handle_command_reply: tid=0 2024-04-26T23:02:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.995+0000 7f43e0572740 1 client.15588 shutdown 2024-04-26T23:02:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.995+0000 7f43e0572740 1 -- 172.21.15.27:0/2656441581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e6fe0 msgr2=0x5595b32ece20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.995+0000 7f43e0572740 1 --2- 172.21.15.27:0/2656441581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e6fe0 0x5595b32ece20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f43d0012fd0 tx=0x7f43d0014040 comp rx=0 tx=0).stop 2024-04-26T23:02:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.995+0000 7f43e0572740 1 -- 172.21.15.27:0/2656441581 shutdown_connections 2024-04-26T23:02:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.995+0000 7f43e0572740 1 --2- 172.21.15.27:0/2656441581 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5595b3378ef0 0x5595b337b310 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.995+0000 7f43e0572740 1 --2- 172.21.15.27:0/2656441581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5595b32e7900 0x5595b3300910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.995+0000 7f43e0572740 1 --2- 172.21.15.27:0/2656441581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595b32f21c0 0x5595b32eb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.995+0000 7f43e0572740 1 --2- 172.21.15.27:0/2656441581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595b32e6fe0 0x5595b32ece20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.995+0000 7f43e0572740 1 -- 172.21.15.27:0/2656441581 >> 172.21.15.27:0/2656441581 conn(0x5595b32e34f0 msgr2=0x5595b32ee260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.995+0000 7f43bbe00640 0 client.15588 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.996+0000 7f43e0572740 1 -- 172.21.15.27:0/2656441581 shutdown_connections 2024-04-26T23:02:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.996+0000 7f43e0572740 1 -- 172.21.15.27:0/2656441581 wait complete. 2024-04-26T23:02:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.996+0000 7f43e0572740 20 client.15588 trim_cache size 0 max 16384 2024-04-26T23:02:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T23:02:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T23:02:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T23:02:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T23:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.996+0000 7f43d7e00640 1 -- 172.21.15.27:0/1308980269 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f43bc0a3d00 msgr2=0x7f43bc0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.996+0000 7f43d7e00640 1 --2- 172.21.15.27:0/1308980269 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f43bc0a3d00 0x7f43bc0a61c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f43c80ee640 tx=0x7f43c8009260 comp rx=0 tx=0).stop 2024-04-26T23:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.996+0000 7f43d7e00640 1 -- 172.21.15.27:0/1308980269 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43d8103910 msgr2=0x7f43d819b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.996+0000 7f43d7e00640 1 --2- 172.21.15.27:0/1308980269 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43d8103910 0x7f43d819b190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f43c8005ab0 tx=0x7f43c8003c10 comp rx=0 tx=0).stop 2024-04-26T23:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.997+0000 7f43d7e00640 1 -- 172.21.15.27:0/1308980269 shutdown_connections 2024-04-26T23:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.997+0000 7f43d7e00640 1 --2- 172.21.15.27:0/1308980269 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f43bc0a3d00 0x7f43bc0a61c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.997+0000 7f43d7e00640 1 --2- 172.21.15.27:0/1308980269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d810cd30 0x7f43d819fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.997+0000 7f43d7e00640 1 --2- 172.21.15.27:0/1308980269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43d8104230 0x7f43d819b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.997+0000 7f43d7e00640 1 --2- 172.21.15.27:0/1308980269 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43d8103910 0x7f43d819b190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.997+0000 7f43d7e00640 1 -- 172.21.15.27:0/1308980269 >> 172.21.15.27:0/1308980269 conn(0x7f43d80fd110 msgr2=0x7f43d8101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.997+0000 7f43d7e00640 1 -- 172.21.15.27:0/1308980269 shutdown_connections 2024-04-26T23:02:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:00.997+0000 7f43d7e00640 1 -- 172.21.15.27:0/1308980269 wait complete. 2024-04-26T23:02:01.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:00 smithi080 ceph-mon[32324]: pgmap v105: 129 pgs: 129 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 147 op/s 2024-04-26T23:02:01.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:00 smithi094 ceph-mon[32889]: pgmap v105: 129 pgs: 129 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 147 op/s 2024-04-26T23:02:01.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:00 smithi027 ceph-mon[28034]: pgmap v105: 129 pgs: 129 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 147 op/s 2024-04-26T23:02:03.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:02 smithi080 ceph-mon[32324]: pgmap v106: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 257 op/s 2024-04-26T23:02:03.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:02 smithi094 ceph-mon[32889]: pgmap v106: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 257 op/s 2024-04-26T23:02:03.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:02 smithi027 ceph-mon[28034]: pgmap v106: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 257 op/s 2024-04-26T23:02:05.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:04 smithi080 ceph-mon[32324]: pgmap v107: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-04-26T23:02:05.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:04 smithi094 ceph-mon[32889]: pgmap v107: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-04-26T23:02:05.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:04 smithi027 ceph-mon[28034]: pgmap v107: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-04-26T23:02:07.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:06 smithi094 ceph-mon[32889]: pgmap v108: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-04-26T23:02:07.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:06 smithi094 ceph-mon[32889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:02:07.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:06 smithi027 ceph-mon[28034]: pgmap v108: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-04-26T23:02:07.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:06 smithi027 ceph-mon[28034]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:02:07.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:06 smithi080 ceph-mon[32324]: pgmap v108: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-04-26T23:02:07.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:06 smithi080 ceph-mon[32324]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:02:09.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:08 smithi094 ceph-mon[32889]: pgmap v109: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 220 op/s 2024-04-26T23:02:09.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:08 smithi027 ceph-mon[28034]: pgmap v109: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 220 op/s 2024-04-26T23:02:09.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:08 smithi080 ceph-mon[32324]: pgmap v109: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 220 op/s 2024-04-26T23:02:11.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:10 smithi027 ceph-mon[28034]: pgmap v110: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 662 KiB/s wr, 188 op/s 2024-04-26T23:02:11.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:10 smithi080 ceph-mon[32324]: pgmap v110: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 662 KiB/s wr, 188 op/s 2024-04-26T23:02:11.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:10 smithi094 ceph-mon[32889]: pgmap v110: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 662 KiB/s wr, 188 op/s 2024-04-26T23:02:12.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:11 smithi027 ceph-mon[28034]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:02:12.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:11 smithi080 ceph-mon[32324]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:02:12.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:11 smithi094 ceph-mon[32889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:02:13.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:12 smithi027 ceph-mon[28034]: pgmap v111: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1013 KiB/s wr, 225 op/s 2024-04-26T23:02:13.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:12 smithi080 ceph-mon[32324]: pgmap v111: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1013 KiB/s wr, 225 op/s 2024-04-26T23:02:13.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:12 smithi094 ceph-mon[32889]: pgmap v111: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1013 KiB/s wr, 225 op/s 2024-04-26T23:02:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.165+0000 7fdd97e00640 1 -- 172.21.15.27:0/534021048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd9810ee50 msgr2=0x7fdd9810f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.165+0000 7fdd97e00640 1 --2- 172.21.15.27:0/534021048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd9810ee50 0x7fdd9810f230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009900 tx=0x7fdd8802a480 comp rx=0 tx=0).stop 2024-04-26T23:02:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.165+0000 7fdd97e00640 1 -- 172.21.15.27:0/534021048 shutdown_connections 2024-04-26T23:02:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.165+0000 7fdd97e00640 1 --2- 172.21.15.27:0/534021048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd98109730 0x7fdd98115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.165+0000 7fdd97e00640 1 --2- 172.21.15.27:0/534021048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd98108d90 0x7fdd981091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.165+0000 7fdd97e00640 1 --2- 172.21.15.27:0/534021048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd9810ee50 0x7fdd9810f230 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.165+0000 7fdd97e00640 1 -- 172.21.15.27:0/534021048 >> 172.21.15.27:0/534021048 conn(0x7fdd980784e0 msgr2=0x7fdd9807a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.165+0000 7fdd97e00640 1 -- 172.21.15.27:0/534021048 shutdown_connections 2024-04-26T23:02:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.165+0000 7fdd97e00640 1 -- 172.21.15.27:0/534021048 wait complete. 2024-04-26T23:02:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.166+0000 7fdd97e00640 1 Processor -- start 2024-04-26T23:02:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.167+0000 7fdd97e00640 1 -- start start 2024-04-26T23:02:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.167+0000 7fdd97e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd98108d90 0x7fdd9819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.167+0000 7fdd97e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd98109730 0x7fdd9819fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.167+0000 7fdd97e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd9810ee50 0x7fdd981a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.167+0000 7fdd97e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98118fc0 con 0x7fdd98108d90 2024-04-26T23:02:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.167+0000 7fdd97e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98118e40 con 0x7fdd98109730 2024-04-26T23:02:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.167+0000 7fdd97e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98119140 con 0x7fdd9810ee50 2024-04-26T23:02:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.167+0000 7fdd96a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd98108d90 0x7fdd9819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.167+0000 7fdd96000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd98109730 0x7fdd9819fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.167+0000 7fdd97400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd9810ee50 0x7fdd981a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.167+0000 7fdd96a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd98108d90 0x7fdd9819f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55200/0 (socket says 172.21.15.27:55200) 2024-04-26T23:02:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.167+0000 7fdd96a00640 1 -- 172.21.15.27:0/3300547788 learned_addr learned my addr 172.21.15.27:0/3300547788 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.168+0000 7fdd97400640 1 -- 172.21.15.27:0/3300547788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd98109730 msgr2=0x7fdd9819fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.168+0000 7fdd97400640 1 --2- 172.21.15.27:0/3300547788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd98109730 0x7fdd9819fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.168+0000 7fdd97400640 1 -- 172.21.15.27:0/3300547788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd98108d90 msgr2=0x7fdd9819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.168+0000 7fdd97400640 1 --2- 172.21.15.27:0/3300547788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd98108d90 0x7fdd9819f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.168+0000 7fdd97400640 1 -- 172.21.15.27:0/3300547788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd981a44b0 con 0x7fdd9810ee50 2024-04-26T23:02:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.168+0000 7fdd96000640 1 --2- 172.21.15.27:0/3300547788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd98109730 0x7fdd9819fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:02:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.168+0000 7fdd97400640 1 --2- 172.21.15.27:0/3300547788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd9810ee50 0x7fdd981a3d80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009fd0 tx=0x7fdd9000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.169+0000 7fdd8f400640 1 -- 172.21.15.27:0/3300547788 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9000ce20 con 0x7fdd9810ee50 2024-04-26T23:02:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.169+0000 7fdd97e00640 1 -- 172.21.15.27:0/3300547788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd981a47a0 con 0x7fdd9810ee50 2024-04-26T23:02:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.169+0000 7fdd8f400640 1 -- 172.21.15.27:0/3300547788 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd90005020 con 0x7fdd9810ee50 2024-04-26T23:02:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.169+0000 7fdd8f400640 1 -- 172.21.15.27:0/3300547788 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd90010690 con 0x7fdd9810ee50 2024-04-26T23:02:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.169+0000 7fdd97e00640 1 -- 172.21.15.27:0/3300547788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd981ac030 con 0x7fdd9810ee50 2024-04-26T23:02:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.173+0000 7fdda0ba8740 1 Processor -- start 2024-04-26T23:02:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.174+0000 7fdd8f400640 1 -- 172.21.15.27:0/3300547788 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdd900054d0 con 0x7fdd9810ee50 2024-04-26T23:02:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.174+0000 7fdda0ba8740 1 -- start start 2024-04-26T23:02:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.175+0000 7fdda0ba8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c65fe0 0x55af81c663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.175+0000 7fdda0ba8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c66900 0x55af81c70c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.175+0000 7fdda0ba8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 0x55af81c735b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.175+0000 7fdda0ba8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55af81bcb380 con 0x55af81c66900 2024-04-26T23:02:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.175+0000 7fdda0ba8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55af81b67c30 con 0x55af81c65fe0 2024-04-26T23:02:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.175+0000 7fdda0ba8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55af81bc7340 con 0x55af81c711c0 2024-04-26T23:02:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.175+0000 7fdd96000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c65fe0 0x55af81c663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.175+0000 7fdd96000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c65fe0 0x55af81c663c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:37788/0 (socket says 172.21.15.27:37788) 2024-04-26T23:02:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.175+0000 7fdd96000640 1 -- 172.21.15.27:0/399710757 learned_addr learned my addr 172.21.15.27:0/399710757 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.175+0000 7fdd8f400640 1 --2- 172.21.15.27:0/3300547788 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fdd7c0a3d00 0x7fdd7c0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.175+0000 7fdd96000640 1 --2- 172.21.15.27:0/399710757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 0x55af81c735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.176+0000 7fdd8f400640 1 -- 172.21.15.27:0/3300547788 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fdd900d1310 con 0x7fdd9810ee50 2024-04-26T23:02:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.176+0000 7fdd96a00640 1 --2- 172.21.15.27:0/399710757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c66900 0x55af81c70c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.176+0000 7fdd96000640 1 -- 172.21.15.27:0/399710757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c65fe0 msgr2=0x55af81c663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.176+0000 7fdd96000640 1 --2- 172.21.15.27:0/399710757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c65fe0 0x55af81c663c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.176+0000 7fdd96000640 1 -- 172.21.15.27:0/399710757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c66900 msgr2=0x55af81c70c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.176+0000 7fdd96000640 1 --2- 172.21.15.27:0/399710757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c66900 0x55af81c70c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.176+0000 7fdd96000640 1 -- 172.21.15.27:0/399710757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af81b89940 con 0x55af81c711c0 2024-04-26T23:02:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.177+0000 7fdd96000640 1 --2- 172.21.15.27:0/399710757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 0x55af81c735b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdd84011be0 tx=0x7fdd84005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9294b93e79cae977 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.177+0000 7fdd97400640 1 --2- 172.21.15.27:0/3300547788 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fdd7c0a3d00 0x7fdd7c0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.177+0000 7fdd83e00640 1 -- 172.21.15.27:0/399710757 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84011cd0 con 0x55af81c711c0 2024-04-26T23:02:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.177+0000 7fdd83e00640 1 -- 172.21.15.27:0/399710757 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd840118a0 con 0x55af81c711c0 2024-04-26T23:02:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.177+0000 7fdd83e00640 1 -- 172.21.15.27:0/399710757 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400e3c0 con 0x55af81c711c0 2024-04-26T23:02:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.178+0000 7fdda0ba8740 1 -- 172.21.15.27:0/399710757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 msgr2=0x55af81c735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.178+0000 7fdd97400640 1 --2- 172.21.15.27:0/3300547788 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fdd7c0a3d00 0x7fdd7c0a61c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdd900992d0 tx=0x7fdd90005f70 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.178+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/399710757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 0x55af81c735b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdd84011be0 tx=0x7fdd84005f90 comp rx=0 tx=0).stop 2024-04-26T23:02:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.178+0000 7fdda0ba8740 1 -- 172.21.15.27:0/399710757 shutdown_connections 2024-04-26T23:02:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.178+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/399710757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 0x55af81c735b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.178+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/399710757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c66900 0x55af81c70c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.178+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/399710757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c65fe0 0x55af81c663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.178+0000 7fdda0ba8740 1 -- 172.21.15.27:0/399710757 >> 172.21.15.27:0/399710757 conn(0x55af81c624f0 msgr2=0x55af81c628d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.178+0000 7fdda0ba8740 1 -- 172.21.15.27:0/399710757 shutdown_connections 2024-04-26T23:02:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.178+0000 7fdda0ba8740 1 -- 172.21.15.27:0/399710757 wait complete. 2024-04-26T23:02:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.179+0000 7fdda0ba8740 1 -- start start 2024-04-26T23:02:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.179+0000 7fdda0ba8740 1 Processor -- start 2024-04-26T23:02:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.180+0000 7fdda0ba8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c65fe0 0x55af81c6e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.180+0000 7fdda0ba8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 0x55af81c6e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.180+0000 7fdda0ba8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 0x55af81c6eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.180+0000 7fdda0ba8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55af81c84e00 con 0x55af81c65fe0 2024-04-26T23:02:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.180+0000 7fdda0ba8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55af81b52c60 con 0x55af81c66900 2024-04-26T23:02:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.180+0000 7fdda0ba8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55af81c84f80 con 0x55af81c711c0 2024-04-26T23:02:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.180+0000 7fdda0ba8740 10 client.0 _ll_register_callbacks cb 0x55af81b7ee50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:02:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.181+0000 7fdd96000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c65fe0 0x55af81c6e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.181+0000 7fdd96000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c65fe0 0x55af81c6e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55220/0 (socket says 172.21.15.27:55220) 2024-04-26T23:02:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.181+0000 7fdd96000640 1 -- 172.21.15.27:0/3756659993 learned_addr learned my addr 172.21.15.27:0/3756659993 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.181+0000 7fdd96a00640 1 --2- 172.21.15.27:0/3756659993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 0x55af81c6e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.181+0000 7fdd96000640 1 --2- 172.21.15.27:0/3756659993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 0x55af81c6eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.181+0000 7fdd96000640 1 -- 172.21.15.27:0/3756659993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 msgr2=0x55af81c6eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.181+0000 7fdd96000640 1 --2- 172.21.15.27:0/3756659993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 0x55af81c6eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.181+0000 7fdd96000640 1 -- 172.21.15.27:0/3756659993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 msgr2=0x55af81c6e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.181+0000 7fdd96000640 1 --2- 172.21.15.27:0/3756659993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 0x55af81c6e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.181+0000 7fdd96000640 1 -- 172.21.15.27:0/3756659993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af81b61510 con 0x55af81c65fe0 2024-04-26T23:02:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.181+0000 7fdd96000640 1 --2- 172.21.15.27:0/3756659993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c65fe0 0x55af81c6e0f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009d20 tx=0x7fdd8401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d428032f4d35a457 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.181+0000 7fdd80c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T23:02:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.182+0000 7fdd80c00640 1 -- 172.21.15.27:0/3756659993 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84024040 con 0x55af81c65fe0 2024-04-26T23:02:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.182+0000 7fdd80c00640 1 -- 172.21.15.27:0/3756659993 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fdd84011240 con 0x55af81c65fe0 2024-04-26T23:02:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.182+0000 7fdda0ba8740 1 -- 172.21.15.27:0/3756659993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd840118a0 con 0x55af81c65fe0 2024-04-26T23:02:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.182+0000 7fdd80c00640 1 -- 172.21.15.27:0/3756659993 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdd8401e3a0 con 0x55af81c65fe0 2024-04-26T23:02:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.182+0000 7fdda0ba8740 10 client.15600 fetch_fsmap learned FSMap version 30 2024-04-26T23:02:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.182+0000 7fdda0ba8740 1 -- 172.21.15.27:0/3756659993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af818dd210 con 0x55af81c65fe0 2024-04-26T23:02:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.182+0000 7fdd80c00640 1 -- 172.21.15.27:0/3756659993 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd840118a0 con 0x55af81c65fe0 2024-04-26T23:02:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.183+0000 7fdd80c00640 1 -- 172.21.15.27:0/3756659993 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdd8402a040 con 0x55af81c65fe0 2024-04-26T23:02:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.183+0000 7fdda0ba8740 10 client.15600 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:02:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.183+0000 7fdda0ba8740 10 client.15600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:02:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.183+0000 7fdda0ba8740 20 client.15600 populate_metadata read hostname 'smithi027' 2024-04-26T23:02:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.183+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/3756659993 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55af81c91e10 0x55af81c94200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.183+0000 7fdda0ba8740 4 client.15600 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:02:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.183+0000 7fdda0ba8740 1 -- 172.21.15.27:0/3756659993 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af81bd8f90 con 0x55af81c91e10 2024-04-26T23:02:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.183+0000 7fdd96a00640 1 --2- 172.21.15.27:0/3756659993 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55af81c91e10 0x55af81c94200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.184+0000 7fdd96a00640 1 --2- 172.21.15.27:0/3756659993 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55af81c91e10 0x55af81c94200 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ccb64e4676c76bd server_cookie=c5b6f8d4c83c0638 in_seq=0 out_seq=0 2024-04-26T23:02:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.184+0000 7fdd80c00640 10 client.15600 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.185+0000 7fdd80c00640 1 -- 172.21.15.27:0/3756659993 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af81bd8f90 con 0x55af81c91e10 2024-04-26T23:02:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.185+0000 7fdd80c00640 10 client.15600 handle_command_reply: tid=0 2024-04-26T23:02:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.185+0000 7fdda0ba8740 1 client.15600 shutdown 2024-04-26T23:02:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.185+0000 7fdda0ba8740 1 -- 172.21.15.27:0/3756659993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c65fe0 msgr2=0x55af81c6e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.185+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/3756659993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c65fe0 0x55af81c6e0f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009d20 tx=0x7fdd8401a040 comp rx=0 tx=0).stop 2024-04-26T23:02:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.185+0000 7fdda0ba8740 1 -- 172.21.15.27:0/3756659993 shutdown_connections 2024-04-26T23:02:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.185+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/3756659993 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55af81c91e10 0x55af81c94200 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.185+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/3756659993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 0x55af81c6e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.185+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/3756659993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 0x55af81c6eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.185+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/3756659993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c65fe0 0x55af81c6e0f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.185+0000 7fdda0ba8740 1 -- 172.21.15.27:0/3756659993 >> 172.21.15.27:0/3756659993 conn(0x55af81c624f0 msgr2=0x55af81c68270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.186+0000 7fdd80c00640 0 client.15600 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.186+0000 7fdda0ba8740 1 -- 172.21.15.27:0/3756659993 shutdown_connections 2024-04-26T23:02:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.186+0000 7fdda0ba8740 1 -- 172.21.15.27:0/3756659993 wait complete. 2024-04-26T23:02:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.186+0000 7fdda0ba8740 20 client.15600 trim_cache size 0 max 16384 2024-04-26T23:02:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.205+0000 7fdda0ba8740 1 Processor -- start 2024-04-26T23:02:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.205+0000 7fdda0ba8740 1 -- start start 2024-04-26T23:02:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.205+0000 7fdda0ba8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c65fe0 0x55af81c689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.205+0000 7fdda0ba8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 0x55af81c6d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.205+0000 7fdda0ba8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c711c0 0x55af81c6e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.205+0000 7fdda0ba8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981b0460 con 0x55af81c711c0 2024-04-26T23:02:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.205+0000 7fdda0ba8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981adc70 con 0x55af81c66900 2024-04-26T23:02:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.205+0000 7fdda0ba8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98083250 con 0x55af81c65fe0 2024-04-26T23:02:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.206+0000 7fdd96a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 0x55af81c6d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.206+0000 7fdd96a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 0x55af81c6d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:37806/0 (socket says 172.21.15.27:37806) 2024-04-26T23:02:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.206+0000 7fdd96a00640 1 -- 172.21.15.27:0/115698769 learned_addr learned my addr 172.21.15.27:0/115698769 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.206+0000 7fdd96000640 1 --2- 172.21.15.27:0/115698769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c65fe0 0x55af81c689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.206+0000 7fdd96a00640 1 -- 172.21.15.27:0/115698769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c65fe0 msgr2=0x55af81c689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.206+0000 7fdd96a00640 1 --2- 172.21.15.27:0/115698769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c65fe0 0x55af81c689f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.206+0000 7fdd96000640 1 --2- 172.21.15.27:0/115698769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c711c0 0x55af81c6e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.206+0000 7fdd96a00640 1 -- 172.21.15.27:0/115698769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c711c0 msgr2=0x55af81c6e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.206+0000 7fdd96a00640 1 --2- 172.21.15.27:0/115698769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c711c0 0x55af81c6e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.206+0000 7fdd96a00640 1 -- 172.21.15.27:0/115698769 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af81c66730 con 0x55af81c66900 2024-04-26T23:02:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.206+0000 7fdd96000640 1 --2- 172.21.15.27:0/115698769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c711c0 0x55af81c6e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:02:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.207+0000 7fdd96a00640 1 --2- 172.21.15.27:0/115698769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 0x55af81c6d250 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdd88031440 tx=0x7fdd880058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c75eea38b34dfad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.207+0000 7fdd83400640 1 -- 172.21.15.27:0/115698769 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8804a030 con 0x55af81c66900 2024-04-26T23:02:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.207+0000 7fdd83400640 1 -- 172.21.15.27:0/115698769 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd88031950 con 0x55af81c66900 2024-04-26T23:02:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.207+0000 7fdd83400640 1 -- 172.21.15.27:0/115698769 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88035040 con 0x55af81c66900 2024-04-26T23:02:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.208+0000 7fdda0ba8740 1 -- 172.21.15.27:0/115698769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 msgr2=0x55af81c6d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.208+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/115698769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 0x55af81c6d250 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdd88031440 tx=0x7fdd880058d0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.208+0000 7fdda0ba8740 1 -- 172.21.15.27:0/115698769 shutdown_connections 2024-04-26T23:02:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.208+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/115698769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 0x55af81c6d250 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.208+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/115698769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c711c0 0x55af81c6e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.208+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/115698769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c65fe0 0x55af81c689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.208+0000 7fdda0ba8740 1 -- 172.21.15.27:0/115698769 >> 172.21.15.27:0/115698769 conn(0x55af81c624f0 msgr2=0x55af81c6f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.208+0000 7fdda0ba8740 1 -- 172.21.15.27:0/115698769 shutdown_connections 2024-04-26T23:02:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.208+0000 7fdda0ba8740 1 -- 172.21.15.27:0/115698769 wait complete. 2024-04-26T23:02:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.209+0000 7fdda0ba8740 1 -- start start 2024-04-26T23:02:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.209+0000 7fdda0ba8740 1 Processor -- start 2024-04-26T23:02:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdda0ba8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c65fe0 0x55af81c85380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdda0ba8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 0x55af81c858c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdda0ba8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 0x55af81cf6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdda0ba8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981ac200 con 0x55af81c65fe0 2024-04-26T23:02:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdda0ba8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981ad5a0 con 0x55af81c66900 2024-04-26T23:02:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdda0ba8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981ac800 con 0x55af81c711c0 2024-04-26T23:02:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdda0ba8740 10 client.0 _ll_register_callbacks cb 0x7fdd84005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:02:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdd96a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 0x55af81c858c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdd96a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 0x55af81c858c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:37814/0 (socket says 172.21.15.27:37814) 2024-04-26T23:02:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdd96a00640 1 -- 172.21.15.27:0/2349737514 learned_addr learned my addr 172.21.15.27:0/2349737514 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdd96000640 1 --2- 172.21.15.27:0/2349737514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c65fe0 0x55af81c85380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdd96a00640 1 -- 172.21.15.27:0/2349737514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 msgr2=0x55af81cf6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdd96000640 1 --2- 172.21.15.27:0/2349737514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 0x55af81cf6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdd96a00640 1 --2- 172.21.15.27:0/2349737514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 0x55af81cf6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdd96a00640 1 -- 172.21.15.27:0/2349737514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c65fe0 msgr2=0x55af81c85380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdd96a00640 1 --2- 172.21.15.27:0/2349737514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c65fe0 0x55af81c85380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdd96a00640 1 -- 172.21.15.27:0/2349737514 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af819a9620 con 0x55af81c66900 2024-04-26T23:02:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.210+0000 7fdd96000640 1 --2- 172.21.15.27:0/2349737514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 0x55af81cf6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:02:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.211+0000 7fdd96a00640 1 --2- 172.21.15.27:0/2349737514 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 0x55af81c858c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009230 tx=0x7fdd88004560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8b7cf7a74517a2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.211+0000 7fdd80c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T23:02:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.211+0000 7fdd80c00640 1 -- 172.21.15.27:0/2349737514 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88053070 con 0x55af81c66900 2024-04-26T23:02:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.211+0000 7fdd80c00640 1 -- 172.21.15.27:0/2349737514 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fdd88005220 con 0x55af81c66900 2024-04-26T23:02:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.211+0000 7fdda0ba8740 1 -- 172.21.15.27:0/2349737514 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55af81bd2e00 con 0x55af81c66900 2024-04-26T23:02:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.211+0000 7fdd80c00640 1 -- 172.21.15.27:0/2349737514 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdd88035070 con 0x55af81c66900 2024-04-26T23:02:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.211+0000 7fdda0ba8740 10 client.25742 fetch_fsmap learned FSMap version 30 2024-04-26T23:02:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.211+0000 7fdda0ba8740 1 -- 172.21.15.27:0/2349737514 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af81b85cb0 con 0x55af81c66900 2024-04-26T23:02:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.212+0000 7fdd80c00640 1 -- 172.21.15.27:0/2349737514 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55af81bd2e00 con 0x55af81c66900 2024-04-26T23:02:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.212+0000 7fdd80c00640 1 -- 172.21.15.27:0/2349737514 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdd880035f0 con 0x55af81c66900 2024-04-26T23:02:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.212+0000 7fdda0ba8740 10 client.25742 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:02:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.212+0000 7fdda0ba8740 10 client.25742 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:02:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.212+0000 7fdda0ba8740 20 client.25742 populate_metadata read hostname 'smithi027' 2024-04-26T23:02:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.212+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/2349737514 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55af81cf7c70 0x55af81cfa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.212+0000 7fdda0ba8740 4 client.25742 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:02:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.212+0000 7fdda0ba8740 1 -- 172.21.15.27:0/2349737514 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af81b4e4b0 con 0x55af81cf7c70 2024-04-26T23:02:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.212+0000 7fdd96000640 1 --2- 172.21.15.27:0/2349737514 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55af81cf7c70 0x55af81cfa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.213+0000 7fdd96000640 1 --2- 172.21.15.27:0/2349737514 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55af81cf7c70 0x55af81cfa090 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51ac5334ca6586b2 server_cookie=502a8393eda01d9c in_seq=0 out_seq=0 2024-04-26T23:02:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.213+0000 7fdd80c00640 10 client.25742 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.342+0000 7fdd80c00640 1 -- 172.21.15.27:0/2349737514 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af81b4e4b0 con 0x55af81cf7c70 2024-04-26T23:02:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.342+0000 7fdd80c00640 10 client.25742 handle_command_reply: tid=0 2024-04-26T23:02:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.342+0000 7fdda0ba8740 1 client.25742 shutdown 2024-04-26T23:02:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.343+0000 7fdda0ba8740 1 -- 172.21.15.27:0/2349737514 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 msgr2=0x55af81c858c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.343+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/2349737514 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 0x55af81c858c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009230 tx=0x7fdd88004560 comp rx=0 tx=0).stop 2024-04-26T23:02:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.343+0000 7fdda0ba8740 1 -- 172.21.15.27:0/2349737514 shutdown_connections 2024-04-26T23:02:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.343+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/2349737514 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55af81cf7c70 0x55af81cfa090 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.343+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/2349737514 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af81c66900 0x55af81c858c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.343+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/2349737514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af81c711c0 0x55af81cf6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.343+0000 7fdd80c00640 0 client.25742 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.343+0000 7fdda0ba8740 1 --2- 172.21.15.27:0/2349737514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af81c65fe0 0x55af81c85380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.343+0000 7fdda0ba8740 1 -- 172.21.15.27:0/2349737514 >> 172.21.15.27:0/2349737514 conn(0x55af81c624f0 msgr2=0x55af81c6f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.343+0000 7fdda0ba8740 1 -- 172.21.15.27:0/2349737514 shutdown_connections 2024-04-26T23:02:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.343+0000 7fdda0ba8740 1 -- 172.21.15.27:0/2349737514 wait complete. 2024-04-26T23:02:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.343+0000 7fdda0ba8740 20 client.25742 trim_cache size 0 max 16384 2024-04-26T23:02:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T23:02:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T23:02:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T23:02:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T23:02:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.344+0000 7fdd97e00640 1 -- 172.21.15.27:0/3300547788 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fdd7c0a3d00 msgr2=0x7fdd7c0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.344+0000 7fdd97e00640 1 --2- 172.21.15.27:0/3300547788 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fdd7c0a3d00 0x7fdd7c0a61c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdd900992d0 tx=0x7fdd90005f70 comp rx=0 tx=0).stop 2024-04-26T23:02:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.344+0000 7fdd97e00640 1 -- 172.21.15.27:0/3300547788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd9810ee50 msgr2=0x7fdd981a3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.344+0000 7fdd97e00640 1 --2- 172.21.15.27:0/3300547788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd9810ee50 0x7fdd981a3d80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009fd0 tx=0x7fdd9000ef10 comp rx=0 tx=0).stop 2024-04-26T23:02:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.344+0000 7fdd97e00640 1 -- 172.21.15.27:0/3300547788 shutdown_connections 2024-04-26T23:02:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.344+0000 7fdd97e00640 1 --2- 172.21.15.27:0/3300547788 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fdd7c0a3d00 0x7fdd7c0a61c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.344+0000 7fdd97e00640 1 --2- 172.21.15.27:0/3300547788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd98109730 0x7fdd9819fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.344+0000 7fdd97e00640 1 --2- 172.21.15.27:0/3300547788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd9810ee50 0x7fdd981a3d80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.344+0000 7fdd97e00640 1 --2- 172.21.15.27:0/3300547788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd98108d90 0x7fdd9819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.344+0000 7fdd97e00640 1 -- 172.21.15.27:0/3300547788 >> 172.21.15.27:0/3300547788 conn(0x7fdd980784e0 msgr2=0x7fdd98079e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.344+0000 7fdd97e00640 1 -- 172.21.15.27:0/3300547788 shutdown_connections 2024-04-26T23:02:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:14.344+0000 7fdd97e00640 1 -- 172.21.15.27:0/3300547788 wait complete. 2024-04-26T23:02:15.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:14 smithi027 ceph-mon[28034]: pgmap v112: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 990 KiB/s wr, 118 op/s 2024-04-26T23:02:15.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:14 smithi080 ceph-mon[32324]: pgmap v112: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 990 KiB/s wr, 118 op/s 2024-04-26T23:02:15.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:14 smithi094 ceph-mon[32889]: pgmap v112: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 990 KiB/s wr, 118 op/s 2024-04-26T23:02:16.484 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T23:02:16.484 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T23:02:16.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:16 smithi027 ceph-mon[28034]: pgmap v113: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 859 KiB/s wr, 88 op/s 2024-04-26T23:02:17.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:16 smithi080 ceph-mon[32324]: pgmap v113: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 859 KiB/s wr, 88 op/s 2024-04-26T23:02:17.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:16 smithi094 ceph-mon[32889]: pgmap v113: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 859 KiB/s wr, 88 op/s 2024-04-26T23:02:19.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:18 smithi027 ceph-mon[28034]: pgmap v114: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 216 op/s 2024-04-26T23:02:19.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:18 smithi080 ceph-mon[32324]: pgmap v114: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 216 op/s 2024-04-26T23:02:19.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:18 smithi094 ceph-mon[32889]: pgmap v114: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 216 op/s 2024-04-26T23:02:21.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:20 smithi027 ceph-mon[28034]: pgmap v115: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 959 KiB/s wr, 168 op/s 2024-04-26T23:02:21.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:20 smithi080 ceph-mon[32324]: pgmap v115: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 959 KiB/s wr, 168 op/s 2024-04-26T23:02:21.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:20 smithi094 ceph-mon[32889]: pgmap v115: 129 pgs: 129 active+clean; 1.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 959 KiB/s wr, 168 op/s 2024-04-26T23:02:23.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:22 smithi027 ceph-mon[28034]: pgmap v116: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 255 op/s 2024-04-26T23:02:23.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:22 smithi080 ceph-mon[32324]: pgmap v116: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 255 op/s 2024-04-26T23:02:23.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:22 smithi094 ceph-mon[32889]: pgmap v116: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 255 op/s 2024-04-26T23:02:25.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:24 smithi027 ceph-mon[28034]: pgmap v117: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 996 KiB/s wr, 219 op/s 2024-04-26T23:02:25.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:24 smithi080 ceph-mon[32324]: pgmap v117: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 996 KiB/s wr, 219 op/s 2024-04-26T23:02:25.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:24 smithi094 ceph-mon[32889]: pgmap v117: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 996 KiB/s wr, 219 op/s 2024-04-26T23:02:27.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:26 smithi027 ceph-mon[28034]: pgmap v118: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 800 KiB/s wr, 215 op/s 2024-04-26T23:02:27.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:26 smithi080 ceph-mon[32324]: pgmap v118: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 800 KiB/s wr, 215 op/s 2024-04-26T23:02:27.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:26 smithi094 ceph-mon[32889]: pgmap v118: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 800 KiB/s wr, 215 op/s 2024-04-26T23:02:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.508+0000 7fc684a00640 1 -- 172.21.15.27:0/3486608869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680101520 msgr2=0x7fc680101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.508+0000 7fc684a00640 1 --2- 172.21.15.27:0/3486608869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680101520 0x7fc680101900 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc670009900 tx=0x7fc67002a3f0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.508+0000 7fc684a00640 1 -- 172.21.15.27:0/3486608869 shutdown_connections 2024-04-26T23:02:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.508+0000 7fc684a00640 1 --2- 172.21.15.27:0/3486608869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc68010f0d0 0x7fc6801114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.508+0000 7fc684a00640 1 --2- 172.21.15.27:0/3486608869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc680101ed0 0x7fc68010eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.508+0000 7fc684a00640 1 --2- 172.21.15.27:0/3486608869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680101520 0x7fc680101900 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.508+0000 7fc684a00640 1 -- 172.21.15.27:0/3486608869 >> 172.21.15.27:0/3486608869 conn(0x7fc6800fd110 msgr2=0x7fc6800ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.508+0000 7fc684a00640 1 -- 172.21.15.27:0/3486608869 shutdown_connections 2024-04-26T23:02:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.508+0000 7fc684a00640 1 -- 172.21.15.27:0/3486608869 wait complete. 2024-04-26T23:02:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.510+0000 7fc684a00640 1 Processor -- start 2024-04-26T23:02:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.510+0000 7fc684a00640 1 -- start start 2024-04-26T23:02:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.511+0000 7fc684a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680101520 0x7fc680103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.511+0000 7fc684a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc680101ed0 0x7fc680103cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.511+0000 7fc684a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc68010f0d0 0x7fc6801041f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.511+0000 7fc684a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc680114c10 con 0x7fc680101ed0 2024-04-26T23:02:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.511+0000 7fc684a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc680114a90 con 0x7fc68010f0d0 2024-04-26T23:02:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.511+0000 7fc684a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc680114d90 con 0x7fc680101520 2024-04-26T23:02:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.511+0000 7fc67f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680101520 0x7fc680103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.511+0000 7fc67fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc68010f0d0 0x7fc6801041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.511+0000 7fc67f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680101520 0x7fc680103770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:57578/0 (socket says 172.21.15.27:57578) 2024-04-26T23:02:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.511+0000 7fc67f400640 1 -- 172.21.15.27:0/633880189 learned_addr learned my addr 172.21.15.27:0/633880189 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.511+0000 7fc67ea00640 1 --2- 172.21.15.27:0/633880189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc680101ed0 0x7fc680103cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.512+0000 7fc67fe00640 1 -- 172.21.15.27:0/633880189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680101520 msgr2=0x7fc680103770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.512+0000 7fc67fe00640 1 --2- 172.21.15.27:0/633880189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680101520 0x7fc680103770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.512+0000 7fc67fe00640 1 -- 172.21.15.27:0/633880189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc680101ed0 msgr2=0x7fc680103cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.512+0000 7fc67fe00640 1 --2- 172.21.15.27:0/633880189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc680101ed0 0x7fc680103cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.512+0000 7fc67fe00640 1 -- 172.21.15.27:0/633880189 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6801ab820 con 0x7fc68010f0d0 2024-04-26T23:02:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.512+0000 7fc67f400640 1 --2- 172.21.15.27:0/633880189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680101520 0x7fc680103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:02:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.512+0000 7fc67fe00640 1 --2- 172.21.15.27:0/633880189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc68010f0d0 0x7fc6801041f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc678005fd0 tx=0x7fc67800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.512+0000 7fc677e00640 1 -- 172.21.15.27:0/633880189 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67800cdf0 con 0x7fc68010f0d0 2024-04-26T23:02:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.512+0000 7fc677e00640 1 -- 172.21.15.27:0/633880189 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc678005020 con 0x7fc68010f0d0 2024-04-26T23:02:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.512+0000 7fc684a00640 1 -- 172.21.15.27:0/633880189 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6801abb10 con 0x7fc68010f0d0 2024-04-26T23:02:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.512+0000 7fc677e00640 1 -- 172.21.15.27:0/633880189 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678010690 con 0x7fc68010f0d0 2024-04-26T23:02:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.513+0000 7fc684a00640 1 -- 172.21.15.27:0/633880189 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6801ac060 con 0x7fc68010f0d0 2024-04-26T23:02:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.517+0000 7fc677e00640 1 -- 172.21.15.27:0/633880189 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc6780054d0 con 0x7fc68010f0d0 2024-04-26T23:02:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.517+0000 7fc688fc1740 1 Processor -- start 2024-04-26T23:02:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.517+0000 7fc688fc1740 1 -- start start 2024-04-26T23:02:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.517+0000 7fc688fc1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 0x563ce86c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.517+0000 7fc688fc1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c5900 0x563ce86cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.517+0000 7fc688fc1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86d01c0 0x563ce86d25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.517+0000 7fc688fc1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563ce862a380 con 0x563ce86c5900 2024-04-26T23:02:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.517+0000 7fc688fc1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x563ce85c6c30 con 0x563ce86c4fe0 2024-04-26T23:02:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.517+0000 7fc688fc1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563ce8626340 con 0x563ce86d01c0 2024-04-26T23:02:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.518+0000 7fc67ea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 0x563ce86c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.518+0000 7fc67f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c5900 0x563ce86cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.518+0000 7fc67ea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 0x563ce86c53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:51006/0 (socket says 172.21.15.27:51006) 2024-04-26T23:02:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.518+0000 7fc67ea00640 1 -- 172.21.15.27:0/2133427115 learned_addr learned my addr 172.21.15.27:0/2133427115 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.518+0000 7fc67ea00640 1 --2- 172.21.15.27:0/2133427115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86d01c0 0x563ce86d25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.518+0000 7fc67f400640 1 -- 172.21.15.27:0/2133427115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86d01c0 msgr2=0x563ce86d25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.518+0000 7fc67f400640 1 --2- 172.21.15.27:0/2133427115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86d01c0 0x563ce86d25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.519+0000 7fc67f400640 1 -- 172.21.15.27:0/2133427115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 msgr2=0x563ce86c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.519+0000 7fc67f400640 1 --2- 172.21.15.27:0/2133427115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 0x563ce86c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.519+0000 7fc67f400640 1 -- 172.21.15.27:0/2133427115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce85e8940 con 0x563ce86c5900 2024-04-26T23:02:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.519+0000 7fc67ea00640 1 --2- 172.21.15.27:0/2133427115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86d01c0 0x563ce86d25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:02:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.519+0000 7fc677e00640 1 --2- 172.21.15.27:0/633880189 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc6640a3d00 0x7fc6640a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.519+0000 7fc67f400640 1 --2- 172.21.15.27:0/2133427115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c5900 0x563ce86cfc80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc670004770 tx=0x7fc670032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463775362d03d23b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.519+0000 7fc677e00640 1 -- 172.21.15.27:0/633880189 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fc6780d1340 con 0x7fc68010f0d0 2024-04-26T23:02:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.520+0000 7fc66be00640 1 -- 172.21.15.27:0/2133427115 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc670035040 con 0x563ce86c5900 2024-04-26T23:02:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.520+0000 7fc66be00640 1 -- 172.21.15.27:0/2133427115 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6700317a0 con 0x563ce86c5900 2024-04-26T23:02:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.520+0000 7fc66be00640 1 -- 172.21.15.27:0/2133427115 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6700047a0 con 0x563ce86c5900 2024-04-26T23:02:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.520+0000 7fc67ea00640 1 --2- 172.21.15.27:0/633880189 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc6640a3d00 0x7fc6640a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.521+0000 7fc688fc1740 1 -- 172.21.15.27:0/2133427115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c5900 msgr2=0x563ce86cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.521+0000 7fc67ea00640 1 --2- 172.21.15.27:0/633880189 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc6640a3d00 0x7fc6640a61c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x563ce86c2910 tx=0x7fc66c015040 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.521+0000 7fc688fc1740 1 --2- 172.21.15.27:0/2133427115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c5900 0x563ce86cfc80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc670004770 tx=0x7fc670032ed0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.521+0000 7fc688fc1740 1 -- 172.21.15.27:0/2133427115 shutdown_connections 2024-04-26T23:02:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.521+0000 7fc688fc1740 1 --2- 172.21.15.27:0/2133427115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86d01c0 0x563ce86d25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.521+0000 7fc688fc1740 1 --2- 172.21.15.27:0/2133427115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c5900 0x563ce86cfc80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.521+0000 7fc688fc1740 1 --2- 172.21.15.27:0/2133427115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 0x563ce86c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.521+0000 7fc688fc1740 1 -- 172.21.15.27:0/2133427115 >> 172.21.15.27:0/2133427115 conn(0x563ce86c14f0 msgr2=0x563ce86c18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.521+0000 7fc688fc1740 1 -- 172.21.15.27:0/2133427115 shutdown_connections 2024-04-26T23:02:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.521+0000 7fc688fc1740 1 -- 172.21.15.27:0/2133427115 wait complete. 2024-04-26T23:02:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.522+0000 7fc688fc1740 1 -- start start 2024-04-26T23:02:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.522+0000 7fc688fc1740 1 Processor -- start 2024-04-26T23:02:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.523+0000 7fc688fc1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 0x563ce86cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.523+0000 7fc688fc1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86c5900 0x563ce86cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.523+0000 7fc688fc1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86d01c0 0x563ce86cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.523+0000 7fc688fc1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563ce86d2d90 con 0x563ce86d01c0 2024-04-26T23:02:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.523+0000 7fc688fc1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x563ce85b1c60 con 0x563ce86c4fe0 2024-04-26T23:02:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.523+0000 7fc688fc1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563ce86d2f10 con 0x563ce86c5900 2024-04-26T23:02:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.523+0000 7fc688fc1740 10 client.0 _ll_register_callbacks cb 0x563ce85dde50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:02:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.524+0000 7fc67ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86c5900 0x563ce86cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.524+0000 7fc67ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86c5900 0x563ce86cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:57592/0 (socket says 172.21.15.27:57592) 2024-04-26T23:02:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.524+0000 7fc67ea00640 1 -- 172.21.15.27:0/2834588126 learned_addr learned my addr 172.21.15.27:0/2834588126 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.524+0000 7fc67f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 0x563ce86cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.524+0000 7fc67fe00640 1 --2- 172.21.15.27:0/2834588126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86d01c0 0x563ce86cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.524+0000 7fc67ea00640 1 -- 172.21.15.27:0/2834588126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 msgr2=0x563ce86cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.524+0000 7fc67ea00640 1 --2- 172.21.15.27:0/2834588126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 0x563ce86cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.524+0000 7fc67ea00640 1 -- 172.21.15.27:0/2834588126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86d01c0 msgr2=0x563ce86cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.524+0000 7fc67ea00640 1 --2- 172.21.15.27:0/2834588126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86d01c0 0x563ce86cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.524+0000 7fc67ea00640 1 -- 172.21.15.27:0/2834588126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ce85c0510 con 0x563ce86c5900 2024-04-26T23:02:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.524+0000 7fc67ea00640 1 --2- 172.21.15.27:0/2834588126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86c5900 0x563ce86cd690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00fb90 tx=0x7fc66c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=992d92b59941b1a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.524+0000 7fc668c00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T23:02:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.525+0000 7fc668c00640 1 -- 172.21.15.27:0/2834588126 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c011cc0 con 0x563ce86c5900 2024-04-26T23:02:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.525+0000 7fc688fc1740 1 -- 172.21.15.27:0/2834588126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6700317a0 con 0x563ce86c5900 2024-04-26T23:02:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.525+0000 7fc668c00640 1 -- 172.21.15.27:0/2834588126 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fc66c004020 con 0x563ce86c5900 2024-04-26T23:02:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.525+0000 7fc668c00640 1 -- 172.21.15.27:0/2834588126 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc66c031050 con 0x563ce86c5900 2024-04-26T23:02:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.525+0000 7fc688fc1740 10 client.25798 fetch_fsmap learned FSMap version 30 2024-04-26T23:02:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.525+0000 7fc688fc1740 1 -- 172.21.15.27:0/2834588126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563ce833c210 con 0x563ce86c5900 2024-04-26T23:02:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.526+0000 7fc668c00640 1 -- 172.21.15.27:0/2834588126 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6700317a0 con 0x563ce86c5900 2024-04-26T23:02:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.526+0000 7fc668c00640 1 -- 172.21.15.27:0/2834588126 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc66c022420 con 0x563ce86c5900 2024-04-26T23:02:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.526+0000 7fc688fc1740 10 client.25798 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:02:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.526+0000 7fc688fc1740 10 client.25798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:02:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.526+0000 7fc688fc1740 20 client.25798 populate_metadata read hostname 'smithi027' 2024-04-26T23:02:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.526+0000 7fc688fc1740 1 --2- 172.21.15.27:0/2834588126 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x563ce86dff60 0x563ce86e2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.526+0000 7fc688fc1740 4 client.25798 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:02:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.526+0000 7fc688fc1740 1 -- 172.21.15.27:0/2834588126 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ce8637f90 con 0x563ce86dff60 2024-04-26T23:02:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.526+0000 7fc67f400640 1 --2- 172.21.15.27:0/2834588126 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x563ce86dff60 0x563ce86e2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.527+0000 7fc67f400640 1 --2- 172.21.15.27:0/2834588126 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x563ce86dff60 0x563ce86e2370 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d52801af169a84 server_cookie=113f3ddf58f19473 in_seq=0 out_seq=0 2024-04-26T23:02:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.527+0000 7fc668c00640 10 client.25798 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.528+0000 7fc668c00640 1 -- 172.21.15.27:0/2834588126 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ce8637f90 con 0x563ce86dff60 2024-04-26T23:02:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.528+0000 7fc668c00640 10 client.25798 handle_command_reply: tid=0 2024-04-26T23:02:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.528+0000 7fc688fc1740 1 client.25798 shutdown 2024-04-26T23:02:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.528+0000 7fc688fc1740 1 -- 172.21.15.27:0/2834588126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86c5900 msgr2=0x563ce86cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.528+0000 7fc688fc1740 1 --2- 172.21.15.27:0/2834588126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86c5900 0x563ce86cd690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00fb90 tx=0x7fc66c005f90 comp rx=0 tx=0).stop 2024-04-26T23:02:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.528+0000 7fc688fc1740 1 -- 172.21.15.27:0/2834588126 shutdown_connections 2024-04-26T23:02:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.528+0000 7fc688fc1740 1 --2- 172.21.15.27:0/2834588126 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x563ce86dff60 0x563ce86e2370 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.529+0000 7fc688fc1740 1 --2- 172.21.15.27:0/2834588126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86d01c0 0x563ce86cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.529+0000 7fc688fc1740 1 --2- 172.21.15.27:0/2834588126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86c5900 0x563ce86cd690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.529+0000 7fc688fc1740 1 --2- 172.21.15.27:0/2834588126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 0x563ce86cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.529+0000 7fc668c00640 0 client.25798 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.529+0000 7fc688fc1740 1 -- 172.21.15.27:0/2834588126 >> 172.21.15.27:0/2834588126 conn(0x563ce86c14f0 msgr2=0x563ce86c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.529+0000 7fc688fc1740 1 -- 172.21.15.27:0/2834588126 shutdown_connections 2024-04-26T23:02:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.529+0000 7fc688fc1740 1 -- 172.21.15.27:0/2834588126 wait complete. 2024-04-26T23:02:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.529+0000 7fc688fc1740 20 client.25798 trim_cache size 0 max 16384 2024-04-26T23:02:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.561+0000 7fc688fc1740 1 Processor -- start 2024-04-26T23:02:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.561+0000 7fc688fc1740 1 -- start start 2024-04-26T23:02:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.561+0000 7fc688fc1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 0x563ce86dbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.561+0000 7fc688fc1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c5900 0x563ce86cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.561+0000 7fc688fc1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86d01c0 0x563ce86e7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.561+0000 7fc688fc1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801b8d50 con 0x563ce86c5900 2024-04-26T23:02:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.561+0000 7fc688fc1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801b6560 con 0x563ce86c4fe0 2024-04-26T23:02:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.561+0000 7fc688fc1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801b43b0 con 0x563ce86d01c0 2024-04-26T23:02:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.562+0000 7fc67f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 0x563ce86dbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.562+0000 7fc67f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 0x563ce86dbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:51024/0 (socket says 172.21.15.27:51024) 2024-04-26T23:02:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.562+0000 7fc67fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86d01c0 0x563ce86e7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.562+0000 7fc67f400640 1 -- 172.21.15.27:0/3560465469 learned_addr learned my addr 172.21.15.27:0/3560465469 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.562+0000 7fc67ea00640 1 --2- 172.21.15.27:0/3560465469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c5900 0x563ce86cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.562+0000 7fc67fe00640 1 -- 172.21.15.27:0/3560465469 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 msgr2=0x563ce86dbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.562+0000 7fc67fe00640 1 --2- 172.21.15.27:0/3560465469 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 0x563ce86dbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.562+0000 7fc67fe00640 1 -- 172.21.15.27:0/3560465469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c5900 msgr2=0x563ce86cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.562+0000 7fc67fe00640 1 --2- 172.21.15.27:0/3560465469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c5900 0x563ce86cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.562+0000 7fc67fe00640 1 -- 172.21.15.27:0/3560465469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce86c5730 con 0x563ce86d01c0 2024-04-26T23:02:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.562+0000 7fc67f400640 1 --2- 172.21.15.27:0/3560465469 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 0x563ce86dbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:02:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.562+0000 7fc67ea00640 1 --2- 172.21.15.27:0/3560465469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c5900 0x563ce86cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:02:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.562+0000 7fc67fe00640 1 --2- 172.21.15.27:0/3560465469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86d01c0 0x563ce86e7c80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc6780ce5f0 tx=0x7fc67809d540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ff786b2477ca651 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.563+0000 7fc66b400640 1 -- 172.21.15.27:0/3560465469 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6780b5070 con 0x563ce86d01c0 2024-04-26T23:02:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.563+0000 7fc66b400640 1 -- 172.21.15.27:0/3560465469 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6780a11f0 con 0x563ce86d01c0 2024-04-26T23:02:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.563+0000 7fc66b400640 1 -- 172.21.15.27:0/3560465469 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67809f030 con 0x563ce86d01c0 2024-04-26T23:02:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.563+0000 7fc688fc1740 1 -- 172.21.15.27:0/3560465469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86d01c0 msgr2=0x563ce86e7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.563+0000 7fc688fc1740 1 --2- 172.21.15.27:0/3560465469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86d01c0 0x563ce86e7c80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc6780ce5f0 tx=0x7fc67809d540 comp rx=0 tx=0).stop 2024-04-26T23:02:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.564+0000 7fc688fc1740 1 -- 172.21.15.27:0/3560465469 shutdown_connections 2024-04-26T23:02:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.564+0000 7fc688fc1740 1 --2- 172.21.15.27:0/3560465469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86d01c0 0x563ce86e7c80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.564+0000 7fc688fc1740 1 --2- 172.21.15.27:0/3560465469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c5900 0x563ce86cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.564+0000 7fc688fc1740 1 --2- 172.21.15.27:0/3560465469 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86c4fe0 0x563ce86dbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.564+0000 7fc688fc1740 1 -- 172.21.15.27:0/3560465469 >> 172.21.15.27:0/3560465469 conn(0x563ce86c14f0 msgr2=0x563ce86ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.564+0000 7fc688fc1740 1 -- 172.21.15.27:0/3560465469 shutdown_connections 2024-04-26T23:02:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.564+0000 7fc688fc1740 1 -- 172.21.15.27:0/3560465469 wait complete. 2024-04-26T23:02:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.565+0000 7fc688fc1740 1 -- start start 2024-04-26T23:02:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.565+0000 7fc688fc1740 1 Processor -- start 2024-04-26T23:02:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.565+0000 7fc688fc1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c4fe0 0x563ce86d3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.565+0000 7fc688fc1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86c5900 0x563ce86d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.565+0000 7fc688fc1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86d01c0 0x563ce86d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.565+0000 7fc688fc1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801b4a20 con 0x563ce86c4fe0 2024-04-26T23:02:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.565+0000 7fc688fc1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801b5e90 con 0x563ce86d01c0 2024-04-26T23:02:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.565+0000 7fc688fc1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801b50f0 con 0x563ce86c5900 2024-04-26T23:02:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.565+0000 7fc688fc1740 10 client.0 _ll_register_callbacks cb 0x7fc66c013dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:02:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.566+0000 7fc67ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86c5900 0x563ce86d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.566+0000 7fc67fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86d01c0 0x563ce86d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.566+0000 7fc67ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86c5900 0x563ce86d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:57598/0 (socket says 172.21.15.27:57598) 2024-04-26T23:02:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.566+0000 7fc67fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86d01c0 0x563ce86d4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:51028/0 (socket says 172.21.15.27:51028) 2024-04-26T23:02:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.566+0000 7fc67ea00640 1 -- 172.21.15.27:0/3390442541 learned_addr learned my addr 172.21.15.27:0/3390442541 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.566+0000 7fc67f400640 1 --2- 172.21.15.27:0/3390442541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c4fe0 0x563ce86d3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.566+0000 7fc67fe00640 1 -- 172.21.15.27:0/3390442541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86c5900 msgr2=0x563ce86d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.566+0000 7fc67fe00640 1 --2- 172.21.15.27:0/3390442541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86c5900 0x563ce86d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.566+0000 7fc67fe00640 1 -- 172.21.15.27:0/3390442541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c4fe0 msgr2=0x563ce86d3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.566+0000 7fc67fe00640 1 --2- 172.21.15.27:0/3390442541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c4fe0 0x563ce86d3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.566+0000 7fc67fe00640 1 -- 172.21.15.27:0/3390442541 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ce8408620 con 0x563ce86d01c0 2024-04-26T23:02:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.566+0000 7fc67ea00640 1 --2- 172.21.15.27:0/3390442541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86c5900 0x563ce86d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:02:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.566+0000 7fc67f400640 1 --2- 172.21.15.27:0/3390442541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c4fe0 0x563ce86d3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:02:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.566+0000 7fc67fe00640 1 --2- 172.21.15.27:0/3390442541 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86d01c0 0x563ce86d4770 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x563ce8638010 tx=0x7fc6780a5ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1ddf92440ab19d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.566+0000 7fc668c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T23:02:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.567+0000 7fc668c00640 1 -- 172.21.15.27:0/3390442541 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67809f070 con 0x563ce86d01c0 2024-04-26T23:02:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.567+0000 7fc668c00640 1 -- 172.21.15.27:0/3390442541 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fc6780a5820 con 0x563ce86d01c0 2024-04-26T23:02:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.567+0000 7fc688fc1740 1 -- 172.21.15.27:0/3390442541 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6780a11f0 con 0x563ce86d01c0 2024-04-26T23:02:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.567+0000 7fc668c00640 1 -- 172.21.15.27:0/3390442541 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc6780b5070 con 0x563ce86d01c0 2024-04-26T23:02:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.567+0000 7fc688fc1740 10 client.25760 fetch_fsmap learned FSMap version 30 2024-04-26T23:02:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.567+0000 7fc688fc1740 1 -- 172.21.15.27:0/3390442541 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563ce85e4cb0 con 0x563ce86d01c0 2024-04-26T23:02:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.567+0000 7fc668c00640 1 -- 172.21.15.27:0/3390442541 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc678098b90 con 0x563ce86d01c0 2024-04-26T23:02:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.567+0000 7fc668c00640 1 -- 172.21.15.27:0/3390442541 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc6780a2380 con 0x563ce86d01c0 2024-04-26T23:02:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.568+0000 7fc688fc1740 10 client.25760 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:02:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.568+0000 7fc688fc1740 10 client.25760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:02:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.568+0000 7fc688fc1740 20 client.25760 populate_metadata read hostname 'smithi027' 2024-04-26T23:02:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.568+0000 7fc688fc1740 1 --2- 172.21.15.27:0/3390442541 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x563ce8746cf0 0x563ce8749110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.568+0000 7fc688fc1740 4 client.25760 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:02:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.568+0000 7fc688fc1740 1 -- 172.21.15.27:0/3390442541 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fc6780a11f0 con 0x563ce8746cf0 2024-04-26T23:02:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.568+0000 7fc67f400640 1 --2- 172.21.15.27:0/3390442541 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x563ce8746cf0 0x563ce8749110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.569+0000 7fc67f400640 1 --2- 172.21.15.27:0/3390442541 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x563ce8746cf0 0x563ce8749110 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=449d683061d50eb server_cookie=245620fb93c6e3c5 in_seq=0 out_seq=0 2024-04-26T23:02:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.569+0000 7fc668c00640 10 client.25760 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.588+0000 7fc668c00640 1 -- 172.21.15.27:0/3390442541 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fc6780a11f0 con 0x563ce8746cf0 2024-04-26T23:02:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.588+0000 7fc668c00640 10 client.25760 handle_command_reply: tid=0 2024-04-26T23:02:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.588+0000 7fc688fc1740 1 client.25760 shutdown 2024-04-26T23:02:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.588+0000 7fc688fc1740 1 -- 172.21.15.27:0/3390442541 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86d01c0 msgr2=0x563ce86d4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.588+0000 7fc688fc1740 1 --2- 172.21.15.27:0/3390442541 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86d01c0 0x563ce86d4770 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x563ce8638010 tx=0x7fc6780a5ab0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.588+0000 7fc688fc1740 1 -- 172.21.15.27:0/3390442541 shutdown_connections 2024-04-26T23:02:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.588+0000 7fc688fc1740 1 --2- 172.21.15.27:0/3390442541 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x563ce8746cf0 0x563ce8749110 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.588+0000 7fc688fc1740 1 --2- 172.21.15.27:0/3390442541 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563ce86d01c0 0x563ce86d4770 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.588+0000 7fc688fc1740 1 --2- 172.21.15.27:0/3390442541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ce86c5900 0x563ce86d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.588+0000 7fc668c00640 0 client.25760 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.588+0000 7fc688fc1740 1 --2- 172.21.15.27:0/3390442541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce86c4fe0 0x563ce86d3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.588+0000 7fc688fc1740 1 -- 172.21.15.27:0/3390442541 >> 172.21.15.27:0/3390442541 conn(0x563ce86c14f0 msgr2=0x563ce86c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.589+0000 7fc688fc1740 1 -- 172.21.15.27:0/3390442541 shutdown_connections 2024-04-26T23:02:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.589+0000 7fc688fc1740 1 -- 172.21.15.27:0/3390442541 wait complete. 2024-04-26T23:02:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.589+0000 7fc688fc1740 20 client.25760 trim_cache size 0 max 16384 2024-04-26T23:02:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T23:02:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T23:02:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T23:02:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T23:02:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.590+0000 7fc684a00640 1 -- 172.21.15.27:0/633880189 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc6640a3d00 msgr2=0x7fc6640a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.590+0000 7fc684a00640 1 --2- 172.21.15.27:0/633880189 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc6640a3d00 0x7fc6640a61c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x563ce86c2910 tx=0x7fc66c015040 comp rx=0 tx=0).stop 2024-04-26T23:02:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.590+0000 7fc684a00640 1 -- 172.21.15.27:0/633880189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc68010f0d0 msgr2=0x7fc6801041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.590+0000 7fc684a00640 1 --2- 172.21.15.27:0/633880189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc68010f0d0 0x7fc6801041f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc678005fd0 tx=0x7fc67800ef10 comp rx=0 tx=0).stop 2024-04-26T23:02:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.590+0000 7fc684a00640 1 -- 172.21.15.27:0/633880189 shutdown_connections 2024-04-26T23:02:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.590+0000 7fc684a00640 1 --2- 172.21.15.27:0/633880189 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc6640a3d00 0x7fc6640a61c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.590+0000 7fc684a00640 1 --2- 172.21.15.27:0/633880189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc68010f0d0 0x7fc6801041f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.590+0000 7fc684a00640 1 --2- 172.21.15.27:0/633880189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc680101ed0 0x7fc680103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.590+0000 7fc684a00640 1 --2- 172.21.15.27:0/633880189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680101520 0x7fc680103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.590+0000 7fc684a00640 1 -- 172.21.15.27:0/633880189 >> 172.21.15.27:0/633880189 conn(0x7fc6800fd110 msgr2=0x7fc6800ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.591+0000 7fc684a00640 1 -- 172.21.15.27:0/633880189 shutdown_connections 2024-04-26T23:02:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:27.591+0000 7fc684a00640 1 -- 172.21.15.27:0/633880189 wait complete. 2024-04-26T23:02:29.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:28 smithi027 ceph-mon[28034]: pgmap v119: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 298 op/s 2024-04-26T23:02:29.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:28 smithi080 ceph-mon[32324]: pgmap v119: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 298 op/s 2024-04-26T23:02:29.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:28 smithi094 ceph-mon[32889]: pgmap v119: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 298 op/s 2024-04-26T23:02:31.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:30 smithi027 ceph-mon[28034]: pgmap v120: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 747 KiB/s wr, 170 op/s 2024-04-26T23:02:31.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:30 smithi080 ceph-mon[32324]: pgmap v120: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 747 KiB/s wr, 170 op/s 2024-04-26T23:02:31.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:30 smithi094 ceph-mon[32889]: pgmap v120: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 747 KiB/s wr, 170 op/s 2024-04-26T23:02:33.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:32 smithi027 ceph-mon[28034]: pgmap v121: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 285 op/s 2024-04-26T23:02:33.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:32 smithi080 ceph-mon[32324]: pgmap v121: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 285 op/s 2024-04-26T23:02:33.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:32 smithi094 ceph-mon[32889]: pgmap v121: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 285 op/s 2024-04-26T23:02:35.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:34 smithi027 ceph-mon[28034]: pgmap v122: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 760 KiB/s wr, 197 op/s 2024-04-26T23:02:35.274 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:34 smithi080 ceph-mon[32324]: pgmap v122: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 760 KiB/s wr, 197 op/s 2024-04-26T23:02:35.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:34 smithi094 ceph-mon[32889]: pgmap v122: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 760 KiB/s wr, 197 op/s 2024-04-26T23:02:37.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:36 smithi080 ceph-mon[32324]: pgmap v123: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 760 KiB/s wr, 197 op/s 2024-04-26T23:02:37.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:36 smithi094 ceph-mon[32889]: pgmap v123: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 760 KiB/s wr, 197 op/s 2024-04-26T23:02:37.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:36 smithi027 ceph-mon[28034]: pgmap v123: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 760 KiB/s wr, 197 op/s 2024-04-26T23:02:39.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:38 smithi080 ceph-mon[32324]: pgmap v124: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 306 op/s 2024-04-26T23:02:39.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:38 smithi094 ceph-mon[32889]: pgmap v124: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 306 op/s 2024-04-26T23:02:39.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:38 smithi027 ceph-mon[28034]: pgmap v124: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 306 op/s 2024-04-26T23:02:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.746+0000 7f209a800640 1 -- 172.21.15.27:0/930774309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2094076e60 msgr2=0x7f2094077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.746+0000 7f209a800640 1 --2- 172.21.15.27:0/930774309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2094076e60 0x7f2094077240 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2084009cd0 tx=0x7f208402a8f0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.746+0000 7f209a800640 1 -- 172.21.15.27:0/930774309 shutdown_connections 2024-04-26T23:02:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.746+0000 7f209a800640 1 --2- 172.21.15.27:0/930774309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20941137d0 0x7f2094115bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.746+0000 7f209a800640 1 --2- 172.21.15.27:0/930774309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2094077780 0x7f2094113290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.746+0000 7f209a800640 1 --2- 172.21.15.27:0/930774309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2094076e60 0x7f2094077240 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.746+0000 7f209a800640 1 -- 172.21.15.27:0/930774309 >> 172.21.15.27:0/930774309 conn(0x7f2094100600 msgr2=0x7f2094102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.746+0000 7f209a800640 1 -- 172.21.15.27:0/930774309 shutdown_connections 2024-04-26T23:02:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.746+0000 7f209a800640 1 -- 172.21.15.27:0/930774309 wait complete. 2024-04-26T23:02:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.747+0000 7f209a800640 1 Processor -- start 2024-04-26T23:02:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.747+0000 7f209a800640 1 -- start start 2024-04-26T23:02:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.748+0000 7f209a800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2094076e60 0x7f20941a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.748+0000 7f209a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2094077780 0x7f20941a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.748+0000 7f209a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20941137d0 0x7f209419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.748+0000 7f209a800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f20941191f0 con 0x7f20941137d0 2024-04-26T23:02:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.748+0000 7f209a800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2094119070 con 0x7f2094076e60 2024-04-26T23:02:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.748+0000 7f209a800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2094119370 con 0x7f2094077780 2024-04-26T23:02:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.748+0000 7f2093e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20941137d0 0x7f209419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.748+0000 7f2093400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2094076e60 0x7f20941a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.748+0000 7f208aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2094077780 0x7f20941a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.748+0000 7f2093e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20941137d0 0x7f209419f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35226/0 (socket says 172.21.15.27:35226) 2024-04-26T23:02:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.748+0000 7f2093e00640 1 -- 172.21.15.27:0/2285631075 learned_addr learned my addr 172.21.15.27:0/2285631075 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.748+0000 7f208aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2094077780 0x7f20941a5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:47966/0 (socket says 172.21.15.27:47966) 2024-04-26T23:02:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.748+0000 7f2093400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2094076e60 0x7f20941a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:52232/0 (socket says 172.21.15.27:52232) 2024-04-26T23:02:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.749+0000 7f2093e00640 1 -- 172.21.15.27:0/2285631075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2094077780 msgr2=0x7f20941a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.749+0000 7f2093e00640 1 --2- 172.21.15.27:0/2285631075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2094077780 0x7f20941a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.749+0000 7f2093e00640 1 -- 172.21.15.27:0/2285631075 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2094076e60 msgr2=0x7f20941a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.749+0000 7f2093e00640 1 --2- 172.21.15.27:0/2285631075 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2094076e60 0x7f20941a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.749+0000 7f2093e00640 1 -- 172.21.15.27:0/2285631075 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209419fd70 con 0x7f20941137d0 2024-04-26T23:02:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.749+0000 7f2093400640 1 --2- 172.21.15.27:0/2285631075 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2094076e60 0x7f20941a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:02:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.749+0000 7f2093e00640 1 --2- 172.21.15.27:0/2285631075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20941137d0 0x7f209419f5f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f208c00e970 tx=0x7f208c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.750+0000 7f2090c00640 1 -- 172.21.15.27:0/2285631075 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c00ccc0 con 0x7f20941137d0 2024-04-26T23:02:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.750+0000 7f2090c00640 1 -- 172.21.15.27:0/2285631075 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f208c0054d0 con 0x7f20941137d0 2024-04-26T23:02:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.750+0000 7f209a800640 1 -- 172.21.15.27:0/2285631075 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20941a0060 con 0x7f20941137d0 2024-04-26T23:02:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.750+0000 7f2090c00640 1 -- 172.21.15.27:0/2285631075 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c010430 con 0x7f20941137d0 2024-04-26T23:02:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.750+0000 7f209a800640 1 -- 172.21.15.27:0/2285631075 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20941ac2b0 con 0x7f20941137d0 2024-04-26T23:02:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.754+0000 7f209d5dd740 1 Processor -- start 2024-04-26T23:02:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.755+0000 7f2090c00640 1 -- 172.21.15.27:0/2285631075 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f208c005000 con 0x7f20941137d0 2024-04-26T23:02:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.756+0000 7f209d5dd740 1 -- start start 2024-04-26T23:02:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.756+0000 7f209d5dd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 0x55c47177f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.756+0000 7f209d5dd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 0x55c471789a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.756+0000 7f209d5dd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c471789fb0 0x55c47178c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.756+0000 7f209d5dd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c4716e4380 con 0x55c47177f6f0 2024-04-26T23:02:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.756+0000 7f209d5dd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55c471680c30 con 0x55c47177edd0 2024-04-26T23:02:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.756+0000 7f209d5dd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c4716e0340 con 0x55c471789fb0 2024-04-26T23:02:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.756+0000 7f208aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 0x55c47177f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.757+0000 7f208aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 0x55c47177f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:52238/0 (socket says 172.21.15.27:52238) 2024-04-26T23:02:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.757+0000 7f208aa00640 1 -- 172.21.15.27:0/1543742846 learned_addr learned my addr 172.21.15.27:0/1543742846 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.757+0000 7f208aa00640 1 --2- 172.21.15.27:0/1543742846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c471789fb0 0x55c47178c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.757+0000 7f208aa00640 1 -- 172.21.15.27:0/1543742846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c471789fb0 msgr2=0x55c47178c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.757+0000 7f208aa00640 1 --2- 172.21.15.27:0/1543742846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c471789fb0 0x55c47178c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.757+0000 7f208aa00640 1 -- 172.21.15.27:0/1543742846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 msgr2=0x55c471789a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.757+0000 7f208aa00640 1 --2- 172.21.15.27:0/1543742846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 0x55c471789a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.757+0000 7f208aa00640 1 -- 172.21.15.27:0/1543742846 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4716a2940 con 0x55c47177edd0 2024-04-26T23:02:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.757+0000 7f208aa00640 1 --2- 172.21.15.27:0/1543742846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 0x55c47177f1b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f207c009fd0 tx=0x7f207c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e82aefba15193a0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.758+0000 7f2090c00640 1 --2- 172.21.15.27:0/2285631075 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f20780a3d10 0x7f20780a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.758+0000 7f2090c00640 1 -- 172.21.15.27:0/2285631075 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f208c0d0eb0 con 0x7f20941137d0 2024-04-26T23:02:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.758+0000 7f2093400640 1 --2- 172.21.15.27:0/2285631075 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f20780a3d10 0x7f20780a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.758+0000 7f2088c00640 1 -- 172.21.15.27:0/1543742846 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c005020 con 0x55c47177edd0 2024-04-26T23:02:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.758+0000 7f2088c00640 1 -- 172.21.15.27:0/1543742846 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f207c0047a0 con 0x55c47177edd0 2024-04-26T23:02:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.758+0000 7f2088c00640 1 -- 172.21.15.27:0/1543742846 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c00f9f0 con 0x55c47177edd0 2024-04-26T23:02:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.759+0000 7f209d5dd740 1 -- 172.21.15.27:0/1543742846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 msgr2=0x55c47177f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.760+0000 7f209d5dd740 1 --2- 172.21.15.27:0/1543742846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 0x55c47177f1b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f207c009fd0 tx=0x7f207c005850 comp rx=0 tx=0).stop 2024-04-26T23:02:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.760+0000 7f2093400640 1 --2- 172.21.15.27:0/2285631075 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f20780a3d10 0x7f20780a61d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2084000f80 tx=0x7f2084009360 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.760+0000 7f209d5dd740 1 -- 172.21.15.27:0/1543742846 shutdown_connections 2024-04-26T23:02:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.760+0000 7f209d5dd740 1 --2- 172.21.15.27:0/1543742846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c471789fb0 0x55c47178c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.760+0000 7f209d5dd740 1 --2- 172.21.15.27:0/1543742846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 0x55c471789a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.760+0000 7f209d5dd740 1 --2- 172.21.15.27:0/1543742846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 0x55c47177f1b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.760+0000 7f209d5dd740 1 -- 172.21.15.27:0/1543742846 >> 172.21.15.27:0/1543742846 conn(0x55c47177b2e0 msgr2=0x55c47177b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.760+0000 7f209d5dd740 1 -- 172.21.15.27:0/1543742846 shutdown_connections 2024-04-26T23:02:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.760+0000 7f209d5dd740 1 -- 172.21.15.27:0/1543742846 wait complete. 2024-04-26T23:02:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.760+0000 7f209d5dd740 1 -- start start 2024-04-26T23:02:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.761+0000 7f209d5dd740 1 Processor -- start 2024-04-26T23:02:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.762+0000 7f209d5dd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 0x55c471786f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.762+0000 7f209d5dd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 0x55c471787470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.762+0000 7f209d5dd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c471789fb0 0x55c4717879b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.762+0000 7f209d5dd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c47178cb20 con 0x55c47177f6f0 2024-04-26T23:02:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.762+0000 7f209d5dd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55c47166bc60 con 0x55c47177edd0 2024-04-26T23:02:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.762+0000 7f209d5dd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c47178cca0 con 0x55c471789fb0 2024-04-26T23:02:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.762+0000 7f209d5dd740 10 client.0 _ll_register_callbacks cb 0x55c471697e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:02:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.762+0000 7f208aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 0x55c471787470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.762+0000 7f2093400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 0x55c471786f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.762+0000 7f2093e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c471789fb0 0x55c4717879b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.762+0000 7f208aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 0x55c471787470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35244/0 (socket says 172.21.15.27:35244) 2024-04-26T23:02:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.762+0000 7f208aa00640 1 -- 172.21.15.27:0/2060796286 learned_addr learned my addr 172.21.15.27:0/2060796286 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.762+0000 7f2093400640 1 -- 172.21.15.27:0/2060796286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c471789fb0 msgr2=0x55c4717879b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.762+0000 7f2093400640 1 --2- 172.21.15.27:0/2060796286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c471789fb0 0x55c4717879b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.762+0000 7f2093400640 1 -- 172.21.15.27:0/2060796286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 msgr2=0x55c471787470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.762+0000 7f2093400640 1 --2- 172.21.15.27:0/2060796286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 0x55c471787470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.763+0000 7f2093400640 1 -- 172.21.15.27:0/2060796286 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c47167a510 con 0x55c47177edd0 2024-04-26T23:02:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.763+0000 7f2093400640 1 --2- 172.21.15.27:0/2060796286 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 0x55c471786f30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2084009ca0 tx=0x7f208402a8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d179dc7c6910d8d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.763+0000 7f2081600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T23:02:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.763+0000 7f2081600640 1 -- 172.21.15.27:0/2060796286 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084004240 con 0x55c47177edd0 2024-04-26T23:02:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.763+0000 7f2081600640 1 -- 172.21.15.27:0/2060796286 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f20840043e0 con 0x55c47177edd0 2024-04-26T23:02:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.763+0000 7f209d5dd740 1 -- 172.21.15.27:0/2060796286 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f207c0047a0 con 0x55c47177edd0 2024-04-26T23:02:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.764+0000 7f2081600640 1 -- 172.21.15.27:0/2060796286 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f20840045e0 con 0x55c47177edd0 2024-04-26T23:02:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.764+0000 7f209d5dd740 10 client.25772 fetch_fsmap learned FSMap version 30 2024-04-26T23:02:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.764+0000 7f209d5dd740 1 -- 172.21.15.27:0/2060796286 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c4713f6210 con 0x55c47177edd0 2024-04-26T23:02:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.764+0000 7f2081600640 1 -- 172.21.15.27:0/2060796286 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f207c0047a0 con 0x55c47177edd0 2024-04-26T23:02:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.764+0000 7f2081600640 1 -- 172.21.15.27:0/2060796286 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f208404e020 con 0x55c47177edd0 2024-04-26T23:02:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.764+0000 7f209d5dd740 10 client.25772 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:02:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.764+0000 7f209d5dd740 10 client.25772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:02:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.764+0000 7f209d5dd740 20 client.25772 populate_metadata read hostname 'smithi027' 2024-04-26T23:02:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.764+0000 7f209d5dd740 1 --2- 172.21.15.27:0/2060796286 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55c471799cb0 0x55c47179c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.764+0000 7f209d5dd740 4 client.25772 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:02:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.765+0000 7f209d5dd740 1 -- 172.21.15.27:0/2060796286 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4716f1f90 con 0x55c471799cb0 2024-04-26T23:02:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.765+0000 7f208aa00640 1 --2- 172.21.15.27:0/2060796286 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55c471799cb0 0x55c47179c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.766+0000 7f208aa00640 1 --2- 172.21.15.27:0/2060796286 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55c471799cb0 0x55c47179c130 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=750733e203a3659d server_cookie=4b1fd8f8033cf358 in_seq=0 out_seq=0 2024-04-26T23:02:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.766+0000 7f2081600640 10 client.25772 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.766+0000 7f2081600640 1 -- 172.21.15.27:0/2060796286 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4716f1f90 con 0x55c471799cb0 2024-04-26T23:02:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.766+0000 7f2081600640 10 client.25772 handle_command_reply: tid=0 2024-04-26T23:02:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.767+0000 7f209d5dd740 1 client.25772 shutdown 2024-04-26T23:02:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.767+0000 7f209d5dd740 1 -- 172.21.15.27:0/2060796286 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 msgr2=0x55c471786f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.767+0000 7f209d5dd740 1 --2- 172.21.15.27:0/2060796286 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 0x55c471786f30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2084009ca0 tx=0x7f208402a8f0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.767+0000 7f209d5dd740 1 -- 172.21.15.27:0/2060796286 shutdown_connections 2024-04-26T23:02:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.767+0000 7f209d5dd740 1 --2- 172.21.15.27:0/2060796286 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55c471799cb0 0x55c47179c130 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.767+0000 7f209d5dd740 1 --2- 172.21.15.27:0/2060796286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c471789fb0 0x55c4717879b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.767+0000 7f209d5dd740 1 --2- 172.21.15.27:0/2060796286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 0x55c471787470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.767+0000 7f209d5dd740 1 --2- 172.21.15.27:0/2060796286 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 0x55c471786f30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.767+0000 7f209d5dd740 1 -- 172.21.15.27:0/2060796286 >> 172.21.15.27:0/2060796286 conn(0x55c47177b2e0 msgr2=0x55c471781060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.767+0000 7f2081600640 0 client.25772 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.767+0000 7f209d5dd740 1 -- 172.21.15.27:0/2060796286 shutdown_connections 2024-04-26T23:02:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.767+0000 7f209d5dd740 1 -- 172.21.15.27:0/2060796286 wait complete. 2024-04-26T23:02:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.767+0000 7f209d5dd740 20 client.25772 trim_cache size 0 max 16384 2024-04-26T23:02:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.785+0000 7f209d5dd740 1 Processor -- start 2024-04-26T23:02:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.785+0000 7f209d5dd740 1 -- start start 2024-04-26T23:02:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.786+0000 7f209d5dd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 0x55c471795a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.786+0000 7f209d5dd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 0x55c47179d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.786+0000 7f209d5dd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c471789fb0 0x55c471788d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.786+0000 7f209d5dd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f20941b0780 con 0x55c47177f6f0 2024-04-26T23:02:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.786+0000 7f209d5dd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f20941adf90 con 0x55c47177edd0 2024-04-26T23:02:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.786+0000 7f209d5dd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f209410e420 con 0x55c471789fb0 2024-04-26T23:02:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.786+0000 7f208aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 0x55c47179d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.786+0000 7f208aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 0x55c47179d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35260/0 (socket says 172.21.15.27:35260) 2024-04-26T23:02:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.786+0000 7f208aa00640 1 -- 172.21.15.27:0/847272229 learned_addr learned my addr 172.21.15.27:0/847272229 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.786+0000 7f2093400640 1 --2- 172.21.15.27:0/847272229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 0x55c471795a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.786+0000 7f2093e00640 1 --2- 172.21.15.27:0/847272229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c471789fb0 0x55c471788d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.787+0000 7f208aa00640 1 -- 172.21.15.27:0/847272229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c471789fb0 msgr2=0x55c471788d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.787+0000 7f208aa00640 1 --2- 172.21.15.27:0/847272229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c471789fb0 0x55c471788d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.787+0000 7f208aa00640 1 -- 172.21.15.27:0/847272229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 msgr2=0x55c471795a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.787+0000 7f208aa00640 1 --2- 172.21.15.27:0/847272229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 0x55c471795a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.787+0000 7f208aa00640 1 -- 172.21.15.27:0/847272229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4714c2620 con 0x55c47177f6f0 2024-04-26T23:02:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.787+0000 7f2093400640 1 --2- 172.21.15.27:0/847272229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 0x55c471795a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:02:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.787+0000 7f208aa00640 1 --2- 172.21.15.27:0/847272229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 0x55c47179d2a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f207c0117f0 tx=0x7f207c009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decbf734c6790958 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.787+0000 7f2083e00640 1 -- 172.21.15.27:0/847272229 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c054070 con 0x55c47177f6f0 2024-04-26T23:02:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.788+0000 7f2083e00640 1 -- 172.21.15.27:0/847272229 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f207c005630 con 0x55c47177f6f0 2024-04-26T23:02:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.788+0000 7f2083e00640 1 -- 172.21.15.27:0/847272229 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c04f3d0 con 0x55c47177f6f0 2024-04-26T23:02:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.788+0000 7f209d5dd740 1 -- 172.21.15.27:0/847272229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 msgr2=0x55c47179d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.788+0000 7f209d5dd740 1 --2- 172.21.15.27:0/847272229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 0x55c47179d2a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f207c0117f0 tx=0x7f207c009f70 comp rx=0 tx=0).stop 2024-04-26T23:02:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.788+0000 7f209d5dd740 1 -- 172.21.15.27:0/847272229 shutdown_connections 2024-04-26T23:02:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.788+0000 7f209d5dd740 1 --2- 172.21.15.27:0/847272229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c471789fb0 0x55c471788d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.788+0000 7f209d5dd740 1 --2- 172.21.15.27:0/847272229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177f6f0 0x55c47179d2a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.788+0000 7f209d5dd740 1 --2- 172.21.15.27:0/847272229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c47177edd0 0x55c471795a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.788+0000 7f209d5dd740 1 -- 172.21.15.27:0/847272229 >> 172.21.15.27:0/847272229 conn(0x55c47177b2e0 msgr2=0x55c471788650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.789+0000 7f209d5dd740 1 -- 172.21.15.27:0/847272229 shutdown_connections 2024-04-26T23:02:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.789+0000 7f209d5dd740 1 -- 172.21.15.27:0/847272229 wait complete. 2024-04-26T23:02:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.789+0000 7f209d5dd740 1 -- start start 2024-04-26T23:02:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.789+0000 7f209d5dd740 1 Processor -- start 2024-04-26T23:02:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.790+0000 7f209d5dd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177edd0 0x55c471791f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.790+0000 7f209d5dd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c47177f6f0 0x55c471792460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.790+0000 7f209d5dd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c471789fb0 0x55c471783350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.790+0000 7f209d5dd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f20941ac540 con 0x55c47177edd0 2024-04-26T23:02:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.790+0000 7f209d5dd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f20941ad8c0 con 0x55c471789fb0 2024-04-26T23:02:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.790+0000 7f209d5dd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20941acb20 con 0x55c47177f6f0 2024-04-26T23:02:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.790+0000 7f209d5dd740 10 client.0 _ll_register_callbacks cb 0x7f208c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:02:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.790+0000 7f208aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c47177f6f0 0x55c471792460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.790+0000 7f2093400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177edd0 0x55c471791f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.790+0000 7f208aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c47177f6f0 0x55c471792460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:47998/0 (socket says 172.21.15.27:47998) 2024-04-26T23:02:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.790+0000 7f208aa00640 1 -- 172.21.15.27:0/2821005418 learned_addr learned my addr 172.21.15.27:0/2821005418 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.790+0000 7f2093e00640 1 --2- 172.21.15.27:0/2821005418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c471789fb0 0x55c471783350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.791+0000 7f208aa00640 1 -- 172.21.15.27:0/2821005418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c471789fb0 msgr2=0x55c471783350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.791+0000 7f208aa00640 1 --2- 172.21.15.27:0/2821005418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c471789fb0 0x55c471783350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.791+0000 7f208aa00640 1 -- 172.21.15.27:0/2821005418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177edd0 msgr2=0x55c471791f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.791+0000 7f208aa00640 1 --2- 172.21.15.27:0/2821005418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177edd0 0x55c471791f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.791+0000 7f208aa00640 1 -- 172.21.15.27:0/2821005418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c47169ecb0 con 0x55c47177f6f0 2024-04-26T23:02:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.791+0000 7f208aa00640 1 --2- 172.21.15.27:0/2821005418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c47177f6f0 0x55c471792460 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f207c010010 tx=0x7f207c011040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=940515bf639bab3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.791+0000 7f2081600640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T23:02:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.791+0000 7f2081600640 1 -- 172.21.15.27:0/2821005418 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c054040 con 0x55c47177f6f0 2024-04-26T23:02:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.791+0000 7f209d5dd740 1 -- 172.21.15.27:0/2821005418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c4716ebe00 con 0x55c47177f6f0 2024-04-26T23:02:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.791+0000 7f2081600640 1 -- 172.21.15.27:0/2821005418 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f207c011230 con 0x55c47177f6f0 2024-04-26T23:02:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.792+0000 7f2081600640 1 -- 172.21.15.27:0/2821005418 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f207c005180 con 0x55c47177f6f0 2024-04-26T23:02:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.792+0000 7f209d5dd740 10 client.25822 fetch_fsmap learned FSMap version 30 2024-04-26T23:02:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.792+0000 7f209d5dd740 1 -- 172.21.15.27:0/2821005418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c471784c10 con 0x55c47177f6f0 2024-04-26T23:02:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.792+0000 7f2081600640 1 -- 172.21.15.27:0/2821005418 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c4716ebe00 con 0x55c47177f6f0 2024-04-26T23:02:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.792+0000 7f2081600640 1 -- 172.21.15.27:0/2821005418 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f207c048040 con 0x55c47177f6f0 2024-04-26T23:02:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.792+0000 7f209d5dd740 10 client.25822 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:02:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.792+0000 7f209d5dd740 10 client.25822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:02:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.793+0000 7f209d5dd740 20 client.25822 populate_metadata read hostname 'smithi027' 2024-04-26T23:02:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.793+0000 7f209d5dd740 1 --2- 172.21.15.27:0/2821005418 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55c471811260 0x55c471813650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.793+0000 7f209d5dd740 4 client.25822 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:02:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.793+0000 7f209d5dd740 1 -- 172.21.15.27:0/2821005418 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4716674b0 con 0x55c471811260 2024-04-26T23:02:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.793+0000 7f2093400640 1 --2- 172.21.15.27:0/2821005418 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55c471811260 0x55c471813650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.793+0000 7f2093400640 1 --2- 172.21.15.27:0/2821005418 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55c471811260 0x55c471813650 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d098d7370252c15b server_cookie=226fe1d934ed0e3 in_seq=0 out_seq=0 2024-04-26T23:02:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:40.793+0000 7f2081600640 10 client.25822 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.247+0000 7f2081600640 1 -- 172.21.15.27:0/2821005418 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c4716674b0 con 0x55c471811260 2024-04-26T23:02:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.247+0000 7f2081600640 10 client.25822 handle_command_reply: tid=0 2024-04-26T23:02:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.247+0000 7f209d5dd740 1 client.25822 shutdown 2024-04-26T23:02:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.247+0000 7f209d5dd740 1 -- 172.21.15.27:0/2821005418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c47177f6f0 msgr2=0x55c471792460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.247+0000 7f209d5dd740 1 --2- 172.21.15.27:0/2821005418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c47177f6f0 0x55c471792460 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f207c010010 tx=0x7f207c011040 comp rx=0 tx=0).stop 2024-04-26T23:02:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.247+0000 7f209d5dd740 1 -- 172.21.15.27:0/2821005418 shutdown_connections 2024-04-26T23:02:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.247+0000 7f209d5dd740 1 --2- 172.21.15.27:0/2821005418 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55c471811260 0x55c471813650 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.247+0000 7f209d5dd740 1 --2- 172.21.15.27:0/2821005418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c471789fb0 0x55c471783350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.247+0000 7f209d5dd740 1 --2- 172.21.15.27:0/2821005418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c47177f6f0 0x55c471792460 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.247+0000 7f209d5dd740 1 --2- 172.21.15.27:0/2821005418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c47177edd0 0x55c471791f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.247+0000 7f209d5dd740 1 -- 172.21.15.27:0/2821005418 >> 172.21.15.27:0/2821005418 conn(0x55c47177b2e0 msgr2=0x55c47177a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.247+0000 7f2081600640 0 client.25822 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.248+0000 7f209d5dd740 1 -- 172.21.15.27:0/2821005418 shutdown_connections 2024-04-26T23:02:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.248+0000 7f209d5dd740 1 -- 172.21.15.27:0/2821005418 wait complete. 2024-04-26T23:02:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.248+0000 7f209d5dd740 20 client.25822 trim_cache size 0 max 16384 2024-04-26T23:02:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T23:02:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T23:02:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T23:02:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T23:02:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.249+0000 7f209a800640 1 -- 172.21.15.27:0/2285631075 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f20780a3d10 msgr2=0x7f20780a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.249+0000 7f209a800640 1 --2- 172.21.15.27:0/2285631075 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f20780a3d10 0x7f20780a61d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2084000f80 tx=0x7f2084009360 comp rx=0 tx=0).stop 2024-04-26T23:02:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.249+0000 7f209a800640 1 -- 172.21.15.27:0/2285631075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20941137d0 msgr2=0x7f209419f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.249+0000 7f209a800640 1 --2- 172.21.15.27:0/2285631075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20941137d0 0x7f209419f5f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f208c00e970 tx=0x7f208c00ee30 comp rx=0 tx=0).stop 2024-04-26T23:02:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.249+0000 7f209a800640 1 -- 172.21.15.27:0/2285631075 shutdown_connections 2024-04-26T23:02:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.249+0000 7f209a800640 1 --2- 172.21.15.27:0/2285631075 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f20780a3d10 0x7f20780a61d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.249+0000 7f209a800640 1 --2- 172.21.15.27:0/2285631075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20941137d0 0x7f209419f5f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.249+0000 7f209a800640 1 --2- 172.21.15.27:0/2285631075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2094077780 0x7f20941a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.249+0000 7f209a800640 1 --2- 172.21.15.27:0/2285631075 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2094076e60 0x7f20941a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.249+0000 7f209a800640 1 -- 172.21.15.27:0/2285631075 >> 172.21.15.27:0/2285631075 conn(0x7f2094100600 msgr2=0x7f2094114e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.249+0000 7f209a800640 1 -- 172.21.15.27:0/2285631075 shutdown_connections 2024-04-26T23:02:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:41.250+0000 7f209a800640 1 -- 172.21.15.27:0/2285631075 wait complete. 2024-04-26T23:02:41.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:40 smithi080 ceph-mon[32324]: pgmap v125: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 740 KiB/s wr, 223 op/s 2024-04-26T23:02:41.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:40 smithi094 ceph-mon[32889]: pgmap v125: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 740 KiB/s wr, 223 op/s 2024-04-26T23:02:41.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:40 smithi027 ceph-mon[28034]: pgmap v125: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 740 KiB/s wr, 223 op/s 2024-04-26T23:02:43.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:42 smithi027 ceph-mon[28034]: pgmap v126: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 492 op/s 2024-04-26T23:02:43.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:42 smithi080 ceph-mon[32324]: pgmap v126: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 492 op/s 2024-04-26T23:02:43.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:42 smithi094 ceph-mon[32889]: pgmap v126: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 492 op/s 2024-04-26T23:02:45.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:44 smithi080 ceph-mon[32324]: pgmap v127: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 817 KiB/s wr, 377 op/s 2024-04-26T23:02:45.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:44 smithi094 ceph-mon[32889]: pgmap v127: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 817 KiB/s wr, 377 op/s 2024-04-26T23:02:45.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:44 smithi027 ceph-mon[28034]: pgmap v127: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 817 KiB/s wr, 377 op/s 2024-04-26T23:02:47.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:46 smithi080 ceph-mon[32324]: pgmap v128: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 817 KiB/s wr, 377 op/s 2024-04-26T23:02:47.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:46 smithi094 ceph-mon[32889]: pgmap v128: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 817 KiB/s wr, 377 op/s 2024-04-26T23:02:47.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:46 smithi027 ceph-mon[28034]: pgmap v128: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 817 KiB/s wr, 377 op/s 2024-04-26T23:02:49.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:48 smithi080 ceph-mon[32324]: pgmap v129: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 526 op/s 2024-04-26T23:02:49.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:48 smithi094 ceph-mon[32889]: pgmap v129: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 526 op/s 2024-04-26T23:02:49.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:48 smithi027 ceph-mon[28034]: pgmap v129: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 526 op/s 2024-04-26T23:02:51.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:50 smithi080 ceph-mon[32324]: pgmap v130: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1011 KiB/s wr, 417 op/s 2024-04-26T23:02:51.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:50 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:02:51.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:50 smithi094 ceph-mon[32889]: pgmap v130: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1011 KiB/s wr, 417 op/s 2024-04-26T23:02:51.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:50 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:02:51.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:50 smithi027 ceph-mon[28034]: pgmap v130: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1011 KiB/s wr, 417 op/s 2024-04-26T23:02:51.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:50 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:02:52.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:51 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:02:52.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:51 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:02:52.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:51 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:02:52.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:51 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:02:52.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:51 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:02:52.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:51 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:02:52.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:51 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:02:52.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:51 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:02:52.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:51 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:02:53.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:52 smithi080 ceph-mon[32324]: pgmap v131: 129 pgs: 129 active+clean; 1.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 591 op/s 2024-04-26T23:02:53.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:52 smithi094 ceph-mon[32889]: pgmap v131: 129 pgs: 129 active+clean; 1.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 591 op/s 2024-04-26T23:02:53.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:52 smithi027 ceph-mon[28034]: pgmap v131: 129 pgs: 129 active+clean; 1.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 591 op/s 2024-04-26T23:02:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.381+0000 7fc431c00640 1 -- 172.21.15.27:0/2104240223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc42c108760 msgr2=0x7fc42c108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.381+0000 7fc431c00640 1 --2- 172.21.15.27:0/2104240223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc42c108760 0x7fc42c108b40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009930 tx=0x7fc41c02a420 comp rx=0 tx=0).stop 2024-04-26T23:02:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.382+0000 7fc431c00640 1 -- 172.21.15.27:0/2104240223 shutdown_connections 2024-04-26T23:02:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.382+0000 7fc431c00640 1 --2- 172.21.15.27:0/2104240223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc42c102750 0x7fc42c102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.382+0000 7fc431c00640 1 --2- 172.21.15.27:0/2104240223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc42c103110 0x7fc42c10f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.382+0000 7fc431c00640 1 --2- 172.21.15.27:0/2104240223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc42c108760 0x7fc42c108b40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.382+0000 7fc431c00640 1 -- 172.21.15.27:0/2104240223 >> 172.21.15.27:0/2104240223 conn(0x7fc42c0fe240 msgr2=0x7fc42c100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.382+0000 7fc431c00640 1 -- 172.21.15.27:0/2104240223 shutdown_connections 2024-04-26T23:02:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.382+0000 7fc431c00640 1 -- 172.21.15.27:0/2104240223 wait complete. 2024-04-26T23:02:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.383+0000 7fc431c00640 1 Processor -- start 2024-04-26T23:02:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.383+0000 7fc431c00640 1 -- start start 2024-04-26T23:02:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.383+0000 7fc431c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc42c102750 0x7fc42c19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.383+0000 7fc431c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc42c103110 0x7fc42c19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.383+0000 7fc431c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc42c108760 0x7fc42c19fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.383+0000 7fc431c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c112a20 con 0x7fc42c103110 2024-04-26T23:02:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.383+0000 7fc431c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1128a0 con 0x7fc42c102750 2024-04-26T23:02:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.383+0000 7fc431c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c112ba0 con 0x7fc42c108760 2024-04-26T23:02:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.384+0000 7fc42be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc42c102750 0x7fc42c19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.384+0000 7fc42be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc42c102750 0x7fc42c19b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:43556/0 (socket says 172.21.15.27:43556) 2024-04-26T23:02:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.384+0000 7fc42b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc42c103110 0x7fc42c19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.384+0000 7fc42be00640 1 -- 172.21.15.27:0/2881029145 learned_addr learned my addr 172.21.15.27:0/2881029145 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.384+0000 7fc431200640 1 --2- 172.21.15.27:0/2881029145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc42c108760 0x7fc42c19fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.384+0000 7fc42be00640 1 -- 172.21.15.27:0/2881029145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc42c108760 msgr2=0x7fc42c19fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.384+0000 7fc42be00640 1 --2- 172.21.15.27:0/2881029145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc42c108760 0x7fc42c19fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.384+0000 7fc42be00640 1 -- 172.21.15.27:0/2881029145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc42c103110 msgr2=0x7fc42c19b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.384+0000 7fc42be00640 1 --2- 172.21.15.27:0/2881029145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc42c103110 0x7fc42c19b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.384+0000 7fc42be00640 1 -- 172.21.15.27:0/2881029145 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c1a01a0 con 0x7fc42c102750 2024-04-26T23:02:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.384+0000 7fc42be00640 1 --2- 172.21.15.27:0/2881029145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc42c102750 0x7fc42c19b1b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc41c005940 tx=0x7fc41c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.385+0000 7fc428c00640 1 -- 172.21.15.27:0/2881029145 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c03f030 con 0x7fc42c102750 2024-04-26T23:02:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.385+0000 7fc431c00640 1 -- 172.21.15.27:0/2881029145 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c1a0430 con 0x7fc42c102750 2024-04-26T23:02:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.385+0000 7fc428c00640 1 -- 172.21.15.27:0/2881029145 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc41c0044c0 con 0x7fc42c102750 2024-04-26T23:02:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.385+0000 7fc428c00640 1 -- 172.21.15.27:0/2881029145 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c035070 con 0x7fc42c102750 2024-04-26T23:02:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.385+0000 7fc431c00640 1 -- 172.21.15.27:0/2881029145 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c1a7d20 con 0x7fc42c102750 2024-04-26T23:02:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.388+0000 7fc428c00640 1 -- 172.21.15.27:0/2881029145 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc41c042030 con 0x7fc42c102750 2024-04-26T23:02:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.389+0000 7fc428c00640 1 --2- 172.21.15.27:0/2881029145 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc4100a3d00 0x7fc4100a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.389+0000 7fc435f95740 1 Processor -- start 2024-04-26T23:02:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.389+0000 7fc428c00640 1 -- 172.21.15.27:0/2881029145 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fc41c0f0fd0 con 0x7fc42c102750 2024-04-26T23:02:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.390+0000 7fc42b400640 1 --2- 172.21.15.27:0/2881029145 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc4100a3d00 0x7fc4100a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.390+0000 7fc435f95740 1 -- start start 2024-04-26T23:02:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.390+0000 7fc435f95740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b94149cfe0 0x55b94149d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.390+0000 7fc435f95740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 0x55b9414a7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.390+0000 7fc435f95740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9414a81c0 0x55b9414aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.390+0000 7fc435f95740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b941402380 con 0x55b94149d900 2024-04-26T23:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.390+0000 7fc435f95740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55b94139ec30 con 0x55b94149cfe0 2024-04-26T23:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.390+0000 7fc435f95740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b9413fe340 con 0x55b9414a81c0 2024-04-26T23:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.390+0000 7fc42be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b94149cfe0 0x55b94149d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.390+0000 7fc42be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b94149cfe0 0x55b94149d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:43566/0 (socket says 172.21.15.27:43566) 2024-04-26T23:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.390+0000 7fc42be00640 1 -- 172.21.15.27:0/3514831080 learned_addr learned my addr 172.21.15.27:0/3514831080 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.390+0000 7fc431200640 1 --2- 172.21.15.27:0/3514831080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9414a81c0 0x55b9414aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.391+0000 7fc42b400640 1 --2- 172.21.15.27:0/3514831080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 0x55b9414a7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.391+0000 7fc42be00640 1 -- 172.21.15.27:0/3514831080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9414a81c0 msgr2=0x55b9414aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.391+0000 7fc42be00640 1 --2- 172.21.15.27:0/3514831080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9414a81c0 0x55b9414aa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.391+0000 7fc42be00640 1 -- 172.21.15.27:0/3514831080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 msgr2=0x55b9414a7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.391+0000 7fc42be00640 1 --2- 172.21.15.27:0/3514831080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 0x55b9414a7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.391+0000 7fc42be00640 1 -- 172.21.15.27:0/3514831080 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9413c0940 con 0x55b94149cfe0 2024-04-26T23:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.391+0000 7fc431200640 1 --2- 172.21.15.27:0/3514831080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9414a81c0 0x55b9414aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.391+0000 7fc42b400640 1 --2- 172.21.15.27:0/2881029145 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc4100a3d00 0x7fc4100a61c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc42c076be0 tx=0x7fc418011440 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.391+0000 7fc42be00640 1 --2- 172.21.15.27:0/3514831080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b94149cfe0 0x55b94149d3c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0b9550 tx=0x7fc41c009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d59a18b799347722 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.391+0000 7fc421600640 1 -- 172.21.15.27:0/3514831080 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c049040 con 0x55b94149cfe0 2024-04-26T23:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.391+0000 7fc421600640 1 -- 172.21.15.27:0/3514831080 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc41c0dd070 con 0x55b94149cfe0 2024-04-26T23:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.391+0000 7fc421600640 1 -- 172.21.15.27:0/3514831080 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c0c52b0 con 0x55b94149cfe0 2024-04-26T23:02:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.392+0000 7fc435f95740 1 -- 172.21.15.27:0/3514831080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b94149cfe0 msgr2=0x55b94149d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.392+0000 7fc435f95740 1 --2- 172.21.15.27:0/3514831080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b94149cfe0 0x55b94149d3c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0b9550 tx=0x7fc41c009680 comp rx=0 tx=0).stop 2024-04-26T23:02:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.392+0000 7fc435f95740 1 -- 172.21.15.27:0/3514831080 shutdown_connections 2024-04-26T23:02:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.392+0000 7fc435f95740 1 --2- 172.21.15.27:0/3514831080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9414a81c0 0x55b9414aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.392+0000 7fc435f95740 1 --2- 172.21.15.27:0/3514831080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 0x55b9414a7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.392+0000 7fc435f95740 1 --2- 172.21.15.27:0/3514831080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b94149cfe0 0x55b94149d3c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.392+0000 7fc435f95740 1 -- 172.21.15.27:0/3514831080 >> 172.21.15.27:0/3514831080 conn(0x55b9414994f0 msgr2=0x55b9414998d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.392+0000 7fc435f95740 1 -- 172.21.15.27:0/3514831080 shutdown_connections 2024-04-26T23:02:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.392+0000 7fc435f95740 1 -- 172.21.15.27:0/3514831080 wait complete. 2024-04-26T23:02:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.393+0000 7fc435f95740 1 -- start start 2024-04-26T23:02:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.393+0000 7fc435f95740 1 Processor -- start 2024-04-26T23:02:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc435f95740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b94149cfe0 0x55b9414a5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc435f95740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 0x55b9414a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc435f95740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9414a81c0 0x55b9414a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc435f95740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b9414aabb0 con 0x55b94149d900 2024-04-26T23:02:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc435f95740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55b941389c60 con 0x55b94149cfe0 2024-04-26T23:02:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc435f95740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b9414aad30 con 0x55b9414a81c0 2024-04-26T23:02:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc435f95740 10 client.0 _ll_register_callbacks cb 0x55b9413b5e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:02:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc42be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b94149cfe0 0x55b9414a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc42be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b94149cfe0 0x55b9414a5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:43576/0 (socket says 172.21.15.27:43576) 2024-04-26T23:02:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc42be00640 1 -- 172.21.15.27:0/2905240573 learned_addr learned my addr 172.21.15.27:0/2905240573 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc42b400640 1 --2- 172.21.15.27:0/2905240573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 0x55b9414a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc431200640 1 --2- 172.21.15.27:0/2905240573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9414a81c0 0x55b9414a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc42be00640 1 -- 172.21.15.27:0/2905240573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9414a81c0 msgr2=0x55b9414a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc42be00640 1 --2- 172.21.15.27:0/2905240573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9414a81c0 0x55b9414a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc42be00640 1 -- 172.21.15.27:0/2905240573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 msgr2=0x55b9414a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc42be00640 1 --2- 172.21.15.27:0/2905240573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 0x55b9414a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc42be00640 1 -- 172.21.15.27:0/2905240573 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b941398510 con 0x55b94149cfe0 2024-04-26T23:02:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc431200640 1 --2- 172.21.15.27:0/2905240573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9414a81c0 0x55b9414a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:02:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.394+0000 7fc42b400640 1 --2- 172.21.15.27:0/2905240573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 0x55b9414a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:02:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.395+0000 7fc42be00640 1 --2- 172.21.15.27:0/2905240573 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b94149cfe0 0x55b9414a5120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0c5210 tx=0x7fc41c02a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baa3fdf95e60e342 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.395+0000 7fc416000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T23:02:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.395+0000 7fc416000640 1 -- 172.21.15.27:0/2905240573 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c049050 con 0x55b94149cfe0 2024-04-26T23:02:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.395+0000 7fc416000640 1 -- 172.21.15.27:0/2905240573 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fc41c0c52b0 con 0x55b94149cfe0 2024-04-26T23:02:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.395+0000 7fc435f95740 1 -- 172.21.15.27:0/2905240573 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc41c0dd070 con 0x55b94149cfe0 2024-04-26T23:02:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.395+0000 7fc416000640 1 -- 172.21.15.27:0/2905240573 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc41c0c54b0 con 0x55b94149cfe0 2024-04-26T23:02:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.395+0000 7fc435f95740 10 client.25790 fetch_fsmap learned FSMap version 30 2024-04-26T23:02:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.395+0000 7fc435f95740 1 -- 172.21.15.27:0/2905240573 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b941114210 con 0x55b94149cfe0 2024-04-26T23:02:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.395+0000 7fc416000640 1 -- 172.21.15.27:0/2905240573 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc41c0dd070 con 0x55b94149cfe0 2024-04-26T23:02:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.396+0000 7fc416000640 1 -- 172.21.15.27:0/2905240573 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc41c0d8020 con 0x55b94149cfe0 2024-04-26T23:02:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.396+0000 7fc435f95740 10 client.25790 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:02:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.396+0000 7fc435f95740 10 client.25790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:02:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.396+0000 7fc435f95740 20 client.25790 populate_metadata read hostname 'smithi027' 2024-04-26T23:02:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.396+0000 7fc435f95740 1 --2- 172.21.15.27:0/2905240573 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55b9414c0220 0x55b9414c2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.396+0000 7fc435f95740 4 client.25790 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:02:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.396+0000 7fc435f95740 1 -- 172.21.15.27:0/2905240573 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b94140ff90 con 0x55b9414c0220 2024-04-26T23:02:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.396+0000 7fc42b400640 1 --2- 172.21.15.27:0/2905240573 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55b9414c0220 0x55b9414c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.397+0000 7fc416000640 10 client.25790 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.396+0000 7fc42b400640 1 --2- 172.21.15.27:0/2905240573 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55b9414c0220 0x55b9414c2630 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5a489ca34153b8c server_cookie=eedbe4a491940c32 in_seq=0 out_seq=0 2024-04-26T23:02:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.397+0000 7fc416000640 1 -- 172.21.15.27:0/2905240573 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b94140ff90 con 0x55b9414c0220 2024-04-26T23:02:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.397+0000 7fc416000640 10 client.25790 handle_command_reply: tid=0 2024-04-26T23:02:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.397+0000 7fc435f95740 1 client.25790 shutdown 2024-04-26T23:02:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.398+0000 7fc435f95740 1 -- 172.21.15.27:0/2905240573 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b94149cfe0 msgr2=0x55b9414a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.398+0000 7fc435f95740 1 --2- 172.21.15.27:0/2905240573 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b94149cfe0 0x55b9414a5120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0c5210 tx=0x7fc41c02a420 comp rx=0 tx=0).stop 2024-04-26T23:02:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.398+0000 7fc435f95740 1 -- 172.21.15.27:0/2905240573 shutdown_connections 2024-04-26T23:02:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.398+0000 7fc435f95740 1 --2- 172.21.15.27:0/2905240573 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55b9414c0220 0x55b9414c2630 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.398+0000 7fc435f95740 1 --2- 172.21.15.27:0/2905240573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9414a81c0 0x55b9414a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.398+0000 7fc435f95740 1 --2- 172.21.15.27:0/2905240573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 0x55b9414a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.398+0000 7fc416000640 0 client.25790 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.398+0000 7fc435f95740 1 --2- 172.21.15.27:0/2905240573 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b94149cfe0 0x55b9414a5120 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.398+0000 7fc435f95740 1 -- 172.21.15.27:0/2905240573 >> 172.21.15.27:0/2905240573 conn(0x55b9414994f0 msgr2=0x55b94149f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.398+0000 7fc435f95740 1 -- 172.21.15.27:0/2905240573 shutdown_connections 2024-04-26T23:02:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.398+0000 7fc435f95740 1 -- 172.21.15.27:0/2905240573 wait complete. 2024-04-26T23:02:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.398+0000 7fc435f95740 20 client.25790 trim_cache size 0 max 16384 2024-04-26T23:02:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.416+0000 7fc435f95740 1 Processor -- start 2024-04-26T23:02:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.416+0000 7fc435f95740 1 -- start start 2024-04-26T23:02:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.417+0000 7fc435f95740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b94149cfe0 0x55b9414b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.417+0000 7fc435f95740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 0x55b9414c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.417+0000 7fc435f95740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9414a81c0 0x55b9414c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.417+0000 7fc435f95740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1b0510 con 0x55b94149d900 2024-04-26T23:02:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.417+0000 7fc435f95740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1add20 con 0x55b9414a81c0 2024-04-26T23:02:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.417+0000 7fc435f95740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1abb70 con 0x55b94149cfe0 2024-04-26T23:02:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.417+0000 7fc42be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b94149cfe0 0x55b9414b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.417+0000 7fc431200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9414a81c0 0x55b9414c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.417+0000 7fc42b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 0x55b9414c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.417+0000 7fc42be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b94149cfe0 0x55b9414b39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:33734/0 (socket says 172.21.15.27:33734) 2024-04-26T23:02:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.417+0000 7fc42be00640 1 -- 172.21.15.27:0/2037815932 learned_addr learned my addr 172.21.15.27:0/2037815932 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.417+0000 7fc431200640 1 -- 172.21.15.27:0/2037815932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b94149cfe0 msgr2=0x55b9414b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.418+0000 7fc431200640 1 --2- 172.21.15.27:0/2037815932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b94149cfe0 0x55b9414b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.418+0000 7fc431200640 1 -- 172.21.15.27:0/2037815932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 msgr2=0x55b9414c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.418+0000 7fc431200640 1 --2- 172.21.15.27:0/2037815932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 0x55b9414c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.418+0000 7fc431200640 1 -- 172.21.15.27:0/2037815932 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9411e0620 con 0x55b9414a81c0 2024-04-26T23:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.418+0000 7fc42be00640 1 --2- 172.21.15.27:0/2037815932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b94149cfe0 0x55b9414b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.418+0000 7fc42b400640 1 --2- 172.21.15.27:0/2037815932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 0x55b9414c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.418+0000 7fc431200640 1 --2- 172.21.15.27:0/2037815932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9414a81c0 0x55b9414c8480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55b94149a680 tx=0x7fc424003ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62e4b1051cb26821 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.418+0000 7fc420c00640 1 -- 172.21.15.27:0/2037815932 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4240045b0 con 0x55b9414a81c0 2024-04-26T23:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.418+0000 7fc420c00640 1 -- 172.21.15.27:0/2037815932 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc424004750 con 0x55b9414a81c0 2024-04-26T23:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.418+0000 7fc420c00640 1 -- 172.21.15.27:0/2037815932 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42401c470 con 0x55b9414a81c0 2024-04-26T23:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.419+0000 7fc435f95740 1 -- 172.21.15.27:0/2037815932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9414a81c0 msgr2=0x55b9414c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.419+0000 7fc435f95740 1 --2- 172.21.15.27:0/2037815932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9414a81c0 0x55b9414c8480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55b94149a680 tx=0x7fc424003ed0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.419+0000 7fc435f95740 1 -- 172.21.15.27:0/2037815932 shutdown_connections 2024-04-26T23:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.419+0000 7fc435f95740 1 --2- 172.21.15.27:0/2037815932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9414a81c0 0x55b9414c8480 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.419+0000 7fc435f95740 1 --2- 172.21.15.27:0/2037815932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149d900 0x55b9414c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.419+0000 7fc435f95740 1 --2- 172.21.15.27:0/2037815932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b94149cfe0 0x55b9414b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.419+0000 7fc435f95740 1 -- 172.21.15.27:0/2037815932 >> 172.21.15.27:0/2037815932 conn(0x55b9414994f0 msgr2=0x55b9414a0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.419+0000 7fc435f95740 1 -- 172.21.15.27:0/2037815932 shutdown_connections 2024-04-26T23:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.419+0000 7fc435f95740 1 -- 172.21.15.27:0/2037815932 wait complete. 2024-04-26T23:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.420+0000 7fc435f95740 1 -- start start 2024-04-26T23:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.420+0000 7fc435f95740 1 Processor -- start 2024-04-26T23:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.420+0000 7fc435f95740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149cfe0 0x55b9414afe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.420+0000 7fc435f95740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b94149d900 0x55b9414b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.420+0000 7fc435f95740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9414a81c0 0x55b9414ab570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.420+0000 7fc435f95740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1ac1e0 con 0x55b94149cfe0 2024-04-26T23:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.420+0000 7fc435f95740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1ad650 con 0x55b9414a81c0 2024-04-26T23:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.420+0000 7fc435f95740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1ac8b0 con 0x55b94149d900 2024-04-26T23:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.421+0000 7fc435f95740 10 client.0 _ll_register_callbacks cb 0x7fc41c0bc7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.421+0000 7fc42b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b94149d900 0x55b9414b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.421+0000 7fc431200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9414a81c0 0x55b9414ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.421+0000 7fc42b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b94149d900 0x55b9414b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:33744/0 (socket says 172.21.15.27:33744) 2024-04-26T23:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.421+0000 7fc42b400640 1 -- 172.21.15.27:0/632557228 learned_addr learned my addr 172.21.15.27:0/632557228 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.421+0000 7fc42be00640 1 --2- 172.21.15.27:0/632557228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149cfe0 0x55b9414afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.421+0000 7fc431200640 1 -- 172.21.15.27:0/632557228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b94149d900 msgr2=0x55b9414b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.421+0000 7fc431200640 1 --2- 172.21.15.27:0/632557228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b94149d900 0x55b9414b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.421+0000 7fc431200640 1 -- 172.21.15.27:0/632557228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149cfe0 msgr2=0x55b9414afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.421+0000 7fc431200640 1 --2- 172.21.15.27:0/632557228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149cfe0 0x55b9414afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.421+0000 7fc431200640 1 -- 172.21.15.27:0/632557228 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9413bccb0 con 0x55b9414a81c0 2024-04-26T23:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.421+0000 7fc42be00640 1 --2- 172.21.15.27:0/632557228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149cfe0 0x55b9414afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:02:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.422+0000 7fc431200640 1 --2- 172.21.15.27:0/632557228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9414a81c0 0x55b9414ab570 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc424006010 tx=0x7fc424005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3f0355a6ec0881 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:02:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.422+0000 7fc416000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T23:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.422+0000 7fc416000640 1 -- 172.21.15.27:0/632557228 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42402a060 con 0x55b9414a81c0 2024-04-26T23:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.422+0000 7fc416000640 1 -- 172.21.15.27:0/632557228 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fc424004c50 con 0x55b9414a81c0 2024-04-26T23:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.422+0000 7fc435f95740 1 -- 172.21.15.27:0/632557228 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc424004750 con 0x55b9414a81c0 2024-04-26T23:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.422+0000 7fc416000640 1 -- 172.21.15.27:0/632557228 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc4240045b0 con 0x55b9414a81c0 2024-04-26T23:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.422+0000 7fc435f95740 10 client.25796 fetch_fsmap learned FSMap version 30 2024-04-26T23:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.422+0000 7fc435f95740 1 -- 172.21.15.27:0/632557228 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b941526260 con 0x55b9414a81c0 2024-04-26T23:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.422+0000 7fc416000640 1 -- 172.21.15.27:0/632557228 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc424004750 con 0x55b9414a81c0 2024-04-26T23:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.423+0000 7fc416000640 1 -- 172.21.15.27:0/632557228 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc424024030 con 0x55b9414a81c0 2024-04-26T23:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.423+0000 7fc435f95740 10 client.25796 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.423+0000 7fc435f95740 10 client.25796 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.423+0000 7fc435f95740 20 client.25796 populate_metadata read hostname 'smithi027' 2024-04-26T23:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.423+0000 7fc435f95740 1 --2- 172.21.15.27:0/632557228 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55b941526960 0x55b941528d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.423+0000 7fc435f95740 4 client.25796 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.423+0000 7fc435f95740 1 -- 172.21.15.27:0/632557228 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b941409e00 con 0x55b941526960 2024-04-26T23:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.423+0000 7fc42be00640 1 --2- 172.21.15.27:0/632557228 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55b941526960 0x55b941528d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.424+0000 7fc42be00640 1 --2- 172.21.15.27:0/632557228 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55b941526960 0x55b941528d80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d372b4c0d1bbdc5 server_cookie=6008b88538f31032 in_seq=0 out_seq=0 2024-04-26T23:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.424+0000 7fc416000640 10 client.25796 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.507+0000 7fc416000640 1 -- 172.21.15.27:0/632557228 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b941409e00 con 0x55b941526960 2024-04-26T23:02:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.507+0000 7fc416000640 10 client.25796 handle_command_reply: tid=0 2024-04-26T23:02:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.508+0000 7fc435f95740 1 client.25796 shutdown 2024-04-26T23:02:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.508+0000 7fc435f95740 1 -- 172.21.15.27:0/632557228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9414a81c0 msgr2=0x55b9414ab570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.508+0000 7fc435f95740 1 --2- 172.21.15.27:0/632557228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9414a81c0 0x55b9414ab570 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc424006010 tx=0x7fc424005250 comp rx=0 tx=0).stop 2024-04-26T23:02:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.508+0000 7fc435f95740 1 -- 172.21.15.27:0/632557228 shutdown_connections 2024-04-26T23:02:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.508+0000 7fc435f95740 1 --2- 172.21.15.27:0/632557228 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x55b941526960 0x55b941528d80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.508+0000 7fc435f95740 1 --2- 172.21.15.27:0/632557228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9414a81c0 0x55b9414ab570 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.508+0000 7fc435f95740 1 --2- 172.21.15.27:0/632557228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b94149d900 0x55b9414b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.508+0000 7fc416000640 0 client.25796 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:02:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.508+0000 7fc435f95740 1 --2- 172.21.15.27:0/632557228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b94149cfe0 0x55b9414afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.508+0000 7fc435f95740 1 -- 172.21.15.27:0/632557228 >> 172.21.15.27:0/632557228 conn(0x55b9414994f0 msgr2=0x55b94149e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.508+0000 7fc435f95740 1 -- 172.21.15.27:0/632557228 shutdown_connections 2024-04-26T23:02:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.508+0000 7fc435f95740 1 -- 172.21.15.27:0/632557228 wait complete. 2024-04-26T23:02:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.508+0000 7fc435f95740 20 client.25796 trim_cache size 0 max 16384 2024-04-26T23:02:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T23:02:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T23:02:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T23:02:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T23:02:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.509+0000 7fc431c00640 1 -- 172.21.15.27:0/2881029145 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc4100a3d00 msgr2=0x7fc4100a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.509+0000 7fc431c00640 1 --2- 172.21.15.27:0/2881029145 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc4100a3d00 0x7fc4100a61c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc42c076be0 tx=0x7fc418011440 comp rx=0 tx=0).stop 2024-04-26T23:02:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.509+0000 7fc431c00640 1 -- 172.21.15.27:0/2881029145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc42c102750 msgr2=0x7fc42c19b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:02:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.509+0000 7fc431c00640 1 --2- 172.21.15.27:0/2881029145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc42c102750 0x7fc42c19b1b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc41c005940 tx=0x7fc41c0040c0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.509+0000 7fc431c00640 1 -- 172.21.15.27:0/2881029145 shutdown_connections 2024-04-26T23:02:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.509+0000 7fc431c00640 1 --2- 172.21.15.27:0/2881029145 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc4100a3d00 0x7fc4100a61c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.509+0000 7fc431c00640 1 --2- 172.21.15.27:0/2881029145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc42c108760 0x7fc42c19fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.509+0000 7fc431c00640 1 --2- 172.21.15.27:0/2881029145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc42c103110 0x7fc42c19b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.509+0000 7fc431c00640 1 --2- 172.21.15.27:0/2881029145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc42c102750 0x7fc42c19b1b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:02:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.509+0000 7fc431c00640 1 -- 172.21.15.27:0/2881029145 >> 172.21.15.27:0/2881029145 conn(0x7fc42c0fe240 msgr2=0x7fc42c10d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:02:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.509+0000 7fc431c00640 1 -- 172.21.15.27:0/2881029145 shutdown_connections 2024-04-26T23:02:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:02:54.509+0000 7fc431c00640 1 -- 172.21.15.27:0/2881029145 wait complete. 2024-04-26T23:02:55.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:54 smithi080 ceph-mon[32324]: pgmap v132: 129 pgs: 129 active+clean; 1.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 323 op/s 2024-04-26T23:02:55.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:54 smithi094 ceph-mon[32889]: pgmap v132: 129 pgs: 129 active+clean; 1.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 323 op/s 2024-04-26T23:02:55.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:54 smithi027 ceph-mon[28034]: pgmap v132: 129 pgs: 129 active+clean; 1.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 323 op/s 2024-04-26T23:02:57.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:56 smithi080 ceph-mon[32324]: pgmap v133: 129 pgs: 129 active+clean; 1.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 323 op/s 2024-04-26T23:02:57.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:56 smithi094 ceph-mon[32889]: pgmap v133: 129 pgs: 129 active+clean; 1.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 323 op/s 2024-04-26T23:02:57.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:56 smithi027 ceph-mon[28034]: pgmap v133: 129 pgs: 129 active+clean; 1.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 323 op/s 2024-04-26T23:02:59.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:02:58 smithi080 ceph-mon[32324]: pgmap v134: 129 pgs: 129 active+clean; 1.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 496 op/s 2024-04-26T23:02:59.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:02:58 smithi094 ceph-mon[32889]: pgmap v134: 129 pgs: 129 active+clean; 1.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 496 op/s 2024-04-26T23:02:59.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:02:58 smithi027 ceph-mon[28034]: pgmap v134: 129 pgs: 129 active+clean; 1.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 496 op/s 2024-04-26T23:03:01.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:00 smithi080 ceph-mon[32324]: pgmap v135: 129 pgs: 129 active+clean; 1.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-04-26T23:03:01.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:00 smithi094 ceph-mon[32889]: pgmap v135: 129 pgs: 129 active+clean; 1.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-04-26T23:03:01.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:00 smithi027 ceph-mon[28034]: pgmap v135: 129 pgs: 129 active+clean; 1.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-04-26T23:03:03.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:02 smithi080 ceph-mon[32324]: pgmap v136: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 532 op/s 2024-04-26T23:03:03.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:02 smithi094 ceph-mon[32889]: pgmap v136: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 532 op/s 2024-04-26T23:03:03.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:02 smithi027 ceph-mon[28034]: pgmap v136: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 532 op/s 2024-04-26T23:03:05.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:04 smithi080 ceph-mon[32324]: pgmap v137: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.1 MiB/s wr, 357 op/s 2024-04-26T23:03:05.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:04 smithi094 ceph-mon[32889]: pgmap v137: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.1 MiB/s wr, 357 op/s 2024-04-26T23:03:05.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:04 smithi027 ceph-mon[28034]: pgmap v137: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.1 MiB/s wr, 357 op/s 2024-04-26T23:03:07.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:06 smithi080 ceph-mon[32324]: pgmap v138: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.1 MiB/s wr, 357 op/s 2024-04-26T23:03:07.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:06 smithi094 ceph-mon[32889]: pgmap v138: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.1 MiB/s wr, 357 op/s 2024-04-26T23:03:07.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:06 smithi027 ceph-mon[28034]: pgmap v138: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.1 MiB/s wr, 357 op/s 2024-04-26T23:03:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.663+0000 7fe948c00640 1 -- 172.21.15.27:0/1877307471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe944102910 msgr2=0x7fe944102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.663+0000 7fe948c00640 1 --2- 172.21.15.27:0/1877307471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe944102910 0x7fe944102d90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe930009890 tx=0x7fe93002a360 comp rx=0 tx=0).stop 2024-04-26T23:03:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.663+0000 7fe948c00640 1 -- 172.21.15.27:0/1877307471 shutdown_connections 2024-04-26T23:03:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.663+0000 7fe948c00640 1 --2- 172.21.15.27:0/1877307471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9441032d0 0x7fe94410f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.663+0000 7fe948c00640 1 --2- 172.21.15.27:0/1877307471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe944102910 0x7fe944102d90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.663+0000 7fe948c00640 1 --2- 172.21.15.27:0/1877307471 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe944108920 0x7fe944108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.663+0000 7fe948c00640 1 -- 172.21.15.27:0/1877307471 >> 172.21.15.27:0/1877307471 conn(0x7fe9440fe3c0 msgr2=0x7fe9441007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.663+0000 7fe948c00640 1 -- 172.21.15.27:0/1877307471 shutdown_connections 2024-04-26T23:03:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.664+0000 7fe948c00640 1 -- 172.21.15.27:0/1877307471 wait complete. 2024-04-26T23:03:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.665+0000 7fe948c00640 1 Processor -- start 2024-04-26T23:03:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.666+0000 7fe948c00640 1 -- start start 2024-04-26T23:03:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.666+0000 7fe948c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe944102910 0x7fe94410f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.668+0000 7fe948c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9441032d0 0x7fe94410c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.668+0000 7fe948c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe944108920 0x7fe94410cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.668+0000 7fe948c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe944076870 con 0x7fe944108920 2024-04-26T23:03:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.668+0000 7fe948c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe9440766f0 con 0x7fe9441032d0 2024-04-26T23:03:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.668+0000 7fe948c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe9440769f0 con 0x7fe944102910 2024-04-26T23:03:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.669+0000 7fe941e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe944102910 0x7fe94410f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.669+0000 7fe941400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9441032d0 0x7fe94410c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.670+0000 7fe941e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe944102910 0x7fe94410f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:54282/0 (socket says 172.21.15.27:54282) 2024-04-26T23:03:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.670+0000 7fe941e00640 1 -- 172.21.15.27:0/4092295285 learned_addr learned my addr 172.21.15.27:0/4092295285 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:03:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.670+0000 7fe941e00640 1 -- 172.21.15.27:0/4092295285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9441032d0 msgr2=0x7fe94410c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.670+0000 7fe941e00640 1 --2- 172.21.15.27:0/4092295285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9441032d0 0x7fe94410c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.670+0000 7fe941e00640 1 -- 172.21.15.27:0/4092295285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe944108920 msgr2=0x7fe94410cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.670+0000 7fe941e00640 1 --2- 172.21.15.27:0/4092295285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe944108920 0x7fe94410cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.670+0000 7fe941e00640 1 -- 172.21.15.27:0/4092295285 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9441abac0 con 0x7fe944102910 2024-04-26T23:03:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.670+0000 7fe941400640 1 --2- 172.21.15.27:0/4092295285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9441032d0 0x7fe94410c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:03:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.671+0000 7fe941e00640 1 --2- 172.21.15.27:0/4092295285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe944102910 0x7fe94410f7d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe93400e9c0 tx=0x7fe93400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.671+0000 7fe93aa00640 1 -- 172.21.15.27:0/4092295285 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93400cde0 con 0x7fe944102910 2024-04-26T23:03:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.671+0000 7fe93aa00640 1 -- 172.21.15.27:0/4092295285 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9340035f0 con 0x7fe944102910 2024-04-26T23:03:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.671+0000 7fe93aa00640 1 -- 172.21.15.27:0/4092295285 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934010690 con 0x7fe944102910 2024-04-26T23:03:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.671+0000 7fe948c00640 1 -- 172.21.15.27:0/4092295285 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9441abd50 con 0x7fe944102910 2024-04-26T23:03:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.672+0000 7fe948c00640 1 -- 172.21.15.27:0/4092295285 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9441ac1c0 con 0x7fe944102910 2024-04-26T23:03:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.678+0000 7fe93aa00640 1 -- 172.21.15.27:0/4092295285 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe9340037a0 con 0x7fe944102910 2024-04-26T23:03:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.680+0000 7fe94b5fd740 1 Processor -- start 2024-04-26T23:03:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.681+0000 7fe94b5fd740 1 -- start start 2024-04-26T23:03:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.683+0000 7fe94b5fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2119dd0 0x5638a211a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.683+0000 7fe94b5fd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 0x5638a2124a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.684+0000 7fe942800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2119dd0 0x5638a211a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.684+0000 7fe94b5fd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2124fb0 0x5638a21273a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.684+0000 7fe94b5fd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5638a207f380 con 0x5638a2124fb0 2024-04-26T23:03:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.684+0000 7fe94b5fd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5638a201bc30 con 0x5638a211a6f0 2024-04-26T23:03:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.684+0000 7fe94b5fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5638a207b340 con 0x5638a2119dd0 2024-04-26T23:03:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.684+0000 7fe941e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 0x5638a2124a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.684+0000 7fe942800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2119dd0 0x5638a211a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:54298/0 (socket says 172.21.15.27:54298) 2024-04-26T23:03:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.684+0000 7fe942800640 1 -- 172.21.15.27:0/2811134606 learned_addr learned my addr 172.21.15.27:0/2811134606 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:03:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.684+0000 7fe941e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 0x5638a2124a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:36638/0 (socket says 172.21.15.27:36638) 2024-04-26T23:03:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.685+0000 7fe941400640 1 --2- 172.21.15.27:0/2811134606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2124fb0 0x5638a21273a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.685+0000 7fe941e00640 1 -- 172.21.15.27:0/2811134606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2119dd0 msgr2=0x5638a211a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.685+0000 7fe941e00640 1 --2- 172.21.15.27:0/2811134606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2119dd0 0x5638a211a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.685+0000 7fe941e00640 1 -- 172.21.15.27:0/2811134606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2124fb0 msgr2=0x5638a21273a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.685+0000 7fe941e00640 1 --2- 172.21.15.27:0/2811134606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2124fb0 0x5638a21273a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.685+0000 7fe941e00640 1 -- 172.21.15.27:0/2811134606 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638a203d940 con 0x5638a211a6f0 2024-04-26T23:03:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.685+0000 7fe942800640 1 --2- 172.21.15.27:0/2811134606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2119dd0 0x5638a211a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:03:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.685+0000 7fe93aa00640 1 --2- 172.21.15.27:0/4092295285 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe9280a3d90 0x7fe9280a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.686+0000 7fe941400640 1 --2- 172.21.15.27:0/2811134606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2124fb0 0x5638a21273a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:03:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.686+0000 7fe93aa00640 1 -- 172.21.15.27:0/4092295285 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fe9340d0f60 con 0x7fe944102910 2024-04-26T23:03:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.686+0000 7fe941e00640 1 --2- 172.21.15.27:0/2811134606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 0x5638a2124a70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe9340ce270 tx=0x7fe9340d9570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c24ee68b8acb5ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.686+0000 7fe92f000640 1 -- 172.21.15.27:0/2811134606 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934098d60 con 0x5638a211a6f0 2024-04-26T23:03:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.686+0000 7fe92f000640 1 -- 172.21.15.27:0/2811134606 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe93409d1c0 con 0x5638a211a6f0 2024-04-26T23:03:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.686+0000 7fe92f000640 1 -- 172.21.15.27:0/2811134606 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93409c420 con 0x5638a211a6f0 2024-04-26T23:03:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.687+0000 7fe942800640 1 --2- 172.21.15.27:0/4092295285 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe9280a3d90 0x7fe9280a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.688+0000 7fe94b5fd740 1 -- 172.21.15.27:0/2811134606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 msgr2=0x5638a2124a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.688+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/2811134606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 0x5638a2124a70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe9340ce270 tx=0x7fe9340d9570 comp rx=0 tx=0).stop 2024-04-26T23:03:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.689+0000 7fe942800640 1 --2- 172.21.15.27:0/4092295285 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe9280a3d90 0x7fe9280a6250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0049d0 tx=0x7fe93c0098e0 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.689+0000 7fe94b5fd740 1 -- 172.21.15.27:0/2811134606 shutdown_connections 2024-04-26T23:03:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.689+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/2811134606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 0x5638a2124a70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.689+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/2811134606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2124fb0 0x5638a21273a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.689+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/2811134606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2119dd0 0x5638a211a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.689+0000 7fe94b5fd740 1 -- 172.21.15.27:0/2811134606 >> 172.21.15.27:0/2811134606 conn(0x5638a21162e0 msgr2=0x5638a21166c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.690+0000 7fe94b5fd740 1 -- 172.21.15.27:0/2811134606 shutdown_connections 2024-04-26T23:03:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.690+0000 7fe94b5fd740 1 -- 172.21.15.27:0/2811134606 wait complete. 2024-04-26T23:03:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.690+0000 7fe94b5fd740 1 -- start start 2024-04-26T23:03:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.691+0000 7fe94b5fd740 1 Processor -- start 2024-04-26T23:03:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.692+0000 7fe94b5fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2119dd0 0x5638a2121f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.692+0000 7fe94b5fd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 0x5638a2122470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.692+0000 7fe94b5fd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2124fb0 0x5638a21229b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.692+0000 7fe94b5fd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5638a2127be0 con 0x5638a2124fb0 2024-04-26T23:03:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.692+0000 7fe94b5fd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5638a2006c60 con 0x5638a211a6f0 2024-04-26T23:03:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.692+0000 7fe94b5fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5638a2127d60 con 0x5638a2119dd0 2024-04-26T23:03:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.692+0000 7fe94b5fd740 10 client.0 _ll_register_callbacks cb 0x5638a2032e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:03:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.692+0000 7fe941e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2119dd0 0x5638a2121f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.693+0000 7fe941e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2119dd0 0x5638a2121f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:54308/0 (socket says 172.21.15.27:54308) 2024-04-26T23:03:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.693+0000 7fe941e00640 1 -- 172.21.15.27:0/2457117648 learned_addr learned my addr 172.21.15.27:0/2457117648 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:03:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.693+0000 7fe941400640 1 --2- 172.21.15.27:0/2457117648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 0x5638a2122470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.693+0000 7fe941e00640 1 -- 172.21.15.27:0/2457117648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 msgr2=0x5638a2122470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.693+0000 7fe941e00640 1 --2- 172.21.15.27:0/2457117648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 0x5638a2122470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.693+0000 7fe941e00640 1 -- 172.21.15.27:0/2457117648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2124fb0 msgr2=0x5638a21229b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:03:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.694+0000 7fe942800640 1 --2- 172.21.15.27:0/2457117648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2124fb0 0x5638a21229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.694+0000 7fe941e00640 1 --2- 172.21.15.27:0/2457117648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2124fb0 0x5638a21229b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.694+0000 7fe941e00640 1 -- 172.21.15.27:0/2457117648 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638a2015510 con 0x5638a2119dd0 2024-04-26T23:03:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.694+0000 7fe942800640 1 --2- 172.21.15.27:0/2457117648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2124fb0 0x5638a21229b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:03:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.695+0000 7fe941e00640 1 --2- 172.21.15.27:0/2457117648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2119dd0 0x5638a2121f30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe9340ce870 tx=0x7fe9340ce270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e8f0b178808eb20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.695+0000 7fe923400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T23:03:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.695+0000 7fe923400640 1 -- 172.21.15.27:0/2457117648 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93409c600 con 0x5638a2119dd0 2024-04-26T23:03:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.696+0000 7fe923400640 1 -- 172.21.15.27:0/2457117648 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fe934003ac0 con 0x5638a2119dd0 2024-04-26T23:03:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.696+0000 7fe94b5fd740 1 -- 172.21.15.27:0/2457117648 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe93409d1c0 con 0x5638a2119dd0 2024-04-26T23:03:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.697+0000 7fe923400640 1 -- 172.21.15.27:0/2457117648 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe93409c890 con 0x5638a2119dd0 2024-04-26T23:03:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.699+0000 7fe94b5fd740 10 client.25849 fetch_fsmap learned FSMap version 30 2024-04-26T23:03:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.699+0000 7fe94b5fd740 1 -- 172.21.15.27:0/2457117648 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5638a1d91210 con 0x5638a2119dd0 2024-04-26T23:03:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.699+0000 7fe923400640 1 -- 172.21.15.27:0/2457117648 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe93409d1c0 con 0x5638a2119dd0 2024-04-26T23:03:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.701+0000 7fe923400640 1 -- 172.21.15.27:0/2457117648 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe9340d9d10 con 0x5638a2119dd0 2024-04-26T23:03:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.701+0000 7fe94b5fd740 10 client.25849 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:03:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.701+0000 7fe94b5fd740 10 client.25849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:03:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.702+0000 7fe94b5fd740 20 client.25849 populate_metadata read hostname 'smithi027' 2024-04-26T23:03:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.702+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/2457117648 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5638a2134ae0 0x5638a2136ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.702+0000 7fe94b5fd740 4 client.25849 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:03:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.702+0000 7fe94b5fd740 1 -- 172.21.15.27:0/2457117648 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638a208cf90 con 0x5638a2134ae0 2024-04-26T23:03:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.702+0000 7fe941400640 1 --2- 172.21.15.27:0/2457117648 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5638a2134ae0 0x5638a2136ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.703+0000 7fe941400640 1 --2- 172.21.15.27:0/2457117648 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5638a2134ae0 0x5638a2136ed0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdba213116cdc335 server_cookie=c4cc4d616eaa9d0a in_seq=0 out_seq=0 2024-04-26T23:03:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.703+0000 7fe923400640 10 client.25849 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:03:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.704+0000 7fe923400640 1 -- 172.21.15.27:0/2457117648 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638a208cf90 con 0x5638a2134ae0 2024-04-26T23:03:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.704+0000 7fe923400640 10 client.25849 handle_command_reply: tid=0 2024-04-26T23:03:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.704+0000 7fe94b5fd740 1 client.25849 shutdown 2024-04-26T23:03:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.705+0000 7fe94b5fd740 1 -- 172.21.15.27:0/2457117648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2119dd0 msgr2=0x5638a2121f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.705+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/2457117648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2119dd0 0x5638a2121f30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe9340ce870 tx=0x7fe9340ce270 comp rx=0 tx=0).stop 2024-04-26T23:03:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.706+0000 7fe94b5fd740 1 -- 172.21.15.27:0/2457117648 shutdown_connections 2024-04-26T23:03:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.706+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/2457117648 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5638a2134ae0 0x5638a2136ed0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.706+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/2457117648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 0x5638a2122470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.706+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/2457117648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2124fb0 0x5638a21229b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.706+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/2457117648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2119dd0 0x5638a2121f30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.706+0000 7fe94b5fd740 1 -- 172.21.15.27:0/2457117648 >> 172.21.15.27:0/2457117648 conn(0x5638a21162e0 msgr2=0x5638a211c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.706+0000 7fe923400640 0 client.25849 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:03:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.707+0000 7fe94b5fd740 1 -- 172.21.15.27:0/2457117648 shutdown_connections 2024-04-26T23:03:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.707+0000 7fe94b5fd740 1 -- 172.21.15.27:0/2457117648 wait complete. 2024-04-26T23:03:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.707+0000 7fe94b5fd740 20 client.25849 trim_cache size 0 max 16384 2024-04-26T23:03:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.745+0000 7fe94b5fd740 1 Processor -- start 2024-04-26T23:03:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.746+0000 7fe94b5fd740 1 -- start start 2024-04-26T23:03:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.746+0000 7fe94b5fd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a2119dd0 0x5638a211c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.746+0000 7fe94b5fd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a211a6f0 0x5638a21152c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.746+0000 7fe94b5fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2124fb0 0x5638a2139ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.746+0000 7fe94b5fd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9441b9080 con 0x5638a211a6f0 2024-04-26T23:03:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.746+0000 7fe94b5fd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe9441b6890 con 0x5638a2119dd0 2024-04-26T23:03:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.746+0000 7fe94b5fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe9441b46e0 con 0x5638a2124fb0 2024-04-26T23:03:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.747+0000 7fe941e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a2119dd0 0x5638a211c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.747+0000 7fe941e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a2119dd0 0x5638a211c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:36656/0 (socket says 172.21.15.27:36656) 2024-04-26T23:03:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.747+0000 7fe941e00640 1 -- 172.21.15.27:0/3863119045 learned_addr learned my addr 172.21.15.27:0/3863119045 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:03:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.747+0000 7fe941400640 1 --2- 172.21.15.27:0/3863119045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a211a6f0 0x5638a21152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.747+0000 7fe941e00640 1 -- 172.21.15.27:0/3863119045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2124fb0 msgr2=0x5638a2139ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.747+0000 7fe941e00640 1 --2- 172.21.15.27:0/3863119045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2124fb0 0x5638a2139ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.747+0000 7fe941e00640 1 -- 172.21.15.27:0/3863119045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a211a6f0 msgr2=0x5638a21152c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.747+0000 7fe941e00640 1 --2- 172.21.15.27:0/3863119045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a211a6f0 0x5638a21152c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.747+0000 7fe941e00640 1 -- 172.21.15.27:0/3863119045 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638a1e5d620 con 0x5638a2119dd0 2024-04-26T23:03:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.747+0000 7fe941400640 1 --2- 172.21.15.27:0/3863119045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a211a6f0 0x5638a21152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:03:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.748+0000 7fe941e00640 1 --2- 172.21.15.27:0/3863119045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a2119dd0 0x5638a211c790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe934099380 tx=0x7fe93409afd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1692e0a12f67ebfe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.748+0000 7fe92e600640 1 -- 172.21.15.27:0/3863119045 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93409c600 con 0x5638a2119dd0 2024-04-26T23:03:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.748+0000 7fe92e600640 1 -- 172.21.15.27:0/3863119045 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe93409d1c0 con 0x5638a2119dd0 2024-04-26T23:03:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.748+0000 7fe92e600640 1 -- 172.21.15.27:0/3863119045 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934098d60 con 0x5638a2119dd0 2024-04-26T23:03:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.749+0000 7fe94b5fd740 1 -- 172.21.15.27:0/3863119045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a2119dd0 msgr2=0x5638a211c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.749+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/3863119045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a2119dd0 0x5638a211c790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe934099380 tx=0x7fe93409afd0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.750+0000 7fe94b5fd740 1 -- 172.21.15.27:0/3863119045 shutdown_connections 2024-04-26T23:03:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.750+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/3863119045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2124fb0 0x5638a2139ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.750+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/3863119045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a211a6f0 0x5638a21152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.750+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/3863119045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a2119dd0 0x5638a211c790 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.750+0000 7fe94b5fd740 1 -- 172.21.15.27:0/3863119045 >> 172.21.15.27:0/3863119045 conn(0x5638a21162e0 msgr2=0x5638a2114bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.750+0000 7fe94b5fd740 1 -- 172.21.15.27:0/3863119045 shutdown_connections 2024-04-26T23:03:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.750+0000 7fe94b5fd740 1 -- 172.21.15.27:0/3863119045 wait complete. 2024-04-26T23:03:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.751+0000 7fe94b5fd740 1 -- start start 2024-04-26T23:03:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.751+0000 7fe94b5fd740 1 Processor -- start 2024-04-26T23:03:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.752+0000 7fe94b5fd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2119dd0 0x5638a21378b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.752+0000 7fe94b5fd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 0x5638a2137df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.752+0000 7fe94b5fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2124fb0 0x5638a2138330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.752+0000 7fe94b5fd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9441b4d50 con 0x5638a2119dd0 2024-04-26T23:03:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.752+0000 7fe94b5fd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe9441b61c0 con 0x5638a211a6f0 2024-04-26T23:03:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.752+0000 7fe94b5fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe9441b5420 con 0x5638a2124fb0 2024-04-26T23:03:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.752+0000 7fe94b5fd740 10 client.0 _ll_register_callbacks cb 0x7fe9340d4dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:03:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.752+0000 7fe942800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2124fb0 0x5638a2138330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.752+0000 7fe942800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2124fb0 0x5638a2138330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:54326/0 (socket says 172.21.15.27:54326) 2024-04-26T23:03:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.752+0000 7fe942800640 1 -- 172.21.15.27:0/828406449 learned_addr learned my addr 172.21.15.27:0/828406449 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:03:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.753+0000 7fe941e00640 1 --2- 172.21.15.27:0/828406449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2119dd0 0x5638a21378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.753+0000 7fe941400640 1 --2- 172.21.15.27:0/828406449 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 0x5638a2137df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.753+0000 7fe941400640 1 -- 172.21.15.27:0/828406449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2124fb0 msgr2=0x5638a2138330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.753+0000 7fe941400640 1 --2- 172.21.15.27:0/828406449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2124fb0 0x5638a2138330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.753+0000 7fe941400640 1 -- 172.21.15.27:0/828406449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2119dd0 msgr2=0x5638a21378b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.753+0000 7fe941400640 1 --2- 172.21.15.27:0/828406449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2119dd0 0x5638a21378b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.753+0000 7fe941400640 1 -- 172.21.15.27:0/828406449 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638a2039cb0 con 0x5638a211a6f0 2024-04-26T23:03:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.754+0000 7fe941400640 1 --2- 172.21.15.27:0/828406449 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 0x5638a2137df0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe93002a950 tx=0x7fe9300095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc8b169dacca3243 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.754+0000 7fe923400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T23:03:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.754+0000 7fe923400640 1 -- 172.21.15.27:0/828406449 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930037040 con 0x5638a211a6f0 2024-04-26T23:03:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.754+0000 7fe923400640 1 -- 172.21.15.27:0/828406449 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fe930030de0 con 0x5638a211a6f0 2024-04-26T23:03:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.754+0000 7fe94b5fd740 1 -- 172.21.15.27:0/828406449 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe93409d1c0 con 0x5638a211a6f0 2024-04-26T23:03:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.754+0000 7fe923400640 1 -- 172.21.15.27:0/828406449 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe930004040 con 0x5638a211a6f0 2024-04-26T23:03:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.754+0000 7fe94b5fd740 10 client.25814 fetch_fsmap learned FSMap version 30 2024-04-26T23:03:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.754+0000 7fe94b5fd740 1 -- 172.21.15.27:0/828406449 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5638a219ad10 con 0x5638a211a6f0 2024-04-26T23:03:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.755+0000 7fe923400640 1 -- 172.21.15.27:0/828406449 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe93409d1c0 con 0x5638a211a6f0 2024-04-26T23:03:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.755+0000 7fe923400640 1 -- 172.21.15.27:0/828406449 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe930032a60 con 0x5638a211a6f0 2024-04-26T23:03:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.755+0000 7fe94b5fd740 10 client.25814 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:03:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.755+0000 7fe94b5fd740 10 client.25814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:03:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.755+0000 7fe94b5fd740 20 client.25814 populate_metadata read hostname 'smithi027' 2024-04-26T23:03:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.755+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/828406449 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5638a219b4a0 0x5638a219d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.755+0000 7fe94b5fd740 4 client.25814 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:03:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.755+0000 7fe94b5fd740 1 -- 172.21.15.27:0/828406449 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638a2086e00 con 0x5638a219b4a0 2024-04-26T23:03:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.757+0000 7fe941e00640 1 --2- 172.21.15.27:0/828406449 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5638a219b4a0 0x5638a219d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.758+0000 7fe941e00640 1 --2- 172.21.15.27:0/828406449 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5638a219b4a0 0x5638a219d8c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c6c92a0e675fa0 server_cookie=2e2799c61b2377e0 in_seq=0 out_seq=0 2024-04-26T23:03:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.758+0000 7fe923400640 10 client.25814 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:03:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.772+0000 7fe923400640 1 -- 172.21.15.27:0/828406449 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638a2086e00 con 0x5638a219b4a0 2024-04-26T23:03:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.772+0000 7fe923400640 10 client.25814 handle_command_reply: tid=0 2024-04-26T23:03:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.772+0000 7fe94b5fd740 1 client.25814 shutdown 2024-04-26T23:03:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.772+0000 7fe94b5fd740 1 -- 172.21.15.27:0/828406449 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 msgr2=0x5638a2137df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.772+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/828406449 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 0x5638a2137df0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe93002a950 tx=0x7fe9300095b0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.772+0000 7fe94b5fd740 1 -- 172.21.15.27:0/828406449 shutdown_connections 2024-04-26T23:03:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.772+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/828406449 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5638a219b4a0 0x5638a219d8c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.772+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/828406449 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5638a211a6f0 0x5638a2137df0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.773+0000 7fe923400640 0 client.25814 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:03:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.773+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/828406449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638a2124fb0 0x5638a2138330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.773+0000 7fe94b5fd740 1 --2- 172.21.15.27:0/828406449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5638a2119dd0 0x5638a21378b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.773+0000 7fe94b5fd740 1 -- 172.21.15.27:0/828406449 >> 172.21.15.27:0/828406449 conn(0x5638a21162e0 msgr2=0x5638a2130840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.773+0000 7fe94b5fd740 1 -- 172.21.15.27:0/828406449 shutdown_connections 2024-04-26T23:03:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.773+0000 7fe94b5fd740 1 -- 172.21.15.27:0/828406449 wait complete. 2024-04-26T23:03:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.773+0000 7fe94b5fd740 20 client.25814 trim_cache size 0 max 16384 2024-04-26T23:03:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T23:03:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T23:03:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T23:03:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T23:03:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.774+0000 7fe948c00640 1 -- 172.21.15.27:0/4092295285 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe9280a3d90 msgr2=0x7fe9280a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.774+0000 7fe948c00640 1 --2- 172.21.15.27:0/4092295285 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe9280a3d90 0x7fe9280a6250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0049d0 tx=0x7fe93c0098e0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.774+0000 7fe948c00640 1 -- 172.21.15.27:0/4092295285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe944102910 msgr2=0x7fe94410f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.774+0000 7fe948c00640 1 --2- 172.21.15.27:0/4092295285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe944102910 0x7fe94410f7d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe93400e9c0 tx=0x7fe93400ee80 comp rx=0 tx=0).stop 2024-04-26T23:03:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.774+0000 7fe948c00640 1 -- 172.21.15.27:0/4092295285 shutdown_connections 2024-04-26T23:03:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.774+0000 7fe948c00640 1 --2- 172.21.15.27:0/4092295285 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe9280a3d90 0x7fe9280a6250 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.774+0000 7fe948c00640 1 --2- 172.21.15.27:0/4092295285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9441032d0 0x7fe94410c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.774+0000 7fe948c00640 1 --2- 172.21.15.27:0/4092295285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe944108920 0x7fe94410cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.774+0000 7fe948c00640 1 --2- 172.21.15.27:0/4092295285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe944102910 0x7fe94410f7d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.774+0000 7fe948c00640 1 -- 172.21.15.27:0/4092295285 >> 172.21.15.27:0/4092295285 conn(0x7fe9440fe3c0 msgr2=0x7fe9440fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.774+0000 7fe948c00640 1 -- 172.21.15.27:0/4092295285 shutdown_connections 2024-04-26T23:03:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:07.775+0000 7fe948c00640 1 -- 172.21.15.27:0/4092295285 wait complete. 2024-04-26T23:03:09.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:08 smithi080 ceph-mon[32324]: pgmap v139: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 432 op/s 2024-04-26T23:03:09.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:08 smithi094 ceph-mon[32889]: pgmap v139: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 432 op/s 2024-04-26T23:03:09.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:08 smithi027 ceph-mon[28034]: pgmap v139: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 432 op/s 2024-04-26T23:03:11.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:10 smithi080 ceph-mon[32324]: pgmap v140: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 258 op/s 2024-04-26T23:03:11.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:10 smithi094 ceph-mon[32889]: pgmap v140: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 258 op/s 2024-04-26T23:03:11.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:10 smithi027 ceph-mon[28034]: pgmap v140: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 258 op/s 2024-04-26T23:03:13.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:12 smithi080 ceph-mon[32324]: pgmap v141: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 320 op/s 2024-04-26T23:03:13.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:12 smithi094 ceph-mon[32889]: pgmap v141: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 320 op/s 2024-04-26T23:03:13.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:12 smithi027 ceph-mon[28034]: pgmap v141: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 320 op/s 2024-04-26T23:03:15.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:14 smithi080 ceph-mon[32324]: pgmap v142: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 900 KiB/s wr, 135 op/s 2024-04-26T23:03:15.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:14 smithi094 ceph-mon[32889]: pgmap v142: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 900 KiB/s wr, 135 op/s 2024-04-26T23:03:15.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:14 smithi027 ceph-mon[28034]: pgmap v142: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 900 KiB/s wr, 135 op/s 2024-04-26T23:03:17.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:16 smithi080 ceph-mon[32324]: pgmap v143: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 901 KiB/s wr, 135 op/s 2024-04-26T23:03:17.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:16 smithi094 ceph-mon[32889]: pgmap v143: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 901 KiB/s wr, 135 op/s 2024-04-26T23:03:17.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:16 smithi027 ceph-mon[28034]: pgmap v143: 129 pgs: 129 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 901 KiB/s wr, 135 op/s 2024-04-26T23:03:19.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:18 smithi080 ceph-mon[32324]: pgmap v144: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2024-04-26T23:03:19.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:18 smithi094 ceph-mon[32889]: pgmap v144: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2024-04-26T23:03:19.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:18 smithi027 ceph-mon[28034]: pgmap v144: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2024-04-26T23:03:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.984+0000 7f18e2800640 1 -- 172.21.15.27:0/1162580934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18dc10f300 msgr2=0x7f18dc1116f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.984+0000 7f18e2800640 1 --2- 172.21.15.27:0/1162580934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18dc10f300 0x7f18dc1116f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009be0 tx=0x7f18d402a6c0 comp rx=0 tx=0).stop 2024-04-26T23:03:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.985+0000 7f18e2800640 1 -- 172.21.15.27:0/1162580934 shutdown_connections 2024-04-26T23:03:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.985+0000 7f18e2800640 1 --2- 172.21.15.27:0/1162580934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc102100 0x7f18dc10edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.985+0000 7f18e2800640 1 --2- 172.21.15.27:0/1162580934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18dc10f300 0x7f18dc1116f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.985+0000 7f18e2800640 1 --2- 172.21.15.27:0/1162580934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18dc101750 0x7f18dc101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.985+0000 7f18e2800640 1 -- 172.21.15.27:0/1162580934 >> 172.21.15.27:0/1162580934 conn(0x7f18dc0fd350 msgr2=0x7f18dc0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.985+0000 7f18e2800640 1 -- 172.21.15.27:0/1162580934 shutdown_connections 2024-04-26T23:03:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.985+0000 7f18e2800640 1 -- 172.21.15.27:0/1162580934 wait complete. 2024-04-26T23:03:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.987+0000 7f18e2800640 1 Processor -- start 2024-04-26T23:03:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.988+0000 7f18e2800640 1 -- start start 2024-04-26T23:03:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.988+0000 7f18e2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18dc101750 0x7f18dc103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.988+0000 7f18e2800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc102100 0x7f18dc103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.988+0000 7f18e2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18dc10f300 0x7f18dc1044b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.989+0000 7f18daa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc102100 0x7f18dc103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.989+0000 7f18e2800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc114e60 con 0x7f18dc10f300 2024-04-26T23:03:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.989+0000 7f18e2800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc114ce0 con 0x7f18dc102100 2024-04-26T23:03:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.989+0000 7f18e2800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc114fe0 con 0x7f18dc101750 2024-04-26T23:03:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.989+0000 7f18daa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc102100 0x7f18dc103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:40714/0 (socket says 172.21.15.27:40714) 2024-04-26T23:03:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.989+0000 7f18daa00640 1 -- 172.21.15.27:0/1601741007 learned_addr learned my addr 172.21.15.27:0/1601741007 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:03:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.989+0000 7f18dbe00640 1 --2- 172.21.15.27:0/1601741007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18dc10f300 0x7f18dc1044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.989+0000 7f18db400640 1 --2- 172.21.15.27:0/1601741007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18dc101750 0x7f18dc103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.990+0000 7f18dbe00640 1 -- 172.21.15.27:0/1601741007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18dc101750 msgr2=0x7f18dc103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.990+0000 7f18dbe00640 1 --2- 172.21.15.27:0/1601741007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18dc101750 0x7f18dc103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.990+0000 7f18dbe00640 1 -- 172.21.15.27:0/1601741007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc102100 msgr2=0x7f18dc103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.990+0000 7f18dbe00640 1 --2- 172.21.15.27:0/1601741007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc102100 0x7f18dc103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.990+0000 7f18dbe00640 1 -- 172.21.15.27:0/1601741007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18dc1ab9d0 con 0x7f18dc10f300 2024-04-26T23:03:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.990+0000 7f18db400640 1 --2- 172.21.15.27:0/1601741007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18dc101750 0x7f18dc103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:03:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.990+0000 7f18daa00640 1 --2- 172.21.15.27:0/1601741007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc102100 0x7f18dc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:03:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.990+0000 7f18dbe00640 1 --2- 172.21.15.27:0/1601741007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18dc10f300 0x7f18dc1044b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f18d402acb0 tx=0x7f18d4003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.991+0000 7f18d3e00640 1 -- 172.21.15.27:0/1601741007 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4040020 con 0x7f18dc10f300 2024-04-26T23:03:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.991+0000 7f18e2800640 1 -- 172.21.15.27:0/1601741007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18dc1abc60 con 0x7f18dc10f300 2024-04-26T23:03:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.991+0000 7f18e2800640 1 -- 172.21.15.27:0/1601741007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18dc1ac1a0 con 0x7f18dc10f300 2024-04-26T23:03:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.991+0000 7f18d3e00640 1 -- 172.21.15.27:0/1601741007 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18d40318f0 con 0x7f18dc10f300 2024-04-26T23:03:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.991+0000 7f18d3e00640 1 -- 172.21.15.27:0/1601741007 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4036040 con 0x7f18dc10f300 2024-04-26T23:03:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.996+0000 7f18e57c3740 1 Processor -- start 2024-04-26T23:03:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.996+0000 7f18e57c3740 1 -- start start 2024-04-26T23:03:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.996+0000 7f18e57c3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b7dd0 0x562b307b81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.996+0000 7f18e57c3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307b86f0 0x562b307c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.996+0000 7f18e57c3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307c2fb0 0x562b307c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.996+0000 7f18e57c3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562b3071d380 con 0x562b307b86f0 2024-04-26T23:03:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.996+0000 7f18e57c3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x562b306b9c30 con 0x562b307c2fb0 2024-04-26T23:03:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.996+0000 7f18e57c3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562b30719340 con 0x562b307b7dd0 2024-04-26T23:03:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.997+0000 7f18db400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307b86f0 0x562b307c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.998+0000 7f18daa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b7dd0 0x562b307b81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.998+0000 7f18daa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b7dd0 0x562b307b81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:41788/0 (socket says 172.21.15.27:41788) 2024-04-26T23:03:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.998+0000 7f18daa00640 1 -- 172.21.15.27:0/2100690890 learned_addr learned my addr 172.21.15.27:0/2100690890 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:03:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.998+0000 7f18daa00640 1 --2- 172.21.15.27:0/2100690890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307c2fb0 0x562b307c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.999+0000 7f18db400640 1 -- 172.21.15.27:0/2100690890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b7dd0 msgr2=0x562b307b81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.999+0000 7f18db400640 1 --2- 172.21.15.27:0/2100690890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b7dd0 0x562b307b81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.999+0000 7f18db400640 1 -- 172.21.15.27:0/2100690890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307c2fb0 msgr2=0x562b307c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.999+0000 7f18db400640 1 --2- 172.21.15.27:0/2100690890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307c2fb0 0x562b307c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:20.999+0000 7f18db400640 1 -- 172.21.15.27:0/2100690890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b306db940 con 0x562b307b86f0 2024-04-26T23:03:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.000+0000 7f18db400640 1 --2- 172.21.15.27:0/2100690890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307b86f0 0x562b307c2a70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f18cc011440 tx=0x7f18cc011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef24c7aaf301f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.000+0000 7f18d0c00640 1 -- 172.21.15.27:0/2100690890 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc02f070 con 0x562b307b86f0 2024-04-26T23:03:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.001+0000 7f18d0c00640 1 -- 172.21.15.27:0/2100690890 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18cc004310 con 0x562b307b86f0 2024-04-26T23:03:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.001+0000 7f18d0c00640 1 -- 172.21.15.27:0/2100690890 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc00f7c0 con 0x562b307b86f0 2024-04-26T23:03:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.001+0000 7f18d3e00640 1 -- 172.21.15.27:0/1601741007 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f18d4043020 con 0x7f18dc10f300 2024-04-26T23:03:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.002+0000 7f18e57c3740 1 -- 172.21.15.27:0/2100690890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307b86f0 msgr2=0x562b307c2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.002+0000 7f18e57c3740 1 --2- 172.21.15.27:0/2100690890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307b86f0 0x562b307c2a70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f18cc011440 tx=0x7f18cc011900 comp rx=0 tx=0).stop 2024-04-26T23:03:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.003+0000 7f18e57c3740 1 -- 172.21.15.27:0/2100690890 shutdown_connections 2024-04-26T23:03:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.003+0000 7f18e57c3740 1 --2- 172.21.15.27:0/2100690890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307c2fb0 0x562b307c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.003+0000 7f18e57c3740 1 --2- 172.21.15.27:0/2100690890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307b86f0 0x562b307c2a70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.003+0000 7f18e57c3740 1 --2- 172.21.15.27:0/2100690890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b7dd0 0x562b307b81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.003+0000 7f18e57c3740 1 -- 172.21.15.27:0/2100690890 >> 172.21.15.27:0/2100690890 conn(0x562b307b42e0 msgr2=0x562b307b46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.004+0000 7f18e57c3740 1 -- 172.21.15.27:0/2100690890 shutdown_connections 2024-04-26T23:03:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.004+0000 7f18e57c3740 1 -- 172.21.15.27:0/2100690890 wait complete. 2024-04-26T23:03:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.004+0000 7f18e57c3740 1 -- start start 2024-04-26T23:03:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.005+0000 7f18e57c3740 1 Processor -- start 2024-04-26T23:03:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.006+0000 7f18e57c3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307b7dd0 0x562b307bfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.006+0000 7f18e57c3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b86f0 0x562b307c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.006+0000 7f18e57c3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307c2fb0 0x562b307c0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.006+0000 7f18e57c3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562b307c5be0 con 0x562b307c2fb0 2024-04-26T23:03:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.006+0000 7f18e57c3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x562b306a4c60 con 0x562b307b7dd0 2024-04-26T23:03:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.006+0000 7f18e57c3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562b307c5d60 con 0x562b307b86f0 2024-04-26T23:03:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.006+0000 7f18d3e00640 1 --2- 172.21.15.27:0/1601741007 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f18c00a3d70 0x7f18c00a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.006+0000 7f18e57c3740 10 client.0 _ll_register_callbacks cb 0x562b306d0e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:03:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.006+0000 7f18d3e00640 1 -- 172.21.15.27:0/1601741007 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f18d40f1590 con 0x7f18dc10f300 2024-04-26T23:03:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.007+0000 7f18daa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307c2fb0 0x562b307c0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.007+0000 7f18db400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b86f0 0x562b307c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.007+0000 7f18db400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b86f0 0x562b307c0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:41794/0 (socket says 172.21.15.27:41794) 2024-04-26T23:03:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.007+0000 7f18daa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307c2fb0 0x562b307c0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60126/0 (socket says 172.21.15.27:60126) 2024-04-26T23:03:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.007+0000 7f18db400640 1 -- 172.21.15.27:0/2247755390 learned_addr learned my addr 172.21.15.27:0/2247755390 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:03:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.008+0000 7f18daa00640 1 --2- 172.21.15.27:0/2247755390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307b7dd0 0x562b307bfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.008+0000 7f18dbe00640 1 --2- 172.21.15.27:0/1601741007 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f18c00a3d70 0x7f18c00a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.008+0000 7f18db400640 1 -- 172.21.15.27:0/2247755390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307b7dd0 msgr2=0x562b307bfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.008+0000 7f18db400640 1 --2- 172.21.15.27:0/2247755390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307b7dd0 0x562b307bfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.008+0000 7f18db400640 1 -- 172.21.15.27:0/2247755390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307c2fb0 msgr2=0x562b307c0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.008+0000 7f18db400640 1 --2- 172.21.15.27:0/2247755390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307c2fb0 0x562b307c0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.008+0000 7f18db400640 1 -- 172.21.15.27:0/2247755390 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b306b3510 con 0x562b307b86f0 2024-04-26T23:03:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.008+0000 7f18daa00640 1 --2- 172.21.15.27:0/2247755390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307c2fb0 0x562b307c0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:03:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.009+0000 7f18db400640 1 --2- 172.21.15.27:0/2247755390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b86f0 0x562b307c0430 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f18cc006010 tx=0x7f18cc005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d693c90a269be921 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.009+0000 7f18c5600640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-26T23:03:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.009+0000 7f18c5600640 1 -- 172.21.15.27:0/2247755390 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc039020 con 0x562b307b86f0 2024-04-26T23:03:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.009+0000 7f18c5600640 1 -- 172.21.15.27:0/2247755390 <== mon.1 v2:172.21.15.94:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f18cc0048d0 con 0x562b307b86f0 2024-04-26T23:03:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.010+0000 7f18e57c3740 1 -- 172.21.15.27:0/2247755390 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18cc004310 con 0x562b307b86f0 2024-04-26T23:03:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.010+0000 7f18c5600640 1 -- 172.21.15.27:0/2247755390 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18cc02f030 con 0x562b307b86f0 2024-04-26T23:03:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.010+0000 7f18dbe00640 1 --2- 172.21.15.27:0/1601741007 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f18c00a3d70 0x7f18c00a6230 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18d40b94f0 tx=0x7f18d40eef90 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.010+0000 7f18e57c3740 10 client.25864 fetch_fsmap learned FSMap version 30 2024-04-26T23:03:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.010+0000 7f18e57c3740 1 -- 172.21.15.27:0/2247755390 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562b3042f210 con 0x562b307b86f0 2024-04-26T23:03:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.011+0000 7f18c5600640 1 -- 172.21.15.27:0/2247755390 <== mon.1 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18cc004310 con 0x562b307b86f0 2024-04-26T23:03:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.011+0000 7f18c5600640 1 -- 172.21.15.27:0/2247755390 <== mon.1 v2:172.21.15.94:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f18cc003b80 con 0x562b307b86f0 2024-04-26T23:03:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.011+0000 7f18e57c3740 10 client.25864 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:03:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.011+0000 7f18e57c3740 10 client.25864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:03:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.011+0000 7f18e57c3740 20 client.25864 populate_metadata read hostname 'smithi027' 2024-04-26T23:03:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.011+0000 7f18e57c3740 1 --2- 172.21.15.27:0/2247755390 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x562b307d2bf0 0x562b307d4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.012+0000 7f18e57c3740 4 client.25864 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:03:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.012+0000 7f18e57c3740 1 -- 172.21.15.27:0/2247755390 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b3072af90 con 0x562b307d2bf0 2024-04-26T23:03:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.012+0000 7f18daa00640 1 --2- 172.21.15.27:0/2247755390 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x562b307d2bf0 0x562b307d4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.013+0000 7f18c5600640 10 client.25864 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:03:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.013+0000 7f18daa00640 1 --2- 172.21.15.27:0/2247755390 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x562b307d2bf0 0x562b307d4fe0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7e5b622f49dfd10 server_cookie=eb03fbee7915a1a5 in_seq=0 out_seq=0 2024-04-26T23:03:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.014+0000 7f18c5600640 1 -- 172.21.15.27:0/2247755390 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b3072af90 con 0x562b307d2bf0 2024-04-26T23:03:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.014+0000 7f18c5600640 10 client.25864 handle_command_reply: tid=0 2024-04-26T23:03:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.014+0000 7f18e57c3740 1 client.25864 shutdown 2024-04-26T23:03:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.016+0000 7f18e57c3740 1 -- 172.21.15.27:0/2247755390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b86f0 msgr2=0x562b307c0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.016+0000 7f18e57c3740 1 --2- 172.21.15.27:0/2247755390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b86f0 0x562b307c0430 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f18cc006010 tx=0x7f18cc005720 comp rx=0 tx=0).stop 2024-04-26T23:03:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.016+0000 7f18e57c3740 1 -- 172.21.15.27:0/2247755390 shutdown_connections 2024-04-26T23:03:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.016+0000 7f18e57c3740 1 --2- 172.21.15.27:0/2247755390 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x562b307d2bf0 0x562b307d4fe0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.016+0000 7f18e57c3740 1 --2- 172.21.15.27:0/2247755390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307c2fb0 0x562b307c0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.016+0000 7f18e57c3740 1 --2- 172.21.15.27:0/2247755390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b86f0 0x562b307c0430 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.016+0000 7f18e57c3740 1 --2- 172.21.15.27:0/2247755390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307b7dd0 0x562b307bfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.016+0000 7f18e57c3740 1 -- 172.21.15.27:0/2247755390 >> 172.21.15.27:0/2247755390 conn(0x562b307b42e0 msgr2=0x562b307ba010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.017+0000 7f18c5600640 0 client.25864 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:03:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.017+0000 7f18e57c3740 1 -- 172.21.15.27:0/2247755390 shutdown_connections 2024-04-26T23:03:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.017+0000 7f18e57c3740 1 -- 172.21.15.27:0/2247755390 wait complete. 2024-04-26T23:03:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.018+0000 7f18e57c3740 20 client.25864 trim_cache size 0 max 16384 2024-04-26T23:03:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.061+0000 7f18e57c3740 1 Processor -- start 2024-04-26T23:03:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.062+0000 7f18e57c3740 1 -- start start 2024-04-26T23:03:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.062+0000 7f18e57c3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307b7dd0 0x562b307ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.062+0000 7f18e57c3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b86f0 0x562b307c0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.062+0000 7f18e57c3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307c2fb0 0x562b307beda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.062+0000 7f18e57c3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc012b90 con 0x562b307c2fb0 2024-04-26T23:03:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.062+0000 7f18e57c3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc0103a0 con 0x562b307b7dd0 2024-04-26T23:03:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.062+0000 7f18e57c3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc00e1f0 con 0x562b307b86f0 2024-04-26T23:03:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.063+0000 7f18db400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b86f0 0x562b307c0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.063+0000 7f18db400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b86f0 0x562b307c0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:41800/0 (socket says 172.21.15.27:41800) 2024-04-26T23:03:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.063+0000 7f18db400640 1 -- 172.21.15.27:0/3539425446 learned_addr learned my addr 172.21.15.27:0/3539425446 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:03:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.063+0000 7f18daa00640 1 --2- 172.21.15.27:0/3539425446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307c2fb0 0x562b307beda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.063+0000 7f18db400640 1 -- 172.21.15.27:0/3539425446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307b7dd0 msgr2=0x562b307ceb00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:03:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.063+0000 7f18db400640 1 --2- 172.21.15.27:0/3539425446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307b7dd0 0x562b307ceb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.063+0000 7f18db400640 1 -- 172.21.15.27:0/3539425446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307c2fb0 msgr2=0x562b307beda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.063+0000 7f18db400640 1 --2- 172.21.15.27:0/3539425446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307c2fb0 0x562b307beda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.063+0000 7f18db400640 1 -- 172.21.15.27:0/3539425446 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b307bad80 con 0x562b307b86f0 2024-04-26T23:03:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.064+0000 7f18db400640 1 --2- 172.21.15.27:0/3539425446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b86f0 0x562b307c0790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009f30 tx=0x7f18cc011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d40518c165279667 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.064+0000 7f18c7e00640 1 -- 172.21.15.27:0/3539425446 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc039070 con 0x562b307b86f0 2024-04-26T23:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.064+0000 7f18c7e00640 1 -- 172.21.15.27:0/3539425446 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18cc030040 con 0x562b307b86f0 2024-04-26T23:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.064+0000 7f18c7e00640 1 -- 172.21.15.27:0/3539425446 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc00faf0 con 0x562b307b86f0 2024-04-26T23:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.065+0000 7f18e57c3740 1 -- 172.21.15.27:0/3539425446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b86f0 msgr2=0x562b307c0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.065+0000 7f18e57c3740 1 --2- 172.21.15.27:0/3539425446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b86f0 0x562b307c0790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009f30 tx=0x7f18cc011900 comp rx=0 tx=0).stop 2024-04-26T23:03:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.067+0000 7f18e57c3740 1 -- 172.21.15.27:0/3539425446 shutdown_connections 2024-04-26T23:03:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.067+0000 7f18e57c3740 1 --2- 172.21.15.27:0/3539425446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307c2fb0 0x562b307beda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.067+0000 7f18e57c3740 1 --2- 172.21.15.27:0/3539425446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307b86f0 0x562b307c0790 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.067+0000 7f18e57c3740 1 --2- 172.21.15.27:0/3539425446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307b7dd0 0x562b307ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.067+0000 7f18e57c3740 1 -- 172.21.15.27:0/3539425446 >> 172.21.15.27:0/3539425446 conn(0x562b307b42e0 msgr2=0x562b307be690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.067+0000 7f18e57c3740 1 -- 172.21.15.27:0/3539425446 shutdown_connections 2024-04-26T23:03:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.068+0000 7f18e57c3740 1 -- 172.21.15.27:0/3539425446 wait complete. 2024-04-26T23:03:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.069+0000 7f18e57c3740 1 -- start start 2024-04-26T23:03:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.069+0000 7f18e57c3740 1 Processor -- start 2024-04-26T23:03:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.071+0000 7f18e57c3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307b7dd0 0x562b307cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.071+0000 7f18e57c3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307b86f0 0x562b307bc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.071+0000 7f18e57c3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307c2fb0 0x562b307bd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.071+0000 7f18e57c3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc00e860 con 0x562b307b86f0 2024-04-26T23:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.071+0000 7f18e57c3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc00fcd0 con 0x562b307b7dd0 2024-04-26T23:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.071+0000 7f18e57c3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc00ef30 con 0x562b307c2fb0 2024-04-26T23:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.071+0000 7f18e57c3740 10 client.0 _ll_register_callbacks cb 0x7f18cc02d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.072+0000 7f18daa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307b7dd0 0x562b307cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.072+0000 7f18daa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307b7dd0 0x562b307cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:40750/0 (socket says 172.21.15.27:40750) 2024-04-26T23:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.072+0000 7f18daa00640 1 -- 172.21.15.27:0/2048862527 learned_addr learned my addr 172.21.15.27:0/2048862527 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.072+0000 7f18daa00640 1 --2- 172.21.15.27:0/2048862527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307c2fb0 0x562b307bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.073+0000 7f18db400640 1 --2- 172.21.15.27:0/2048862527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307b86f0 0x562b307bc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.073+0000 7f18daa00640 1 -- 172.21.15.27:0/2048862527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307c2fb0 msgr2=0x562b307bd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.073+0000 7f18daa00640 1 --2- 172.21.15.27:0/2048862527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307c2fb0 0x562b307bd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.073+0000 7f18daa00640 1 -- 172.21.15.27:0/2048862527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307b86f0 msgr2=0x562b307bc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.073+0000 7f18daa00640 1 --2- 172.21.15.27:0/2048862527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307b86f0 0x562b307bc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.073+0000 7f18daa00640 1 -- 172.21.15.27:0/2048862527 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b304fb620 con 0x562b307b7dd0 2024-04-26T23:03:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.073+0000 7f18daa00640 1 --2- 172.21.15.27:0/2048862527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307b7dd0 0x562b307cb530 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x562b307b5470 tx=0x7f18c800bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cca2ee1486f50de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.074+0000 7f18c5600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T23:03:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.074+0000 7f18c5600640 1 -- 172.21.15.27:0/2048862527 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c8010c00 con 0x562b307b7dd0 2024-04-26T23:03:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.074+0000 7f18c5600640 1 -- 172.21.15.27:0/2048862527 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7f18c80131d0 con 0x562b307b7dd0 2024-04-26T23:03:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.074+0000 7f18e57c3740 1 -- 172.21.15.27:0/2048862527 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18cc030040 con 0x562b307b7dd0 2024-04-26T23:03:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.075+0000 7f18c5600640 1 -- 172.21.15.27:0/2048862527 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18c8010c00 con 0x562b307b7dd0 2024-04-26T23:03:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.075+0000 7f18e57c3740 10 client.25826 fetch_fsmap learned FSMap version 30 2024-04-26T23:03:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.075+0000 7f18e57c3740 1 -- 172.21.15.27:0/2048862527 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562b306d7cb0 con 0x562b307b7dd0 2024-04-26T23:03:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.076+0000 7f18c5600640 1 -- 172.21.15.27:0/2048862527 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18cc030040 con 0x562b307b7dd0 2024-04-26T23:03:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.076+0000 7f18c5600640 1 -- 172.21.15.27:0/2048862527 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f18c801f670 con 0x562b307b7dd0 2024-04-26T23:03:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.078+0000 7f18e57c3740 10 client.25826 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:03:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.078+0000 7f18e57c3740 10 client.25826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:03:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.078+0000 7f18e57c3740 20 client.25826 populate_metadata read hostname 'smithi027' 2024-04-26T23:03:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.078+0000 7f18e57c3740 1 --2- 172.21.15.27:0/2048862527 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x562b30839570 0x562b3083b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.078+0000 7f18e57c3740 4 client.25826 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:03:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.078+0000 7f18e57c3740 1 -- 172.21.15.27:0/2048862527 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b30724e00 con 0x562b30839570 2024-04-26T23:03:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.078+0000 7f18db400640 1 --2- 172.21.15.27:0/2048862527 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x562b30839570 0x562b3083b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.080+0000 7f18db400640 1 --2- 172.21.15.27:0/2048862527 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x562b30839570 0x562b3083b990 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b840095c074f92c7 server_cookie=b0a8bbd4e8e70cf8 in_seq=0 out_seq=0 2024-04-26T23:03:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.080+0000 7f18c5600640 10 client.25826 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:03:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.105+0000 7f18c5600640 1 -- 172.21.15.27:0/2048862527 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562b30724e00 con 0x562b30839570 2024-04-26T23:03:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.105+0000 7f18c5600640 10 client.25826 handle_command_reply: tid=0 2024-04-26T23:03:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.106+0000 7f18e57c3740 1 client.25826 shutdown 2024-04-26T23:03:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.106+0000 7f18e57c3740 1 -- 172.21.15.27:0/2048862527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307b7dd0 msgr2=0x562b307cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.106+0000 7f18e57c3740 1 --2- 172.21.15.27:0/2048862527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307b7dd0 0x562b307cb530 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x562b307b5470 tx=0x7f18c800bbc0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.107+0000 7f18e57c3740 1 -- 172.21.15.27:0/2048862527 shutdown_connections 2024-04-26T23:03:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.107+0000 7f18e57c3740 1 --2- 172.21.15.27:0/2048862527 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x562b30839570 0x562b3083b990 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.107+0000 7f18e57c3740 1 --2- 172.21.15.27:0/2048862527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b307c2fb0 0x562b307bd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.107+0000 7f18e57c3740 1 --2- 172.21.15.27:0/2048862527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b307b86f0 0x562b307bc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.107+0000 7f18e57c3740 1 --2- 172.21.15.27:0/2048862527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562b307b7dd0 0x562b307cb530 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.107+0000 7f18c5600640 0 client.25826 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:03:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.107+0000 7f18e57c3740 1 -- 172.21.15.27:0/2048862527 >> 172.21.15.27:0/2048862527 conn(0x562b307b42e0 msgr2=0x562b307bef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.108+0000 7f18e57c3740 1 -- 172.21.15.27:0/2048862527 shutdown_connections 2024-04-26T23:03:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.108+0000 7f18e57c3740 1 -- 172.21.15.27:0/2048862527 wait complete. 2024-04-26T23:03:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.108+0000 7f18e57c3740 20 client.25826 trim_cache size 0 max 16384 2024-04-26T23:03:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T23:03:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T23:03:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T23:03:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T23:03:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.109+0000 7f18e2800640 1 -- 172.21.15.27:0/1601741007 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f18c00a3d70 msgr2=0x7f18c00a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.109+0000 7f18e2800640 1 --2- 172.21.15.27:0/1601741007 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f18c00a3d70 0x7f18c00a6230 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18d40b94f0 tx=0x7f18d40eef90 comp rx=0 tx=0).stop 2024-04-26T23:03:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.109+0000 7f18e2800640 1 -- 172.21.15.27:0/1601741007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18dc10f300 msgr2=0x7f18dc1044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.109+0000 7f18e2800640 1 --2- 172.21.15.27:0/1601741007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18dc10f300 0x7f18dc1044b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f18d402acb0 tx=0x7f18d4003cc0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.110+0000 7f18e2800640 1 -- 172.21.15.27:0/1601741007 shutdown_connections 2024-04-26T23:03:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.110+0000 7f18e2800640 1 --2- 172.21.15.27:0/1601741007 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f18c00a3d70 0x7f18c00a6230 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.110+0000 7f18e2800640 1 --2- 172.21.15.27:0/1601741007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc102100 0x7f18dc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.110+0000 7f18e2800640 1 --2- 172.21.15.27:0/1601741007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18dc10f300 0x7f18dc1044b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.110+0000 7f18e2800640 1 --2- 172.21.15.27:0/1601741007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18dc101750 0x7f18dc103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.110+0000 7f18e2800640 1 -- 172.21.15.27:0/1601741007 >> 172.21.15.27:0/1601741007 conn(0x7f18dc0fd350 msgr2=0x7f18dc0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.110+0000 7f18e2800640 1 -- 172.21.15.27:0/1601741007 shutdown_connections 2024-04-26T23:03:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:21.110+0000 7f18e2800640 1 -- 172.21.15.27:0/1601741007 wait complete. 2024-04-26T23:03:21.273 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:20 smithi080 ceph-mon[32324]: pgmap v145: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 857 KiB/s wr, 114 op/s 2024-04-26T23:03:21.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:20 smithi094 ceph-mon[32889]: pgmap v145: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 857 KiB/s wr, 114 op/s 2024-04-26T23:03:21.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:20 smithi027 ceph-mon[28034]: pgmap v145: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 857 KiB/s wr, 114 op/s 2024-04-26T23:03:23.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:22 smithi080 ceph-mon[32324]: pgmap v146: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-04-26T23:03:23.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:22 smithi094 ceph-mon[32889]: pgmap v146: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-04-26T23:03:23.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:22 smithi027 ceph-mon[28034]: pgmap v146: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-04-26T23:03:25.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:24 smithi080 ceph-mon[32324]: pgmap v147: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 935 KiB/s wr, 124 op/s 2024-04-26T23:03:25.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:24 smithi094 ceph-mon[32889]: pgmap v147: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 935 KiB/s wr, 124 op/s 2024-04-26T23:03:25.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:24 smithi027 ceph-mon[28034]: pgmap v147: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 935 KiB/s wr, 124 op/s 2024-04-26T23:03:27.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:27 smithi080 ceph-mon[32324]: pgmap v148: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 935 KiB/s wr, 124 op/s 2024-04-26T23:03:27.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:27 smithi094 ceph-mon[32889]: pgmap v148: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 935 KiB/s wr, 124 op/s 2024-04-26T23:03:27.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:27 smithi027 ceph-mon[28034]: pgmap v148: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 935 KiB/s wr, 124 op/s 2024-04-26T23:03:29.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:29 smithi080 ceph-mon[32324]: pgmap v149: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-04-26T23:03:29.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:29 smithi094 ceph-mon[32889]: pgmap v149: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-04-26T23:03:29.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:29 smithi027 ceph-mon[28034]: pgmap v149: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-04-26T23:03:31.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:31 smithi080 ceph-mon[32324]: pgmap v150: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-04-26T23:03:31.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:31 smithi094 ceph-mon[32889]: pgmap v150: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-04-26T23:03:31.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:31 smithi027 ceph-mon[28034]: pgmap v150: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-04-26T23:03:33.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:33 smithi094 ceph-mon[32889]: pgmap v151: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2024-04-26T23:03:33.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:33 smithi027 ceph-mon[28034]: pgmap v151: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2024-04-26T23:03:33.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:33 smithi080 ceph-mon[32324]: pgmap v151: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2024-04-26T23:03:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.266+0000 7fb1f2400640 1 -- 172.21.15.27:0/4243813522 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1ec10f0a0 msgr2=0x7fb1ec111490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.266+0000 7fb1f2400640 1 --2- 172.21.15.27:0/4243813522 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1ec10f0a0 0x7fb1ec111490 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009be0 tx=0x7fb1e402a6c0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.266+0000 7fb1f2400640 1 -- 172.21.15.27:0/4243813522 shutdown_connections 2024-04-26T23:03:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.266+0000 7fb1f2400640 1 --2- 172.21.15.27:0/4243813522 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1ec10f0a0 0x7fb1ec111490 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.266+0000 7fb1f2400640 1 --2- 172.21.15.27:0/4243813522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1ec101ea0 0x7fb1ec10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.266+0000 7fb1f2400640 1 --2- 172.21.15.27:0/4243813522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1ec1014f0 0x7fb1ec1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.266+0000 7fb1f2400640 1 -- 172.21.15.27:0/4243813522 >> 172.21.15.27:0/4243813522 conn(0x7fb1ec0fd110 msgr2=0x7fb1ec0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.266+0000 7fb1f2400640 1 -- 172.21.15.27:0/4243813522 shutdown_connections 2024-04-26T23:03:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.266+0000 7fb1f2400640 1 -- 172.21.15.27:0/4243813522 wait complete. 2024-04-26T23:03:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.267+0000 7fb1f2400640 1 Processor -- start 2024-04-26T23:03:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.267+0000 7fb1f2400640 1 -- start start 2024-04-26T23:03:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.267+0000 7fb1f2400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1ec1014f0 0x7fb1ec19b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.268+0000 7fb1f2400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1ec101ea0 0x7fb1ec19b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.268+0000 7fb1f2400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1ec10f0a0 0x7fb1ec19f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.268+0000 7fb1f2400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec114c00 con 0x7fb1ec1014f0 2024-04-26T23:03:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.268+0000 7fb1f2400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec114a80 con 0x7fb1ec10f0a0 2024-04-26T23:03:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.268+0000 7fb1f2400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec114d80 con 0x7fb1ec101ea0 2024-04-26T23:03:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.268+0000 7fb1f1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1ec1014f0 0x7fb1ec19b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.268+0000 7fb1f1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1ec1014f0 0x7fb1ec19b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52908/0 (socket says 172.21.15.27:52908) 2024-04-26T23:03:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.268+0000 7fb1f1000640 1 -- 172.21.15.27:0/465659205 learned_addr learned my addr 172.21.15.27:0/465659205 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:03:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.268+0000 7fb1f1a00640 1 --2- 172.21.15.27:0/465659205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1ec10f0a0 0x7fb1ec19f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.268+0000 7fb1e3e00640 1 --2- 172.21.15.27:0/465659205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1ec101ea0 0x7fb1ec19b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.268+0000 7fb1f1000640 1 -- 172.21.15.27:0/465659205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1ec101ea0 msgr2=0x7fb1ec19b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.268+0000 7fb1f1000640 1 --2- 172.21.15.27:0/465659205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1ec101ea0 0x7fb1ec19b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.268+0000 7fb1f1000640 1 -- 172.21.15.27:0/465659205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1ec10f0a0 msgr2=0x7fb1ec19f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.268+0000 7fb1f1000640 1 --2- 172.21.15.27:0/465659205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1ec10f0a0 0x7fb1ec19f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.268+0000 7fb1f1000640 1 -- 172.21.15.27:0/465659205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ec1a00f0 con 0x7fb1ec1014f0 2024-04-26T23:03:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.268+0000 7fb1f1a00640 1 --2- 172.21.15.27:0/465659205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1ec10f0a0 0x7fb1ec19f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:03:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.269+0000 7fb1f1000640 1 --2- 172.21.15.27:0/465659205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1ec1014f0 0x7fb1ec19b100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400e9f0 tx=0x7fb1d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.269+0000 7fb1ea000640 1 -- 172.21.15.27:0/465659205 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400ccc0 con 0x7fb1ec1014f0 2024-04-26T23:03:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.269+0000 7fb1f2400640 1 -- 172.21.15.27:0/465659205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ec1a03e0 con 0x7fb1ec1014f0 2024-04-26T23:03:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.269+0000 7fb1ea000640 1 -- 172.21.15.27:0/465659205 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1d40054d0 con 0x7fb1ec1014f0 2024-04-26T23:03:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.269+0000 7fb1ea000640 1 -- 172.21.15.27:0/465659205 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4010430 con 0x7fb1ec1014f0 2024-04-26T23:03:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.269+0000 7fb1f2400640 1 -- 172.21.15.27:0/465659205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ec1a7c70 con 0x7fb1ec1014f0 2024-04-26T23:03:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.273+0000 7fb1f6966740 1 Processor -- start 2024-04-26T23:03:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.273+0000 7fb1f6966740 1 -- start start 2024-04-26T23:03:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.273+0000 7fb1ea000640 1 -- 172.21.15.27:0/465659205 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb1d40046c0 con 0x7fb1ec1014f0 2024-04-26T23:03:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.274+0000 7fb1f6966740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0ea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.274+0000 7fb1f6966740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0ea900 0x5624eb0f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.274+0000 7fb1f6966740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0f51c0 0x5624eb0f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.274+0000 7fb1f6966740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5624eb04f380 con 0x5624eb0f51c0 2024-04-26T23:03:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.274+0000 7fb1f6966740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5624eafebc30 con 0x5624eb0e9fe0 2024-04-26T23:03:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.274+0000 7fb1f6966740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5624eb04b340 con 0x5624eb0ea900 2024-04-26T23:03:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.274+0000 7fb1f1000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0ea900 0x5624eb0f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.274+0000 7fb1f1a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0ea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.274+0000 7fb1f1000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0ea900 0x5624eb0f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:47728/0 (socket says 172.21.15.27:47728) 2024-04-26T23:03:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.274+0000 7fb1f1000640 1 -- 172.21.15.27:0/3322167101 learned_addr learned my addr 172.21.15.27:0/3322167101 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:03:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.274+0000 7fb1e3e00640 1 --2- 172.21.15.27:0/3322167101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0f51c0 0x5624eb0f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.275+0000 7fb1f1000640 1 -- 172.21.15.27:0/3322167101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 msgr2=0x5624eb0ea3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.275+0000 7fb1f1000640 1 --2- 172.21.15.27:0/3322167101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0ea3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.275+0000 7fb1f1000640 1 -- 172.21.15.27:0/3322167101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0f51c0 msgr2=0x5624eb0f75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.275+0000 7fb1f1000640 1 --2- 172.21.15.27:0/3322167101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0f51c0 0x5624eb0f75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.275+0000 7fb1f1000640 1 -- 172.21.15.27:0/3322167101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624eb00d940 con 0x5624eb0ea900 2024-04-26T23:03:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.275+0000 7fb1f1a00640 1 --2- 172.21.15.27:0/3322167101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0ea3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:03:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.275+0000 7fb1e3e00640 1 --2- 172.21.15.27:0/3322167101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0f51c0 0x5624eb0f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:03:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.275+0000 7fb1f1000640 1 --2- 172.21.15.27:0/3322167101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0ea900 0x5624eb0f4c80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40ce0d0 tx=0x7fb1d40d4f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9daf31d3bb265092 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.275+0000 7fb1ea000640 1 --2- 172.21.15.27:0/465659205 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fb1cc0a3d80 0x7fb1cc0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.276+0000 7fb1ea000640 1 -- 172.21.15.27:0/465659205 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fb1d40d0e50 con 0x7fb1ec1014f0 2024-04-26T23:03:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.276+0000 7fb1e2000640 1 -- 172.21.15.27:0/3322167101 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d40e4070 con 0x5624eb0ea900 2024-04-26T23:03:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.276+0000 7fb1e2000640 1 -- 172.21.15.27:0/3322167101 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1d4005680 con 0x5624eb0ea900 2024-04-26T23:03:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.276+0000 7fb1f1a00640 1 --2- 172.21.15.27:0/465659205 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fb1cc0a3d80 0x7fb1cc0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.276+0000 7fb1e2000640 1 -- 172.21.15.27:0/3322167101 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d409c4f0 con 0x5624eb0ea900 2024-04-26T23:03:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.277+0000 7fb1f6966740 1 -- 172.21.15.27:0/3322167101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0ea900 msgr2=0x5624eb0f4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.277+0000 7fb1f6966740 1 --2- 172.21.15.27:0/3322167101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0ea900 0x5624eb0f4c80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40ce0d0 tx=0x7fb1d40d4f00 comp rx=0 tx=0).stop 2024-04-26T23:03:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.277+0000 7fb1f1a00640 1 --2- 172.21.15.27:0/465659205 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fb1cc0a3d80 0x7fb1cc0a6240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb1e402abe0 tx=0x7fb1e4005e30 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.277+0000 7fb1f6966740 1 -- 172.21.15.27:0/3322167101 shutdown_connections 2024-04-26T23:03:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.277+0000 7fb1f6966740 1 --2- 172.21.15.27:0/3322167101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0f51c0 0x5624eb0f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.277+0000 7fb1f6966740 1 --2- 172.21.15.27:0/3322167101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0ea900 0x5624eb0f4c80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.277+0000 7fb1f6966740 1 --2- 172.21.15.27:0/3322167101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0ea3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.277+0000 7fb1f6966740 1 -- 172.21.15.27:0/3322167101 >> 172.21.15.27:0/3322167101 conn(0x5624eb0e64f0 msgr2=0x5624eb0e68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.277+0000 7fb1f6966740 1 -- 172.21.15.27:0/3322167101 shutdown_connections 2024-04-26T23:03:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.278+0000 7fb1f6966740 1 -- 172.21.15.27:0/3322167101 wait complete. 2024-04-26T23:03:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.278+0000 7fb1f6966740 1 -- start start 2024-04-26T23:03:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.278+0000 7fb1f6966740 1 Processor -- start 2024-04-26T23:03:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.279+0000 7fb1f6966740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0f2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.279+0000 7fb1f6966740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0ea900 0x5624eb0f2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.279+0000 7fb1f6966740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0f51c0 0x5624eb0f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.279+0000 7fb1f6966740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5624eb0f7df0 con 0x5624eb0ea900 2024-04-26T23:03:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.279+0000 7fb1f6966740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5624eafd6c60 con 0x5624eb0e9fe0 2024-04-26T23:03:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.279+0000 7fb1f6966740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5624eb0f7f70 con 0x5624eb0f51c0 2024-04-26T23:03:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.279+0000 7fb1f6966740 10 client.0 _ll_register_callbacks cb 0x5624eb002e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:03:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.279+0000 7fb1f1000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.279+0000 7fb1f1000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0f2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:37214/0 (socket says 172.21.15.27:37214) 2024-04-26T23:03:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.279+0000 7fb1f1000640 1 -- 172.21.15.27:0/3060563533 learned_addr learned my addr 172.21.15.27:0/3060563533 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:03:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.279+0000 7fb1f1a00640 1 --2- 172.21.15.27:0/3060563533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0f51c0 0x5624eb0f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.279+0000 7fb1e3e00640 1 --2- 172.21.15.27:0/3060563533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0ea900 0x5624eb0f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.279+0000 7fb1f1000640 1 -- 172.21.15.27:0/3060563533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0f51c0 msgr2=0x5624eb0f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.279+0000 7fb1f1000640 1 --2- 172.21.15.27:0/3060563533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0f51c0 0x5624eb0f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.279+0000 7fb1f1000640 1 -- 172.21.15.27:0/3060563533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0ea900 msgr2=0x5624eb0f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.279+0000 7fb1f1000640 1 --2- 172.21.15.27:0/3060563533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0ea900 0x5624eb0f2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.280+0000 7fb1f1000640 1 -- 172.21.15.27:0/3060563533 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624eafe5510 con 0x5624eb0e9fe0 2024-04-26T23:03:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.280+0000 7fb1e3e00640 1 --2- 172.21.15.27:0/3060563533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0ea900 0x5624eb0f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:03:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.280+0000 7fb1f1000640 1 --2- 172.21.15.27:0/3060563533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0f2120 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40ce210 tx=0x7fb1d4004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0e437f80a9f6c09 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.280+0000 7fb1daa00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T23:03:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.280+0000 7fb1daa00640 1 -- 172.21.15.27:0/3060563533 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d409c310 con 0x5624eb0e9fe0 2024-04-26T23:03:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.280+0000 7fb1daa00640 1 -- 172.21.15.27:0/3060563533 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fb1d409cc10 con 0x5624eb0e9fe0 2024-04-26T23:03:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.280+0000 7fb1f6966740 1 -- 172.21.15.27:0/3060563533 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1d4005680 con 0x5624eb0e9fe0 2024-04-26T23:03:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.280+0000 7fb1daa00640 1 -- 172.21.15.27:0/3060563533 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb1d409c4f0 con 0x5624eb0e9fe0 2024-04-26T23:03:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.280+0000 7fb1f6966740 10 client.25838 fetch_fsmap learned FSMap version 30 2024-04-26T23:03:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.280+0000 7fb1f6966740 1 -- 172.21.15.27:0/3060563533 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5624ead61210 con 0x5624eb0e9fe0 2024-04-26T23:03:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.281+0000 7fb1daa00640 1 -- 172.21.15.27:0/3060563533 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1d4005680 con 0x5624eb0e9fe0 2024-04-26T23:03:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.281+0000 7fb1daa00640 1 -- 172.21.15.27:0/3060563533 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb1d409dce0 con 0x5624eb0e9fe0 2024-04-26T23:03:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.281+0000 7fb1f6966740 10 client.25838 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:03:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.281+0000 7fb1f6966740 10 client.25838 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:03:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.281+0000 7fb1f6966740 20 client.25838 populate_metadata read hostname 'smithi027' 2024-04-26T23:03:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.281+0000 7fb1f6966740 1 --2- 172.21.15.27:0/3060563533 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5624eb10d5c0 0x5624eb10f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.281+0000 7fb1f6966740 4 client.25838 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:03:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.281+0000 7fb1f6966740 1 -- 172.21.15.27:0/3060563533 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624eb05cf90 con 0x5624eb10d5c0 2024-04-26T23:03:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.281+0000 7fb1e3e00640 1 --2- 172.21.15.27:0/3060563533 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5624eb10d5c0 0x5624eb10f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.282+0000 7fb1e3e00640 1 --2- 172.21.15.27:0/3060563533 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5624eb10d5c0 0x5624eb10f9b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c7382c0754cfe0a server_cookie=ca9ec4d450a0c19b in_seq=0 out_seq=0 2024-04-26T23:03:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.282+0000 7fb1daa00640 10 client.25838 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:03:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.283+0000 7fb1daa00640 1 -- 172.21.15.27:0/3060563533 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5624eb05cf90 con 0x5624eb10d5c0 2024-04-26T23:03:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.283+0000 7fb1daa00640 10 client.25838 handle_command_reply: tid=0 2024-04-26T23:03:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.283+0000 7fb1f6966740 1 client.25838 shutdown 2024-04-26T23:03:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.283+0000 7fb1f6966740 1 -- 172.21.15.27:0/3060563533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 msgr2=0x5624eb0f2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.283+0000 7fb1f6966740 1 --2- 172.21.15.27:0/3060563533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0f2120 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40ce210 tx=0x7fb1d4004a60 comp rx=0 tx=0).stop 2024-04-26T23:03:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.283+0000 7fb1f6966740 1 -- 172.21.15.27:0/3060563533 shutdown_connections 2024-04-26T23:03:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.283+0000 7fb1f6966740 1 --2- 172.21.15.27:0/3060563533 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5624eb10d5c0 0x5624eb10f9b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.283+0000 7fb1f6966740 1 --2- 172.21.15.27:0/3060563533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0f51c0 0x5624eb0f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.283+0000 7fb1f6966740 1 --2- 172.21.15.27:0/3060563533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0ea900 0x5624eb0f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.283+0000 7fb1f6966740 1 --2- 172.21.15.27:0/3060563533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0f2120 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.283+0000 7fb1daa00640 0 client.25838 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:03:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.283+0000 7fb1f6966740 1 -- 172.21.15.27:0/3060563533 >> 172.21.15.27:0/3060563533 conn(0x5624eb0e64f0 msgr2=0x5624eb0ec220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.283+0000 7fb1f6966740 1 -- 172.21.15.27:0/3060563533 shutdown_connections 2024-04-26T23:03:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.283+0000 7fb1f6966740 1 -- 172.21.15.27:0/3060563533 wait complete. 2024-04-26T23:03:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.284+0000 7fb1f6966740 20 client.25838 trim_cache size 0 max 16384 2024-04-26T23:03:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.302+0000 7fb1f6966740 1 Processor -- start 2024-04-26T23:03:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.302+0000 7fb1f6966740 1 -- start start 2024-04-26T23:03:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.302+0000 7fb1f6966740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.303+0000 7fb1f6966740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0ea900 0x5624eb1129d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.303+0000 7fb1f6966740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0f51c0 0x5624eb112f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.303+0000 7fb1f6966740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec1b04e0 con 0x5624eb0f51c0 2024-04-26T23:03:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.303+0000 7fb1f6966740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec1adcf0 con 0x5624eb0e9fe0 2024-04-26T23:03:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.303+0000 7fb1f6966740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec1abb60 con 0x5624eb0ea900 2024-04-26T23:03:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.303+0000 7fb1f1000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.303+0000 7fb1f1000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0f3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:37220/0 (socket says 172.21.15.27:37220) 2024-04-26T23:03:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.303+0000 7fb1f1000640 1 -- 172.21.15.27:0/625677742 learned_addr learned my addr 172.21.15.27:0/625677742 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:03:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.303+0000 7fb1f1a00640 1 --2- 172.21.15.27:0/625677742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0f51c0 0x5624eb112f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.303+0000 7fb1e3e00640 1 --2- 172.21.15.27:0/625677742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0ea900 0x5624eb1129d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.303+0000 7fb1f1000640 1 -- 172.21.15.27:0/625677742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0ea900 msgr2=0x5624eb1129d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.303+0000 7fb1f1000640 1 --2- 172.21.15.27:0/625677742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0ea900 0x5624eb1129d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.303+0000 7fb1f1000640 1 -- 172.21.15.27:0/625677742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0f51c0 msgr2=0x5624eb112f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.303+0000 7fb1f1000640 1 --2- 172.21.15.27:0/625677742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0f51c0 0x5624eb112f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.303+0000 7fb1f1000640 1 -- 172.21.15.27:0/625677742 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624eae2d620 con 0x5624eb0e9fe0 2024-04-26T23:03:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.303+0000 7fb1e3e00640 1 --2- 172.21.15.27:0/625677742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0ea900 0x5624eb1129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:03:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.304+0000 7fb1f1000640 1 --2- 172.21.15.27:0/625677742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0f3f50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40ce760 tx=0x7fb1d40d4f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f02370450cb58b67 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.304+0000 7fb1e1600640 1 -- 172.21.15.27:0/625677742 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4098c50 con 0x5624eb0e9fe0 2024-04-26T23:03:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.304+0000 7fb1e1600640 1 -- 172.21.15.27:0/625677742 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1d40b5030 con 0x5624eb0e9fe0 2024-04-26T23:03:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.304+0000 7fb1e1600640 1 -- 172.21.15.27:0/625677742 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d40d4770 con 0x5624eb0e9fe0 2024-04-26T23:03:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.305+0000 7fb1f6966740 1 -- 172.21.15.27:0/625677742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 msgr2=0x5624eb0f3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.305+0000 7fb1f6966740 1 --2- 172.21.15.27:0/625677742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0f3f50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40ce760 tx=0x7fb1d40d4f00 comp rx=0 tx=0).stop 2024-04-26T23:03:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.305+0000 7fb1f6966740 1 -- 172.21.15.27:0/625677742 shutdown_connections 2024-04-26T23:03:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.305+0000 7fb1f6966740 1 --2- 172.21.15.27:0/625677742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0f51c0 0x5624eb112f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.305+0000 7fb1f6966740 1 --2- 172.21.15.27:0/625677742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0ea900 0x5624eb1129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.305+0000 7fb1f6966740 1 --2- 172.21.15.27:0/625677742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0e9fe0 0x5624eb0f3f50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.305+0000 7fb1f6966740 1 -- 172.21.15.27:0/625677742 >> 172.21.15.27:0/625677742 conn(0x5624eb0e64f0 msgr2=0x5624eb0f3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.305+0000 7fb1f6966740 1 -- 172.21.15.27:0/625677742 shutdown_connections 2024-04-26T23:03:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.305+0000 7fb1f6966740 1 -- 172.21.15.27:0/625677742 wait complete. 2024-04-26T23:03:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.305+0000 7fb1f6966740 1 -- start start 2024-04-26T23:03:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.306+0000 7fb1f6966740 1 Processor -- start 2024-04-26T23:03:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.306+0000 7fb1f6966740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0e9fe0 0x5624eb1104f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.306+0000 7fb1f6966740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0ea900 0x5624eb110a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.306+0000 7fb1f6966740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0f51c0 0x5624eb110f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.306+0000 7fb1f6966740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec1ac1b0 con 0x5624eb0e9fe0 2024-04-26T23:03:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.306+0000 7fb1f6966740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec1ad620 con 0x5624eb0ea900 2024-04-26T23:03:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.306+0000 7fb1f6966740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec1ac880 con 0x5624eb0f51c0 2024-04-26T23:03:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.306+0000 7fb1f6966740 10 client.0 _ll_register_callbacks cb 0x7fb1d40e2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:03:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.307+0000 7fb1e3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0ea900 0x5624eb110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.307+0000 7fb1e3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0ea900 0x5624eb110a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:37230/0 (socket says 172.21.15.27:37230) 2024-04-26T23:03:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.307+0000 7fb1f1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0e9fe0 0x5624eb1104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.307+0000 7fb1e3e00640 1 -- 172.21.15.27:0/3269570331 learned_addr learned my addr 172.21.15.27:0/3269570331 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:03:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.307+0000 7fb1f1a00640 1 --2- 172.21.15.27:0/3269570331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0f51c0 0x5624eb110f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.307+0000 7fb1e3e00640 1 -- 172.21.15.27:0/3269570331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0f51c0 msgr2=0x5624eb110f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.307+0000 7fb1e3e00640 1 --2- 172.21.15.27:0/3269570331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0f51c0 0x5624eb110f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.307+0000 7fb1e3e00640 1 -- 172.21.15.27:0/3269570331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0e9fe0 msgr2=0x5624eb1104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:03:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.307+0000 7fb1e3e00640 1 --2- 172.21.15.27:0/3269570331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0e9fe0 0x5624eb1104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.307+0000 7fb1e3e00640 1 -- 172.21.15.27:0/3269570331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624eb009cb0 con 0x5624eb0ea900 2024-04-26T23:03:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.307+0000 7fb1f1a00640 1 --2- 172.21.15.27:0/3269570331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0f51c0 0x5624eb110f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:03:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.307+0000 7fb1e3e00640 1 --2- 172.21.15.27:0/3269570331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0ea900 0x5624eb110a30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009c80 tx=0x7fb1dc0108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=141b4eb1c92a2bbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:03:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.307+0000 7fb1daa00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-26T23:03:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.308+0000 7fb1daa00640 1 -- 172.21.15.27:0/3269570331 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc0203c0 con 0x5624eb0ea900 2024-04-26T23:03:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.308+0000 7fb1daa00640 1 -- 172.21.15.27:0/3269570331 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9612+0+0 (secure 0 0 0) 0x7fb1dc020560 con 0x5624eb0ea900 2024-04-26T23:03:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.308+0000 7fb1f6966740 1 -- 172.21.15.27:0/3269570331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1d40b5030 con 0x5624eb0ea900 2024-04-26T23:03:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.308+0000 7fb1daa00640 1 -- 172.21.15.27:0/3269570331 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb1dc02f050 con 0x5624eb0ea900 2024-04-26T23:03:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.308+0000 7fb1f6966740 10 client.25844 fetch_fsmap learned FSMap version 30 2024-04-26T23:03:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.308+0000 7fb1f6966740 1 -- 172.21.15.27:0/3269570331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5624eb173a80 con 0x5624eb0ea900 2024-04-26T23:03:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.308+0000 7fb1daa00640 1 -- 172.21.15.27:0/3269570331 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1d40b5030 con 0x5624eb0ea900 2024-04-26T23:03:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.308+0000 7fb1daa00640 1 -- 172.21.15.27:0/3269570331 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb1dc01d380 con 0x5624eb0ea900 2024-04-26T23:03:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.308+0000 7fb1f6966740 10 client.25844 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:03:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.309+0000 7fb1f6966740 10 client.25844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:03:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.309+0000 7fb1f6966740 20 client.25844 populate_metadata read hostname 'smithi027' 2024-04-26T23:03:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.309+0000 7fb1f6966740 1 --2- 172.21.15.27:0/3269570331 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5624eb174290 0x5624eb1766b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:03:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.309+0000 7fb1f6966740 4 client.25844 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:03:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.309+0000 7fb1f6966740 1 -- 172.21.15.27:0/3269570331 --> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624eb056e00 con 0x5624eb174290 2024-04-26T23:03:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.309+0000 7fb1f1000640 1 --2- 172.21.15.27:0/3269570331 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5624eb174290 0x5624eb1766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:03:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.309+0000 7fb1f1000640 1 --2- 172.21.15.27:0/3269570331 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5624eb174290 0x5624eb1766b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ec60a033c5f10bb server_cookie=dd0f7c2853061073 in_seq=0 out_seq=0 2024-04-26T23:03:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.309+0000 7fb1daa00640 10 client.25844 ms_handle_connect on v2:172.21.15.80:6838/95952000 2024-04-26T23:03:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.856+0000 7fb1daa00640 1 -- 172.21.15.27:0/3269570331 <== mds.0 v2:172.21.15.80:6838/95952000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5624eb056e00 con 0x5624eb174290 2024-04-26T23:03:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.856+0000 7fb1daa00640 10 client.25844 handle_command_reply: tid=0 2024-04-26T23:03:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.857+0000 7fb1f6966740 1 client.25844 shutdown 2024-04-26T23:03:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.857+0000 7fb1f6966740 1 -- 172.21.15.27:0/3269570331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0ea900 msgr2=0x5624eb110a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.857+0000 7fb1f6966740 1 --2- 172.21.15.27:0/3269570331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0ea900 0x5624eb110a30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009c80 tx=0x7fb1dc0108c0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.857+0000 7fb1f6966740 1 -- 172.21.15.27:0/3269570331 shutdown_connections 2024-04-26T23:03:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.857+0000 7fb1f6966740 1 --2- 172.21.15.27:0/3269570331 >> [v2:172.21.15.80:6838/95952000,v1:172.21.15.80:6839/95952000] conn(0x5624eb174290 0x5624eb1766b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.857+0000 7fb1f6966740 1 --2- 172.21.15.27:0/3269570331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5624eb0ea900 0x5624eb110a30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.857+0000 7fb1f6966740 1 --2- 172.21.15.27:0/3269570331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624eb0f51c0 0x5624eb110f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.857+0000 7fb1f6966740 1 --2- 172.21.15.27:0/3269570331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624eb0e9fe0 0x5624eb1104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.857+0000 7fb1daa00640 0 client.25844 ms_handle_reset on v2:172.21.15.80:6838/95952000 2024-04-26T23:03:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.857+0000 7fb1f6966740 1 -- 172.21.15.27:0/3269570331 >> 172.21.15.27:0/3269570331 conn(0x5624eb0e64f0 msgr2=0x5624eb0f0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.857+0000 7fb1f6966740 1 -- 172.21.15.27:0/3269570331 shutdown_connections 2024-04-26T23:03:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.857+0000 7fb1f6966740 1 -- 172.21.15.27:0/3269570331 wait complete. 2024-04-26T23:03:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.858+0000 7fb1f6966740 20 client.25844 trim_cache size 0 max 16384 2024-04-26T23:03:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-04-26T23:03:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-04-26T23:03:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-04-26T23:03:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-04-26T23:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.858+0000 7fb1f2400640 1 -- 172.21.15.27:0/465659205 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fb1cc0a3d80 msgr2=0x7fb1cc0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.858+0000 7fb1f2400640 1 --2- 172.21.15.27:0/465659205 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fb1cc0a3d80 0x7fb1cc0a6240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb1e402abe0 tx=0x7fb1e4005e30 comp rx=0 tx=0).stop 2024-04-26T23:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.858+0000 7fb1f2400640 1 -- 172.21.15.27:0/465659205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1ec1014f0 msgr2=0x7fb1ec19b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.858+0000 7fb1f2400640 1 --2- 172.21.15.27:0/465659205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1ec1014f0 0x7fb1ec19b100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400e9f0 tx=0x7fb1d400eeb0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.859+0000 7fb1f2400640 1 -- 172.21.15.27:0/465659205 shutdown_connections 2024-04-26T23:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.859+0000 7fb1f2400640 1 --2- 172.21.15.27:0/465659205 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fb1cc0a3d80 0x7fb1cc0a6240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.859+0000 7fb1f2400640 1 --2- 172.21.15.27:0/465659205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1ec10f0a0 0x7fb1ec19f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.859+0000 7fb1f2400640 1 --2- 172.21.15.27:0/465659205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1ec101ea0 0x7fb1ec19b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.859+0000 7fb1f2400640 1 --2- 172.21.15.27:0/465659205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1ec1014f0 0x7fb1ec19b100 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.859+0000 7fb1f2400640 1 -- 172.21.15.27:0/465659205 >> 172.21.15.27:0/465659205 conn(0x7fb1ec0fd110 msgr2=0x7fb1ec0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:03:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.859+0000 7fb1f2400640 1 -- 172.21.15.27:0/465659205 shutdown_connections 2024-04-26T23:03:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-04-26T23:03:34.859+0000 7fb1f2400640 1 -- 172.21.15.27:0/465659205 wait complete. 2024-04-26T23:03:35.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:35 smithi094 ceph-mon[32889]: pgmap v152: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-04-26T23:03:35.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:35 smithi027 ceph-mon[28034]: pgmap v152: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-04-26T23:03:35.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:35 smithi080 ceph-mon[32324]: pgmap v152: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-04-26T23:03:37.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:37 smithi094 ceph-mon[32889]: pgmap v153: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-04-26T23:03:37.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:37 smithi027 ceph-mon[28034]: pgmap v153: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-04-26T23:03:37.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:37 smithi080 ceph-mon[32324]: pgmap v153: 129 pgs: 129 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-04-26T23:03:39.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:39 smithi027 ceph-mon[28034]: pgmap v154: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 579 op/s 2024-04-26T23:03:39.485 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-26T23:03:39.486 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-26T23:03:39.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:39 smithi080 ceph-mon[32324]: pgmap v154: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 579 op/s 2024-04-26T23:03:39.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:39 smithi094 ceph-mon[32889]: pgmap v154: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 579 op/s 2024-04-26T23:03:40.752 DEBUG:teuthology.parallel:result is None 2024-04-26T23:03:40.752 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T23:03:40.838 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T23:03:40.838 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-26T23:03:40.849 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T23:03:40.875 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-26T23:03:40.884 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-26T23:03:40.885 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T23:03:40.885 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T23:03:40.885 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-26T23:03:41.013 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-26T23:03:41.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:41 smithi027 ceph-mon[28034]: pgmap v155: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 507 op/s 2024-04-26T23:03:41.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:41 smithi080 ceph-mon[32324]: pgmap v155: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 507 op/s 2024-04-26T23:03:41.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:41 smithi094 ceph-mon[32889]: pgmap v155: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 507 op/s 2024-04-26T23:03:43.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:43 smithi027 ceph-mon[28034]: pgmap v156: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 595 op/s 2024-04-26T23:03:43.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:43 smithi080 ceph-mon[32324]: pgmap v156: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 595 op/s 2024-04-26T23:03:43.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:43 smithi094 ceph-mon[32889]: pgmap v156: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 595 op/s 2024-04-26T23:03:45.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:45 smithi027 ceph-mon[28034]: pgmap v157: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 974 KiB/s wr, 511 op/s 2024-04-26T23:03:45.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:45 smithi080 ceph-mon[32324]: pgmap v157: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 974 KiB/s wr, 511 op/s 2024-04-26T23:03:45.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:45 smithi094 ceph-mon[32889]: pgmap v157: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 974 KiB/s wr, 511 op/s 2024-04-26T23:03:47.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:47 smithi027 ceph-mon[28034]: pgmap v158: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 974 KiB/s wr, 511 op/s 2024-04-26T23:03:47.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:47 smithi080 ceph-mon[32324]: pgmap v158: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 974 KiB/s wr, 511 op/s 2024-04-26T23:03:47.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:47 smithi094 ceph-mon[32889]: pgmap v158: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 974 KiB/s wr, 511 op/s 2024-04-26T23:03:49.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:49 smithi027 ceph-mon[28034]: pgmap v159: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 982 KiB/s wr, 516 op/s 2024-04-26T23:03:49.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:49 smithi080 ceph-mon[32324]: pgmap v159: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 982 KiB/s wr, 516 op/s 2024-04-26T23:03:49.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:49 smithi094 ceph-mon[32889]: pgmap v159: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 982 KiB/s wr, 516 op/s 2024-04-26T23:03:51.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:51 smithi027 ceph-mon[28034]: pgmap v160: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 282 KiB/s wr, 92 op/s 2024-04-26T23:03:51.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:51 smithi080 ceph-mon[32324]: pgmap v160: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 282 KiB/s wr, 92 op/s 2024-04-26T23:03:51.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:51 smithi094 ceph-mon[32889]: pgmap v160: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 282 KiB/s wr, 92 op/s 2024-04-26T23:03:52.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:52 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:03:52.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:52 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:03:52.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:52 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:03:53.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:53 smithi027 ceph-mon[28034]: pgmap v161: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 285 KiB/s wr, 96 op/s 2024-04-26T23:03:53.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:53 smithi080 ceph-mon[32324]: pgmap v161: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 285 KiB/s wr, 96 op/s 2024-04-26T23:03:53.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:53 smithi094 ceph-mon[32889]: pgmap v161: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 285 KiB/s wr, 96 op/s 2024-04-26T23:03:54.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:54 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:03:54.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:54 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:03:54.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:54 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:03:54.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:54 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:03:54.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:54 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:03:54.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:54 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:03:54.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:54 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:03:54.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:54 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:03:54.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:54 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:03:55.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:55 smithi027 ceph-mon[28034]: pgmap v162: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T23:03:55.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:55 smithi080 ceph-mon[32324]: pgmap v162: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T23:03:55.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:55 smithi094 ceph-mon[32889]: pgmap v162: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T23:03:57.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:57 smithi027 ceph-mon[28034]: pgmap v163: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T23:03:57.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:57 smithi080 ceph-mon[32324]: pgmap v163: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T23:03:57.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:57 smithi094 ceph-mon[32889]: pgmap v163: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T23:03:58.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:03:58 smithi080 ceph-mon[32324]: pgmap v164: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-26T23:03:58.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:03:58 smithi094 ceph-mon[32889]: pgmap v164: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-26T23:03:58.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:03:58 smithi027 ceph-mon[28034]: pgmap v164: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-26T23:04:01.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:00 smithi027 ceph-mon[28034]: pgmap v165: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-04-26T23:04:01.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:00 smithi080 ceph-mon[32324]: pgmap v165: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-04-26T23:04:01.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:00 smithi094 ceph-mon[32889]: pgmap v165: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-04-26T23:04:03.073 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:02 smithi080 ceph-mon[32324]: pgmap v166: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-04-26T23:04:03.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:02 smithi027 ceph-mon[28034]: pgmap v166: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-04-26T23:04:03.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:02 smithi094 ceph-mon[32889]: pgmap v166: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-04-26T23:04:05.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:04 smithi027 ceph-mon[28034]: pgmap v167: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T23:04:05.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:04 smithi080 ceph-mon[32324]: pgmap v167: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T23:04:05.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:04 smithi094 ceph-mon[32889]: pgmap v167: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T23:04:07.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:06 smithi027 ceph-mon[28034]: pgmap v168: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T23:04:07.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:06 smithi080 ceph-mon[32324]: pgmap v168: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T23:04:07.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:06 smithi094 ceph-mon[32889]: pgmap v168: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T23:04:09.117 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:08 smithi027 ceph-mon[28034]: pgmap v169: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T23:04:09.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:08 smithi080 ceph-mon[32324]: pgmap v169: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T23:04:09.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:08 smithi094 ceph-mon[32889]: pgmap v169: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T23:04:11.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:10 smithi027 ceph-mon[28034]: pgmap v170: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-26T23:04:11.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:10 smithi080 ceph-mon[32324]: pgmap v170: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-26T23:04:11.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:10 smithi094 ceph-mon[32889]: pgmap v170: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-26T23:04:13.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:12 smithi027 ceph-mon[28034]: pgmap v171: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T23:04:13.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:12 smithi080 ceph-mon[32324]: pgmap v171: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T23:04:13.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:12 smithi094 ceph-mon[32889]: pgmap v171: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T23:04:15.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:14 smithi027 ceph-mon[28034]: pgmap v172: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T23:04:15.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:14 smithi080 ceph-mon[32324]: pgmap v172: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T23:04:15.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:14 smithi094 ceph-mon[32889]: pgmap v172: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T23:04:17.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:16 smithi027 ceph-mon[28034]: pgmap v173: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T23:04:17.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:16 smithi080 ceph-mon[32324]: pgmap v173: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T23:04:17.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:16 smithi094 ceph-mon[32889]: pgmap v173: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T23:04:19.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:18 smithi027 ceph-mon[28034]: pgmap v174: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T23:04:19.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:18 smithi080 ceph-mon[32324]: pgmap v174: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T23:04:19.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:18 smithi094 ceph-mon[32889]: pgmap v174: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T23:04:21.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:20 smithi027 ceph-mon[28034]: pgmap v175: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:04:21.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:20 smithi080 ceph-mon[32324]: pgmap v175: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:04:21.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:20 smithi094 ceph-mon[32889]: pgmap v175: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:04:22.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:22 smithi080 ceph-mon[32324]: pgmap v176: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-26T23:04:22.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:22 smithi094 ceph-mon[32889]: pgmap v176: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-26T23:04:22.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:22 smithi027 ceph-mon[28034]: pgmap v176: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-26T23:04:25.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:24 smithi027 ceph-mon[28034]: pgmap v177: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:25.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:24 smithi080 ceph-mon[32324]: pgmap v177: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:25.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:24 smithi094 ceph-mon[32889]: pgmap v177: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:27.087 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:26 smithi094 ceph-mon[32889]: pgmap v178: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:27.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:26 smithi027 ceph-mon[28034]: pgmap v178: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:27.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:26 smithi080 ceph-mon[32324]: pgmap v178: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:29.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:28 smithi027 ceph-mon[28034]: pgmap v179: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:29.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:28 smithi080 ceph-mon[32324]: pgmap v179: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:29.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:28 smithi094 ceph-mon[32889]: pgmap v179: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:31.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:30 smithi027 ceph-mon[28034]: pgmap v180: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:31.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:30 smithi080 ceph-mon[32324]: pgmap v180: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:31.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:30 smithi094 ceph-mon[32889]: pgmap v180: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:33.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:32 smithi027 ceph-mon[28034]: pgmap v181: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:33.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:32 smithi080 ceph-mon[32324]: pgmap v181: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:33.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:32 smithi094 ceph-mon[32889]: pgmap v181: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:35.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:34 smithi027 ceph-mon[28034]: pgmap v182: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:35.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:34 smithi080 ceph-mon[32324]: pgmap v182: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:35.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:34 smithi094 ceph-mon[32889]: pgmap v182: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:37.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:36 smithi027 ceph-mon[28034]: pgmap v183: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:37.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:36 smithi080 ceph-mon[32324]: pgmap v183: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:37.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:36 smithi094 ceph-mon[32889]: pgmap v183: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:39.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:38 smithi027 ceph-mon[28034]: pgmap v184: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:39.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:38 smithi080 ceph-mon[32324]: pgmap v184: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:39.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:38 smithi094 ceph-mon[32889]: pgmap v184: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:41.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:40 smithi027 ceph-mon[28034]: pgmap v185: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:41.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:40 smithi080 ceph-mon[32324]: pgmap v185: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:41.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:40 smithi094 ceph-mon[32889]: pgmap v185: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:43.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:42 smithi027 ceph-mon[28034]: pgmap v186: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:43.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:42 smithi080 ceph-mon[32324]: pgmap v186: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:43.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:42 smithi094 ceph-mon[32889]: pgmap v186: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:45.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:44 smithi027 ceph-mon[28034]: pgmap v187: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:45.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:44 smithi080 ceph-mon[32324]: pgmap v187: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:45.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:44 smithi094 ceph-mon[32889]: pgmap v187: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:47.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:46 smithi080 ceph-mon[32324]: pgmap v188: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:47.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:46 smithi094 ceph-mon[32889]: pgmap v188: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:47.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:46 smithi027 ceph-mon[28034]: pgmap v188: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:49.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:48 smithi080 ceph-mon[32324]: pgmap v189: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:49.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:48 smithi094 ceph-mon[32889]: pgmap v189: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:49.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:48 smithi027 ceph-mon[28034]: pgmap v189: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:51.274 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:50 smithi080 ceph-mon[32324]: pgmap v190: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:51.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:50 smithi094 ceph-mon[32889]: pgmap v190: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:51.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:50 smithi027 ceph-mon[28034]: pgmap v190: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:53.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:52 smithi080 ceph-mon[32324]: pgmap v191: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:53.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:52 smithi094 ceph-mon[32889]: pgmap v191: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:53.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:52 smithi027 ceph-mon[28034]: pgmap v191: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:54.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:53 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:04:54.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:53 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:04:54.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:53 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:04:55.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:54 smithi080 ceph-mon[32324]: pgmap v192: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:55.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:54 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:04:55.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:54 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:04:55.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:54 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:04:55.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:54 smithi094 ceph-mon[32889]: pgmap v192: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:55.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:54 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:04:55.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:54 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:04:55.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:54 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:04:55.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:54 smithi027 ceph-mon[28034]: pgmap v192: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:55.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:54 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:04:55.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:54 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:04:55.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:54 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:04:57.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:56 smithi080 ceph-mon[32324]: pgmap v193: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:57.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:56 smithi094 ceph-mon[32889]: pgmap v193: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:57.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:56 smithi027 ceph-mon[28034]: pgmap v193: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:04:59.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:04:58 smithi080 ceph-mon[32324]: pgmap v194: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:59.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:04:58 smithi094 ceph-mon[32889]: pgmap v194: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:04:59.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:04:58 smithi027 ceph-mon[28034]: pgmap v194: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:01.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:00 smithi080 ceph-mon[32324]: pgmap v195: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:01.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:00 smithi094 ceph-mon[32889]: pgmap v195: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:01.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:00 smithi027 ceph-mon[28034]: pgmap v195: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:03.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:02 smithi080 ceph-mon[32324]: pgmap v196: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T23:05:03.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:02 smithi094 ceph-mon[32889]: pgmap v196: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T23:05:03.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:02 smithi027 ceph-mon[28034]: pgmap v196: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T23:05:05.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:04 smithi080 ceph-mon[32324]: pgmap v197: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-26T23:05:05.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:04 smithi094 ceph-mon[32889]: pgmap v197: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-26T23:05:05.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:04 smithi027 ceph-mon[28034]: pgmap v197: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-26T23:05:07.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:06 smithi080 ceph-mon[32324]: pgmap v198: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-26T23:05:07.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:06 smithi094 ceph-mon[32889]: pgmap v198: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-26T23:05:07.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:06 smithi027 ceph-mon[28034]: pgmap v198: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-26T23:05:09.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:08 smithi080 ceph-mon[32324]: pgmap v199: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T23:05:09.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:08 smithi094 ceph-mon[32889]: pgmap v199: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T23:05:09.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:08 smithi027 ceph-mon[28034]: pgmap v199: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T23:05:11.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:10 smithi080 ceph-mon[32324]: pgmap v200: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T23:05:11.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:10 smithi094 ceph-mon[32889]: pgmap v200: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T23:05:11.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:10 smithi027 ceph-mon[28034]: pgmap v200: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T23:05:13.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:12 smithi080 ceph-mon[32324]: pgmap v201: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T23:05:13.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:12 smithi094 ceph-mon[32889]: pgmap v201: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T23:05:13.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:12 smithi027 ceph-mon[28034]: pgmap v201: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T23:05:15.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:14 smithi080 ceph-mon[32324]: pgmap v202: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T23:05:15.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:14 smithi094 ceph-mon[32889]: pgmap v202: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T23:05:15.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:14 smithi027 ceph-mon[28034]: pgmap v202: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T23:05:17.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:16 smithi080 ceph-mon[32324]: pgmap v203: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T23:05:17.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:16 smithi094 ceph-mon[32889]: pgmap v203: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T23:05:17.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:16 smithi027 ceph-mon[28034]: pgmap v203: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T23:05:19.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:18 smithi080 ceph-mon[32324]: pgmap v204: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T23:05:19.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:18 smithi094 ceph-mon[32889]: pgmap v204: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T23:05:19.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:18 smithi027 ceph-mon[28034]: pgmap v204: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T23:05:21.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:20 smithi080 ceph-mon[32324]: pgmap v205: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:05:21.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:20 smithi094 ceph-mon[32889]: pgmap v205: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:05:21.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:20 smithi027 ceph-mon[28034]: pgmap v205: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:05:23.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:22 smithi080 ceph-mon[32324]: pgmap v206: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-26T23:05:23.293 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:22 smithi094 ceph-mon[32889]: pgmap v206: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-26T23:05:23.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:22 smithi027 ceph-mon[28034]: pgmap v206: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-26T23:05:25.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:24 smithi080 ceph-mon[32324]: pgmap v207: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:25.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:24 smithi094 ceph-mon[32889]: pgmap v207: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:25.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:24 smithi027 ceph-mon[28034]: pgmap v207: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:27.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:26 smithi080 ceph-mon[32324]: pgmap v208: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:27.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:26 smithi094 ceph-mon[32889]: pgmap v208: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:27.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:26 smithi027 ceph-mon[28034]: pgmap v208: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:29.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:28 smithi080 ceph-mon[32324]: pgmap v209: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:29.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:28 smithi094 ceph-mon[32889]: pgmap v209: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:29.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:28 smithi027 ceph-mon[28034]: pgmap v209: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:31.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:30 smithi080 ceph-mon[32324]: pgmap v210: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:31.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:30 smithi094 ceph-mon[32889]: pgmap v210: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:31.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:30 smithi027 ceph-mon[28034]: pgmap v210: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:33.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:33 smithi080 ceph-mon[32324]: pgmap v211: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:33.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:33 smithi094 ceph-mon[32889]: pgmap v211: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:33.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:33 smithi027 ceph-mon[28034]: pgmap v211: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:35.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:35 smithi080 ceph-mon[32324]: pgmap v212: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:35.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:35 smithi094 ceph-mon[32889]: pgmap v212: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:35.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:35 smithi027 ceph-mon[28034]: pgmap v212: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:37.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:37 smithi080 ceph-mon[32324]: pgmap v213: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:37.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:37 smithi094 ceph-mon[32889]: pgmap v213: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:37.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:37 smithi027 ceph-mon[28034]: pgmap v213: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:38.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:38 smithi027 ceph-mon[28034]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:05:38.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:38 smithi080 ceph-mon[32324]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:05:38.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:38 smithi094 ceph-mon[32889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:05:39.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:39 smithi027 ceph-mon[28034]: pgmap v214: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:39.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:39 smithi080 ceph-mon[32324]: pgmap v214: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:39.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:39 smithi094 ceph-mon[32889]: pgmap v214: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:41.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:41 smithi027 ceph-mon[28034]: pgmap v215: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:41.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:41 smithi080 ceph-mon[32324]: pgmap v215: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:41.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:41 smithi094 ceph-mon[32889]: pgmap v215: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:43.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:43 smithi027 ceph-mon[28034]: pgmap v216: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:43.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:43 smithi080 ceph-mon[32324]: pgmap v216: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:43.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:43 smithi094 ceph-mon[32889]: pgmap v216: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:45.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:45 smithi027 ceph-mon[28034]: pgmap v217: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:45.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:45 smithi027 ceph-mon[28034]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:05:45.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:45 smithi080 ceph-mon[32324]: pgmap v217: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:45.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:45 smithi080 ceph-mon[32324]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:05:45.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:45 smithi094 ceph-mon[32889]: pgmap v217: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:45.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:45 smithi094 ceph-mon[32889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:05:47.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:47 smithi027 ceph-mon[28034]: pgmap v218: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:47.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:47 smithi080 ceph-mon[32324]: pgmap v218: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:47.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:47 smithi094 ceph-mon[32889]: pgmap v218: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:49.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:49 smithi027 ceph-mon[28034]: pgmap v219: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:49.523 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:49 smithi080 ceph-mon[32324]: pgmap v219: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:49.770 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:49 smithi094 ceph-mon[32889]: pgmap v219: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:51.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:51 smithi027 ceph-mon[28034]: pgmap v220: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:51.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:51 smithi080 ceph-mon[32324]: pgmap v220: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:51.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:51 smithi094 ceph-mon[32889]: pgmap v220: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:53.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:53 smithi027 ceph-mon[28034]: pgmap v221: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:53.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:53 smithi080 ceph-mon[32324]: pgmap v221: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:53.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:53 smithi094 ceph-mon[32889]: pgmap v221: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:55.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:55 smithi027 ceph-mon[28034]: pgmap v222: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:55.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:55 smithi080 ceph-mon[32324]: pgmap v222: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:55.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:55 smithi094 ceph-mon[32889]: pgmap v222: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:56.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:56 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:05:56.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:56 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:05:56.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:56 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:05:57.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:57 smithi027 ceph-mon[28034]: pgmap v223: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:57.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:57 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:05:57.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:57 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:05:57.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:57 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:05:57.435 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:57 smithi094 ceph-mon[32889]: pgmap v223: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:57.435 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:57 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:05:57.435 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:57 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:05:57.435 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:57 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:05:57.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:57 smithi080 ceph-mon[32324]: pgmap v223: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:05:57.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:57 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:05:57.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:57 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:05:57.522 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:57 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:05:59.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:05:59 smithi027 ceph-mon[28034]: pgmap v224: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:59.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:05:59 smithi080 ceph-mon[32324]: pgmap v224: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:05:59.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:05:59 smithi094 ceph-mon[32889]: pgmap v224: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:01.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:01 smithi027 ceph-mon[28034]: pgmap v225: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:01.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:01 smithi080 ceph-mon[32324]: pgmap v225: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:01.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:01 smithi094 ceph-mon[32889]: pgmap v225: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:02.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:02 smithi080 ceph-mon[32324]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:06:02.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:02 smithi080 ceph-mon[32324]: pgmap v226: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T23:06:02.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:02 smithi094 ceph-mon[32889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:06:02.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:02 smithi094 ceph-mon[32889]: pgmap v226: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T23:06:02.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:02 smithi027 ceph-mon[28034]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:06:02.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:02 smithi027 ceph-mon[28034]: pgmap v226: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T23:06:04.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:04 smithi027 ceph-mon[28034]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:06:04.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:04 smithi027 ceph-mon[28034]: pgmap v227: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T23:06:05.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:04 smithi080 ceph-mon[32324]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:06:05.021 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:04 smithi080 ceph-mon[32324]: pgmap v227: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T23:06:05.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:04 smithi094 ceph-mon[32889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:06:05.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:04 smithi094 ceph-mon[32889]: pgmap v227: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T23:06:07.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:06 smithi027 ceph-mon[28034]: pgmap v228: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T23:06:07.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:06 smithi080 ceph-mon[32324]: pgmap v228: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T23:06:07.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:06 smithi094 ceph-mon[32889]: pgmap v228: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T23:06:09.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:08 smithi080 ceph-mon[32324]: pgmap v229: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T23:06:09.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:08 smithi094 ceph-mon[32889]: pgmap v229: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T23:06:09.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:08 smithi027 ceph-mon[28034]: pgmap v229: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T23:06:10.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:09 smithi080 ceph-mon[32324]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T23:06:10.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:09 smithi094 ceph-mon[32889]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T23:06:10.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:09 smithi027 ceph-mon[28034]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T23:06:11.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:10 smithi080 ceph-mon[32324]: pgmap v231: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T23:06:11.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:10 smithi094 ceph-mon[32889]: pgmap v231: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T23:06:11.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:10 smithi027 ceph-mon[28034]: pgmap v231: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T23:06:13.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:12 smithi080 ceph-mon[32324]: pgmap v232: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:06:13.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:12 smithi094 ceph-mon[32889]: pgmap v232: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:06:13.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:12 smithi027 ceph-mon[28034]: pgmap v232: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:06:15.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:14 smithi027 ceph-mon[28034]: pgmap v233: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:06:15.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:14 smithi080 ceph-mon[32324]: pgmap v233: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:06:15.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:14 smithi094 ceph-mon[32889]: pgmap v233: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:06:17.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:16 smithi080 ceph-mon[32324]: pgmap v234: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:06:17.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:16 smithi080 ceph-mon[32324]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T23:06:17.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:16 smithi094 ceph-mon[32889]: pgmap v234: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:06:17.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:16 smithi094 ceph-mon[32889]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T23:06:17.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:16 smithi027 ceph-mon[28034]: pgmap v234: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:06:17.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:16 smithi027 ceph-mon[28034]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T23:06:18.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:18 smithi080 ceph-mon[32324]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:06:18.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:18 smithi080 ceph-mon[32324]: pgmap v236: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2024-04-26T23:06:18.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:18 smithi094 ceph-mon[32889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:06:18.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:18 smithi094 ceph-mon[32889]: pgmap v236: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2024-04-26T23:06:18.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:18 smithi027 ceph-mon[28034]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:06:18.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:18 smithi027 ceph-mon[28034]: pgmap v236: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2024-04-26T23:06:21.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:20 smithi080 ceph-mon[32324]: pgmap v237: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:06:21.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:20 smithi094 ceph-mon[32889]: pgmap v237: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:06:21.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:20 smithi027 ceph-mon[28034]: pgmap v237: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:06:23.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:22 smithi080 ceph-mon[32324]: pgmap v238: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:06:23.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:22 smithi094 ceph-mon[32889]: pgmap v238: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:06:23.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:22 smithi027 ceph-mon[28034]: pgmap v238: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:06:25.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:24 smithi080 ceph-mon[32324]: pgmap v239: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:06:25.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:24 smithi094 ceph-mon[32889]: pgmap v239: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:06:25.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:24 smithi027 ceph-mon[28034]: pgmap v239: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:06:27.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:26 smithi080 ceph-mon[32324]: pgmap v240: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:06:27.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:26 smithi094 ceph-mon[32889]: pgmap v240: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:06:27.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:26 smithi027 ceph-mon[28034]: pgmap v240: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:06:29.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:28 smithi080 ceph-mon[32324]: pgmap v241: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 90 B/s wr, 8 op/s 2024-04-26T23:06:29.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:28 smithi094 ceph-mon[32889]: pgmap v241: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 90 B/s wr, 8 op/s 2024-04-26T23:06:29.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:28 smithi027 ceph-mon[28034]: pgmap v241: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 90 B/s wr, 8 op/s 2024-04-26T23:06:31.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:30 smithi080 ceph-mon[32324]: pgmap v242: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:31.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:30 smithi094 ceph-mon[32889]: pgmap v242: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:31.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:30 smithi027 ceph-mon[28034]: pgmap v242: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:32.273 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:31 smithi080 ceph-mon[32324]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:06:32.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:31 smithi094 ceph-mon[32889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:06:32.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:31 smithi027 ceph-mon[28034]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:06:33.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:32 smithi080 ceph-mon[32324]: pgmap v243: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:33.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:32 smithi094 ceph-mon[32889]: pgmap v243: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:33.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:32 smithi027 ceph-mon[28034]: pgmap v243: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:35.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:34 smithi080 ceph-mon[32324]: pgmap v244: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T23:06:35.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:34 smithi094 ceph-mon[32889]: pgmap v244: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T23:06:35.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:34 smithi027 ceph-mon[28034]: pgmap v244: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T23:06:37.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:36 smithi080 ceph-mon[32324]: pgmap v245: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T23:06:37.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:36 smithi094 ceph-mon[32889]: pgmap v245: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T23:06:37.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:36 smithi027 ceph-mon[28034]: pgmap v245: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T23:06:39.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:38 smithi080 ceph-mon[32324]: pgmap v246: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:39.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:38 smithi094 ceph-mon[32889]: pgmap v246: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:39.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:38 smithi027 ceph-mon[28034]: pgmap v246: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:41.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:40 smithi080 ceph-mon[32324]: pgmap v247: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T23:06:41.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:40 smithi094 ceph-mon[32889]: pgmap v247: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T23:06:41.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:40 smithi027 ceph-mon[28034]: pgmap v247: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-26T23:06:43.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:42 smithi080 ceph-mon[32324]: pgmap v248: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:43.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:42 smithi094 ceph-mon[32889]: pgmap v248: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:43.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:42 smithi027 ceph-mon[28034]: pgmap v248: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:45.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:44 smithi080 ceph-mon[32324]: pgmap v249: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:45.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:44 smithi094 ceph-mon[32889]: pgmap v249: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:45.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:44 smithi027 ceph-mon[28034]: pgmap v249: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:47.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:46 smithi080 ceph-mon[32324]: pgmap v250: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:47.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:46 smithi094 ceph-mon[32889]: pgmap v250: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:47.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:46 smithi027 ceph-mon[28034]: pgmap v250: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:49.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:48 smithi080 ceph-mon[32324]: pgmap v251: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:49.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:48 smithi094 ceph-mon[32889]: pgmap v251: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:49.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:48 smithi027 ceph-mon[28034]: pgmap v251: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:51.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:50 smithi080 ceph-mon[32324]: pgmap v252: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:51.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:50 smithi094 ceph-mon[32889]: pgmap v252: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:51.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:50 smithi027 ceph-mon[28034]: pgmap v252: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:53.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:52 smithi080 ceph-mon[32324]: pgmap v253: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:53.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:52 smithi094 ceph-mon[32889]: pgmap v253: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:53.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:52 smithi027 ceph-mon[28034]: pgmap v253: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:55.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:54 smithi080 ceph-mon[32324]: pgmap v254: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:55.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:54 smithi094 ceph-mon[32889]: pgmap v254: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:55.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:54 smithi027 ceph-mon[28034]: pgmap v254: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:57.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:56 smithi080 ceph-mon[32324]: pgmap v255: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:57.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:56 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:06:57.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:56 smithi094 ceph-mon[32889]: pgmap v255: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:57.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:56 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:06:57.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:56 smithi027 ceph-mon[28034]: pgmap v255: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:06:57.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:56 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:06:59.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:58 smithi080 ceph-mon[32324]: pgmap v256: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:59.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:58 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:06:59.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:58 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:06:59.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:06:58 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:06:59.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:58 smithi094 ceph-mon[32889]: pgmap v256: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:59.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:58 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:06:59.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:58 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:06:59.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:06:58 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:06:59.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:58 smithi027 ceph-mon[28034]: pgmap v256: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:06:59.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:58 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:06:59.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:58 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:06:59.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:06:58 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:07:01.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:00 smithi080 ceph-mon[32324]: pgmap v257: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:01.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:00 smithi094 ceph-mon[32889]: pgmap v257: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:01.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:00 smithi027 ceph-mon[28034]: pgmap v257: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:03.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:02 smithi080 ceph-mon[32324]: pgmap v258: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T23:07:03.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:02 smithi094 ceph-mon[32889]: pgmap v258: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T23:07:03.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:02 smithi027 ceph-mon[28034]: pgmap v258: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T23:07:05.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:04 smithi080 ceph-mon[32324]: pgmap v259: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T23:07:05.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:04 smithi094 ceph-mon[32889]: pgmap v259: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T23:07:05.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:04 smithi027 ceph-mon[28034]: pgmap v259: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T23:07:07.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:07 smithi080 ceph-mon[32324]: pgmap v260: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T23:07:07.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:07 smithi094 ceph-mon[32889]: pgmap v260: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T23:07:07.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:07 smithi027 ceph-mon[28034]: pgmap v260: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T23:07:09.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:09 smithi080 ceph-mon[32324]: pgmap v261: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T23:07:09.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:09 smithi094 ceph-mon[32889]: pgmap v261: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T23:07:09.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:09 smithi027 ceph-mon[28034]: pgmap v261: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T23:07:10.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:10 smithi094 ceph-mon[32889]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T23:07:10.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:10 smithi027 ceph-mon[28034]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T23:07:10.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:10 smithi080 ceph-mon[32324]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T23:07:11.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:11 smithi094 ceph-mon[32889]: pgmap v263: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T23:07:11.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:11 smithi027 ceph-mon[28034]: pgmap v263: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T23:07:11.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:11 smithi080 ceph-mon[32324]: pgmap v263: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T23:07:13.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:13 smithi027 ceph-mon[28034]: pgmap v264: 129 pgs: 19 active+clean+snaptrim, 48 active+clean, 62 active+clean+snaptrim_wait; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:07:13.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:13 smithi080 ceph-mon[32324]: pgmap v264: 129 pgs: 19 active+clean+snaptrim, 48 active+clean, 62 active+clean+snaptrim_wait; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:07:13.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:13 smithi094 ceph-mon[32889]: pgmap v264: 129 pgs: 19 active+clean+snaptrim, 48 active+clean, 62 active+clean+snaptrim_wait; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:07:15.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:15 smithi027 ceph-mon[28034]: pgmap v265: 129 pgs: 19 active+clean+snaptrim, 48 active+clean, 62 active+clean+snaptrim_wait; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:07:15.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:15 smithi080 ceph-mon[32324]: pgmap v265: 129 pgs: 19 active+clean+snaptrim, 48 active+clean, 62 active+clean+snaptrim_wait; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:07:15.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:15 smithi094 ceph-mon[32889]: pgmap v265: 129 pgs: 19 active+clean+snaptrim, 48 active+clean, 62 active+clean+snaptrim_wait; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:07:17.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:17 smithi027 ceph-mon[28034]: pgmap v266: 129 pgs: 19 active+clean+snaptrim, 48 active+clean, 62 active+clean+snaptrim_wait; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:07:17.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:17 smithi080 ceph-mon[32324]: pgmap v266: 129 pgs: 19 active+clean+snaptrim, 48 active+clean, 62 active+clean+snaptrim_wait; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:07:17.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:17 smithi094 ceph-mon[32889]: pgmap v266: 129 pgs: 19 active+clean+snaptrim, 48 active+clean, 62 active+clean+snaptrim_wait; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:07:19.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:19 smithi027 ceph-mon[28034]: pgmap v267: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:07:19.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:19 smithi080 ceph-mon[32324]: pgmap v267: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:07:19.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:19 smithi094 ceph-mon[32889]: pgmap v267: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:07:21.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:21 smithi027 ceph-mon[28034]: pgmap v268: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 189 B/s wr, 5 op/s 2024-04-26T23:07:21.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:21 smithi080 ceph-mon[32324]: pgmap v268: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 189 B/s wr, 5 op/s 2024-04-26T23:07:21.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:21 smithi094 ceph-mon[32889]: pgmap v268: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 189 B/s wr, 5 op/s 2024-04-26T23:07:22.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:22 smithi080 ceph-mon[32324]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T23:07:22.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:22 smithi080 ceph-mon[32324]: pgmap v270: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:07:22.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:22 smithi094 ceph-mon[32889]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T23:07:22.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:22 smithi094 ceph-mon[32889]: pgmap v270: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:07:22.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:22 smithi027 ceph-mon[28034]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T23:07:22.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:22 smithi027 ceph-mon[28034]: pgmap v270: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:07:25.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:24 smithi080 ceph-mon[32324]: pgmap v271: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:07:25.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:24 smithi094 ceph-mon[32889]: pgmap v271: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:07:25.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:24 smithi027 ceph-mon[28034]: pgmap v271: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:07:27.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:26 smithi080 ceph-mon[32324]: pgmap v272: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:07:27.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:26 smithi094 ceph-mon[32889]: pgmap v272: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:07:27.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:26 smithi027 ceph-mon[28034]: pgmap v272: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:07:29.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:28 smithi080 ceph-mon[32324]: pgmap v273: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:07:29.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:28 smithi094 ceph-mon[32889]: pgmap v273: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:07:29.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:28 smithi027 ceph-mon[28034]: pgmap v273: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:07:31.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:30 smithi080 ceph-mon[32324]: pgmap v274: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:07:31.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:30 smithi094 ceph-mon[32889]: pgmap v274: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:07:31.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:30 smithi027 ceph-mon[28034]: pgmap v274: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:07:33.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:32 smithi080 ceph-mon[32324]: pgmap v275: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T23:07:33.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:32 smithi094 ceph-mon[32889]: pgmap v275: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T23:07:33.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:32 smithi027 ceph-mon[28034]: pgmap v275: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T23:07:35.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:34 smithi080 ceph-mon[32324]: pgmap v276: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:35.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:34 smithi094 ceph-mon[32889]: pgmap v276: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:35.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:34 smithi027 ceph-mon[28034]: pgmap v276: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:37.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:36 smithi080 ceph-mon[32324]: pgmap v277: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:37.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:36 smithi094 ceph-mon[32889]: pgmap v277: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:37.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:36 smithi027 ceph-mon[28034]: pgmap v277: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:39.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:38 smithi080 ceph-mon[32324]: pgmap v278: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:07:39.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:38 smithi094 ceph-mon[32889]: pgmap v278: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:07:39.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:38 smithi027 ceph-mon[28034]: pgmap v278: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:07:41.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:40 smithi080 ceph-mon[32324]: pgmap v279: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:41.305 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:40 smithi094 ceph-mon[32889]: pgmap v279: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:41.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:40 smithi027 ceph-mon[28034]: pgmap v279: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:43.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:42 smithi080 ceph-mon[32324]: pgmap v280: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:07:43.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:42 smithi094 ceph-mon[32889]: pgmap v280: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:07:43.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:42 smithi027 ceph-mon[28034]: pgmap v280: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:07:45.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:44 smithi080 ceph-mon[32324]: pgmap v281: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:45.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:44 smithi094 ceph-mon[32889]: pgmap v281: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:45.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:44 smithi027 ceph-mon[28034]: pgmap v281: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:47.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:46 smithi080 ceph-mon[32324]: pgmap v282: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:47.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:46 smithi094 ceph-mon[32889]: pgmap v282: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:47.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:46 smithi027 ceph-mon[28034]: pgmap v282: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:49.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:48 smithi080 ceph-mon[32324]: pgmap v283: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:07:49.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:48 smithi094 ceph-mon[32889]: pgmap v283: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:07:49.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:48 smithi027 ceph-mon[28034]: pgmap v283: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:07:51.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:50 smithi080 ceph-mon[32324]: pgmap v284: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:51.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:50 smithi094 ceph-mon[32889]: pgmap v284: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:51.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:50 smithi027 ceph-mon[28034]: pgmap v284: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:53.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:52 smithi080 ceph-mon[32324]: pgmap v285: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:07:53.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:52 smithi094 ceph-mon[32889]: pgmap v285: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:07:53.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:52 smithi027 ceph-mon[28034]: pgmap v285: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:07:55.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:54 smithi080 ceph-mon[32324]: pgmap v286: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:55.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:54 smithi094 ceph-mon[32889]: pgmap v286: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:55.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:54 smithi027 ceph-mon[28034]: pgmap v286: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:57.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:56 smithi080 ceph-mon[32324]: pgmap v287: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:57.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:56 smithi094 ceph-mon[32889]: pgmap v287: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:57.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:56 smithi027 ceph-mon[28034]: pgmap v287: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:07:59.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:58 smithi080 ceph-mon[32324]: pgmap v288: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:07:59.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:58 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:07:59.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:58 smithi094 ceph-mon[32889]: pgmap v288: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:07:59.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:58 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:07:59.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:58 smithi027 ceph-mon[28034]: pgmap v288: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:07:59.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:58 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:08:00.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:59 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:08:00.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:59 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:08:00.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:07:59 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:08:00.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:59 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:08:00.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:59 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:08:00.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:07:59 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:08:00.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:59 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:08:00.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:59 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:08:00.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:07:59 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:08:01.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:00 smithi080 ceph-mon[32324]: pgmap v289: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:08:01.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:00 smithi094 ceph-mon[32889]: pgmap v289: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:08:01.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:00 smithi027 ceph-mon[28034]: pgmap v289: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:08:03.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:03 smithi080 ceph-mon[32324]: pgmap v290: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T23:08:03.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:03 smithi094 ceph-mon[32889]: pgmap v290: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T23:08:03.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:03 smithi027 ceph-mon[28034]: pgmap v290: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T23:08:05.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:05 smithi080 ceph-mon[32324]: pgmap v291: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T23:08:05.287 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:05 smithi094 ceph-mon[32889]: pgmap v291: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T23:08:05.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:05 smithi027 ceph-mon[28034]: pgmap v291: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T23:08:07.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:07 smithi080 ceph-mon[32324]: pgmap v292: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T23:08:07.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:07 smithi094 ceph-mon[32889]: pgmap v292: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T23:08:07.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:07 smithi027 ceph-mon[28034]: pgmap v292: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T23:08:09.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:09 smithi080 ceph-mon[32324]: pgmap v293: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T23:08:09.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:09 smithi094 ceph-mon[32889]: pgmap v293: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T23:08:09.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:09 smithi027 ceph-mon[28034]: pgmap v293: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T23:08:10.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:10 smithi027 ceph-mon[28034]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T23:08:10.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:10 smithi080 ceph-mon[32324]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T23:08:10.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:10 smithi094 ceph-mon[32889]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T23:08:11.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:11 smithi027 ceph-mon[28034]: pgmap v295: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-26T23:08:11.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:11 smithi080 ceph-mon[32324]: pgmap v295: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-26T23:08:11.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:11 smithi094 ceph-mon[32889]: pgmap v295: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-26T23:08:13.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:13 smithi027 ceph-mon[28034]: pgmap v296: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:08:13.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:13 smithi080 ceph-mon[32324]: pgmap v296: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:08:13.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:13 smithi094 ceph-mon[32889]: pgmap v296: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:08:15.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:15 smithi027 ceph-mon[28034]: pgmap v297: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:08:15.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:15 smithi080 ceph-mon[32324]: pgmap v297: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:08:15.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:15 smithi094 ceph-mon[32889]: pgmap v297: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:08:17.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:17 smithi027 ceph-mon[28034]: pgmap v298: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:08:17.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:17 smithi080 ceph-mon[32324]: pgmap v298: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:08:17.536 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:17 smithi094 ceph-mon[32889]: pgmap v298: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:08:19.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:19 smithi027 ceph-mon[28034]: pgmap v299: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 457 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:08:19.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:19 smithi080 ceph-mon[32324]: pgmap v299: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 457 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:08:19.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:19 smithi094 ceph-mon[32889]: pgmap v299: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 457 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:08:21.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:21 smithi027 ceph-mon[28034]: pgmap v300: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 457 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 188 B/s wr, 5 op/s 2024-04-26T23:08:21.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:21 smithi080 ceph-mon[32324]: pgmap v300: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 457 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 188 B/s wr, 5 op/s 2024-04-26T23:08:21.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:21 smithi094 ceph-mon[32889]: pgmap v300: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 457 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 188 B/s wr, 5 op/s 2024-04-26T23:08:23.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:23 smithi027 ceph-mon[28034]: pgmap v301: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T23:08:23.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:23 smithi080 ceph-mon[32324]: pgmap v301: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T23:08:23.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:23 smithi094 ceph-mon[32889]: pgmap v301: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-26T23:08:25.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:25 smithi027 ceph-mon[28034]: pgmap v302: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:08:25.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:25 smithi080 ceph-mon[32324]: pgmap v302: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:08:25.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:25 smithi094 ceph-mon[32889]: pgmap v302: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:08:27.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:27 smithi027 ceph-mon[28034]: pgmap v303: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:08:27.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:27 smithi027 ceph-mon[28034]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T23:08:27.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:27 smithi080 ceph-mon[32324]: pgmap v303: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:08:27.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:27 smithi080 ceph-mon[32324]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T23:08:27.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:27 smithi094 ceph-mon[32889]: pgmap v303: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:08:27.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:27 smithi094 ceph-mon[32889]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T23:08:29.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:29 smithi027 ceph-mon[28034]: pgmap v305: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:08:29.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:29 smithi080 ceph-mon[32324]: pgmap v305: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:08:29.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:29 smithi094 ceph-mon[32889]: pgmap v305: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:08:31.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:31 smithi027 ceph-mon[28034]: pgmap v306: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:08:31.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:31 smithi080 ceph-mon[32324]: pgmap v306: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:08:31.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:31 smithi094 ceph-mon[32889]: pgmap v306: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:08:33.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:33 smithi027 ceph-mon[28034]: pgmap v307: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:08:33.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:33 smithi080 ceph-mon[32324]: pgmap v307: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:08:33.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:33 smithi094 ceph-mon[32889]: pgmap v307: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:08:35.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:35 smithi027 ceph-mon[28034]: pgmap v308: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:08:35.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:35 smithi080 ceph-mon[32324]: pgmap v308: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:08:35.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:35 smithi094 ceph-mon[32889]: pgmap v308: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:08:37.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:37 smithi080 ceph-mon[32324]: pgmap v309: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:08:37.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:37 smithi094 ceph-mon[32889]: pgmap v309: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:08:37.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:37 smithi027 ceph-mon[28034]: pgmap v309: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:08:38.113 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T23:08:38.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.333+0000 7f368b200640 1 -- 172.21.15.27:0/3650530398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f368c1030e0 msgr2=0x7f368c10f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:38.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.333+0000 7f368b200640 1 --2- 172.21.15.27:0/3650530398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f368c1030e0 0x7f368c10f5f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3684009c10 tx=0x7f368402a6f0 comp rx=0 tx=0).stop 2024-04-26T23:08:38.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.333+0000 7f368b200640 1 -- 172.21.15.27:0/3650530398 shutdown_connections 2024-04-26T23:08:38.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.333+0000 7f368b200640 1 --2- 172.21.15.27:0/3650530398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f368c1030e0 0x7f368c10f5f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:38.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.333+0000 7f368b200640 1 --2- 172.21.15.27:0/3650530398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c102720 0x7f368c102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:38.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.333+0000 7f368b200640 1 --2- 172.21.15.27:0/3650530398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c108730 0x7f368c108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:38.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.333+0000 7f368b200640 1 -- 172.21.15.27:0/3650530398 >> 172.21.15.27:0/3650530398 conn(0x7f368c0fe270 msgr2=0x7f368c100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:08:38.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.333+0000 7f368b200640 1 -- 172.21.15.27:0/3650530398 shutdown_connections 2024-04-26T23:08:38.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.333+0000 7f368b200640 1 -- 172.21.15.27:0/3650530398 wait complete. 2024-04-26T23:08:38.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.334+0000 7f368b200640 1 Processor -- start 2024-04-26T23:08:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.335+0000 7f368b200640 1 -- start start 2024-04-26T23:08:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.335+0000 7f368b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c102720 0x7f368c074c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.335+0000 7f368b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c1030e0 0x7f368c075170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.335+0000 7f368b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f368c108730 0x7f368c0756b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.335+0000 7f368b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f368c1129f0 con 0x7f368c1030e0 2024-04-26T23:08:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.335+0000 7f368b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f368c112870 con 0x7f368c102720 2024-04-26T23:08:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.335+0000 7f368b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f368c112b70 con 0x7f368c108730 2024-04-26T23:08:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.335+0000 7f3689400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c1030e0 0x7f368c075170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.335+0000 7f3689400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c1030e0 0x7f368c075170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38756/0 (socket says 172.21.15.27:38756) 2024-04-26T23:08:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.335+0000 7f3689400640 1 -- 172.21.15.27:0/2640273274 learned_addr learned my addr 172.21.15.27:0/2640273274 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:08:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.335+0000 7f3689e00640 1 --2- 172.21.15.27:0/2640273274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c102720 0x7f368c074c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.335+0000 7f368a800640 1 --2- 172.21.15.27:0/2640273274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f368c108730 0x7f368c0756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.336+0000 7f3689400640 1 -- 172.21.15.27:0/2640273274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f368c108730 msgr2=0x7f368c0756b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.336+0000 7f3689400640 1 --2- 172.21.15.27:0/2640273274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f368c108730 0x7f368c0756b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.336+0000 7f3689400640 1 -- 172.21.15.27:0/2640273274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c102720 msgr2=0x7f368c074c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.336+0000 7f3689400640 1 --2- 172.21.15.27:0/2640273274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c102720 0x7f368c074c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.336+0000 7f3689400640 1 -- 172.21.15.27:0/2640273274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c1a7450 con 0x7f368c1030e0 2024-04-26T23:08:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.336+0000 7f3689400640 1 --2- 172.21.15.27:0/2640273274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c1030e0 0x7f368c075170 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f367800e9c0 tx=0x7f367800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:08:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.336+0000 7f3682a00640 1 -- 172.21.15.27:0/2640273274 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367800cde0 con 0x7f368c1030e0 2024-04-26T23:08:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.336+0000 7f3682a00640 1 -- 172.21.15.27:0/2640273274 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36780035f0 con 0x7f368c1030e0 2024-04-26T23:08:38.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.336+0000 7f3682a00640 1 -- 172.21.15.27:0/2640273274 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3678010690 con 0x7f368c1030e0 2024-04-26T23:08:38.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.336+0000 7f368b200640 1 -- 172.21.15.27:0/2640273274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368c1a7740 con 0x7f368c1030e0 2024-04-26T23:08:38.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.337+0000 7f368b200640 1 -- 172.21.15.27:0/2640273274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368c1a7c90 con 0x7f368c1030e0 2024-04-26T23:08:38.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.338+0000 7f368b200640 1 -- 172.21.15.27:0/2640273274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f368c076160 con 0x7f368c1030e0 2024-04-26T23:08:38.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.341+0000 7f3682a00640 1 -- 172.21.15.27:0/2640273274 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3678004000 con 0x7f368c1030e0 2024-04-26T23:08:38.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.342+0000 7f3682a00640 1 --2- 172.21.15.27:0/2640273274 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f36700a3d00 0x7f36700a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:38.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.342+0000 7f3682a00640 1 -- 172.21.15.27:0/2640273274 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9700+0+0 (secure 0 0 0) 0x7f36780d0fe0 con 0x7f368c1030e0 2024-04-26T23:08:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.342+0000 7f3689e00640 1 --2- 172.21.15.27:0/2640273274 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f36700a3d00 0x7f36700a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.343+0000 7f3689e00640 1 --2- 172.21.15.27:0/2640273274 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f36700a3d00 0x7f36700a61c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f367c009b50 tx=0x7f367c005f90 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:08:38.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.347+0000 7f3682a00640 1 -- 172.21.15.27:0/2640273274 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3678009510 con 0x7f368c1030e0 2024-04-26T23:08:38.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.583+0000 7f368b200640 1 -- 172.21.15.27:0/2640273274 --> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f368c10ad00 con 0x7f36700a3d00 2024-04-26T23:08:38.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.596+0000 7f3682a00640 1 -- 172.21.15.27:0/2640273274 <== mgr.25391 v2:172.21.15.27:6800/3058395889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f367c004030 con 0x7f36700a3d00 2024-04-26T23:08:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T22:58:36", "first": "2024-04-26T22:59:00", "last": "2024-04-26T23:08:00", "last_pruned": "2024-04-26T23:08:00", "created_count": 10, "pruned_count": 3, "active": true} 2024-04-26T23:08:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.599+0000 7f368b200640 1 -- 172.21.15.27:0/2640273274 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f36700a3d00 msgr2=0x7f36700a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.599+0000 7f368b200640 1 --2- 172.21.15.27:0/2640273274 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f36700a3d00 0x7f36700a61c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f367c009b50 tx=0x7f367c005f90 comp rx=0 tx=0).stop 2024-04-26T23:08:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.599+0000 7f368b200640 1 -- 172.21.15.27:0/2640273274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c1030e0 msgr2=0x7f368c075170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.600+0000 7f368b200640 1 --2- 172.21.15.27:0/2640273274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c1030e0 0x7f368c075170 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f367800e9c0 tx=0x7f367800ee80 comp rx=0 tx=0).stop 2024-04-26T23:08:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.600+0000 7f368b200640 1 -- 172.21.15.27:0/2640273274 shutdown_connections 2024-04-26T23:08:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.600+0000 7f368b200640 1 --2- 172.21.15.27:0/2640273274 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f36700a3d00 0x7f36700a61c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.600+0000 7f368b200640 1 --2- 172.21.15.27:0/2640273274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f368c108730 0x7f368c0756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.600+0000 7f368b200640 1 --2- 172.21.15.27:0/2640273274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c1030e0 0x7f368c075170 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.600+0000 7f368b200640 1 --2- 172.21.15.27:0/2640273274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c102720 0x7f368c074c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.600+0000 7f368b200640 1 -- 172.21.15.27:0/2640273274 >> 172.21.15.27:0/2640273274 conn(0x7f368c0fe270 msgr2=0x7f368c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:08:38.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.600+0000 7f368b200640 1 -- 172.21.15.27:0/2640273274 shutdown_connections 2024-04-26T23:08:38.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.600+0000 7f368b200640 1 -- 172.21.15.27:0/2640273274 wait complete. 2024-04-26T23:08:38.619 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T23:08:38.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.833+0000 7f926da00640 1 -- 172.21.15.27:0/598233902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9268104960 msgr2=0x7f9268104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:38.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.833+0000 7f926da00640 1 --2- 172.21.15.27:0/598233902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9268104960 0x7f9268104de0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9254009930 tx=0x7f925402a440 comp rx=0 tx=0).stop 2024-04-26T23:08:38.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.833+0000 7f926da00640 1 -- 172.21.15.27:0/598233902 shutdown_connections 2024-04-26T23:08:38.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.833+0000 7f926da00640 1 --2- 172.21.15.27:0/598233902 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9268105320 0x7f9268111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:38.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.833+0000 7f926da00640 1 --2- 172.21.15.27:0/598233902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9268104960 0x7f9268104de0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:38.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.833+0000 7f926da00640 1 --2- 172.21.15.27:0/598233902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f926810a970 0x7f926810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:38.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.833+0000 7f926da00640 1 -- 172.21.15.27:0/598233902 >> 172.21.15.27:0/598233902 conn(0x7f9268100410 msgr2=0x7f9268102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:08:38.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.833+0000 7f926da00640 1 -- 172.21.15.27:0/598233902 shutdown_connections 2024-04-26T23:08:38.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.833+0000 7f926da00640 1 -- 172.21.15.27:0/598233902 wait complete. 2024-04-26T23:08:38.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.834+0000 7f926da00640 1 Processor -- start 2024-04-26T23:08:38.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.835+0000 7f926da00640 1 -- start start 2024-04-26T23:08:38.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.835+0000 7f926da00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9268104960 0x7f926819b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:38.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.835+0000 7f926da00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9268105320 0x7f926819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.835+0000 7f926da00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f926810a970 0x7f926819fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.835+0000 7f926da00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9268114c30 con 0x7f9268104960 2024-04-26T23:08:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.835+0000 7f926da00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9268114ab0 con 0x7f9268105320 2024-04-26T23:08:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.835+0000 7f926da00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9268114db0 con 0x7f926810a970 2024-04-26T23:08:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.835+0000 7f9267e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9268104960 0x7f926819b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.835+0000 7f9267e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9268104960 0x7f926819b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38786/0 (socket says 172.21.15.27:38786) 2024-04-26T23:08:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.835+0000 7f9267e00640 1 -- 172.21.15.27:0/4250868035 learned_addr learned my addr 172.21.15.27:0/4250868035 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:08:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.835+0000 7f9267400640 1 --2- 172.21.15.27:0/4250868035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9268105320 0x7f926819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.835+0000 7f926d000640 1 --2- 172.21.15.27:0/4250868035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f926810a970 0x7f926819fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.836+0000 7f9267e00640 1 -- 172.21.15.27:0/4250868035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f926810a970 msgr2=0x7f926819fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.836+0000 7f9267e00640 1 --2- 172.21.15.27:0/4250868035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f926810a970 0x7f926819fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.836+0000 7f9267e00640 1 -- 172.21.15.27:0/4250868035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9268105320 msgr2=0x7f926819b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.836+0000 7f9267e00640 1 --2- 172.21.15.27:0/4250868035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9268105320 0x7f926819b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.836+0000 7f9267e00640 1 -- 172.21.15.27:0/4250868035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92681a0200 con 0x7f9268104960 2024-04-26T23:08:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.836+0000 7f9267400640 1 --2- 172.21.15.27:0/4250868035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9268105320 0x7f926819b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:08:38.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.836+0000 7f9267e00640 1 --2- 172.21.15.27:0/4250868035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9268104960 0x7f926819b210 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f925800e9f0 tx=0x7f925800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:08:38.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.836+0000 7f9264c00640 1 -- 172.21.15.27:0/4250868035 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925800ccc0 con 0x7f9268104960 2024-04-26T23:08:38.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.837+0000 7f926da00640 1 -- 172.21.15.27:0/4250868035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92681a04f0 con 0x7f9268104960 2024-04-26T23:08:38.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.837+0000 7f9264c00640 1 -- 172.21.15.27:0/4250868035 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f925800ce60 con 0x7f9268104960 2024-04-26T23:08:38.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.837+0000 7f926da00640 1 -- 172.21.15.27:0/4250868035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92681a7d80 con 0x7f9268104960 2024-04-26T23:08:38.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.837+0000 7f9264c00640 1 -- 172.21.15.27:0/4250868035 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258018430 con 0x7f9268104960 2024-04-26T23:08:38.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.838+0000 7f926da00640 1 -- 172.21.15.27:0/4250868035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92680691d0 con 0x7f9268104960 2024-04-26T23:08:38.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.840+0000 7f9264c00640 1 -- 172.21.15.27:0/4250868035 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f92580046c0 con 0x7f9268104960 2024-04-26T23:08:38.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.842+0000 7f9264c00640 1 --2- 172.21.15.27:0/4250868035 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f924c0a3d00 0x7f924c0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:38.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.843+0000 7f9264c00640 1 -- 172.21.15.27:0/4250868035 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9700+0+0 (secure 0 0 0) 0x7f92580d1170 con 0x7f9268104960 2024-04-26T23:08:38.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.843+0000 7f9267400640 1 --2- 172.21.15.27:0/4250868035 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f924c0a3d00 0x7f924c0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:38.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.844+0000 7f9267400640 1 --2- 172.21.15.27:0/4250868035 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f924c0a3d00 0x7f924c0a61c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9254009900 tx=0x7f9254000bc0 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:08:38.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:38.847+0000 7f9264c00640 1 -- 172.21.15.27:0/4250868035 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f925809d310 con 0x7f9268104960 2024-04-26T23:08:39.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.078+0000 7f926da00640 1 -- 172.21.15.27:0/4250868035 --> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f9268062d70 con 0x7f924c0a3d00 2024-04-26T23:08:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.094+0000 7f9264c00640 1 -- 172.21.15.27:0/4250868035 <== mgr.25391 v2:172.21.15.27:6800/3058395889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f9268062d70 con 0x7f924c0a3d00 2024-04-26T23:08:39.095 INFO:teuthology.orchestra.run.smithi027.stdout:/ 1m 6m3h 2024-04-26T23:08:39.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.097+0000 7f926da00640 1 -- 172.21.15.27:0/4250868035 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f924c0a3d00 msgr2=0x7f924c0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:39.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.097+0000 7f926da00640 1 --2- 172.21.15.27:0/4250868035 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f924c0a3d00 0x7f924c0a61c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9254009900 tx=0x7f9254000bc0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.097+0000 7f926da00640 1 -- 172.21.15.27:0/4250868035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9268104960 msgr2=0x7f926819b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:39.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.097+0000 7f926da00640 1 --2- 172.21.15.27:0/4250868035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9268104960 0x7f926819b210 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f925800e9f0 tx=0x7f925800edf0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.097+0000 7f926da00640 1 -- 172.21.15.27:0/4250868035 shutdown_connections 2024-04-26T23:08:39.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.097+0000 7f926da00640 1 --2- 172.21.15.27:0/4250868035 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f924c0a3d00 0x7f924c0a61c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.097+0000 7f926da00640 1 --2- 172.21.15.27:0/4250868035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9268105320 0x7f926819b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.097+0000 7f926da00640 1 --2- 172.21.15.27:0/4250868035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f926810a970 0x7f926819fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.097+0000 7f926da00640 1 --2- 172.21.15.27:0/4250868035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9268104960 0x7f926819b210 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.097+0000 7f926da00640 1 -- 172.21.15.27:0/4250868035 >> 172.21.15.27:0/4250868035 conn(0x7f9268100410 msgr2=0x7f9268100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:08:39.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.098+0000 7f926da00640 1 -- 172.21.15.27:0/4250868035 shutdown_connections 2024-04-26T23:08:39.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.098+0000 7f926da00640 1 -- 172.21.15.27:0/4250868035 wait complete. 2024-04-26T23:08:39.119 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-26T23:08:39.132 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:08:39.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.352+0000 7f828d600640 1 -- 172.21.15.27:0/2846780937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8288101490 msgr2=0x7f8288101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:39.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.352+0000 7f828d600640 1 --2- 172.21.15.27:0/2846780937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8288101490 0x7f8288101870 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8278009900 tx=0x7f827802a410 comp rx=0 tx=0).stop 2024-04-26T23:08:39.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.352+0000 7f828d600640 1 -- 172.21.15.27:0/2846780937 shutdown_connections 2024-04-26T23:08:39.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.352+0000 7f828d600640 1 --2- 172.21.15.27:0/2846780937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8288101e40 0x7f828810eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.352+0000 7f828d600640 1 --2- 172.21.15.27:0/2846780937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f828810f040 0x7f8288111430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.352+0000 7f828d600640 1 --2- 172.21.15.27:0/2846780937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8288101490 0x7f8288101870 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.352+0000 7f828d600640 1 -- 172.21.15.27:0/2846780937 >> 172.21.15.27:0/2846780937 conn(0x7f82880783e0 msgr2=0x7f82880ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:08:39.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.352+0000 7f828d600640 1 -- 172.21.15.27:0/2846780937 shutdown_connections 2024-04-26T23:08:39.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.352+0000 7f828d600640 1 -- 172.21.15.27:0/2846780937 wait complete. 2024-04-26T23:08:39.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.353+0000 7f828d600640 1 Processor -- start 2024-04-26T23:08:39.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.354+0000 7f828d600640 1 -- start start 2024-04-26T23:08:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.354+0000 7f828d600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8288101490 0x7f828819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.354+0000 7f828d600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8288101e40 0x7f828819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.354+0000 7f828d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f828810f040 0x7f828819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.354+0000 7f828d600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8288114c00 con 0x7f8288101490 2024-04-26T23:08:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.354+0000 7f828d600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8288114a80 con 0x7f8288101e40 2024-04-26T23:08:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.354+0000 7f828d600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8288114d80 con 0x7f828810f040 2024-04-26T23:08:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.354+0000 7f8287e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8288101490 0x7f828819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.354+0000 7f8287e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8288101490 0x7f828819afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38816/0 (socket says 172.21.15.27:38816) 2024-04-26T23:08:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.354+0000 7f8287e00640 1 -- 172.21.15.27:0/3836444660 learned_addr learned my addr 172.21.15.27:0/3836444660 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:08:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.354+0000 7f828cc00640 1 --2- 172.21.15.27:0/3836444660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f828810f040 0x7f828819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.354+0000 7f8287400640 1 --2- 172.21.15.27:0/3836444660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8288101e40 0x7f828819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.355+0000 7f8287e00640 1 -- 172.21.15.27:0/3836444660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f828810f040 msgr2=0x7f828819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.355+0000 7f8287e00640 1 --2- 172.21.15.27:0/3836444660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f828810f040 0x7f828819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.355+0000 7f8287e00640 1 -- 172.21.15.27:0/3836444660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8288101e40 msgr2=0x7f828819b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.355+0000 7f8287e00640 1 --2- 172.21.15.27:0/3836444660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8288101e40 0x7f828819b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.355+0000 7f8287e00640 1 -- 172.21.15.27:0/3836444660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828819ff90 con 0x7f8288101490 2024-04-26T23:08:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.355+0000 7f8287e00640 1 --2- 172.21.15.27:0/3836444660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8288101490 0x7f828819afa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8278005ab0 tx=0x7f8278003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:08:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.355+0000 7f8284c00640 1 -- 172.21.15.27:0/3836444660 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827803f070 con 0x7f8288101490 2024-04-26T23:08:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.355+0000 7f8284c00640 1 -- 172.21.15.27:0/3836444660 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8278031b60 con 0x7f8288101490 2024-04-26T23:08:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.355+0000 7f828d600640 1 -- 172.21.15.27:0/3836444660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82881a0220 con 0x7f8288101490 2024-04-26T23:08:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.356+0000 7f8284c00640 1 -- 172.21.15.27:0/3836444660 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278035030 con 0x7f8288101490 2024-04-26T23:08:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.356+0000 7f828d600640 1 -- 172.21.15.27:0/3836444660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82881a7b10 con 0x7f8288101490 2024-04-26T23:08:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.357+0000 7f828d600640 1 -- 172.21.15.27:0/3836444660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82880506c0 con 0x7f8288101490 2024-04-26T23:08:39.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.359+0000 7f8284c00640 1 -- 172.21.15.27:0/3836444660 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8278042070 con 0x7f8288101490 2024-04-26T23:08:39.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.361+0000 7f8284c00640 1 --2- 172.21.15.27:0/3836444660 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f826c0a3e40 0x7f826c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:39.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.361+0000 7f8284c00640 1 -- 172.21.15.27:0/3836444660 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9700+0+0 (secure 0 0 0) 0x7f82780f12e0 con 0x7f8288101490 2024-04-26T23:08:39.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.361+0000 7f8287400640 1 --2- 172.21.15.27:0/3836444660 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f826c0a3e40 0x7f826c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:39.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.362+0000 7f8287400640 1 --2- 172.21.15.27:0/3836444660 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f826c0a3e40 0x7f826c0a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f82740095f0 tx=0x7f8274005f90 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:08:39.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.365+0000 7f8284c00640 1 -- 172.21.15.27:0/3836444660 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f82780bce90 con 0x7f8288101490 2024-04-26T23:08:39.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:39 smithi080 ceph-mon[32324]: pgmap v310: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-26T23:08:39.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:39 smithi094 ceph-mon[32889]: pgmap v310: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-26T23:08:39.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.574+0000 7f828d600640 1 -- 172.21.15.27:0/3836444660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82881a7d10 con 0x7f8288101490 2024-04-26T23:08:39.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.578+0000 7f8284c00640 1 -- 172.21.15.27:0/3836444660 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+25528 (secure 0 0 0) 0x7f82780c1190 con 0x7f8288101490 2024-04-26T23:08:39.579 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T23:08:39.579 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":101,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T23:08:26.530553+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":14,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":14,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4},{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/4202829010":"2024-04-27T22:58:35.543340+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3746496955":"2024-04-27T22:58:35.543340+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/1199904816":"2024-04-27T22:58:35.543340+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3276968774":"2024-04-27T22:58:35.543340+0000","172.21.15.27:0/3449139270":"2024-04-27T22:58:35.543340+0000","172.21.15.27:6801/3636526714":"2024-04-27T22:58:35.543340+0000","172.21.15.27:6800/3636526714":"2024-04-27T22:58:35.543340+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":14,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":14,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:08:39.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.581+0000 7f828d600640 1 -- 172.21.15.27:0/3836444660 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f826c0a3e40 msgr2=0x7f826c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:39.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.581+0000 7f828d600640 1 --2- 172.21.15.27:0/3836444660 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f826c0a3e40 0x7f826c0a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f82740095f0 tx=0x7f8274005f90 comp rx=0 tx=0).stop 2024-04-26T23:08:39.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.581+0000 7f828d600640 1 -- 172.21.15.27:0/3836444660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8288101490 msgr2=0x7f828819afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:39.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.581+0000 7f828d600640 1 --2- 172.21.15.27:0/3836444660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8288101490 0x7f828819afa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8278005ab0 tx=0x7f8278003c10 comp rx=0 tx=0).stop 2024-04-26T23:08:39.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.581+0000 7f828d600640 1 -- 172.21.15.27:0/3836444660 shutdown_connections 2024-04-26T23:08:39.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.581+0000 7f828d600640 1 --2- 172.21.15.27:0/3836444660 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f826c0a3e40 0x7f826c0a6300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.581+0000 7f828d600640 1 --2- 172.21.15.27:0/3836444660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8288101e40 0x7f828819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.581+0000 7f828d600640 1 --2- 172.21.15.27:0/3836444660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f828810f040 0x7f828819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.581+0000 7f828d600640 1 --2- 172.21.15.27:0/3836444660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8288101490 0x7f828819afa0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.581+0000 7f828d600640 1 -- 172.21.15.27:0/3836444660 >> 172.21.15.27:0/3836444660 conn(0x7f82880783e0 msgr2=0x7f8288111220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:08:39.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.581+0000 7f828d600640 1 -- 172.21.15.27:0/3836444660 shutdown_connections 2024-04-26T23:08:39.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.581+0000 7f828d600640 1 -- 172.21.15.27:0/3836444660 wait complete. 2024-04-26T23:08:39.598 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:39 smithi027 ceph-mon[28034]: pgmap v310: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-26T23:08:39.602 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:49:54.482679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:55:22.899174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:55:24.218295+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T22:55:28.394547+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:08:39.603 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:08:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.820+0000 7f0e69e00640 1 -- 172.21.15.27:0/2452192066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e6410edb0 msgr2=0x7f0e6410f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.820+0000 7f0e69e00640 1 --2- 172.21.15.27:0/2452192066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e6410edb0 0x7f0e6410f190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009930 tx=0x7f0e5802a440 comp rx=0 tx=0).stop 2024-04-26T23:08:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.821+0000 7f0e69e00640 1 -- 172.21.15.27:0/2452192066 shutdown_connections 2024-04-26T23:08:39.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.821+0000 7f0e69e00640 1 --2- 172.21.15.27:0/2452192066 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e64108e70 0x7f0e641092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.821+0000 7f0e69e00640 1 --2- 172.21.15.27:0/2452192066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e64109810 0x7f0e64115be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.821+0000 7f0e69e00640 1 --2- 172.21.15.27:0/2452192066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e6410edb0 0x7f0e6410f190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.821+0000 7f0e69e00640 1 -- 172.21.15.27:0/2452192066 >> 172.21.15.27:0/2452192066 conn(0x7f0e64078740 msgr2=0x7f0e64107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:08:39.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.821+0000 7f0e69e00640 1 -- 172.21.15.27:0/2452192066 shutdown_connections 2024-04-26T23:08:39.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.821+0000 7f0e69e00640 1 -- 172.21.15.27:0/2452192066 wait complete. 2024-04-26T23:08:39.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.822+0000 7f0e69e00640 1 Processor -- start 2024-04-26T23:08:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.822+0000 7f0e69e00640 1 -- start start 2024-04-26T23:08:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.823+0000 7f0e69e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e64108e70 0x7f0e6419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.823+0000 7f0e69e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e64109810 0x7f0e6419fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.823+0000 7f0e69e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e6410edb0 0x7f0e641a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.823+0000 7f0e69e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0e64118fe0 con 0x7f0e6410edb0 2024-04-26T23:08:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.823+0000 7f0e69e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0e64118e60 con 0x7f0e64108e70 2024-04-26T23:08:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.823+0000 7f0e69e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0e64119160 con 0x7f0e64109810 2024-04-26T23:08:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.823+0000 7f0e68a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e64108e70 0x7f0e6419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:39.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.823+0000 7f0e69400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e6410edb0 0x7f0e641a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:39.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.823+0000 7f0e63e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e64109810 0x7f0e6419fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:39.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.823+0000 7f0e63e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e64109810 0x7f0e6419fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:38006/0 (socket says 172.21.15.27:38006) 2024-04-26T23:08:39.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.823+0000 7f0e63e00640 1 -- 172.21.15.27:0/3966870918 learned_addr learned my addr 172.21.15.27:0/3966870918 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:08:39.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.823+0000 7f0e68a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e64108e70 0x7f0e6419f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:40380/0 (socket says 172.21.15.27:40380) 2024-04-26T23:08:39.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.824+0000 7f0e63e00640 1 -- 172.21.15.27:0/3966870918 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e64108e70 msgr2=0x7f0e6419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:39.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.824+0000 7f0e63e00640 1 --2- 172.21.15.27:0/3966870918 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e64108e70 0x7f0e6419f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.824+0000 7f0e63e00640 1 -- 172.21.15.27:0/3966870918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e6410edb0 msgr2=0x7f0e641a3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:39.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.824+0000 7f0e63e00640 1 --2- 172.21.15.27:0/3966870918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e6410edb0 0x7f0e641a3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:39.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.824+0000 7f0e63e00640 1 -- 172.21.15.27:0/3966870918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e641a4570 con 0x7f0e64109810 2024-04-26T23:08:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.824+0000 7f0e63e00640 1 --2- 172.21.15.27:0/3966870918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e64109810 0x7f0e6419fac0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000e9f0 tx=0x7f0e5000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:08:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.824+0000 7f0e61600640 1 -- 172.21.15.27:0/3966870918 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5000cd80 con 0x7f0e64109810 2024-04-26T23:08:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.824+0000 7f0e61600640 1 -- 172.21.15.27:0/3966870918 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e500054d0 con 0x7f0e64109810 2024-04-26T23:08:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.824+0000 7f0e61600640 1 -- 172.21.15.27:0/3966870918 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50010690 con 0x7f0e64109810 2024-04-26T23:08:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.825+0000 7f0e69e00640 1 -- 172.21.15.27:0/3966870918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e641a4860 con 0x7f0e64109810 2024-04-26T23:08:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.825+0000 7f0e69e00640 1 -- 172.21.15.27:0/3966870918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e641ac0f0 con 0x7f0e64109810 2024-04-26T23:08:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.826+0000 7f0e69e00640 1 -- 172.21.15.27:0/3966870918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e640506c0 con 0x7f0e64109810 2024-04-26T23:08:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.828+0000 7f0e61600640 1 -- 172.21.15.27:0/3966870918 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0e50004e90 con 0x7f0e64109810 2024-04-26T23:08:39.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.830+0000 7f0e61600640 1 --2- 172.21.15.27:0/3966870918 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f0e4c0a3d20 0x7f0e4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:39.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.830+0000 7f0e61600640 1 -- 172.21.15.27:0/3966870918 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9700+0+0 (secure 0 0 0) 0x7f0e500d10f0 con 0x7f0e64109810 2024-04-26T23:08:39.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.830+0000 7f0e68a00640 1 --2- 172.21.15.27:0/3966870918 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f0e4c0a3d20 0x7f0e4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:39.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.831+0000 7f0e68a00640 1 --2- 172.21.15.27:0/3966870918 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f0e4c0a3d20 0x7f0e4c0a61e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0e58031440 tx=0x7f0e58005a20 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:08:39.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:39.834+0000 7f0e61600640 1 -- 172.21.15.27:0/3966870918 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0e5009d290 con 0x7f0e64109810 2024-04-26T23:08:40.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.045+0000 7f0e69e00640 1 -- 172.21.15.27:0/3966870918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0e64110ca0 con 0x7f0e64109810 2024-04-26T23:08:40.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.045+0000 7f0e61600640 1 -- 172.21.15.27:0/3966870918 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f0e5009cb90 con 0x7f0e64109810 2024-04-26T23:08:40.046 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T23:08:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.048+0000 7f0e69e00640 1 -- 172.21.15.27:0/3966870918 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f0e4c0a3d20 msgr2=0x7f0e4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.048+0000 7f0e69e00640 1 --2- 172.21.15.27:0/3966870918 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f0e4c0a3d20 0x7f0e4c0a61e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0e58031440 tx=0x7f0e58005a20 comp rx=0 tx=0).stop 2024-04-26T23:08:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.048+0000 7f0e69e00640 1 -- 172.21.15.27:0/3966870918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e64109810 msgr2=0x7f0e6419fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.048+0000 7f0e69e00640 1 --2- 172.21.15.27:0/3966870918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e64109810 0x7f0e6419fac0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000e9f0 tx=0x7f0e5000edf0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.048+0000 7f0e69e00640 1 -- 172.21.15.27:0/3966870918 shutdown_connections 2024-04-26T23:08:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.048+0000 7f0e69e00640 1 --2- 172.21.15.27:0/3966870918 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f0e4c0a3d20 0x7f0e4c0a61e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.048+0000 7f0e69e00640 1 --2- 172.21.15.27:0/3966870918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e6410edb0 0x7f0e641a3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.048+0000 7f0e69e00640 1 --2- 172.21.15.27:0/3966870918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e64109810 0x7f0e6419fac0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.048+0000 7f0e69e00640 1 --2- 172.21.15.27:0/3966870918 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e64108e70 0x7f0e6419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.048+0000 7f0e69e00640 1 -- 172.21.15.27:0/3966870918 >> 172.21.15.27:0/3966870918 conn(0x7f0e64078740 msgr2=0x7f0e64079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:08:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.048+0000 7f0e69e00640 1 -- 172.21.15.27:0/3966870918 shutdown_connections 2024-04-26T23:08:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.048+0000 7f0e69e00640 1 -- 172.21.15.27:0/3966870918 wait complete. 2024-04-26T23:08:40.069 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:08:40.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.292+0000 7f6760a00640 1 -- 172.21.15.27:0/2419608589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c1052d0 msgr2=0x7f675c1117e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:40.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.292+0000 7f6760a00640 1 --2- 172.21.15.27:0/2419608589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c1052d0 0x7f675c1117e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6754009cd0 tx=0x7f675402a8f0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.292+0000 7f6760a00640 1 -- 172.21.15.27:0/2419608589 shutdown_connections 2024-04-26T23:08:40.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.292+0000 7f6760a00640 1 --2- 172.21.15.27:0/2419608589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c1052d0 0x7f675c1117e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.292+0000 7f6760a00640 1 --2- 172.21.15.27:0/2419608589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f675c104910 0x7f675c104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.292+0000 7f6760a00640 1 --2- 172.21.15.27:0/2419608589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f675c10a920 0x7f675c10ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.292+0000 7f6760a00640 1 -- 172.21.15.27:0/2419608589 >> 172.21.15.27:0/2419608589 conn(0x7f675c100400 msgr2=0x7f675c102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:08:40.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.292+0000 7f6760a00640 1 -- 172.21.15.27:0/2419608589 shutdown_connections 2024-04-26T23:08:40.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.292+0000 7f6760a00640 1 -- 172.21.15.27:0/2419608589 wait complete. 2024-04-26T23:08:40.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.293+0000 7f6760a00640 1 Processor -- start 2024-04-26T23:08:40.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.293+0000 7f6760a00640 1 -- start start 2024-04-26T23:08:40.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.294+0000 7f6760a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f675c104910 0x7f675c19b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:40.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.294+0000 7f6760a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c1052d0 0x7f675c19b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:40.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.294+0000 7f6760a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f675c10a920 0x7f675c19fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:40.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.294+0000 7f6760a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f675c114c30 con 0x7f675c10a920 2024-04-26T23:08:40.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.294+0000 7f6760a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f675c114ab0 con 0x7f675c104910 2024-04-26T23:08:40.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.294+0000 7f6760a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f675c114db0 con 0x7f675c1052d0 2024-04-26T23:08:40.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.294+0000 7f675b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f675c10a920 0x7f675c19fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:40.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.294+0000 7f675a800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f675c104910 0x7f675c19b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:40.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.294+0000 7f675b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f675c10a920 0x7f675c19fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38846/0 (socket says 172.21.15.27:38846) 2024-04-26T23:08:40.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.294+0000 7f675b200640 1 -- 172.21.15.27:0/1978179277 learned_addr learned my addr 172.21.15.27:0/1978179277 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:08:40.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.294+0000 7f6759e00640 1 --2- 172.21.15.27:0/1978179277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c1052d0 0x7f675c19b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:40.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.295+0000 7f675a800640 1 -- 172.21.15.27:0/1978179277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c1052d0 msgr2=0x7f675c19b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.295+0000 7f675a800640 1 --2- 172.21.15.27:0/1978179277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c1052d0 0x7f675c19b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.295+0000 7f675a800640 1 -- 172.21.15.27:0/1978179277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f675c10a920 msgr2=0x7f675c19fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.295+0000 7f675a800640 1 --2- 172.21.15.27:0/1978179277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f675c10a920 0x7f675c19fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.295+0000 7f675a800640 1 -- 172.21.15.27:0/1978179277 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675c1a0140 con 0x7f675c104910 2024-04-26T23:08:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.295+0000 7f6759e00640 1 --2- 172.21.15.27:0/1978179277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c1052d0 0x7f675c19b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:08:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.295+0000 7f675a800640 1 --2- 172.21.15.27:0/1978179277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f675c104910 0x7f675c19b150 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f674c00e9f0 tx=0x7f674c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:08:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.295+0000 7f6753400640 1 -- 172.21.15.27:0/1978179277 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c00ccc0 con 0x7f675c104910 2024-04-26T23:08:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.295+0000 7f6760a00640 1 -- 172.21.15.27:0/1978179277 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675c1a0430 con 0x7f675c104910 2024-04-26T23:08:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.295+0000 7f6753400640 1 -- 172.21.15.27:0/1978179277 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f674c005020 con 0x7f675c104910 2024-04-26T23:08:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.295+0000 7f6753400640 1 -- 172.21.15.27:0/1978179277 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c004a00 con 0x7f675c104910 2024-04-26T23:08:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.295+0000 7f6760a00640 1 -- 172.21.15.27:0/1978179277 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f675c1a7cc0 con 0x7f675c104910 2024-04-26T23:08:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.297+0000 7f6760a00640 1 -- 172.21.15.27:0/1978179277 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f675c0506c0 con 0x7f675c104910 2024-04-26T23:08:40.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.299+0000 7f6753400640 1 -- 172.21.15.27:0/1978179277 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f674c004ba0 con 0x7f675c104910 2024-04-26T23:08:40.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.300+0000 7f6753400640 1 --2- 172.21.15.27:0/1978179277 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f67400a3da0 0x7f67400a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:40.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.300+0000 7f6753400640 1 -- 172.21.15.27:0/1978179277 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9700+0+0 (secure 0 0 0) 0x7f674c0d0b80 con 0x7f675c104910 2024-04-26T23:08:40.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.300+0000 7f6759e00640 1 --2- 172.21.15.27:0/1978179277 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f67400a3da0 0x7f67400a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:40.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.301+0000 7f6759e00640 1 --2- 172.21.15.27:0/1978179277 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f67400a3da0 0x7f67400a6260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f675c103c10 tx=0x7f6744005f90 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:08:40.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.305+0000 7f6753400640 1 -- 172.21.15.27:0/1978179277 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f674c09cd20 con 0x7f675c104910 2024-04-26T23:08:40.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:40 smithi027 ceph-mon[28034]: from='client.15702 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:08:40.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:40 smithi027 ceph-mon[28034]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:08:40.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:40 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3836444660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:08:40.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:40 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3966870918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:08:40.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.516+0000 7f6760a00640 1 -- 172.21.15.27:0/1978179277 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f675c02dad0 con 0x7f675c104910 2024-04-26T23:08:40.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.516+0000 7f6753400640 1 -- 172.21.15.27:0/1978179277 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7f674c09c620 con 0x7f675c104910 2024-04-26T23:08:40.517 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T23:08:40.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.519+0000 7f6760a00640 1 -- 172.21.15.27:0/1978179277 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f67400a3da0 msgr2=0x7f67400a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.519+0000 7f6760a00640 1 --2- 172.21.15.27:0/1978179277 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f67400a3da0 0x7f67400a6260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f675c103c10 tx=0x7f6744005f90 comp rx=0 tx=0).stop 2024-04-26T23:08:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.519+0000 7f6760a00640 1 -- 172.21.15.27:0/1978179277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f675c104910 msgr2=0x7f675c19b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.519+0000 7f6760a00640 1 --2- 172.21.15.27:0/1978179277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f675c104910 0x7f675c19b150 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f674c00e9f0 tx=0x7f674c00eeb0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.519+0000 7f6760a00640 1 -- 172.21.15.27:0/1978179277 shutdown_connections 2024-04-26T23:08:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.519+0000 7f6760a00640 1 --2- 172.21.15.27:0/1978179277 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f67400a3da0 0x7f67400a6260 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.519+0000 7f6760a00640 1 --2- 172.21.15.27:0/1978179277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f675c10a920 0x7f675c19fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.519+0000 7f6760a00640 1 --2- 172.21.15.27:0/1978179277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c1052d0 0x7f675c19b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.519+0000 7f6760a00640 1 --2- 172.21.15.27:0/1978179277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f675c104910 0x7f675c19b150 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.519+0000 7f6760a00640 1 -- 172.21.15.27:0/1978179277 >> 172.21.15.27:0/1978179277 conn(0x7f675c100400 msgr2=0x7f675c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:08:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.519+0000 7f6760a00640 1 -- 172.21.15.27:0/1978179277 shutdown_connections 2024-04-26T23:08:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.520+0000 7f6760a00640 1 -- 172.21.15.27:0/1978179277 wait complete. 2024-04-26T23:08:40.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:40 smithi080 ceph-mon[32324]: from='client.15702 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:08:40.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:40 smithi080 ceph-mon[32324]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:08:40.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:40 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3836444660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:08:40.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:40 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3966870918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:08:40.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:40 smithi094 ceph-mon[32889]: from='client.15702 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:08:40.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:40 smithi094 ceph-mon[32889]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:08:40.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:40 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3836444660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:08:40.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:40 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3966870918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:08:40.540 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:08:40.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.756+0000 7f79dba00640 1 -- 172.21.15.27:0/3717472665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d4104b30 msgr2=0x7f79d4104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:40.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.756+0000 7f79dba00640 1 --2- 172.21.15.27:0/3717472665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d4104b30 0x7f79d4104fb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009900 tx=0x7f79c002a3f0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.756+0000 7f79dba00640 1 -- 172.21.15.27:0/3717472665 shutdown_connections 2024-04-26T23:08:40.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.756+0000 7f79dba00640 1 --2- 172.21.15.27:0/3717472665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d41054f0 0x7f79d41119b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.756+0000 7f79dba00640 1 --2- 172.21.15.27:0/3717472665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d4104b30 0x7f79d4104fb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.756+0000 7f79dba00640 1 --2- 172.21.15.27:0/3717472665 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79d410abf0 0x7f79d410afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.756+0000 7f79dba00640 1 -- 172.21.15.27:0/3717472665 >> 172.21.15.27:0/3717472665 conn(0x7f79d41005e0 msgr2=0x7f79d4102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:08:40.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.756+0000 7f79dba00640 1 -- 172.21.15.27:0/3717472665 shutdown_connections 2024-04-26T23:08:40.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.756+0000 7f79dba00640 1 -- 172.21.15.27:0/3717472665 wait complete. 2024-04-26T23:08:40.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.757+0000 7f79dba00640 1 Processor -- start 2024-04-26T23:08:40.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.757+0000 7f79dba00640 1 -- start start 2024-04-26T23:08:40.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.758+0000 7f79dba00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d4104b30 0x7f79d419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:40.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.758+0000 7f79dba00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d41054f0 0x7f79d419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:40.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.758+0000 7f79dba00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79d410abf0 0x7f79d419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:40.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.758+0000 7f79dba00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f79d4114df0 con 0x7f79d4104b30 2024-04-26T23:08:40.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.758+0000 7f79dba00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f79d4114c70 con 0x7f79d410abf0 2024-04-26T23:08:40.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.758+0000 7f79dba00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f79d4114f70 con 0x7f79d41054f0 2024-04-26T23:08:40.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.758+0000 7f79d9000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d4104b30 0x7f79d419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:40.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.758+0000 7f79d9000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d4104b30 0x7f79d419b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38864/0 (socket says 172.21.15.27:38864) 2024-04-26T23:08:40.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.758+0000 7f79d9000640 1 -- 172.21.15.27:0/975975470 learned_addr learned my addr 172.21.15.27:0/975975470 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:08:40.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.758+0000 7f79d3e00640 1 --2- 172.21.15.27:0/975975470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d41054f0 0x7f79d419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:40.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.758+0000 7f79d9a00640 1 --2- 172.21.15.27:0/975975470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79d410abf0 0x7f79d419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:40.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.759+0000 7f79d9000640 1 -- 172.21.15.27:0/975975470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d41054f0 msgr2=0x7f79d419b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:40.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.759+0000 7f79d9000640 1 --2- 172.21.15.27:0/975975470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d41054f0 0x7f79d419b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.759+0000 7f79d9000640 1 -- 172.21.15.27:0/975975470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79d410abf0 msgr2=0x7f79d419fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:40.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.759+0000 7f79d9000640 1 --2- 172.21.15.27:0/975975470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79d410abf0 0x7f79d419fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.759+0000 7f79d9000640 1 -- 172.21.15.27:0/975975470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d41a0260 con 0x7f79d4104b30 2024-04-26T23:08:40.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.759+0000 7f79d9000640 1 --2- 172.21.15.27:0/975975470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d4104b30 0x7f79d419b270 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f79c800ea60 tx=0x7f79c800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:08:40.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.759+0000 7f79d1600640 1 -- 172.21.15.27:0/975975470 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c800cde0 con 0x7f79d4104b30 2024-04-26T23:08:40.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.759+0000 7f79d1600640 1 -- 172.21.15.27:0/975975470 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79c80035f0 con 0x7f79d4104b30 2024-04-26T23:08:40.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.759+0000 7f79dba00640 1 -- 172.21.15.27:0/975975470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d41a0550 con 0x7f79d4104b30 2024-04-26T23:08:40.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.759+0000 7f79d1600640 1 -- 172.21.15.27:0/975975470 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c8010690 con 0x7f79d4104b30 2024-04-26T23:08:40.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.759+0000 7f79dba00640 1 -- 172.21.15.27:0/975975470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d41a7de0 con 0x7f79d4104b30 2024-04-26T23:08:40.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.761+0000 7f79dba00640 1 -- 172.21.15.27:0/975975470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d4069c30 con 0x7f79d4104b30 2024-04-26T23:08:40.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.763+0000 7f79d1600640 1 -- 172.21.15.27:0/975975470 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f79c80037a0 con 0x7f79d4104b30 2024-04-26T23:08:40.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.764+0000 7f79d1600640 1 --2- 172.21.15.27:0/975975470 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f79bc0a3da0 0x7f79bc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:40.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.765+0000 7f79d1600640 1 -- 172.21.15.27:0/975975470 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9700+0+0 (secure 0 0 0) 0x7f79c80d0f50 con 0x7f79d4104b30 2024-04-26T23:08:40.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.765+0000 7f79d3e00640 1 --2- 172.21.15.27:0/975975470 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f79bc0a3da0 0x7f79bc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:40.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.766+0000 7f79d3e00640 1 --2- 172.21.15.27:0/975975470 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f79bc0a3da0 0x7f79bc0a6260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79c0000c00 tx=0x7f79c0005e80 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:08:40.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.769+0000 7f79d1600640 1 -- 172.21.15.27:0/975975470 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f79c809d0f0 con 0x7f79d4104b30 2024-04-26T23:08:40.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.976+0000 7f79dba00640 1 -- 172.21.15.27:0/975975470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f79d402dad0 con 0x7f79d4104b30 2024-04-26T23:08:40.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.977+0000 7f79d1600640 1 -- 172.21.15.27:0/975975470 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7f79c809c9f0 con 0x7f79d4104b30 2024-04-26T23:08:40.977 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-26T23:08:40.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.979+0000 7f79dba00640 1 -- 172.21.15.27:0/975975470 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f79bc0a3da0 msgr2=0x7f79bc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:40.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.979+0000 7f79dba00640 1 --2- 172.21.15.27:0/975975470 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f79bc0a3da0 0x7f79bc0a6260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79c0000c00 tx=0x7f79c0005e80 comp rx=0 tx=0).stop 2024-04-26T23:08:40.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.979+0000 7f79dba00640 1 -- 172.21.15.27:0/975975470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d4104b30 msgr2=0x7f79d419b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:40.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.979+0000 7f79dba00640 1 --2- 172.21.15.27:0/975975470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d4104b30 0x7f79d419b270 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f79c800ea60 tx=0x7f79c800ef20 comp rx=0 tx=0).stop 2024-04-26T23:08:40.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.979+0000 7f79dba00640 1 -- 172.21.15.27:0/975975470 shutdown_connections 2024-04-26T23:08:40.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.979+0000 7f79dba00640 1 --2- 172.21.15.27:0/975975470 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f79bc0a3da0 0x7f79bc0a6260 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.979+0000 7f79dba00640 1 --2- 172.21.15.27:0/975975470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79d410abf0 0x7f79d419fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.979+0000 7f79dba00640 1 --2- 172.21.15.27:0/975975470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d41054f0 0x7f79d419b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.979+0000 7f79dba00640 1 --2- 172.21.15.27:0/975975470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d4104b30 0x7f79d419b270 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:40.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.979+0000 7f79dba00640 1 -- 172.21.15.27:0/975975470 >> 172.21.15.27:0/975975470 conn(0x7f79d41005e0 msgr2=0x7f79d4101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:08:40.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.980+0000 7f79dba00640 1 -- 172.21.15.27:0/975975470 shutdown_connections 2024-04-26T23:08:40.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:40.980+0000 7f79dba00640 1 -- 172.21.15.27:0/975975470 wait complete. 2024-04-26T23:08:41.000 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:08:41.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.217+0000 7f68bb200640 1 -- 172.21.15.27:0/417019973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68bc109810 msgr2=0x7f68bc115be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.217+0000 7f68bb200640 1 --2- 172.21.15.27:0/417019973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68bc109810 0x7f68bc115be0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009c10 tx=0x7f68b402a6f0 comp rx=0 tx=0).stop 2024-04-26T23:08:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.217+0000 7f68bb200640 1 -- 172.21.15.27:0/417019973 shutdown_connections 2024-04-26T23:08:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.217+0000 7f68bb200640 1 --2- 172.21.15.27:0/417019973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68bc108e70 0x7f68bc1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.217+0000 7f68bb200640 1 --2- 172.21.15.27:0/417019973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68bc109810 0x7f68bc115be0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.217+0000 7f68bb200640 1 --2- 172.21.15.27:0/417019973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68bc10edb0 0x7f68bc10f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.217+0000 7f68bb200640 1 -- 172.21.15.27:0/417019973 >> 172.21.15.27:0/417019973 conn(0x7f68bc078740 msgr2=0x7f68bc107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:08:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.217+0000 7f68bb200640 1 -- 172.21.15.27:0/417019973 shutdown_connections 2024-04-26T23:08:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.217+0000 7f68bb200640 1 -- 172.21.15.27:0/417019973 wait complete. 2024-04-26T23:08:41.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.218+0000 7f68bb200640 1 Processor -- start 2024-04-26T23:08:41.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.218+0000 7f68bb200640 1 -- start start 2024-04-26T23:08:41.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.218+0000 7f68bb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68bc108e70 0x7f68bc19f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:41.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.218+0000 7f68bb200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68bc109810 0x7f68bc19fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.218+0000 7f68bb200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68bc10edb0 0x7f68bc1a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.218+0000 7f68bb200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc118f90 con 0x7f68bc108e70 2024-04-26T23:08:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.218+0000 7f68bb200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc118e10 con 0x7f68bc10edb0 2024-04-26T23:08:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.218+0000 7f68bb200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc119110 con 0x7f68bc109810 2024-04-26T23:08:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.219+0000 7f68b9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68bc109810 0x7f68bc19fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.219+0000 7f68b9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68bc108e70 0x7f68bc19f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.219+0000 7f68b9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68bc109810 0x7f68bc19fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:38056/0 (socket says 172.21.15.27:38056) 2024-04-26T23:08:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.219+0000 7f68b9400640 1 -- 172.21.15.27:0/2926173439 learned_addr learned my addr 172.21.15.27:0/2926173439 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:08:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.219+0000 7f68ba800640 1 --2- 172.21.15.27:0/2926173439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68bc10edb0 0x7f68bc1a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.219+0000 7f68b9e00640 1 -- 172.21.15.27:0/2926173439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68bc109810 msgr2=0x7f68bc19fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.219+0000 7f68b9e00640 1 --2- 172.21.15.27:0/2926173439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68bc109810 0x7f68bc19fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.219+0000 7f68b9e00640 1 -- 172.21.15.27:0/2926173439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68bc10edb0 msgr2=0x7f68bc1a3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.219+0000 7f68b9e00640 1 --2- 172.21.15.27:0/2926173439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68bc10edb0 0x7f68bc1a3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.219+0000 7f68b9e00640 1 -- 172.21.15.27:0/2926173439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68bc1a4520 con 0x7f68bc108e70 2024-04-26T23:08:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.219+0000 7f68ba800640 1 --2- 172.21.15.27:0/2926173439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68bc10edb0 0x7f68bc1a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:08:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.220+0000 7f68b9e00640 1 --2- 172.21.15.27:0/2926173439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68bc108e70 0x7f68bc19f530 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00e790 tx=0x7f68ac00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:08:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.220+0000 7f68b2a00640 1 -- 172.21.15.27:0/2926173439 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00eea0 con 0x7f68bc108e70 2024-04-26T23:08:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.220+0000 7f68bb200640 1 -- 172.21.15.27:0/2926173439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68bc1a4810 con 0x7f68bc108e70 2024-04-26T23:08:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.220+0000 7f68b2a00640 1 -- 172.21.15.27:0/2926173439 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68ac00ccc0 con 0x7f68bc108e70 2024-04-26T23:08:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.220+0000 7f68b2a00640 1 -- 172.21.15.27:0/2926173439 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac018660 con 0x7f68bc108e70 2024-04-26T23:08:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.220+0000 7f68bb200640 1 -- 172.21.15.27:0/2926173439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68bc1ac0a0 con 0x7f68bc108e70 2024-04-26T23:08:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.221+0000 7f68bb200640 1 -- 172.21.15.27:0/2926173439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68bc0506c0 con 0x7f68bc108e70 2024-04-26T23:08:41.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.223+0000 7f68b2a00640 1 -- 172.21.15.27:0/2926173439 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f68ac010430 con 0x7f68bc108e70 2024-04-26T23:08:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.225+0000 7f68b2a00640 1 --2- 172.21.15.27:0/2926173439 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f68a00a3d20 0x7f68a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.225+0000 7f68b2a00640 1 -- 172.21.15.27:0/2926173439 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9700+0+0 (secure 0 0 0) 0x7f68ac0d0e00 con 0x7f68bc108e70 2024-04-26T23:08:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.225+0000 7f68b9400640 1 --2- 172.21.15.27:0/2926173439 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f68a00a3d20 0x7f68a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.226+0000 7f68b9400640 1 --2- 172.21.15.27:0/2926173439 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f68a00a3d20 0x7f68a00a61e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f68bc10e500 tx=0x7f68a8005f90 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:08:41.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.229+0000 7f68b2a00640 1 -- 172.21.15.27:0/2926173439 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f68ac09c950 con 0x7f68bc108e70 2024-04-26T23:08:41.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:41 smithi027 ceph-mon[28034]: pgmap v311: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:08:41.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:41 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1978179277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:08:41.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:41 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/975975470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:08:41.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.436+0000 7f68bb200640 1 -- 172.21.15.27:0/2926173439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f68bc111260 con 0x7f68bc108e70 2024-04-26T23:08:41.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.437+0000 7f68b2a00640 1 -- 172.21.15.27:0/2926173439 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v101) ==== 103+0+11 (secure 0 0 0) 0x7f68ac0a0c50 con 0x7f68bc108e70 2024-04-26T23:08:41.437 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T23:08:41.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.439+0000 7f68bb200640 1 -- 172.21.15.27:0/2926173439 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f68a00a3d20 msgr2=0x7f68a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:41.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.439+0000 7f68bb200640 1 --2- 172.21.15.27:0/2926173439 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f68a00a3d20 0x7f68a00a61e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f68bc10e500 tx=0x7f68a8005f90 comp rx=0 tx=0).stop 2024-04-26T23:08:41.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.439+0000 7f68bb200640 1 -- 172.21.15.27:0/2926173439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68bc108e70 msgr2=0x7f68bc19f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:41.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.439+0000 7f68bb200640 1 --2- 172.21.15.27:0/2926173439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68bc108e70 0x7f68bc19f530 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00e790 tx=0x7f68ac00ec50 comp rx=0 tx=0).stop 2024-04-26T23:08:41.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.439+0000 7f68bb200640 1 -- 172.21.15.27:0/2926173439 shutdown_connections 2024-04-26T23:08:41.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.439+0000 7f68bb200640 1 --2- 172.21.15.27:0/2926173439 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f68a00a3d20 0x7f68a00a61e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:41.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.439+0000 7f68bb200640 1 --2- 172.21.15.27:0/2926173439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68bc10edb0 0x7f68bc1a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:41.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.439+0000 7f68bb200640 1 --2- 172.21.15.27:0/2926173439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68bc109810 0x7f68bc19fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:41.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.439+0000 7f68bb200640 1 --2- 172.21.15.27:0/2926173439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68bc108e70 0x7f68bc19f530 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:41.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.439+0000 7f68bb200640 1 -- 172.21.15.27:0/2926173439 >> 172.21.15.27:0/2926173439 conn(0x7f68bc078740 msgr2=0x7f68bc079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:08:41.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.440+0000 7f68bb200640 1 -- 172.21.15.27:0/2926173439 shutdown_connections 2024-04-26T23:08:41.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.440+0000 7f68bb200640 1 -- 172.21.15.27:0/2926173439 wait complete. 2024-04-26T23:08:41.457 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-26T23:08:41.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:41 smithi080 ceph-mon[32324]: pgmap v311: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:08:41.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:41 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1978179277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:08:41.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:41 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/975975470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:08:41.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:41 smithi094 ceph-mon[32889]: pgmap v311: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:08:41.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:41 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1978179277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:08:41.535 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:41 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/975975470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:08:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.671+0000 7fe5d6600640 1 -- 172.21.15.27:0/1154410731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d0109a90 msgr2=0x7fe5d010da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.671+0000 7fe5d6600640 1 --2- 172.21.15.27:0/1154410731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d0109a90 0x7fe5d010da80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009900 tx=0x7fe5bc02a3f0 comp rx=0 tx=0).stop 2024-04-26T23:08:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.671+0000 7fe5d6600640 1 -- 172.21.15.27:0/1154410731 shutdown_connections 2024-04-26T23:08:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.671+0000 7fe5d6600640 1 --2- 172.21.15.27:0/1154410731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5d010e160 0x7fe5d0115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.671+0000 7fe5d6600640 1 --2- 172.21.15.27:0/1154410731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d0109a90 0x7fe5d010da80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.671+0000 7fe5d6600640 1 --2- 172.21.15.27:0/1154410731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5d01090e0 0x7fe5d01094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.671+0000 7fe5d6600640 1 -- 172.21.15.27:0/1154410731 >> 172.21.15.27:0/1154410731 conn(0x7fe5d0078700 msgr2=0x7fe5d0107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:08:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.671+0000 7fe5d6600640 1 -- 172.21.15.27:0/1154410731 shutdown_connections 2024-04-26T23:08:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.671+0000 7fe5d6600640 1 -- 172.21.15.27:0/1154410731 wait complete. 2024-04-26T23:08:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.672+0000 7fe5d6600640 1 Processor -- start 2024-04-26T23:08:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.672+0000 7fe5d6600640 1 -- start start 2024-04-26T23:08:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.672+0000 7fe5d6600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5d01090e0 0x7fe5d019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.672+0000 7fe5d6600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d010e160 0x7fe5d019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.672+0000 7fe5d6600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5d01a3ee0 0x7fe5d01a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.672+0000 7fe5d6600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d01190d0 con 0x7fe5d01a3ee0 2024-04-26T23:08:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.672+0000 7fe5d6600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d0118f50 con 0x7fe5d01090e0 2024-04-26T23:08:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.672+0000 7fe5d6600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d0119250 con 0x7fe5d010e160 2024-04-26T23:08:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.673+0000 7fe5d5200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5d01090e0 0x7fe5d019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.673+0000 7fe5d5200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5d01090e0 0x7fe5d019f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:40456/0 (socket says 172.21.15.27:40456) 2024-04-26T23:08:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.673+0000 7fe5d5200640 1 -- 172.21.15.27:0/1689222691 learned_addr learned my addr 172.21.15.27:0/1689222691 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:08:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.673+0000 7fe5cfe00640 1 --2- 172.21.15.27:0/1689222691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d010e160 0x7fe5d019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.673+0000 7fe5d5c00640 1 --2- 172.21.15.27:0/1689222691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5d01a3ee0 0x7fe5d01a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.673+0000 7fe5d5200640 1 -- 172.21.15.27:0/1689222691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d010e160 msgr2=0x7fe5d019fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.673+0000 7fe5d5200640 1 --2- 172.21.15.27:0/1689222691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d010e160 0x7fe5d019fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.673+0000 7fe5d5200640 1 -- 172.21.15.27:0/1689222691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5d01a3ee0 msgr2=0x7fe5d01a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.673+0000 7fe5d5200640 1 --2- 172.21.15.27:0/1689222691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5d01a3ee0 0x7fe5d01a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.673+0000 7fe5d5200640 1 -- 172.21.15.27:0/1689222691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d01a49d0 con 0x7fe5d01090e0 2024-04-26T23:08:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.674+0000 7fe5d5200640 1 --2- 172.21.15.27:0/1689222691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5d01090e0 0x7fe5d019f670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400ea60 tx=0x7fe5c400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:08:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.674+0000 7fe5cd600640 1 -- 172.21.15.27:0/1689222691 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c400cde0 con 0x7fe5d01090e0 2024-04-26T23:08:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.674+0000 7fe5cd600640 1 -- 172.21.15.27:0/1689222691 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5c40035f0 con 0x7fe5d01090e0 2024-04-26T23:08:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.674+0000 7fe5cd600640 1 -- 172.21.15.27:0/1689222691 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4010690 con 0x7fe5d01090e0 2024-04-26T23:08:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.674+0000 7fe5d6600640 1 -- 172.21.15.27:0/1689222691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d01a4cc0 con 0x7fe5d01090e0 2024-04-26T23:08:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.674+0000 7fe5d6600640 1 -- 172.21.15.27:0/1689222691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d01ac430 con 0x7fe5d01090e0 2024-04-26T23:08:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.675+0000 7fe5d6600640 1 -- 172.21.15.27:0/1689222691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d0050730 con 0x7fe5d01090e0 2024-04-26T23:08:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.677+0000 7fe5cd600640 1 -- 172.21.15.27:0/1689222691 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe5c40037a0 con 0x7fe5d01090e0 2024-04-26T23:08:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.679+0000 7fe5cd600640 1 --2- 172.21.15.27:0/1689222691 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe5b80a3d90 0x7fe5b80a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:08:41.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.679+0000 7fe5cd600640 1 -- 172.21.15.27:0/1689222691 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9700+0+0 (secure 0 0 0) 0x7fe5c40d0f20 con 0x7fe5d01090e0 2024-04-26T23:08:41.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.679+0000 7fe5cfe00640 1 --2- 172.21.15.27:0/1689222691 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe5b80a3d90 0x7fe5b80a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:08:41.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.680+0000 7fe5cfe00640 1 --2- 172.21.15.27:0/1689222691 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe5b80a3d90 0x7fe5b80a6250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc02a9e0 tx=0x7fe5bc005a20 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:08:41.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.684+0000 7fe5cd600640 1 -- 172.21.15.27:0/1689222691 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe5c409d0c0 con 0x7fe5d01090e0 2024-04-26T23:08:41.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.947+0000 7fe5d6600640 1 -- 172.21.15.27:0/1689222691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fe5d01a0970 con 0x7fe5d01090e0 2024-04-26T23:08:41.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.959+0000 7fe5cd600640 1 -- 172.21.15.27:0/1689222691 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746062 (secure 0 0 0) 0x7fe5c409c9c0 con 0x7fe5d01090e0 2024-04-26T23:08:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T23:08:41.971 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T23:08:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 26, 2024-04-26T23:08:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "active_gid": 25391, 2024-04-26T23:08:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "active_name": "x", 2024-04-26T23:08:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "active_addrs": { 2024-04-26T23:08:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-04-26T23:08:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-04-26T23:08:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2024-04-26T23:08:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3058395889 2024-04-26T23:08:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-04-26T23:08:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2024-04-26T23:08:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3058395889 2024-04-26T23:08:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T23:08:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: "active_addr": "172.21.15.27:6801/3058395889", 2024-04-26T23:08:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: "active_change": "2024-04-26T22:58:35.543920+0000", 2024-04-26T23:08:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: "active_mgr_features": 4540701547738038271, 2024-04-26T23:08:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: "available": true, 2024-04-26T23:08:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: "standbys": [ 2024-04-26T23:08:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:41.973 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 25388, 2024-04-26T23:08:41.973 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "y", 2024-04-26T23:08:41.973 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_features": 4540701547738038271, 2024-04-26T23:08:41.973 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-04-26T23:08:41.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:41.973 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-04-26T23:08:41.973 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:41.973 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:41.973 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:41.973 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T23:08:41.973 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T23:08:41.973 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:41.973 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.974 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.974 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T23:08:41.974 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.974 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.974 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.974 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T23:08:41.974 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:41.974 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.974 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.974 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:41.974 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:41.974 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:41.974 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.975 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.975 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:41.975 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.975 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.975 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:41.975 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:41.975 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:41.975 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:41.975 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:41.975 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:41.976 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:41.976 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:41.976 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:41.976 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:41.976 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.976 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.976 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:41.976 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:41.977 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:41.977 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.977 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.977 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:41.977 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.977 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.977 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.977 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:41.977 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:41.977 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.977 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.978 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:41.978 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:41.978 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:41.978 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.978 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.978 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:41.978 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.978 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.978 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:41.978 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:41.978 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:41.979 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:41.979 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:41.979 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:41.979 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:41.979 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:41.979 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:41.979 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:41.979 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.979 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.980 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:41.980 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:41.980 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:41.980 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.980 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.980 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:41.980 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.980 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.980 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.980 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:41.980 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:41.981 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.981 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.981 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-04-26T23:08:41.981 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-04-26T23:08:41.981 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:41.981 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.981 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.981 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:41.981 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.981 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.982 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.982 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to", 2024-04-26T23:08:41.982 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:41.982 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.982 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.982 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-04-26T23:08:41.982 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-04-26T23:08:41.982 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:41.982 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.982 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.982 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-04-26T23:08:41.983 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.983 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.983 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.983 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-04-26T23:08:41.983 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:41.983 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.983 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.983 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-04-26T23:08:41.983 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-04-26T23:08:41.983 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:41.983 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.984 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.984 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:41.984 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.984 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.984 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.984 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-04-26T23:08:41.984 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:41.984 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.984 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.984 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-04-26T23:08:41.984 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-04-26T23:08:41.985 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:41.985 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.985 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.985 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:41.985 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.985 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.985 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.985 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-04-26T23:08:41.985 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:41.985 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.985 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.985 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-04-26T23:08:41.985 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-04-26T23:08:41.986 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:41.986 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.986 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.986 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-04-26T23:08:41.986 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.986 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.986 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.986 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-04-26T23:08:41.986 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:41.986 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.986 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.986 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-04-26T23:08:41.986 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-04-26T23:08:41.986 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-04-26T23:08:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-04-26T23:08:41.988 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-04-26T23:08:41.988 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:41.988 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.988 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.988 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:41.988 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.988 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.988 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.988 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T23:08:41.988 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:41.988 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.988 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.988 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-04-26T23:08:41.988 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-04-26T23:08:41.989 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:41.989 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.989 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.989 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:41.989 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.989 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.989 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.989 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-04-26T23:08:41.989 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:41.989 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.989 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.989 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:41.989 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:41.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.990 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:41.990 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-04-26T23:08:41.990 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:41.990 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:41.990 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:41.990 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-04-26T23:08:41.990 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-04-26T23:08:41.990 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:41.990 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.990 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.990 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:41.990 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.990 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.990 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.990 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T23:08:41.991 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:41.991 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.991 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.991 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-04-26T23:08:41.991 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-04-26T23:08:41.991 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:41.991 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.991 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.991 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-04-26T23:08:41.992 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.992 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.992 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.992 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T23:08:41.992 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T23:08:41.992 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.992 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.992 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-04-26T23:08:41.992 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-04-26T23:08:41.992 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T23:08:41.992 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.992 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.992 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T23:08:41.993 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-04-26T23:08:41.993 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-04-26T23:08:41.993 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.993 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T23:08:41.993 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T23:08:41.993 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.993 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.993 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-04-26T23:08:41.993 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-04-26T23:08:41.993 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T23:08:41.993 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.993 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.993 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-04-26T23:08:41.993 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T23:08:41.994 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-04-26T23:08:41.994 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.994 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T23:08:41.994 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:41.994 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.994 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.994 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-04-26T23:08:41.994 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-04-26T23:08:41.994 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:41.995 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.995 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.995 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T23:08:41.995 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.995 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.995 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.995 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T23:08:41.995 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T23:08:41.995 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.995 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.995 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-04-26T23:08:41.995 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-04-26T23:08:41.995 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:41.995 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.996 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.996 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-04-26T23:08:41.996 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-04-26T23:08:41.996 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-04-26T23:08:41.996 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.996 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-04-26T23:08:41.996 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T23:08:41.996 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.996 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.996 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-04-26T23:08:41.996 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-04-26T23:08:41.996 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:41.996 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.996 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.996 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-04-26T23:08:41.997 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.997 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.997 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.997 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T23:08:41.997 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T23:08:41.997 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.997 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.997 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-04-26T23:08:41.997 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-04-26T23:08:41.997 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T23:08:41.997 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.997 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.997 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T23:08:41.997 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-04-26T23:08:41.998 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-04-26T23:08:41.998 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:41.998 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T23:08:41.998 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T23:08:41.998 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:41.998 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:41.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:41.998 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:41.998 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:41.998 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:41.998 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:41.999 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:41.999 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:41.999 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:41.999 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:41.999 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:41.999 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:41.999 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:41.999 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:41.999 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:41.999 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:41.999 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:41.999 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:41.999 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:41.999 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:41.999 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.000 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.000 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.000 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.000 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.000 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.000 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.000 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.000 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.000 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.000 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.001 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.001 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.001 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.001 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.001 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.001 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.001 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.001 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.001 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.001 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.001 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.001 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.001 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.001 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.002 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.002 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.002 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.002 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.002 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.002 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.002 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.002 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.002 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.002 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.002 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.002 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.002 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.003 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.003 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.003 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.003 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.003 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.003 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.003 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.003 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.003 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.003 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.003 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-04-26T23:08:42.003 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-04-26T23:08:42.003 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.004 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.004 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.004 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T23:08:42.004 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.004 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.004 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.004 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T23:08:42.004 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.004 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.004 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.004 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-04-26T23:08:42.004 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-04-26T23:08:42.004 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.004 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.005 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.005 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-04-26T23:08:42.005 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.005 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.005 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.005 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-04-26T23:08:42.005 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-04-26T23:08:42.005 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-04-26T23:08:42.005 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-04-26T23:08:42.005 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-04-26T23:08:42.005 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.005 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-04-26T23:08:42.005 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.005 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.005 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.006 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-04-26T23:08:42.006 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-04-26T23:08:42.006 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.006 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.006 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.006 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.006 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.006 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.006 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.006 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T23:08:42.006 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.006 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.006 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.007 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T23:08:42.007 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T23:08:42.007 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.007 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.007 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.007 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T23:08:42.007 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.007 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.007 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.007 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T23:08:42.007 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.007 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.007 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.007 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-04-26T23:08:42.008 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-04-26T23:08:42.008 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.008 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.008 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.008 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T23:08:42.008 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T23:08:42.008 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.008 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.008 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T23:08:42.008 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T23:08:42.008 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.008 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.008 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-04-26T23:08:42.009 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-04-26T23:08:42.009 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T23:08:42.009 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.009 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.009 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.009 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.009 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.009 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.009 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T23:08:42.009 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.009 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.009 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.009 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.009 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.010 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-04-26T23:08:42.010 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.010 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.010 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.010 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-04-26T23:08:42.010 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-04-26T23:08:42.010 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.010 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.010 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.010 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-04-26T23:08:42.010 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.010 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.010 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.010 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T23:08:42.011 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.011 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.011 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.011 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-04-26T23:08:42.011 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-04-26T23:08:42.011 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.011 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.011 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.011 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-04-26T23:08:42.011 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.011 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.011 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.011 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T23:08:42.011 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.012 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.012 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.012 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-04-26T23:08:42.012 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-04-26T23:08:42.012 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.012 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.012 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.012 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-04-26T23:08:42.012 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.012 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.012 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.012 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T23:08:42.012 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.013 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.013 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.013 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-04-26T23:08:42.013 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-04-26T23:08:42.013 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.013 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.013 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.013 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.013 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.013 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.013 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.013 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T23:08:42.013 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T23:08:42.014 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.014 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.014 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-04-26T23:08:42.014 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-04-26T23:08:42.014 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.014 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.014 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.014 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T23:08:42.014 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.014 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.014 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.014 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T23:08:42.014 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.014 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.015 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.015 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-04-26T23:08:42.015 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T23:08:42.015 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.015 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.015 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.015 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-04-26T23:08:42.015 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.015 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.015 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.015 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T23:08:42.015 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.015 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.016 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.016 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-04-26T23:08:42.016 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-04-26T23:08:42.016 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.016 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.016 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.016 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.016 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.016 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.016 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.016 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-04-26T23:08:42.016 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-04-26T23:08:42.016 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-04-26T23:08:42.016 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.017 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T23:08:42.017 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.017 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.017 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.017 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-04-26T23:08:42.017 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-04-26T23:08:42.017 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.017 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.017 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.017 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.017 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.017 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.017 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.017 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T23:08:42.018 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.018 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.018 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.018 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-04-26T23:08:42.018 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-04-26T23:08:42.018 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.018 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.018 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.018 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.018 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.018 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.018 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.019 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T23:08:42.019 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.019 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.019 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.019 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-04-26T23:08:42.019 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-04-26T23:08:42.019 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.019 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.019 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.019 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.019 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.019 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.019 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.019 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T23:08:42.020 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.020 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.020 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.020 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-04-26T23:08:42.020 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-04-26T23:08:42.020 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.020 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.020 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.020 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T23:08:42.020 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.020 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.020 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.020 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T23:08:42.021 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.021 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.021 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.021 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-04-26T23:08:42.021 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-04-26T23:08:42.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.021 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.021 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T23:08:42.021 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.021 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.021 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.021 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-04-26T23:08:42.021 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.022 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.022 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.022 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-04-26T23:08:42.022 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-04-26T23:08:42.022 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.022 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.022 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.022 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T23:08:42.022 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.022 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.022 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.022 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-04-26T23:08:42.022 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.022 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.023 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.023 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-04-26T23:08:42.023 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-04-26T23:08:42.023 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.023 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.023 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.023 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T23:08:42.023 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.023 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.023 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.023 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T23:08:42.023 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.023 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.023 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.024 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-04-26T23:08:42.024 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-04-26T23:08:42.024 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.024 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.024 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.024 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T23:08:42.024 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.024 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.024 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.025 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-04-26T23:08:42.025 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.025 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.025 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.025 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-04-26T23:08:42.025 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-04-26T23:08:42.025 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.025 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.025 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.025 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T23:08:42.025 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.025 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.025 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-04-26T23:08:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-04-26T23:08:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-04-26T23:08:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T23:08:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-04-26T23:08:42.027 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.027 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.027 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.027 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-04-26T23:08:42.027 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-04-26T23:08:42.027 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.027 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.027 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.027 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T23:08:42.027 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.027 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.027 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.027 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-04-26T23:08:42.027 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.028 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.028 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.028 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-04-26T23:08:42.028 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-04-26T23:08:42.028 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.028 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.028 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.028 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T23:08:42.028 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.028 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.028 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.028 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-04-26T23:08:42.028 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.028 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-04-26T23:08:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-04-26T23:08:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T23:08:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-04-26T23:08:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.030 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.030 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-04-26T23:08:42.030 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-04-26T23:08:42.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.030 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.030 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T23:08:42.030 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.030 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.030 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.030 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T23:08:42.030 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.030 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.031 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.031 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-04-26T23:08:42.031 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-04-26T23:08:42.031 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.031 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.031 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.031 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T23:08:42.031 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.031 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.031 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.031 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-04-26T23:08:42.031 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.031 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.031 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.032 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-04-26T23:08:42.032 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-04-26T23:08:42.032 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.032 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.032 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.032 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T23:08:42.032 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.032 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.032 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.032 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T23:08:42.032 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.032 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.032 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.033 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-04-26T23:08:42.033 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-04-26T23:08:42.033 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.033 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.033 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.033 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T23:08:42.033 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.033 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.033 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.033 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-04-26T23:08:42.033 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.033 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.033 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.033 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-04-26T23:08:42.034 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-04-26T23:08:42.034 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.034 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.034 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.034 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T23:08:42.034 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.034 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.034 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.034 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-04-26T23:08:42.034 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.034 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.034 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.034 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-04-26T23:08:42.034 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-04-26T23:08:42.035 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.035 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.035 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.035 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T23:08:42.035 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.035 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.035 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.035 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-04-26T23:08:42.035 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.035 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.035 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.035 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-04-26T23:08:42.035 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-04-26T23:08:42.035 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.035 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.036 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.036 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.036 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.036 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.036 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.036 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T23:08:42.036 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.036 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.036 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.036 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-04-26T23:08:42.036 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-04-26T23:08:42.036 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.036 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.037 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.037 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T23:08:42.037 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.037 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.037 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.037 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T23:08:42.037 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.037 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.037 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.037 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-04-26T23:08:42.037 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T23:08:42.037 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.037 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.037 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.038 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-04-26T23:08:42.038 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.038 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.038 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.038 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T23:08:42.038 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.038 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.038 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.038 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-04-26T23:08:42.038 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-04-26T23:08:42.038 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.038 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.038 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.039 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-04-26T23:08:42.039 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.039 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.039 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.039 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T23:08:42.039 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.039 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.039 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.039 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-04-26T23:08:42.039 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-04-26T23:08:42.039 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.039 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.040 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.040 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-04-26T23:08:42.040 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.040 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.040 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.040 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-04-26T23:08:42.040 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.040 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.040 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.040 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-04-26T23:08:42.040 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-04-26T23:08:42.040 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.040 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.041 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.041 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.041 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.041 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.041 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.041 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T23:08:42.041 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.041 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.041 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.041 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-04-26T23:08:42.041 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-04-26T23:08:42.041 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.041 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.041 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.042 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T23:08:42.042 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.042 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.042 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.042 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-04-26T23:08:42.042 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.042 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.042 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.042 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-04-26T23:08:42.042 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-04-26T23:08:42.042 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.042 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.043 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.043 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T23:08:42.043 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.043 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.043 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.043 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T23:08:42.043 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.043 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.043 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.043 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-04-26T23:08:42.043 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-04-26T23:08:42.043 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.043 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.043 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.044 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T23:08:42.044 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.044 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.044 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.044 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-04-26T23:08:42.044 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.044 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.044 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.044 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-04-26T23:08:42.044 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-04-26T23:08:42.044 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.044 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.044 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T23:08:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-04-26T23:08:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-04-26T23:08:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.046 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.046 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.046 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.046 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.046 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T23:08:42.046 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.046 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.046 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.046 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.046 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.046 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.046 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.046 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.047 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.047 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.047 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.047 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.047 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.047 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.047 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.047 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.047 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.047 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.047 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.047 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.047 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.047 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.048 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.048 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-04-26T23:08:42.048 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-04-26T23:08:42.048 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.048 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.048 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.048 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.048 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.048 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.048 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.048 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T23:08:42.048 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.048 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.049 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.049 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.049 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.049 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.049 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.049 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.049 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.049 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.049 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.049 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.049 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T23:08:42.049 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.049 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.049 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.050 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.050 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.050 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.050 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.050 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.050 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.050 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.050 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.050 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.050 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.050 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.050 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.050 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.051 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.051 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.051 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.051 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.051 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.051 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.051 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.051 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.051 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.051 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.051 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.051 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.051 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.051 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.052 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.052 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.052 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.052 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.052 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.052 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.052 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T23:08:42.052 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T23:08:42.052 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.052 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.052 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.052 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.052 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.053 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.053 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.053 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T23:08:42.053 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.053 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.053 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.053 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T23:08:42.053 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T23:08:42.053 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.053 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.053 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.053 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-04-26T23:08:42.053 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.053 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.054 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.054 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T23:08:42.054 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.054 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.054 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.054 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-04-26T23:08:42.054 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-04-26T23:08:42.054 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.054 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.054 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.054 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.054 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.054 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.055 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.055 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-04-26T23:08:42.055 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.055 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.055 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.055 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-04-26T23:08:42.055 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-04-26T23:08:42.055 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.055 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.055 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.055 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.055 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.055 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.055 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.056 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T23:08:42.056 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.056 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.056 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.056 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-04-26T23:08:42.056 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-04-26T23:08:42.056 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.056 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.056 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.056 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.056 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.056 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.056 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.056 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-04-26T23:08:42.057 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.057 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.057 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.057 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-04-26T23:08:42.057 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-04-26T23:08:42.057 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.057 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.057 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.057 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-04-26T23:08:42.057 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.057 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.057 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.057 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-04-26T23:08:42.057 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-04-26T23:08:42.058 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.058 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T23:08:42.058 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.058 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.058 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.058 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-04-26T23:08:42.058 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-04-26T23:08:42.058 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.058 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.058 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.058 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-04-26T23:08:42.058 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.058 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.058 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.059 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T23:08:42.059 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.059 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.059 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.059 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-04-26T23:08:42.059 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-04-26T23:08:42.059 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.059 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.059 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.059 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T23:08:42.059 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.059 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.059 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.059 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T23:08:42.060 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.060 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.060 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.060 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-04-26T23:08:42.060 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-04-26T23:08:42.060 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.060 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.060 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.060 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.060 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.060 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.060 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.061 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T23:08:42.061 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.061 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.061 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.061 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-04-26T23:08:42.061 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-04-26T23:08:42.061 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.061 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.061 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.061 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.061 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.061 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.061 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.061 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T23:08:42.062 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.062 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.062 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.062 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-04-26T23:08:42.062 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-04-26T23:08:42.062 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.062 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.062 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.062 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.062 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.062 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.062 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.062 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T23:08:42.062 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.063 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.063 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.063 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-04-26T23:08:42.063 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-04-26T23:08:42.063 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.063 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.063 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.063 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.063 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.063 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.063 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.063 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T23:08:42.063 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.063 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.064 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.064 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-04-26T23:08:42.064 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-04-26T23:08:42.064 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.064 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.064 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.064 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.064 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.064 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.064 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.064 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T23:08:42.064 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.064 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.065 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.065 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-04-26T23:08:42.065 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-04-26T23:08:42.065 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.065 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.065 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.065 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-04-26T23:08:42.065 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.065 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.065 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.065 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-04-26T23:08:42.065 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.065 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.065 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.066 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-04-26T23:08:42.066 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-04-26T23:08:42.066 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.066 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.066 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.066 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.066 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.066 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.066 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.066 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T23:08:42.066 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.066 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.067 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.067 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-04-26T23:08:42.067 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-04-26T23:08:42.067 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.067 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.067 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.067 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.067 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.067 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.067 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.067 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T23:08:42.067 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.067 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.068 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.068 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-04-26T23:08:42.068 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-04-26T23:08:42.068 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.068 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.068 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.068 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.068 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.068 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.068 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.068 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T23:08:42.068 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.069 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.069 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.069 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-04-26T23:08:42.069 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-04-26T23:08:42.069 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.069 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.069 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.069 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.069 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.070 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.070 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.070 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T23:08:42.070 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.070 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.070 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.070 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-04-26T23:08:42.070 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-04-26T23:08:42.070 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.071 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.071 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.071 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.071 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.071 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.071 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.071 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T23:08:42.071 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.071 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.071 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.071 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-04-26T23:08:42.072 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-04-26T23:08:42.072 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.072 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.072 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.072 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.072 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.072 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.072 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.072 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T23:08:42.072 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.072 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.073 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.073 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.073 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.073 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-04-26T23:08:42.073 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.073 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.073 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.073 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.073 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.073 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.073 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.074 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.074 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.074 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.074 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.074 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.074 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.074 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.074 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.074 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.074 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.074 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.074 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.074 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.074 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.075 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.075 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.075 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.075 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.075 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.075 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.075 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.075 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.075 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.075 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.075 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.075 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.075 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.075 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.076 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.076 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.076 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.076 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.076 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.076 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.076 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.076 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.076 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.076 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.076 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.076 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.076 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.076 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.077 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.077 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.077 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.077 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.077 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.077 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.077 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.077 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.077 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.077 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.077 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.077 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.077 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.077 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.078 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.078 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.078 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.078 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.078 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.078 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.078 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-04-26T23:08:42.078 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-04-26T23:08:42.078 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.078 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.078 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.078 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-04-26T23:08:42.079 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.079 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.079 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.079 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T23:08:42.079 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.079 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.079 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.079 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-04-26T23:08:42.079 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-04-26T23:08:42.079 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.079 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.079 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.079 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-04-26T23:08:42.079 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.080 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.080 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.080 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T23:08:42.080 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.080 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.080 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.080 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.080 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.080 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-04-26T23:08:42.080 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.080 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.080 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.080 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T23:08:42.081 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T23:08:42.081 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.081 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.081 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.081 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.081 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.081 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.081 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.081 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.081 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.081 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.081 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.081 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T23:08:42.081 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T23:08:42.081 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.082 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.082 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.082 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.082 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.082 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.082 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.082 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.082 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.082 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.082 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.082 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T23:08:42.082 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T23:08:42.082 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.083 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.083 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.083 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.083 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.083 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.083 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.083 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.083 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.083 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.083 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.083 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-04-26T23:08:42.083 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T23:08:42.083 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.083 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.084 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.084 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.084 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.084 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.084 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.084 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.084 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.084 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.084 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.084 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T23:08:42.084 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T23:08:42.084 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.084 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.085 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.085 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.085 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.085 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.085 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.085 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.085 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.085 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.085 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.085 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T23:08:42.085 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T23:08:42.085 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.085 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.085 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.085 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.086 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.086 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.086 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.086 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.086 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.086 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.086 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.086 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T23:08:42.086 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T23:08:42.086 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.086 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.086 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.087 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.087 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.087 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.087 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.087 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.087 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.087 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.087 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.087 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T23:08:42.087 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T23:08:42.087 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.087 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.087 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.088 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.088 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.088 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.088 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.088 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.088 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.088 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.088 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.088 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T23:08:42.088 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T23:08:42.088 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.088 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.088 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.088 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.089 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.089 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.089 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.089 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.089 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.089 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.089 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.089 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T23:08:42.089 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T23:08:42.089 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.089 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.089 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.089 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.089 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.090 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.090 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.090 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.090 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.090 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.090 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.090 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T23:08:42.090 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T23:08:42.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.090 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.090 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.090 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.091 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.091 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.091 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.091 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.091 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.091 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.091 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.091 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T23:08:42.091 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T23:08:42.091 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.091 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.091 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.091 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.091 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.092 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.092 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.092 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.092 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.092 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.092 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.092 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T23:08:42.092 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T23:08:42.092 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.092 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.092 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.092 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.092 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.092 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.093 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.093 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.093 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.093 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.093 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.093 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T23:08:42.093 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T23:08:42.093 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.093 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.093 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.093 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.093 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.093 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.093 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.094 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.094 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.094 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.094 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.094 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T23:08:42.094 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T23:08:42.094 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.094 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.094 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.094 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T23:08:42.094 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.094 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.094 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.094 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.095 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.095 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.095 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.095 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T23:08:42.095 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T23:08:42.095 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.095 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.095 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.095 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.095 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.095 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.095 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.095 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.095 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.096 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.096 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.096 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-04-26T23:08:42.096 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-04-26T23:08:42.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.096 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.096 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.096 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.096 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.096 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.096 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.096 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.096 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.096 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.097 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.097 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T23:08:42.097 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T23:08:42.097 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.097 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.097 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.097 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T23:08:42.097 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.097 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.097 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.097 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.097 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.097 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.097 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.098 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T23:08:42.098 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T23:08:42.098 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.098 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.098 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.098 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.098 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.098 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.098 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.098 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.098 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.098 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.098 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.099 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T23:08:42.099 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T23:08:42.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.099 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.099 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.099 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.099 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.099 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.099 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.099 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.099 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.099 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.100 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.100 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T23:08:42.100 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T23:08:42.100 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.100 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.100 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.100 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.100 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.100 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.100 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.100 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.100 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.100 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.100 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.101 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T23:08:42.101 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T23:08:42.101 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.101 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.101 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.101 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.101 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.101 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.101 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.101 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.101 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.101 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.102 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.102 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T23:08:42.102 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T23:08:42.102 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.102 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.102 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.102 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-04-26T23:08:42.102 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.102 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.102 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.102 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.102 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.102 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.103 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.103 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T23:08:42.103 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T23:08:42.103 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.103 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.103 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.103 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-04-26T23:08:42.103 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.103 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.103 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.103 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.103 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.104 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.104 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.104 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T23:08:42.104 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T23:08:42.104 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.104 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.104 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.104 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.104 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.104 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.104 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.104 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.104 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.105 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.105 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.105 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T23:08:42.105 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T23:08:42.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.105 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.105 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.105 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.105 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.105 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.105 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.105 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.105 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.105 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.106 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.106 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T23:08:42.106 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T23:08:42.106 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.106 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.106 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.106 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.106 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.106 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.106 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.106 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.106 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.106 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.106 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.107 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T23:08:42.107 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T23:08:42.107 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.107 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.107 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.107 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.107 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.107 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.107 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.107 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.107 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.107 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.107 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.108 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T23:08:42.108 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T23:08:42.108 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.108 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.108 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.108 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.108 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.108 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.108 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.108 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.108 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.109 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.109 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.109 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T23:08:42.109 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T23:08:42.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.109 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.109 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.109 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.109 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.109 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.110 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.110 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.110 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.110 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.110 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.110 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T23:08:42.110 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T23:08:42.110 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.110 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.110 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.110 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.110 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.110 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.110 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.111 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.111 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.111 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.111 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.111 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T23:08:42.111 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T23:08:42.111 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.111 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.111 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.111 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.111 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.111 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.111 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.111 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.112 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.112 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.112 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.112 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T23:08:42.112 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T23:08:42.112 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.112 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.112 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.112 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.112 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.112 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.113 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.113 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.113 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.113 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.113 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.113 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T23:08:42.113 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T23:08:42.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.113 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.113 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.113 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.113 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.113 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.114 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.114 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.114 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.114 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.114 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T23:08:42.114 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T23:08:42.114 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.114 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.114 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.114 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T23:08:42.114 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.114 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.114 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.114 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.115 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.115 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.115 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.115 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T23:08:42.115 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T23:08:42.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.115 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.115 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.115 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.115 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.115 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.115 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.115 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.116 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.116 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.116 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.116 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T23:08:42.116 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T23:08:42.116 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.116 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.116 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.116 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-04-26T23:08:42.116 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.116 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.116 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.116 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.117 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.117 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.117 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.117 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T23:08:42.117 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T23:08:42.117 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.117 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.117 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.117 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-04-26T23:08:42.117 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.117 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.117 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.117 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.117 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.118 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.118 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.118 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T23:08:42.118 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T23:08:42.118 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.118 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.118 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.118 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.118 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.118 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.118 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.118 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.118 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.119 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.119 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.119 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T23:08:42.119 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T23:08:42.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.119 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.119 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.119 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T23:08:42.119 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.119 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.119 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.119 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.119 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.120 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.120 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.120 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T23:08:42.120 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T23:08:42.120 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.120 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.120 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.120 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.120 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.120 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.120 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.120 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.120 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.120 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.120 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.121 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T23:08:42.121 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T23:08:42.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.121 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.121 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.121 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.121 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.121 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.121 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.121 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.121 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.121 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.121 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.122 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T23:08:42.122 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T23:08:42.122 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.122 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.122 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.122 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.122 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.122 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.122 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.122 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.122 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.122 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.122 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.123 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T23:08:42.123 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T23:08:42.123 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.123 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.123 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.123 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T23:08:42.123 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.123 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.123 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.123 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.123 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.124 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.124 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.124 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T23:08:42.124 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T23:08:42.124 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.124 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.124 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.124 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.124 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.124 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.124 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.125 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.125 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.125 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.125 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.125 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T23:08:42.125 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T23:08:42.125 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.125 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.126 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.126 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T23:08:42.126 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.126 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.126 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.126 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.126 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.126 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.126 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.126 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-04-26T23:08:42.127 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-04-26T23:08:42.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.127 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.127 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.127 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.127 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.127 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.127 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.127 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.127 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.127 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.127 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.128 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-04-26T23:08:42.128 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-04-26T23:08:42.128 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.128 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.128 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.128 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.128 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.128 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.128 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.129 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.129 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.129 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.129 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.129 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-04-26T23:08:42.129 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-04-26T23:08:42.129 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.129 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.129 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.129 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.129 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.130 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.130 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.130 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-04-26T23:08:42.130 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.130 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.130 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.130 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-04-26T23:08:42.130 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-04-26T23:08:42.130 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.130 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.131 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.131 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-04-26T23:08:42.131 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.131 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.131 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.131 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.131 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.131 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.131 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.132 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-04-26T23:08:42.132 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-04-26T23:08:42.132 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.132 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.132 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.132 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.132 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.132 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.132 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.132 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.133 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.133 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.133 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.133 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.133 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.133 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.133 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.133 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.133 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.133 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.134 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.134 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.134 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.134 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.134 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.134 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.134 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.134 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.135 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.135 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.135 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.135 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.135 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.135 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.135 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.135 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.135 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.135 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.136 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.136 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.136 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.136 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.136 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.136 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.136 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.136 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.136 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.136 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.137 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.137 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.137 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.137 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.137 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.137 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.137 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.137 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.137 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.137 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.137 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.137 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.138 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.138 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.138 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.138 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.138 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.138 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.138 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.138 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.138 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.139 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.139 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.139 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.139 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.139 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.139 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.139 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.139 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.139 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.139 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.139 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-04-26T23:08:42.139 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-04-26T23:08:42.140 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.140 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.140 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.140 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.140 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.140 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.140 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.140 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-04-26T23:08:42.140 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.140 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.140 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.140 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-04-26T23:08:42.141 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T23:08:42.141 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.141 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.141 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.141 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.141 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.141 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.141 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.141 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.141 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.141 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.141 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.141 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T23:08:42.142 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T23:08:42.142 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.142 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.142 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.142 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-04-26T23:08:42.142 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.142 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.142 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.142 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.142 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.142 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.142 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.143 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T23:08:42.143 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T23:08:42.143 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.143 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.143 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.143 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-04-26T23:08:42.143 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.143 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.143 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.143 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.143 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.143 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.143 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.144 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-04-26T23:08:42.144 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-04-26T23:08:42.144 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.144 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.144 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.144 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.144 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.144 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.144 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.144 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.144 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.144 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.144 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.144 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-04-26T23:08:42.144 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-04-26T23:08:42.145 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.145 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.145 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.145 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-04-26T23:08:42.145 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.145 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.145 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.145 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.145 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.145 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.145 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.145 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-04-26T23:08:42.145 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-04-26T23:08:42.146 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.146 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.146 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.146 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-04-26T23:08:42.146 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.146 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.146 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.146 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.146 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-04-26T23:08:42.146 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.146 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.146 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.146 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.146 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-04-26T23:08:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-04-26T23:08:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-04-26T23:08:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-04-26T23:08:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-04-26T23:08:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-04-26T23:08:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-04-26T23:08:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.149 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.149 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-04-26T23:08:42.149 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.149 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.149 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.149 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-04-26T23:08:42.149 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-04-26T23:08:42.149 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.149 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.149 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.149 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.149 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.149 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.149 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.150 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-04-26T23:08:42.150 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.150 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.150 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.150 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.150 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.150 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.150 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.150 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.150 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.150 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.150 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.150 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.151 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.151 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.151 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.151 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.151 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.151 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.151 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.151 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.151 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.151 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.151 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.151 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.151 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.151 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.152 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.152 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.152 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.152 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.152 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.152 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.152 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.152 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.152 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.152 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.152 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.152 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.152 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.152 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.153 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.153 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.153 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.153 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.153 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.153 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.153 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.153 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.153 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.153 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.153 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.153 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.153 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.153 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.153 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.154 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.154 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.154 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.154 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.154 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.154 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.154 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.154 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.154 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.154 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.154 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.154 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.154 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.154 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.155 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-04-26T23:08:42.155 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-04-26T23:08:42.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.155 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.155 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.155 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-04-26T23:08:42.155 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.155 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.155 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.155 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T23:08:42.155 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.155 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.155 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-04-26T23:08:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-04-26T23:08:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-04-26T23:08:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T23:08:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-04-26T23:08:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-04-26T23:08:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-04-26T23:08:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-04-26T23:08:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-04-26T23:08:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-04-26T23:08:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-04-26T23:08:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T23:08:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-04-26T23:08:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-04-26T23:08:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T23:08:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T23:08:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T23:08:42.160 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.160 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.160 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.160 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T23:08:42.160 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.160 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.160 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.160 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T23:08:42.160 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.160 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.160 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.160 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-04-26T23:08:42.160 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-04-26T23:08:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-04-26T23:08:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T23:08:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.162 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-04-26T23:08:42.162 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.162 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.162 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.162 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.162 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.162 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.162 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.162 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.162 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.162 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.162 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.162 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.162 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.163 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.163 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.163 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.163 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.163 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.163 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.163 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.163 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.163 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.163 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.163 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.163 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.163 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.163 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.164 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.164 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.164 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.164 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.164 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.164 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.164 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.164 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.164 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.164 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.164 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.164 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.164 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.164 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.165 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.165 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.165 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.165 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.165 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.165 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.165 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.165 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.165 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.165 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.165 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.165 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.165 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.165 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.165 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.166 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.166 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.166 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.166 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.166 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.166 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.166 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.166 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.166 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.166 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.166 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.166 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.167 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-04-26T23:08:42.167 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-04-26T23:08:42.167 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.167 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.167 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.167 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-04-26T23:08:42.167 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.167 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.167 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.167 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.167 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.167 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.167 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.167 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-04-26T23:08:42.168 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-04-26T23:08:42.168 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.168 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.168 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.168 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-04-26T23:08:42.168 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.168 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.168 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.168 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.168 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.168 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.168 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.169 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T23:08:42.169 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T23:08:42.169 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.169 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.169 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.169 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T23:08:42.169 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.169 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.169 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.169 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.169 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.169 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.169 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.169 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.169 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.170 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-04-26T23:08:42.170 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-04-26T23:08:42.170 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-04-26T23:08:42.170 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.170 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-04-26T23:08:42.170 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-04-26T23:08:42.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.170 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.170 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.170 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-04-26T23:08:42.170 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.170 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.171 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.171 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T23:08:42.171 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.171 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.171 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.171 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-04-26T23:08:42.171 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-04-26T23:08:42.171 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.171 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.171 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.171 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-04-26T23:08:42.171 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.171 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.171 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.172 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T23:08:42.172 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.172 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.172 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.172 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-04-26T23:08:42.172 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-04-26T23:08:42.172 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.172 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.172 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.172 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.172 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.172 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.172 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.172 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-04-26T23:08:42.173 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.173 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.173 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.173 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T23:08:42.173 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T23:08:42.173 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.173 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.173 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.173 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-04-26T23:08:42.173 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-04-26T23:08:42.173 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.173 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.173 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T23:08:42.174 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.174 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.174 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.174 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.174 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.174 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.174 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.174 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.174 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.174 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.174 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.174 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.175 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.175 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.175 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.175 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.175 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.175 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.175 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.175 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.175 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.175 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.175 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.175 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.175 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.176 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.176 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.176 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.176 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.176 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.176 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.176 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.176 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.176 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.176 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.176 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.176 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.176 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.176 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.177 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.177 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.177 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.177 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.177 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.177 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.177 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.177 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.177 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.177 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.177 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.177 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.177 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.177 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.177 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.178 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.178 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.178 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.178 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.178 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.178 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.178 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.178 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.178 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.178 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.178 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.178 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.178 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.179 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.179 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-04-26T23:08:42.179 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-04-26T23:08:42.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.179 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.179 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.179 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.179 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.179 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.179 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.179 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-04-26T23:08:42.179 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.179 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.179 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.180 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-04-26T23:08:42.180 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-04-26T23:08:42.180 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.180 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.180 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.180 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-04-26T23:08:42.180 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.180 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.180 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.180 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-04-26T23:08:42.180 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.180 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.180 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.181 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-04-26T23:08:42.181 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-04-26T23:08:42.181 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.181 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.181 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.181 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-04-26T23:08:42.181 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.181 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.181 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.181 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T23:08:42.181 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.181 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.181 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.182 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-04-26T23:08:42.182 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-04-26T23:08:42.182 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.182 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.182 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.182 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T23:08:42.182 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T23:08:42.182 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-04-26T23:08:42.182 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.182 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T23:08:42.182 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.182 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.182 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.183 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-04-26T23:08:42.183 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-04-26T23:08:42.183 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.183 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.183 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.183 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.183 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.183 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.183 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.183 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-04-26T23:08:42.183 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.183 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.183 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.184 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-04-26T23:08:42.184 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-04-26T23:08:42.184 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.184 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.184 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.184 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-04-26T23:08:42.184 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.184 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.184 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.184 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T23:08:42.184 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.184 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.185 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.185 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.185 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.185 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-04-26T23:08:42.185 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.185 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.185 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.185 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.185 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.185 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.186 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.186 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.186 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.186 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.186 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.186 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.186 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.186 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.186 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.186 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.186 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.186 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.186 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.187 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.187 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.187 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.187 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.187 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.187 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.187 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.187 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.187 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.187 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.187 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.187 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.187 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.187 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.188 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.188 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.188 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.188 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.188 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.188 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.188 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.188 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.188 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.188 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.188 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.188 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.188 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.188 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.189 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.189 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.189 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.189 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.189 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.189 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.189 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.189 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.189 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.189 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.189 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.189 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.190 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.190 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.190 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.190 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.190 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.190 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.190 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.190 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.190 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.190 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.190 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.190 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.191 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-04-26T23:08:42.191 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.191 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.191 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.191 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.191 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.191 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.191 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.191 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.191 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.191 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.191 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.191 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.191 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.192 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.192 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.192 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.192 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.192 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.192 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.192 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.192 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.192 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.192 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.192 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.192 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.192 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.193 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.193 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.193 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.193 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.193 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.193 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.193 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.193 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.193 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.193 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.193 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.193 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.194 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.194 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.194 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.194 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.194 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.194 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.194 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.194 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.194 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.194 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.194 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.194 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.194 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.195 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.195 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.195 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.195 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.195 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.195 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.195 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.195 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.195 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.195 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.196 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.196 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.196 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.196 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.196 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.196 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.196 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.196 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.196 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.197 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-04-26T23:08:42.197 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.197 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.197 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.197 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-04-26T23:08:42.197 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-04-26T23:08:42.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.197 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.197 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.197 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-04-26T23:08:42.197 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.197 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.197 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.198 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T23:08:42.198 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.198 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.198 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.198 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-04-26T23:08:42.198 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-04-26T23:08:42.198 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.198 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.198 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.198 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.198 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-04-26T23:08:42.198 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.198 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.199 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T23:08:42.199 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.199 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.199 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.199 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.199 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.199 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.199 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.199 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.199 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.199 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.199 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.199 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.200 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.200 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.200 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.200 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.200 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.200 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.200 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.200 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.200 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.200 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.200 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.200 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.200 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.201 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.201 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.201 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.201 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.201 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.201 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.201 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.201 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.201 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.201 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.201 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.201 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.201 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.203 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.203 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.203 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.203 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.203 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.203 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.203 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.203 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.203 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.203 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.203 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.203 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.204 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.204 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.204 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.204 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.204 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-04-26T23:08:42.204 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.204 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.204 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.204 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-04-26T23:08:42.204 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-04-26T23:08:42.204 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.204 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.204 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.205 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-04-26T23:08:42.205 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.205 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.205 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.205 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-04-26T23:08:42.205 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T23:08:42.205 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.205 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.205 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.205 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.205 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.206 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.206 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.206 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.206 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.206 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.206 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.206 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.206 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.206 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.206 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.207 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.207 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.207 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.207 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.207 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.207 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.207 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.207 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.207 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.207 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.207 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.207 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.207 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.208 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.208 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.208 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.208 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.208 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.208 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.208 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.208 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.208 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.208 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.208 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.208 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.209 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.209 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.209 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.209 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.209 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.209 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.209 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.209 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.209 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.209 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.209 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.209 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.209 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.210 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.210 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.210 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.210 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.210 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.210 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.210 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.210 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.210 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.210 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.210 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.211 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.211 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.211 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-04-26T23:08:42.211 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-04-26T23:08:42.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.211 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.211 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.211 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.211 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.211 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.211 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.211 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-04-26T23:08:42.212 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T23:08:42.212 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.212 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.212 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-04-26T23:08:42.212 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-04-26T23:08:42.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.212 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.212 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.212 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-04-26T23:08:42.212 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.212 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.212 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.212 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-04-26T23:08:42.213 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.213 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.213 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.213 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-04-26T23:08:42.213 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-04-26T23:08:42.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.213 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.213 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.213 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-04-26T23:08:42.213 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.214 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.214 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.214 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T23:08:42.214 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.214 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.214 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.214 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-04-26T23:08:42.214 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-04-26T23:08:42.214 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.214 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.215 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.215 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.215 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.215 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.215 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.215 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-04-26T23:08:42.215 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.215 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.215 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.215 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-04-26T23:08:42.215 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-04-26T23:08:42.216 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.216 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.216 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.216 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-04-26T23:08:42.216 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.216 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.216 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.216 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T23:08:42.216 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T23:08:42.216 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.216 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.216 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.216 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.217 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-04-26T23:08:42.217 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.217 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.217 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.217 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.217 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.217 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.217 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.217 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.217 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.217 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.217 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.217 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.217 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.217 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.218 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.218 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.218 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.218 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.218 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.218 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.218 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.218 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.218 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.218 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.218 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.218 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.218 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.219 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.219 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.219 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.219 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.219 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.219 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.219 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.219 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.219 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.219 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.219 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.219 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.219 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.220 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.220 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.220 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.220 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.220 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.220 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.220 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.220 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.220 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.220 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.220 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.220 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.220 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.220 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.221 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.221 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.221 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.221 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.221 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.221 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.221 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.221 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.221 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.221 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.221 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.221 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.221 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.221 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.221 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.222 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-04-26T23:08:42.222 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.222 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.222 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.222 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.222 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.222 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.222 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.222 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.222 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.222 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.223 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.223 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.223 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.223 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.223 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.223 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.223 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.223 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.223 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.223 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.223 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.223 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.223 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.224 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.224 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.224 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.224 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.224 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.224 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.224 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.224 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.224 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.224 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.224 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.224 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.225 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.225 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.225 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.225 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.225 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.225 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.225 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.225 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.225 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.225 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.225 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.225 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.225 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.225 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.226 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.226 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.226 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.226 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.226 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.226 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.226 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.226 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.226 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.226 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.226 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.226 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.226 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.226 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.227 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.227 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.227 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.227 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.227 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.227 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.227 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-04-26T23:08:42.227 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.227 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.227 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.227 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.227 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.227 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.228 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.228 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.228 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.228 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.228 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.228 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.228 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.228 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.228 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.228 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.228 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.228 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.228 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.228 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.228 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.229 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.229 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.229 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.229 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.229 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.229 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.229 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.229 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.229 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.229 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.229 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.229 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.229 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.230 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.230 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.230 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.230 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.230 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.230 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.230 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.230 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.230 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.230 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.230 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.230 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.231 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.231 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.231 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.231 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.231 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.231 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.231 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.231 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.231 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.232 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.232 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.232 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.232 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.232 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.232 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.232 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.232 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.232 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.232 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.232 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.232 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.233 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-04-26T23:08:42.233 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.233 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.233 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.233 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-04-26T23:08:42.233 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-04-26T23:08:42.233 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.234 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.234 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.234 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.234 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.234 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.234 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.235 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T23:08:42.235 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.235 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.235 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.235 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.235 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.235 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.235 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.235 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.235 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.236 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.236 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.236 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.236 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.236 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.236 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.236 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.236 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.236 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.236 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.236 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.237 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.237 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.237 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.237 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.237 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.237 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.237 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.237 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.237 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.237 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.238 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.238 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.238 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.238 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.238 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.238 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.238 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.238 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.238 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.239 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.239 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.239 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.239 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.239 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.239 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.239 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.239 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.239 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.239 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.239 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.240 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.240 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.240 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.240 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.240 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.240 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.240 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.240 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.240 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.240 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.240 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.241 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.241 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.241 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.241 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.241 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.241 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.241 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-04-26T23:08:42.241 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-04-26T23:08:42.241 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.241 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.241 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.241 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.241 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.242 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.242 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.242 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-04-26T23:08:42.242 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-04-26T23:08:42.242 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-04-26T23:08:42.242 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.242 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-04-26T23:08:42.242 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.242 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.242 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.242 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.242 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.243 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-04-26T23:08:42.243 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.243 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.243 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.243 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.243 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.243 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.243 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.243 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.243 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.243 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.243 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.244 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.244 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.244 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.244 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.244 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.244 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.244 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.244 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.244 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.244 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.244 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.245 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.245 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.245 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.245 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.245 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.245 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.245 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.245 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.245 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.246 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.246 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.246 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.246 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.246 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.246 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.246 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.246 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.246 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.246 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.246 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.246 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.246 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.247 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.247 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.247 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.247 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.247 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.247 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.247 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.247 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.247 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.247 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.247 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.247 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.247 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.247 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-04-26T23:08:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.251 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.251 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.251 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.251 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.251 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.251 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.251 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.251 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.251 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.251 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.251 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.251 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.251 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.251 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.252 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.252 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.252 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.252 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.252 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.252 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.252 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.252 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.252 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.252 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.252 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.252 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.252 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.252 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.253 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.253 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.253 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.253 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.253 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.253 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.253 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.253 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.253 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.253 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.253 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.254 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-04-26T23:08:42.254 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.254 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.254 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.254 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.254 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.254 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.254 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.254 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.254 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.254 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.254 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.255 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.255 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.255 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.255 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.255 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.255 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.255 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.255 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.255 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.255 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.255 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.255 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.255 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.256 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.256 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.256 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.256 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.256 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.256 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.256 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.256 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.256 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.256 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.256 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.256 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.257 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.257 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.257 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.257 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.257 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.257 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.257 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.257 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.257 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.257 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.257 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.257 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.257 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.257 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.257 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.258 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.258 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.258 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.258 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.258 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.258 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.258 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.258 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.258 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.258 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.258 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.258 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.258 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.259 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.259 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T23:08:42.259 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T23:08:42.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.259 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.259 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.259 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T23:08:42.259 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.259 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.259 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.259 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.259 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.259 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.259 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.260 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-04-26T23:08:42.260 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-04-26T23:08:42.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.260 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.260 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-04-26T23:08:42.260 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-04-26T23:08:42.260 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.260 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.260 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-04-26T23:08:42.260 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T23:08:42.260 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.260 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.261 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.261 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-04-26T23:08:42.261 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.261 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.261 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.261 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-04-26T23:08:42.261 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-04-26T23:08:42.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.261 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.261 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.261 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.261 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.261 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.262 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T23:08:42.262 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.262 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.262 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.262 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-04-26T23:08:42.262 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-04-26T23:08:42.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.262 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.262 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.262 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.262 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.262 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.262 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.262 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.263 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.263 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.263 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.263 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.263 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.263 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.263 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.263 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.263 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.263 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.263 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.264 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.264 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.264 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.264 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.264 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.264 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.264 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.264 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.264 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.264 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.264 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.265 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.265 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.265 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.265 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.265 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.265 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.265 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.265 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.265 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.265 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.266 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.266 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.266 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.266 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.266 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.266 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.266 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.266 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.266 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.267 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.267 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.267 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.267 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.267 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.267 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.267 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.267 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.267 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.267 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.267 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.268 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.268 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.268 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.268 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.268 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.268 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.268 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.268 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.269 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.269 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.269 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.269 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.269 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-04-26T23:08:42.269 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-04-26T23:08:42.269 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.269 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.269 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.269 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-04-26T23:08:42.269 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.270 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.270 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.270 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-04-26T23:08:42.270 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.270 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.270 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.270 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T23:08:42.270 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T23:08:42.270 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.270 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.271 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.271 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T23:08:42.271 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.271 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.271 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.271 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-04-26T23:08:42.271 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.271 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.271 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.271 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.271 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.271 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-04-26T23:08:42.271 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.272 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.272 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.272 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-04-26T23:08:42.272 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-04-26T23:08:42.272 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.272 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.272 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.272 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.272 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.272 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.272 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.272 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.272 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.273 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.273 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-04-26T23:08:42.273 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-04-26T23:08:42.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.273 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.273 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.273 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.273 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.273 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.273 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T23:08:42.273 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T23:08:42.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.274 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.274 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.274 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.274 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.274 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.274 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.274 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.274 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.274 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.275 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.275 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.275 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.275 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.275 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.275 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.275 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.275 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.275 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.275 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.275 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.276 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.276 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.276 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.276 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.276 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.276 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.276 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.276 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.276 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.276 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.276 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.277 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.277 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.277 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.277 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.277 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.277 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.277 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.277 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.278 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.278 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.278 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.278 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.278 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.278 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.278 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.278 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.278 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.278 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.278 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.279 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.279 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.279 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.279 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.279 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.279 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.279 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.279 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.279 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.279 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.279 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.279 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-04-26T23:08:42.280 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-04-26T23:08:42.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.280 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.280 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.280 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.280 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.280 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.280 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.280 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.280 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.280 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.280 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.280 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T23:08:42.280 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T23:08:42.281 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.281 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.281 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.281 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-04-26T23:08:42.281 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.281 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.281 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.281 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.281 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.281 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.282 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.282 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-04-26T23:08:42.282 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-04-26T23:08:42.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.282 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.282 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.282 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-04-26T23:08:42.283 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.283 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.283 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.283 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.283 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.283 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.283 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T23:08:42.283 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T23:08:42.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.283 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.284 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-04-26T23:08:42.284 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.284 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.284 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.284 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T23:08:42.284 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.284 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.284 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.284 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T23:08:42.284 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T23:08:42.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.284 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-04-26T23:08:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T23:08:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-04-26T23:08:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-04-26T23:08:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.286 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-04-26T23:08:42.286 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.286 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.286 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.286 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.286 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.286 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.286 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.286 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-04-26T23:08:42.286 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-04-26T23:08:42.286 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.286 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.286 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.287 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-04-26T23:08:42.287 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.287 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.287 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.287 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-04-26T23:08:42.287 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-04-26T23:08:42.287 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.287 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.287 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.287 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.287 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.287 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-04-26T23:08:42.287 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-04-26T23:08:42.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.288 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.288 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.288 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-04-26T23:08:42.288 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-04-26T23:08:42.288 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-04-26T23:08:42.288 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.288 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.288 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.288 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.288 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.289 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-04-26T23:08:42.289 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.289 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.289 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.289 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.289 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.289 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.289 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.289 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.289 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.289 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.289 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.289 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.290 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.290 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.290 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.290 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.290 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.290 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.290 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.290 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.290 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.290 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.290 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.290 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.290 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.290 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.291 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.291 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.291 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.291 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.291 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.291 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.291 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.291 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.291 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.291 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.291 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.291 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-04-26T23:08:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-04-26T23:08:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.295 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-04-26T23:08:42.295 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T23:08:42.295 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.295 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.295 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.295 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.295 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.295 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.295 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.295 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.295 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.295 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.295 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.296 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-04-26T23:08:42.296 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-04-26T23:08:42.296 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.296 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.296 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.296 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.296 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.296 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.296 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.296 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.297 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.297 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.297 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.297 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.297 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.297 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-04-26T23:08:42.297 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.297 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.297 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.297 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-04-26T23:08:42.297 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-04-26T23:08:42.297 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.297 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.298 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.298 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.298 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.298 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.298 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.298 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.298 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.298 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.298 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.298 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-04-26T23:08:42.299 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-04-26T23:08:42.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.299 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.299 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.299 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.299 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.299 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.299 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.299 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.299 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.299 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.300 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.300 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.300 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.300 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.300 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.300 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.300 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.300 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.300 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.300 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.300 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.300 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.300 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.300 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.301 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.301 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.301 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.301 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.301 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.301 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.301 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.301 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.301 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.301 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.301 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.301 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.302 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.302 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.302 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.302 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.302 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.302 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.302 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.302 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.302 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.302 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.302 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.302 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.302 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.303 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.303 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.303 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.303 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.303 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.303 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.303 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.303 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.303 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.303 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.303 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.303 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.303 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.303 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.303 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.304 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.304 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.304 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.304 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.304 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.304 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.304 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.304 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.304 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.304 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.304 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.304 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T23:08:42.304 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T23:08:42.304 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.305 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.305 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.305 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.305 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.305 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.305 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.305 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.305 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.305 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.305 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.305 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T23:08:42.305 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T23:08:42.305 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.305 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.306 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.306 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.306 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.306 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.306 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.306 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.306 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.306 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.306 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.306 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.306 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.306 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-04-26T23:08:42.307 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.307 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.307 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.307 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.307 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.307 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.307 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.307 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.307 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.307 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.307 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.307 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.307 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.307 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.307 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.307 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.308 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.308 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.308 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.308 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.308 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.308 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.308 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.308 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.308 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.308 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.308 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.308 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.308 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.309 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.309 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.309 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.309 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.309 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.309 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.309 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.309 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.309 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.309 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.309 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.309 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.309 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.309 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.310 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.310 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.310 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.310 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.310 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.310 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.310 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.310 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.310 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.310 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.310 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.310 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.311 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.311 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.311 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.311 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.311 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.311 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.311 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.311 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.311 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.311 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.311 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.311 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.311 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.312 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.312 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-04-26T23:08:42.312 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.312 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.312 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.312 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.312 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.312 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.312 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.312 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.312 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.312 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.312 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.313 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.313 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.313 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.313 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.313 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.313 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.313 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.313 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.313 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.313 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.313 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.313 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.313 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.313 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.314 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.314 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.314 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.314 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.314 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.314 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.314 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.314 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.314 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.314 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.314 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.314 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.314 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.314 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.315 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.315 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.315 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.315 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.315 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.315 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.315 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.315 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.315 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.315 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.315 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.315 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.315 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.316 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.316 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.316 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.316 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.316 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.316 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.316 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.316 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.316 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.316 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.316 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.316 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.316 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.316 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.317 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.317 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.317 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.318 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-04-26T23:08:42.318 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-04-26T23:08:42.318 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.318 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.318 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.319 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-04-26T23:08:42.319 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.319 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.319 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.319 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T23:08:42.319 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.319 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.319 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.319 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.319 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.320 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-04-26T23:08:42.320 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.320 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.320 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.320 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.320 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.320 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.320 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.320 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.320 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.320 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.321 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.321 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.321 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.321 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.321 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.321 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.321 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.322 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.322 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.322 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.322 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.322 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.322 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.322 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.322 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.323 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.323 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.323 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.323 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.323 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.323 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.323 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.323 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.323 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.324 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.324 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.324 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.324 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.324 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.324 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.324 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.324 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.324 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.324 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.325 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.325 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.325 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.325 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.325 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.325 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.325 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.325 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.326 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.326 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.326 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.326 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.326 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.326 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.326 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.326 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.326 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.327 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.327 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.327 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.327 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.327 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.327 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.327 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-04-26T23:08:42.327 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-04-26T23:08:42.328 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.328 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.328 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.328 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.328 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.328 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.328 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.329 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.329 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.329 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.329 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.329 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-04-26T23:08:42.329 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-04-26T23:08:42.329 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.329 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.329 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.329 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-04-26T23:08:42.329 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.330 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.330 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.330 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.330 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.330 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.330 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.330 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-04-26T23:08:42.330 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-04-26T23:08:42.330 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.330 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.331 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.331 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.331 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.331 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.331 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.331 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.331 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.331 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.331 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.331 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-04-26T23:08:42.331 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-04-26T23:08:42.331 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.331 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.332 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.332 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.332 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.332 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.332 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.332 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.332 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.332 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.332 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-04-26T23:08:42.332 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-04-26T23:08:42.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.332 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.333 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.333 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.333 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.333 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.333 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.333 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.333 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.333 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.333 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.333 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-04-26T23:08:42.333 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-04-26T23:08:42.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.333 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.334 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.334 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.334 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.334 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.334 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.334 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.334 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.334 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.334 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.334 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-04-26T23:08:42.334 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-04-26T23:08:42.334 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.335 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.335 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.335 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.335 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.335 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.335 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.335 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.335 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.335 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.335 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.335 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-04-26T23:08:42.335 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-04-26T23:08:42.335 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.335 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.336 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.336 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.336 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.336 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.336 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.336 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.336 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.336 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.336 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.336 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-04-26T23:08:42.336 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-04-26T23:08:42.336 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.336 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.336 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.337 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.337 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T23:08:42.337 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-04-26T23:08:42.337 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.337 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.337 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.337 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.337 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.337 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-04-26T23:08:42.337 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-04-26T23:08:42.337 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.337 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.337 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.338 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.338 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.338 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.338 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.338 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.338 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.338 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.338 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.338 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-04-26T23:08:42.338 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-04-26T23:08:42.338 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.338 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.338 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.338 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.339 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.339 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.339 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.339 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.339 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.339 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.339 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.339 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-04-26T23:08:42.339 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-04-26T23:08:42.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.339 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.339 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.339 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.340 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.340 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.340 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.340 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.340 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.340 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.340 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.340 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.340 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-04-26T23:08:42.340 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.340 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.340 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.340 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-04-26T23:08:42.341 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-04-26T23:08:42.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.341 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.341 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.341 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.341 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.341 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.341 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T23:08:42.341 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.341 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.341 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.342 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-04-26T23:08:42.342 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-04-26T23:08:42.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.342 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.342 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.342 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.342 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.342 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.342 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-04-26T23:08:42.343 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.343 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.343 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.343 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.343 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.343 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.343 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.343 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.344 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.344 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.344 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.344 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.344 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.344 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.344 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.344 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.344 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.345 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.345 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.345 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.345 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.345 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.345 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.345 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.345 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.345 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.346 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.346 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.346 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.346 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.346 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.346 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.346 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.346 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.346 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.347 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.347 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.347 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.347 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.347 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.347 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.347 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.347 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.347 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.347 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.348 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.348 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.348 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.348 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.348 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.348 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.348 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.348 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.348 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.348 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.349 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.349 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.349 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.349 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.349 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.349 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.349 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.349 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.349 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.349 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.349 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.350 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.350 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.350 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-04-26T23:08:42.350 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.350 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.350 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.350 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.350 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.351 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.351 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.351 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.351 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.351 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.351 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.351 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.351 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.351 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.351 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.351 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.352 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.352 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.352 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.352 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.352 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.352 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.352 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.352 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.352 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.352 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.353 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.353 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.353 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.353 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.353 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.353 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.353 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.353 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.353 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.354 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.354 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.354 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.354 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.354 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.354 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.354 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.354 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.354 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.354 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.355 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.355 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.355 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.355 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.355 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.355 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.355 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.355 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.355 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.355 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.356 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.356 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.356 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.356 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.356 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.356 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.356 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.356 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.356 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.356 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.356 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.356 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.357 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.357 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-04-26T23:08:42.357 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.357 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.357 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.357 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.358 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.358 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.358 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.358 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.358 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.358 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.358 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.358 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.358 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.358 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.358 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.359 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.359 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.359 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.359 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.359 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.359 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.359 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.359 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.359 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.359 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.360 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.360 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.360 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.360 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.360 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.360 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.360 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.360 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.360 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.360 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.360 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.361 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.361 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.361 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.361 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.361 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.361 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.361 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.361 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.361 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.361 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.362 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.362 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.362 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.362 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.362 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.362 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.362 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.362 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.362 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.362 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.362 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.362 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.362 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.363 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.363 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.363 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.363 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.363 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.363 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.363 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.363 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.363 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.363 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.363 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.363 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-04-26T23:08:42.364 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.364 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.364 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.364 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-04-26T23:08:42.364 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-04-26T23:08:42.364 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.364 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.364 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.364 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T23:08:42.364 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.364 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.364 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.364 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.364 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.365 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.365 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.365 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T23:08:42.365 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T23:08:42.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.365 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.365 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.365 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-04-26T23:08:42.365 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.365 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.365 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.365 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.365 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.366 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.366 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.366 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.366 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.366 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.366 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.366 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.366 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.366 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.366 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.366 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.366 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.366 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.367 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.367 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.367 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.367 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.367 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.367 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.367 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.367 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.367 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.367 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.367 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.368 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.368 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.368 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.368 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.368 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.368 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.368 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.368 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.368 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.368 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.368 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.368 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.369 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.369 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.369 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.369 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.369 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.369 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.369 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.369 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.369 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.369 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.369 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.369 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.369 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.370 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.370 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.370 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.370 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.370 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.370 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.370 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.370 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.370 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.370 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.371 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.371 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.371 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.371 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.371 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.371 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.371 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.371 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-04-26T23:08:42.371 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.371 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.371 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.371 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-04-26T23:08:42.372 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-04-26T23:08:42.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.372 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.372 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.372 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.372 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.372 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.372 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.372 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T23:08:42.372 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.372 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.372 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-04-26T23:08:42.373 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-04-26T23:08:42.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.373 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.373 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.373 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.373 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.373 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.373 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.373 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T23:08:42.373 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.373 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.373 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.373 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-04-26T23:08:42.373 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-04-26T23:08:42.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.374 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.374 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.374 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.374 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.374 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.374 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T23:08:42.374 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.374 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.374 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.374 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-04-26T23:08:42.374 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-04-26T23:08:42.375 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.375 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.375 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.375 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.375 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.375 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.375 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.375 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T23:08:42.375 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.375 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.375 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.376 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-04-26T23:08:42.376 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-04-26T23:08:42.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.376 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.376 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.376 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.376 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.376 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.376 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T23:08:42.376 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.376 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.376 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.377 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-04-26T23:08:42.377 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-04-26T23:08:42.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.377 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.377 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.377 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.377 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.377 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.377 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.377 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.377 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.377 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.377 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.378 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-04-26T23:08:42.378 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-04-26T23:08:42.378 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.378 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.378 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.378 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.378 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.378 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.378 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.378 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.379 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.379 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-04-26T23:08:42.379 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-04-26T23:08:42.379 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.379 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.379 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.379 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T23:08:42.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.380 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.380 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.380 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.380 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.380 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.380 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.380 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-04-26T23:08:42.380 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-04-26T23:08:42.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.381 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.381 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.381 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.381 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.381 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.381 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.381 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.381 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.381 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.381 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.382 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T23:08:42.382 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T23:08:42.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.382 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.382 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.382 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-04-26T23:08:42.382 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-04-26T23:08:42.382 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.382 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.383 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.383 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.383 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.383 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.383 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-04-26T23:08:42.383 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-04-26T23:08:42.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.383 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.383 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.383 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-04-26T23:08:42.384 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.384 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.384 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.384 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.384 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.384 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.384 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-04-26T23:08:42.384 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-04-26T23:08:42.385 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.385 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.385 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.385 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.385 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.385 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.385 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.385 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.385 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.385 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.385 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.386 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-04-26T23:08:42.386 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-04-26T23:08:42.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.386 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.386 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.386 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.386 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.386 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.386 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.387 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.387 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.387 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.387 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.387 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.387 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.387 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.387 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.387 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.387 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.388 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.388 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.388 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.388 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.388 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.388 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.388 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.388 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.388 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.388 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.388 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.389 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.389 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.389 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.389 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.389 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.389 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.389 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.390 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.390 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.390 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.390 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.390 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.390 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.390 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.390 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.390 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.391 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.391 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.391 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.391 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.391 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.391 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.391 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.391 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.391 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.391 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.391 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.392 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.392 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.392 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.392 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.392 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.392 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.392 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.392 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.393 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.393 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.393 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.393 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.393 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.393 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.393 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.393 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.393 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-04-26T23:08:42.394 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-04-26T23:08:42.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.394 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.394 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.394 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.394 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.394 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.394 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.394 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.394 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.394 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.395 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.395 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-04-26T23:08:42.395 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-04-26T23:08:42.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.395 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.395 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.395 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.395 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.395 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.395 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.396 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.396 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.396 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.396 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.396 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-04-26T23:08:42.396 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-04-26T23:08:42.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.396 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.396 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.396 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T23:08:42.397 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.397 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.397 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.397 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.397 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.397 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.397 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.397 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.397 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.398 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-04-26T23:08:42.398 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.398 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.398 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.398 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.398 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.398 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.398 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.398 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.399 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.399 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.399 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.399 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.399 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.399 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.399 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.399 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.399 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.399 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.399 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.399 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.399 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.400 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.400 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-04-26T23:08:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-04-26T23:08:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-04-26T23:08:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-04-26T23:08:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T23:08:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-04-26T23:08:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-04-26T23:08:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-04-26T23:08:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-04-26T23:08:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-04-26T23:08:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T23:08:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T23:08:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-04-26T23:08:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T23:08:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T23:08:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-04-26T23:08:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-04-26T23:08:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-04-26T23:08:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T23:08:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-04-26T23:08:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-04-26T23:08:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-04-26T23:08:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T23:08:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T23:08:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T23:08:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-04-26T23:08:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-04-26T23:08:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-04-26T23:08:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-04-26T23:08:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-04-26T23:08:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-04-26T23:08:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-04-26T23:08:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T23:08:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T23:08:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 25408, 2024-04-26T23:08:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "z", 2024-04-26T23:08:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_features": 4540701547738038271, 2024-04-26T23:08:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-04-26T23:08:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-04-26T23:08:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T23:08:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T23:08:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T23:08:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T23:08:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-04-26T23:08:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-04-26T23:08:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to", 2024-04-26T23:08:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-04-26T23:08:42.503 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-04-26T23:08:42.503 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.503 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.503 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.503 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-04-26T23:08:42.503 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.503 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.503 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.503 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-04-26T23:08:42.503 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.503 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.503 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.503 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-04-26T23:08:42.503 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-04-26T23:08:42.503 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.504 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.504 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.504 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.504 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.504 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.504 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.504 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-04-26T23:08:42.504 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.504 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.504 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.504 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-04-26T23:08:42.504 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-04-26T23:08:42.504 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.504 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.505 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.505 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.505 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.505 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.505 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.505 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-04-26T23:08:42.505 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.505 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.505 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.505 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-04-26T23:08:42.505 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-04-26T23:08:42.505 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.505 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.506 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.506 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-04-26T23:08:42.506 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.506 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.506 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.506 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-04-26T23:08:42.506 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.506 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.506 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.506 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-04-26T23:08:42.506 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-04-26T23:08:42.506 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.506 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.507 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.507 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.507 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.507 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.507 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.507 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-04-26T23:08:42.507 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.507 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.507 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.507 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-04-26T23:08:42.507 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-04-26T23:08:42.507 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.507 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.507 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.508 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.508 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.508 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.508 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.508 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T23:08:42.508 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.508 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.508 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.508 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-04-26T23:08:42.508 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-04-26T23:08:42.508 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.508 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.508 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.508 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.509 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.509 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.509 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.509 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-04-26T23:08:42.509 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.509 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.509 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.509 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.509 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.509 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-04-26T23:08:42.509 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.509 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.510 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.510 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-04-26T23:08:42.510 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-04-26T23:08:42.510 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.510 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.510 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.510 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.510 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.510 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.510 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.510 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T23:08:42.510 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.510 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.510 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.511 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-04-26T23:08:42.511 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-04-26T23:08:42.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.511 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.511 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.511 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-04-26T23:08:42.511 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.511 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.511 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.511 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T23:08:42.511 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T23:08:42.511 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.511 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.511 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-04-26T23:08:42.512 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-04-26T23:08:42.512 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T23:08:42.512 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.512 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.512 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T23:08:42.512 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-04-26T23:08:42.512 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-04-26T23:08:42.512 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.512 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T23:08:42.512 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T23:08:42.512 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.512 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.512 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-04-26T23:08:42.512 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-04-26T23:08:42.513 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T23:08:42.513 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.513 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.513 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-04-26T23:08:42.513 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T23:08:42.513 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-04-26T23:08:42.513 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.513 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T23:08:42.513 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.513 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.513 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.513 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-04-26T23:08:42.513 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-04-26T23:08:42.513 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.514 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.514 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.514 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T23:08:42.514 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.514 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.514 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.514 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T23:08:42.514 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T23:08:42.514 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.514 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.514 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-04-26T23:08:42.514 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-04-26T23:08:42.514 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.514 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.515 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.515 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-04-26T23:08:42.515 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-04-26T23:08:42.515 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-04-26T23:08:42.515 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.515 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-04-26T23:08:42.515 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T23:08:42.515 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.515 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.515 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-04-26T23:08:42.515 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-04-26T23:08:42.515 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.515 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.515 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.516 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-04-26T23:08:42.516 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.516 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.516 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.516 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T23:08:42.516 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T23:08:42.516 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.516 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.516 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-04-26T23:08:42.516 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-04-26T23:08:42.516 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T23:08:42.516 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.516 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.516 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T23:08:42.517 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-04-26T23:08:42.517 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-04-26T23:08:42.517 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.517 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T23:08:42.517 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T23:08:42.517 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.517 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.517 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.517 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.517 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.517 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.517 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.518 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.518 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.518 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.518 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.518 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.518 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.518 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.518 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.518 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.518 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.518 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.518 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.518 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.518 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.518 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.519 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.519 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.519 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.519 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.519 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.519 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.519 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.519 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.519 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.519 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.519 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.519 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.519 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.520 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.520 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.520 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.520 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.520 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.520 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.520 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.520 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.520 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.520 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.520 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.520 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.520 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.521 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.521 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.521 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.521 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.521 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.521 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.521 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.521 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.521 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.521 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.521 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.521 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.521 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.521 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.522 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.522 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.522 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.522 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.522 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.522 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.522 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-04-26T23:08:42.522 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-04-26T23:08:42.522 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.522 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.522 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.522 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T23:08:42.522 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.522 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.523 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.523 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T23:08:42.523 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.523 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.523 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.523 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-04-26T23:08:42.523 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-04-26T23:08:42.523 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.523 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.523 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.523 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-04-26T23:08:42.523 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.523 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.524 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.524 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-04-26T23:08:42.524 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-04-26T23:08:42.524 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-04-26T23:08:42.524 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-04-26T23:08:42.524 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-04-26T23:08:42.524 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.524 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-04-26T23:08:42.524 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.524 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.524 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.524 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-04-26T23:08:42.524 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-04-26T23:08:42.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.525 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.525 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.525 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.525 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.525 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.525 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.525 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T23:08:42.525 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.525 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.526 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.526 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T23:08:42.526 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T23:08:42.526 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.526 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.526 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.526 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T23:08:42.526 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.526 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.526 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.526 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T23:08:42.526 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.527 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.527 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.527 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-04-26T23:08:42.527 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-04-26T23:08:42.527 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.527 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.527 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.527 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T23:08:42.527 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T23:08:42.527 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.527 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.528 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T23:08:42.528 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T23:08:42.528 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.528 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.528 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-04-26T23:08:42.528 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-04-26T23:08:42.528 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T23:08:42.528 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.528 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.528 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.528 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.529 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.529 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.529 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T23:08:42.529 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.529 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.529 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.529 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.529 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.530 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-04-26T23:08:42.530 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.530 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.530 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.530 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-04-26T23:08:42.530 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-04-26T23:08:42.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.530 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.530 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.531 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-04-26T23:08:42.531 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.531 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.531 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.531 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T23:08:42.531 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.531 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.531 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.531 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-04-26T23:08:42.531 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-04-26T23:08:42.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.531 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.532 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.532 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-04-26T23:08:42.532 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.532 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.532 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.532 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T23:08:42.532 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.532 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.532 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.532 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-04-26T23:08:42.532 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-04-26T23:08:42.532 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.532 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.532 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.533 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-04-26T23:08:42.533 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.533 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.533 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.533 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T23:08:42.533 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.533 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.533 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.533 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-04-26T23:08:42.533 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-04-26T23:08:42.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.533 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.533 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.533 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.534 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.534 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.534 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.534 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T23:08:42.534 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T23:08:42.534 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.534 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.534 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-04-26T23:08:42.534 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-04-26T23:08:42.534 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.534 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.534 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T23:08:42.535 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.535 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.535 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.535 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T23:08:42.535 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.535 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.535 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.535 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-04-26T23:08:42.535 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T23:08:42.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.535 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.535 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.535 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-04-26T23:08:42.536 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.536 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.536 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.536 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T23:08:42.536 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.536 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.536 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.536 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-04-26T23:08:42.536 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-04-26T23:08:42.536 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.536 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.536 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.536 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.536 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.537 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.537 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.537 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-04-26T23:08:42.537 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-04-26T23:08:42.537 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-04-26T23:08:42.537 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.537 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T23:08:42.537 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.537 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.537 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.537 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-04-26T23:08:42.537 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-04-26T23:08:42.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.537 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.537 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.538 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.538 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.538 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.538 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.538 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T23:08:42.538 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.538 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.538 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.538 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-04-26T23:08:42.538 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-04-26T23:08:42.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.538 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.538 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.539 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.539 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.539 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.539 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.539 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T23:08:42.539 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.539 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.539 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-04-26T23:08:42.539 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-04-26T23:08:42.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.539 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.539 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.539 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.540 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.540 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.540 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.540 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T23:08:42.540 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.540 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.540 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.540 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-04-26T23:08:42.540 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-04-26T23:08:42.540 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.540 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.540 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.540 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T23:08:42.540 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.540 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.541 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.541 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T23:08:42.541 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.541 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.541 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.541 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-04-26T23:08:42.541 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-04-26T23:08:42.541 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.541 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.541 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.541 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T23:08:42.541 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.541 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.541 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.542 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-04-26T23:08:42.542 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.542 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.542 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.542 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-04-26T23:08:42.542 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-04-26T23:08:42.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.542 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.542 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.542 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T23:08:42.542 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.542 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.542 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.543 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-04-26T23:08:42.543 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.543 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.543 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.543 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-04-26T23:08:42.543 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-04-26T23:08:42.543 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.543 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.543 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.543 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T23:08:42.543 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.543 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.543 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.543 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T23:08:42.544 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.544 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.544 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.544 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-04-26T23:08:42.544 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-04-26T23:08:42.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.544 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.544 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.544 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T23:08:42.544 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.544 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.544 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.544 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-04-26T23:08:42.544 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.545 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.545 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.545 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-04-26T23:08:42.545 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-04-26T23:08:42.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.545 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.545 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.545 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T23:08:42.545 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.545 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.545 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.545 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-04-26T23:08:42.545 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.545 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.546 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.546 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-04-26T23:08:42.546 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-04-26T23:08:42.546 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.546 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.546 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.546 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T23:08:42.546 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.546 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.546 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.546 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-04-26T23:08:42.546 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.546 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.547 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.547 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-04-26T23:08:42.547 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-04-26T23:08:42.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.547 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.547 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.547 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T23:08:42.547 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.547 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.547 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.547 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-04-26T23:08:42.548 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.548 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.548 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.548 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-04-26T23:08:42.548 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-04-26T23:08:42.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.548 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.548 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.548 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T23:08:42.548 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.548 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.548 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.548 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-04-26T23:08:42.548 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.549 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.549 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.549 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-04-26T23:08:42.549 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-04-26T23:08:42.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.549 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.549 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.549 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T23:08:42.549 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.549 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.549 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.549 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-04-26T23:08:42.549 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.550 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.550 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.550 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-04-26T23:08:42.550 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-04-26T23:08:42.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.550 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.550 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.550 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T23:08:42.550 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.550 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.550 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.550 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T23:08:42.550 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.551 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.551 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.551 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-04-26T23:08:42.551 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-04-26T23:08:42.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.551 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.551 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.551 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T23:08:42.551 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.551 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.551 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.551 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-04-26T23:08:42.551 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.551 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.552 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.552 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-04-26T23:08:42.552 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-04-26T23:08:42.552 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.552 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.552 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.552 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T23:08:42.552 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.552 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.552 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.552 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T23:08:42.552 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.552 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.552 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.553 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-04-26T23:08:42.553 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-04-26T23:08:42.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.553 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.553 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.553 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T23:08:42.553 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.553 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.553 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.553 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-04-26T23:08:42.553 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.553 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.553 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.553 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-04-26T23:08:42.554 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-04-26T23:08:42.554 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.554 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.554 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.554 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T23:08:42.554 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.554 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.554 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.554 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-04-26T23:08:42.554 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.554 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.554 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.555 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-04-26T23:08:42.555 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-04-26T23:08:42.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.555 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.555 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.555 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T23:08:42.555 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.555 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.555 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.555 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-04-26T23:08:42.555 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.555 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.555 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.555 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-04-26T23:08:42.555 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-04-26T23:08:42.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.556 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.556 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.556 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.556 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.556 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.556 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.556 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T23:08:42.556 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.556 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.556 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.556 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-04-26T23:08:42.556 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-04-26T23:08:42.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.557 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.557 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.557 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T23:08:42.557 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.557 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.557 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.557 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T23:08:42.557 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.557 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.557 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.557 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-04-26T23:08:42.557 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T23:08:42.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.558 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.558 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.558 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-04-26T23:08:42.558 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.558 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.558 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.558 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T23:08:42.558 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.558 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.558 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.558 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-04-26T23:08:42.558 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-04-26T23:08:42.558 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.559 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.559 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.559 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-04-26T23:08:42.559 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.559 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.559 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.559 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T23:08:42.559 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.559 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.559 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.559 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-04-26T23:08:42.559 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-04-26T23:08:42.559 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.559 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.560 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.560 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-04-26T23:08:42.560 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.560 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.560 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.560 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-04-26T23:08:42.560 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.560 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.560 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.560 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-04-26T23:08:42.560 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-04-26T23:08:42.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.560 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.560 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.560 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.561 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.561 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.561 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.561 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T23:08:42.561 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.561 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.561 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.561 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-04-26T23:08:42.561 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-04-26T23:08:42.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.561 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.561 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.561 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T23:08:42.561 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.562 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.562 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.562 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-04-26T23:08:42.562 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.562 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.562 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.562 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-04-26T23:08:42.562 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-04-26T23:08:42.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.562 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.562 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.562 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T23:08:42.562 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.562 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.563 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.563 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T23:08:42.563 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.563 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.563 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.563 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-04-26T23:08:42.563 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-04-26T23:08:42.563 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.563 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.563 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.563 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T23:08:42.563 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.563 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.563 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.564 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-04-26T23:08:42.564 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.564 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.564 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.564 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-04-26T23:08:42.564 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-04-26T23:08:42.564 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.564 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.564 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.564 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.565 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.565 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.565 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.565 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T23:08:42.565 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.565 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.565 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.565 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-04-26T23:08:42.565 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-04-26T23:08:42.565 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.566 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.566 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.566 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.566 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.566 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.566 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.566 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T23:08:42.566 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.566 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.567 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.567 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.567 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.567 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.567 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.567 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.567 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.567 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.567 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.567 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.568 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.568 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.568 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.568 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.568 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.568 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.568 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.568 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.568 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.568 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.568 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-04-26T23:08:42.568 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-04-26T23:08:42.569 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.569 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.569 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.569 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.569 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.569 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.569 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.569 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T23:08:42.569 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.569 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.569 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.569 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.569 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.569 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.570 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.570 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.570 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.570 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.570 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.570 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.570 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T23:08:42.570 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.570 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.570 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.570 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.570 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.570 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.571 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.571 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.571 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.571 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.571 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.571 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.571 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.571 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.571 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.571 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.571 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.571 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.571 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.571 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.572 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.572 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.572 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.572 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.572 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.572 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.572 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.572 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.572 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.572 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.572 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.572 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.572 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.573 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.573 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.573 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.573 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T23:08:42.573 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T23:08:42.573 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.573 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.573 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.573 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.573 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.573 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.573 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.573 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T23:08:42.574 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.574 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.574 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.574 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T23:08:42.574 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T23:08:42.574 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.574 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.574 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.574 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-04-26T23:08:42.574 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.575 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.575 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.575 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T23:08:42.575 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.575 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.575 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.575 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-04-26T23:08:42.575 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-04-26T23:08:42.575 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.575 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.576 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.576 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.576 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.576 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.576 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.576 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-04-26T23:08:42.576 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.576 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.576 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.576 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-04-26T23:08:42.576 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-04-26T23:08:42.576 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.577 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.577 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.577 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.577 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.577 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.577 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.577 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T23:08:42.577 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.577 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.577 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.577 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-04-26T23:08:42.577 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-04-26T23:08:42.577 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.578 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.578 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.578 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.578 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.578 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.578 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.578 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-04-26T23:08:42.578 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.578 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.578 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.578 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-04-26T23:08:42.578 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-04-26T23:08:42.578 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.578 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.579 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.579 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-04-26T23:08:42.579 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.579 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.579 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.579 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-04-26T23:08:42.579 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-04-26T23:08:42.579 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.579 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T23:08:42.579 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.579 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.579 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.579 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-04-26T23:08:42.579 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-04-26T23:08:42.580 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.580 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.580 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.580 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-04-26T23:08:42.580 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.580 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.580 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.580 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T23:08:42.580 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.580 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.580 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.580 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-04-26T23:08:42.580 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-04-26T23:08:42.581 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.581 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.581 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.581 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T23:08:42.581 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.581 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.581 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.581 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T23:08:42.581 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.581 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.581 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.581 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-04-26T23:08:42.581 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-04-26T23:08:42.582 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.582 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.582 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.582 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.582 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.582 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.582 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.582 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T23:08:42.582 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.582 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.582 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.583 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-04-26T23:08:42.583 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-04-26T23:08:42.583 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.583 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.583 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.583 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.583 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.583 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.583 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.583 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T23:08:42.583 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.583 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.584 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.584 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-04-26T23:08:42.584 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-04-26T23:08:42.584 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.584 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.584 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.584 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.584 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.584 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.584 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.584 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T23:08:42.585 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.585 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.585 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.585 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-04-26T23:08:42.585 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-04-26T23:08:42.585 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.585 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.585 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.585 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.585 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.585 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.586 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.586 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T23:08:42.586 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.586 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.586 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.586 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-04-26T23:08:42.586 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-04-26T23:08:42.586 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.586 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.586 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.586 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.587 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.587 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.587 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.587 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T23:08:42.587 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.587 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.587 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.587 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-04-26T23:08:42.587 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-04-26T23:08:42.587 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.587 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.587 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.587 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-04-26T23:08:42.588 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.588 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.588 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.588 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-04-26T23:08:42.588 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.588 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.588 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.588 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-04-26T23:08:42.588 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-04-26T23:08:42.588 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.588 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.588 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.588 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.589 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.589 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.589 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.589 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T23:08:42.589 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.589 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.589 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.589 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-04-26T23:08:42.589 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-04-26T23:08:42.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.589 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.589 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.589 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.589 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.590 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.590 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.590 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T23:08:42.590 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.590 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.590 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.590 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-04-26T23:08:42.590 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-04-26T23:08:42.590 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.590 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.590 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.590 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.590 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.591 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.591 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.591 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T23:08:42.591 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.591 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.591 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.591 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-04-26T23:08:42.591 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-04-26T23:08:42.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.591 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.591 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.591 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.591 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.592 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.592 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.592 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T23:08:42.592 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.592 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.592 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.592 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-04-26T23:08:42.592 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-04-26T23:08:42.592 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.592 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.592 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.592 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.592 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.593 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.593 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.593 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T23:08:42.593 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.593 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.593 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.593 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-04-26T23:08:42.593 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-04-26T23:08:42.593 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.593 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.593 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.593 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.593 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.593 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.593 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.594 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T23:08:42.594 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.594 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.594 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.594 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.594 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.594 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-04-26T23:08:42.594 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.594 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.594 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.594 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.594 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.595 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.595 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.595 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.595 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.595 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.595 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.595 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.595 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.595 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.595 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.595 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.595 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.595 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.595 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.595 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.596 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.596 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.596 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.596 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.596 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.596 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.596 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.596 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.596 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.596 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.596 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.596 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.596 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.597 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.597 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.597 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.597 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.597 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.597 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.597 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.597 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.597 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.597 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.597 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.597 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.597 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.597 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.598 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.598 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.598 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.598 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.598 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.598 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.598 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.598 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.598 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.598 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.598 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.598 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.598 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.599 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.599 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.599 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.599 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.599 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.599 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.599 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.599 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.599 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.599 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-04-26T23:08:42.599 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-04-26T23:08:42.599 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.599 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.599 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.600 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-04-26T23:08:42.600 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.600 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.600 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.600 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T23:08:42.600 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.600 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.600 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.600 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-04-26T23:08:42.600 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-04-26T23:08:42.600 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.600 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.600 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.600 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-04-26T23:08:42.601 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.601 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.601 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.601 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T23:08:42.601 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.601 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.601 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.601 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.601 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.601 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-04-26T23:08:42.601 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.601 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.601 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.602 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T23:08:42.602 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T23:08:42.602 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.602 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.602 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.602 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.602 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.602 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.602 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.602 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.602 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.602 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.602 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.602 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T23:08:42.603 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T23:08:42.603 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.603 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.603 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.603 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.603 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.603 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.603 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.603 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.603 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.603 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.603 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.603 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T23:08:42.604 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T23:08:42.604 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.604 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.604 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.604 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.604 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.604 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.604 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.604 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.604 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.604 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.604 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.604 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-04-26T23:08:42.604 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T23:08:42.605 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.605 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.605 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.605 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.605 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.605 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.605 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.605 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.605 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.605 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.605 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.605 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T23:08:42.605 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T23:08:42.606 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.606 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.606 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.606 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.606 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.606 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.606 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.606 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.606 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.606 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.606 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.606 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T23:08:42.607 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T23:08:42.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.607 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.607 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.607 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.607 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.607 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.607 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.607 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.607 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.607 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.607 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T23:08:42.607 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T23:08:42.608 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.608 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.608 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.608 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.608 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.608 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.608 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.608 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.608 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.608 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.608 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.608 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T23:08:42.608 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T23:08:42.608 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.609 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.609 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.609 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.609 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.609 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.609 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.609 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.609 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.609 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.609 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.609 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T23:08:42.609 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T23:08:42.609 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.610 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.610 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.610 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.610 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.610 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.610 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.610 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.610 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.610 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.610 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.611 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T23:08:42.611 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T23:08:42.611 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.611 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.611 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.611 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.611 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.611 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.611 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.611 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.611 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.611 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.612 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.612 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T23:08:42.612 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T23:08:42.612 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.612 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.612 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.612 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.612 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.612 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.613 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.613 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.613 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.613 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.613 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T23:08:42.613 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T23:08:42.613 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.613 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.613 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.613 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.613 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.614 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.614 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.614 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.614 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.614 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.614 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.614 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T23:08:42.614 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T23:08:42.614 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.614 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.615 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.615 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.615 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.615 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.615 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.615 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.615 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.615 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.615 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.616 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T23:08:42.616 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T23:08:42.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.616 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.616 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.616 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.616 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.616 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.616 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.616 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.616 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.616 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.616 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.617 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T23:08:42.617 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T23:08:42.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.617 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.617 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.617 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T23:08:42.617 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.617 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.617 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.617 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.617 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.617 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.617 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.617 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T23:08:42.618 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T23:08:42.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.618 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.618 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.618 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.618 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.618 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.618 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.618 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.618 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.618 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.618 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.618 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-04-26T23:08:42.619 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-04-26T23:08:42.619 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.619 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.619 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.619 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.619 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.619 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.619 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.619 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.619 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.619 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.619 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.619 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T23:08:42.619 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T23:08:42.620 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.620 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.620 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.620 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T23:08:42.620 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.620 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.620 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.620 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.620 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.620 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.620 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.620 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T23:08:42.620 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T23:08:42.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.621 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.621 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.621 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.621 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.621 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.621 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.621 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.621 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.621 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.621 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.621 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T23:08:42.621 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T23:08:42.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.622 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.622 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.622 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.622 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.622 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.622 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.622 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.622 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.622 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.622 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.622 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T23:08:42.622 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T23:08:42.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.623 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.623 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.623 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.623 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.623 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.623 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.623 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.623 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.623 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.623 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.623 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T23:08:42.623 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T23:08:42.623 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.623 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.624 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.624 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.624 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.624 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.624 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.624 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.624 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.624 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.624 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.624 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T23:08:42.624 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T23:08:42.624 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.624 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.624 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.625 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-04-26T23:08:42.625 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.625 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.625 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.625 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.625 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.625 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.625 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.625 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T23:08:42.625 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T23:08:42.625 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.625 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.625 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.626 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-04-26T23:08:42.626 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.626 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.626 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.626 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.626 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.626 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.626 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.626 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T23:08:42.626 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T23:08:42.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.626 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.626 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.627 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.627 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.627 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.627 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.627 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.627 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.627 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.627 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.627 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T23:08:42.627 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T23:08:42.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.627 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.627 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.627 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.627 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.628 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.628 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.628 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.628 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.628 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.628 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.628 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T23:08:42.628 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T23:08:42.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.628 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.628 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.628 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.628 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.629 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.629 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.629 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.629 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.629 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.629 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.629 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T23:08:42.629 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T23:08:42.629 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.629 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.629 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.629 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.629 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.629 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.629 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.630 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.630 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.630 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.630 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.630 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T23:08:42.630 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T23:08:42.630 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.630 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.630 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.630 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.630 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.630 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.630 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.631 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.631 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.631 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.631 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.631 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T23:08:42.631 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T23:08:42.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.631 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.631 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.631 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.631 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.631 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.631 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.631 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.632 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.632 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.632 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.632 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T23:08:42.632 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T23:08:42.632 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.632 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.632 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.632 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.632 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.632 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.632 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.633 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.633 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.633 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.633 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.633 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T23:08:42.633 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T23:08:42.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.633 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.633 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.633 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.633 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.634 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.634 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.634 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.634 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.634 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.634 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.634 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T23:08:42.634 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T23:08:42.634 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.635 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.635 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.635 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.635 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.635 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.635 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.635 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.635 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.635 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.635 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.636 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T23:08:42.636 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T23:08:42.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.636 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.636 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.636 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.636 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.636 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.636 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.636 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.636 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.636 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.637 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.637 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T23:08:42.637 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T23:08:42.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.637 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.637 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T23:08:42.637 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.637 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.637 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.638 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.638 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.638 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.638 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.638 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T23:08:42.638 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T23:08:42.638 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.638 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.638 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.638 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.639 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.639 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.639 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.639 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.639 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.639 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.639 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.639 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T23:08:42.639 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T23:08:42.640 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.640 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.640 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.640 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-04-26T23:08:42.640 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.640 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.640 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.640 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.640 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.640 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.640 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.641 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T23:08:42.641 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T23:08:42.641 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.641 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.641 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.641 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-04-26T23:08:42.641 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.641 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.641 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.641 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.641 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.641 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.641 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.641 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T23:08:42.642 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T23:08:42.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.642 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.642 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.642 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.642 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.642 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.642 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.642 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.642 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.642 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.642 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.642 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T23:08:42.643 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T23:08:42.643 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.643 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.643 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.643 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T23:08:42.643 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.643 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.643 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.643 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.643 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.643 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.643 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.643 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T23:08:42.643 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T23:08:42.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.644 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.644 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.644 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.644 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.644 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.644 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.644 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.644 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.644 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.644 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.644 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T23:08:42.644 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T23:08:42.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.644 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.645 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.645 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.645 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.645 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.645 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.645 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.645 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.645 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.645 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T23:08:42.645 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T23:08:42.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.645 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.646 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.646 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.646 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.646 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.646 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.646 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.646 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.646 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.646 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T23:08:42.646 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T23:08:42.646 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.646 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.647 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.647 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T23:08:42.647 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.647 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.647 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.647 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.647 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.647 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.647 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.647 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T23:08:42.647 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T23:08:42.647 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.647 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.648 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.648 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.648 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.648 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.648 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.648 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.648 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.648 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.648 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.648 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T23:08:42.648 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T23:08:42.648 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.648 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.648 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.648 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T23:08:42.649 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.649 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.649 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.649 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.649 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.649 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.649 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.649 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-04-26T23:08:42.649 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-04-26T23:08:42.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.649 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.649 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.649 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.649 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.650 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.650 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.650 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.650 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.650 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.650 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.650 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-04-26T23:08:42.650 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-04-26T23:08:42.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.650 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.650 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.650 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.650 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.651 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.651 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.651 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.651 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.651 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.651 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.651 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-04-26T23:08:42.651 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-04-26T23:08:42.651 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.651 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.651 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.651 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.651 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.651 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.652 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.652 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-04-26T23:08:42.652 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.652 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.652 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.652 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-04-26T23:08:42.652 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-04-26T23:08:42.652 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.652 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.652 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.652 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-04-26T23:08:42.652 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.652 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.653 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.653 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.653 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.653 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.653 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.653 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-04-26T23:08:42.653 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-04-26T23:08:42.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.653 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.653 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.653 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.653 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.653 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.654 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.654 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.654 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.654 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.654 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.654 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.654 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.654 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.654 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.654 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.655 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.655 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.655 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.655 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.655 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.655 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.655 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.655 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.655 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.655 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.656 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.656 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.656 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.656 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.656 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.656 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.656 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.656 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.656 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.656 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.656 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.657 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.657 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.657 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.657 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.657 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.657 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.657 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.657 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.657 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.657 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.657 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.657 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.658 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.658 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.658 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.658 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.658 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.658 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.658 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.658 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.658 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.658 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.658 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.658 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.658 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.658 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.659 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.659 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.659 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.659 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.659 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.659 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.659 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.659 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.659 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.659 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.659 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.659 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.659 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-04-26T23:08:42.660 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-04-26T23:08:42.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.660 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.660 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.660 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.660 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.660 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.660 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.660 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-04-26T23:08:42.660 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.660 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.660 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.660 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-04-26T23:08:42.661 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T23:08:42.661 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.661 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.661 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.661 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.661 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.661 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.661 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.661 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.661 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.661 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.661 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.661 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T23:08:42.661 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T23:08:42.662 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.662 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.662 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.662 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-04-26T23:08:42.662 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.662 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.662 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.662 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.662 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.662 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.662 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.662 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T23:08:42.662 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T23:08:42.663 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.663 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.663 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.663 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-04-26T23:08:42.663 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.663 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.663 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.663 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.663 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.663 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.663 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.663 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-04-26T23:08:42.663 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-04-26T23:08:42.663 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.664 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.664 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.664 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.664 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.664 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.664 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.664 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.664 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.664 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.664 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.664 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-04-26T23:08:42.664 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-04-26T23:08:42.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.664 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.665 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.665 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-04-26T23:08:42.665 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.665 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.665 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.665 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.665 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.665 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.665 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.665 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-04-26T23:08:42.665 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-04-26T23:08:42.665 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.665 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.665 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.666 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-04-26T23:08:42.666 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.666 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.666 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.666 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.666 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-04-26T23:08:42.666 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.666 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.666 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.666 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.666 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.666 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-04-26T23:08:42.666 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-04-26T23:08:42.667 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.667 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.667 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.667 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-04-26T23:08:42.667 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-04-26T23:08:42.667 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-04-26T23:08:42.667 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.667 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.667 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.667 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.667 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.667 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-04-26T23:08:42.667 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-04-26T23:08:42.667 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.668 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.668 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.668 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.668 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.668 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.668 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.668 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.668 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.668 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.668 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.668 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.668 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.668 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-04-26T23:08:42.669 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.669 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.669 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.669 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-04-26T23:08:42.669 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-04-26T23:08:42.669 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.669 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.669 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.669 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.669 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.669 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.669 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.670 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-04-26T23:08:42.670 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.670 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.670 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.670 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.670 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.670 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.670 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.670 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.670 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.671 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.671 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.671 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.671 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.671 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.671 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.671 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.671 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.671 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.671 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.671 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.671 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.672 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.672 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.672 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.672 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.672 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.672 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.672 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.672 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.672 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.672 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.673 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.673 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.673 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.673 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.673 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.673 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.673 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.673 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.673 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.673 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.674 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.674 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.674 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.674 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.674 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.674 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.674 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.674 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.674 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.674 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.675 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.675 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.675 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.675 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.675 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.675 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.675 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.675 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.675 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.676 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.676 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.676 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.676 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.676 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.676 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.676 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.676 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.676 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-04-26T23:08:42.676 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-04-26T23:08:42.677 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.677 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.677 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.677 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-04-26T23:08:42.677 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.677 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.677 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.677 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T23:08:42.677 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.677 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.677 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.678 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-04-26T23:08:42.678 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-04-26T23:08:42.678 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.678 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.678 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.678 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-04-26T23:08:42.678 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.678 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.678 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.678 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T23:08:42.679 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.679 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.679 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.679 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-04-26T23:08:42.679 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-04-26T23:08:42.679 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.679 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.679 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.679 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-04-26T23:08:42.679 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.680 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.680 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.680 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-04-26T23:08:42.680 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.680 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.680 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.680 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-04-26T23:08:42.680 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-04-26T23:08:42.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.680 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.681 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.681 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-04-26T23:08:42.681 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.681 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.681 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.681 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T23:08:42.681 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.681 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.681 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.681 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-04-26T23:08:42.681 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-04-26T23:08:42.681 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.682 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.682 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.682 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.682 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.682 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.682 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.682 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T23:08:42.682 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.682 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.682 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.682 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T23:08:42.682 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T23:08:42.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.683 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.683 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.683 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T23:08:42.683 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.683 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.683 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.683 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T23:08:42.683 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.683 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.684 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.684 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-04-26T23:08:42.684 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-04-26T23:08:42.684 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.684 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.684 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.684 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-04-26T23:08:42.684 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.684 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.684 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.684 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T23:08:42.684 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.685 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.685 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.685 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.685 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.685 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-04-26T23:08:42.685 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.685 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.685 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.685 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.685 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.685 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.686 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.686 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.686 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.686 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.686 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.686 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.686 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.686 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.686 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.686 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.686 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.686 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.686 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.686 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.686 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.687 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.689 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.690 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.690 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.690 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.690 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.690 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.690 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.690 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.690 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.690 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.690 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.690 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.690 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.691 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.691 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.691 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.691 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.691 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.691 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.691 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.691 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.691 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.691 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.691 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.691 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.691 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.691 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.691 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.692 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.692 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.692 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.692 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.692 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.692 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.692 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.692 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.692 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.692 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.692 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.692 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.693 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.693 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.693 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.693 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.693 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.693 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.693 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-04-26T23:08:42.693 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-04-26T23:08:42.693 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.693 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.693 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.693 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-04-26T23:08:42.693 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.694 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.694 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.694 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.694 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.694 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.694 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.694 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-04-26T23:08:42.694 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-04-26T23:08:42.694 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.694 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.694 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.694 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-04-26T23:08:42.694 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.694 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.695 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.695 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.695 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.695 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.695 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.695 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T23:08:42.695 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T23:08:42.695 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.695 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.695 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.695 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T23:08:42.695 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.695 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.696 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.696 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.696 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.696 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.696 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.696 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.696 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.696 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-04-26T23:08:42.696 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-04-26T23:08:42.696 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-04-26T23:08:42.696 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.696 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-04-26T23:08:42.696 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-04-26T23:08:42.697 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.697 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.697 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.697 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-04-26T23:08:42.697 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.697 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.697 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.697 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T23:08:42.697 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.697 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.697 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.697 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-04-26T23:08:42.697 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-04-26T23:08:42.698 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.698 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.698 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.698 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-04-26T23:08:42.698 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.698 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.698 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.698 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T23:08:42.698 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.698 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.698 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.698 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-04-26T23:08:42.698 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-04-26T23:08:42.698 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.699 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.699 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.699 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.699 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.699 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.699 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.699 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-04-26T23:08:42.699 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.699 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.699 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.699 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T23:08:42.699 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T23:08:42.699 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.699 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.700 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.700 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-04-26T23:08:42.700 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-04-26T23:08:42.700 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.700 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.700 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T23:08:42.700 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.700 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.700 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.700 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.700 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.700 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.700 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.700 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.701 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.701 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.701 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.701 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.701 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.701 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.701 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.701 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.701 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.701 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.701 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.701 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.701 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.701 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.701 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.702 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.702 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.702 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.702 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.702 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.702 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.702 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.702 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.702 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.702 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.702 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.703 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.703 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.703 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.703 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.703 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.703 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.703 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.703 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.703 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.703 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.703 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.703 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.703 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.703 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.704 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.704 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.704 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.704 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.704 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.704 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.704 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.704 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.704 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.704 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.704 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.704 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.704 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.704 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.705 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.705 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.705 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.705 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.705 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.705 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.705 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.705 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-04-26T23:08:42.705 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-04-26T23:08:42.705 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.705 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.705 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.705 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.706 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.706 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.706 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.706 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-04-26T23:08:42.706 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.706 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.706 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.706 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-04-26T23:08:42.706 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-04-26T23:08:42.706 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.706 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.706 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.706 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-04-26T23:08:42.707 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.707 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.707 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.707 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-04-26T23:08:42.707 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.707 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.707 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.707 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-04-26T23:08:42.707 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-04-26T23:08:42.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.707 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.707 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.707 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-04-26T23:08:42.708 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.708 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.708 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.708 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T23:08:42.708 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.708 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.708 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.708 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-04-26T23:08:42.708 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-04-26T23:08:42.708 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.708 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.708 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.708 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T23:08:42.708 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T23:08:42.709 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-04-26T23:08:42.709 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.709 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T23:08:42.709 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.709 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.709 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.709 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-04-26T23:08:42.709 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-04-26T23:08:42.709 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.709 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.709 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.709 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.709 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.709 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.710 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.710 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-04-26T23:08:42.710 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.710 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.710 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.710 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-04-26T23:08:42.710 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-04-26T23:08:42.710 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.710 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.710 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.710 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-04-26T23:08:42.710 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.710 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.711 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.711 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T23:08:42.711 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.711 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.711 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.711 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.711 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.711 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-04-26T23:08:42.711 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.711 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.711 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.711 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.712 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.712 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.712 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.712 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.712 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.712 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.712 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.712 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.712 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.712 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.712 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.712 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.712 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.712 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.712 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.713 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.713 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.713 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.713 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.713 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.713 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.713 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.713 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.713 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.713 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.713 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.714 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.714 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.714 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.714 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.714 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.714 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.714 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.714 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.714 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.714 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.715 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.715 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.715 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.715 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.715 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.715 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.715 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.715 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.715 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.715 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.715 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.715 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.715 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.715 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.715 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.716 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.716 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.716 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.716 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.716 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.716 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.716 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.716 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.716 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.716 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.716 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.716 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.716 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.716 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.716 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.717 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.717 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-04-26T23:08:42.717 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.717 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.717 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.717 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.717 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.717 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.717 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.717 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.717 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.717 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.717 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.718 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.718 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.718 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.718 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.718 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.718 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.718 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.718 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.718 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.718 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.718 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.718 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.718 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.719 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.719 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.719 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.719 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.719 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.719 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.719 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.719 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.719 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.719 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.719 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.719 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.719 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.719 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.720 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.720 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.720 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.720 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.720 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.720 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.720 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.720 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.720 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.720 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.720 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.720 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.720 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.720 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.720 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.721 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.721 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.721 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.721 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.721 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.721 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.721 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.721 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.721 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.721 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.721 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.721 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.721 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.721 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.722 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.722 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.722 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-04-26T23:08:42.722 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.722 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.722 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.722 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-04-26T23:08:42.722 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-04-26T23:08:42.722 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.722 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.723 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.723 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-04-26T23:08:42.723 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.723 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.723 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.723 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T23:08:42.723 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.723 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.723 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.723 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-04-26T23:08:42.723 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-04-26T23:08:42.723 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.723 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.723 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.724 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.724 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-04-26T23:08:42.724 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.724 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.724 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T23:08:42.724 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.724 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.724 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.724 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.724 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.724 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.724 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.724 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.725 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.725 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.725 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.725 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.725 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.725 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.725 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.725 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.725 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.725 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.725 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.725 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.725 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.725 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.725 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.726 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.726 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.726 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.726 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.726 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.726 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.726 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.726 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.726 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.726 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.726 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.726 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.726 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.727 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.727 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.727 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.727 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.727 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.727 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.727 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.727 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.727 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.727 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.727 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.727 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.727 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.727 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.727 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.728 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.728 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.728 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.728 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.728 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.728 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.728 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.728 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.728 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.728 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.728 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.728 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.728 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.729 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.729 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.729 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.729 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.729 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.729 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.729 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.729 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-04-26T23:08:42.729 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.729 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.729 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.729 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-04-26T23:08:42.730 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-04-26T23:08:42.730 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.730 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.730 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.730 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-04-26T23:08:42.730 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.730 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.730 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.730 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-04-26T23:08:42.730 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T23:08:42.731 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.731 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.731 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.731 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.731 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.731 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.731 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.731 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.731 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.731 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.731 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.732 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.732 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.732 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.732 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.732 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.732 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.732 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.732 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.732 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.732 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.732 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.733 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.733 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.733 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.733 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.733 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.733 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.733 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.733 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.734 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.734 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.734 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.734 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.734 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.734 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.734 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.734 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.734 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.734 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.734 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.735 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.735 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.735 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.735 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.735 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.735 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.735 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.735 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.735 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.735 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.735 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.736 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.736 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.736 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.736 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.736 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.736 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.736 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.736 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.736 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.737 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.737 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.737 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.737 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.737 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.737 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.737 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.737 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-04-26T23:08:42.737 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-04-26T23:08:42.737 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-04-26T23:08:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T23:08:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.739 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-04-26T23:08:42.739 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-04-26T23:08:42.739 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.739 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.739 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.739 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-04-26T23:08:42.739 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.739 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.739 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.739 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-04-26T23:08:42.740 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.740 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.740 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.740 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-04-26T23:08:42.740 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-04-26T23:08:42.740 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.740 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.740 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.740 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-04-26T23:08:42.740 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.741 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.741 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.741 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T23:08:42.741 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.741 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.741 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.741 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-04-26T23:08:42.741 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-04-26T23:08:42.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.741 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.741 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-04-26T23:08:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.742 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-04-26T23:08:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-04-26T23:08:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-04-26T23:08:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T23:08:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T23:08:42.743 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.744 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.744 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.744 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.744 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-04-26T23:08:42.744 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.744 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.744 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.744 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.744 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.745 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.745 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.745 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.745 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.745 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.745 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.745 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.745 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.745 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.745 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.745 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.745 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.746 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.746 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.746 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.746 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.746 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.746 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.746 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.746 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.746 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.746 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.747 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.747 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.747 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.747 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.747 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.747 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.747 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.747 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.747 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.747 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.747 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.748 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.748 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.748 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.748 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.748 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.748 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.748 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.748 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.748 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.748 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.748 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.748 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.749 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.749 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.749 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.749 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.749 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.749 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.749 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.749 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.749 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.749 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.750 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.750 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.750 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.750 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.750 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.750 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.750 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.750 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.750 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.750 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.750 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-04-26T23:08:42.751 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.751 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.751 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.751 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.751 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.751 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.751 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.751 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.751 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.751 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.751 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.751 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.751 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.751 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.751 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.752 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.752 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.752 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.752 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.752 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.752 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.752 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.752 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.752 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.752 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.752 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.752 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.752 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.753 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.753 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.753 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.753 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.753 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.753 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.753 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.753 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.753 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.753 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.753 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.753 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.753 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.753 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.754 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.754 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.754 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.754 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.754 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.754 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.754 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.754 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.754 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.754 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.754 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.754 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.754 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.754 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.755 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-04-26T23:08:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.756 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.757 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.758 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.759 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.760 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-04-26T23:08:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.761 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-04-26T23:08:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-04-26T23:08:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T23:08:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.762 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.763 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.764 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.765 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.765 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.765 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.765 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.765 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.765 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.765 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.765 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.765 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.765 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.765 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.765 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.765 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.766 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.766 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.766 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.766 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.766 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.766 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.766 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.766 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.766 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.766 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.766 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.766 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.766 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.766 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.767 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.767 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.767 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.767 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.767 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.767 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.767 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.767 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.767 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.767 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.767 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.767 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.767 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.768 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-04-26T23:08:42.768 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-04-26T23:08:42.768 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.768 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.768 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.768 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.768 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.768 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.768 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.768 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-04-26T23:08:42.768 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-04-26T23:08:42.768 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-04-26T23:08:42.768 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.768 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-04-26T23:08:42.768 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.769 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.769 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.769 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.769 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.769 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-04-26T23:08:42.769 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.769 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.769 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.769 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.769 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.769 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.770 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.770 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.770 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.770 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.770 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.770 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.770 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.770 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.770 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.770 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.770 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.770 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.770 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.770 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.770 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.771 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.771 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.771 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.771 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.771 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.771 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.771 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.771 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.771 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.771 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.771 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.771 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.771 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.771 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.772 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.772 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.772 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.772 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.772 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.772 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.772 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.772 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.772 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.772 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.772 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.772 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.772 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.772 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.773 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.773 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.773 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.773 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.773 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.773 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.773 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.773 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.773 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.773 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.773 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.773 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.773 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.773 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.774 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.774 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.774 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.774 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.774 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.774 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.774 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.774 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-04-26T23:08:42.774 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.774 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.774 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.774 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.775 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.775 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.775 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.775 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.775 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.775 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.775 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.775 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.775 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.775 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.775 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.775 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.775 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.775 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.775 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.776 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.776 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.776 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.776 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.776 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.776 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.776 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.776 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.776 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.776 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.776 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.776 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.776 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.776 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.777 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.777 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.777 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.777 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.777 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.777 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.777 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.777 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.777 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.777 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.777 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.777 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.777 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.778 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.778 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.778 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.778 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.778 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.778 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.778 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.778 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.778 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.778 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.778 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.778 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.778 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.779 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.779 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.779 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.779 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.779 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.779 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.779 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.779 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.779 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.779 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-04-26T23:08:42.779 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.781 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.781 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.781 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.781 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.781 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.781 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.781 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.781 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.781 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.781 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.781 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.781 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.781 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.781 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.782 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.782 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.782 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.782 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.782 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.782 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.782 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.782 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.782 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.782 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.782 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.782 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.782 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.783 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.783 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.783 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.783 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.783 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.783 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.783 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.783 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.783 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.783 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.783 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.783 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.783 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.783 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T23:08:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T23:08:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T23:08:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-04-26T23:08:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-04-26T23:08:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-04-26T23:08:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-04-26T23:08:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-04-26T23:08:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T23:08:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-04-26T23:08:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-04-26T23:08:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-04-26T23:08:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T23:08:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-04-26T23:08:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-04-26T23:08:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-04-26T23:08:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-04-26T23:08:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-04-26T23:08:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-04-26T23:08:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.795 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T23:08:42.795 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T23:08:42.795 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.795 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.795 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.795 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T23:08:42.795 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.795 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.795 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.795 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-04-26T23:08:42.795 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.795 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.795 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.796 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.796 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.796 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-04-26T23:08:42.796 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.796 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.796 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.796 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-04-26T23:08:42.796 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-04-26T23:08:42.797 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.797 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.797 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.797 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.797 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.797 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.797 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.797 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.797 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.797 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.797 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.798 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-04-26T23:08:42.798 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-04-26T23:08:42.798 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.798 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.798 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.798 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.798 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.798 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.798 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.798 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T23:08:42.799 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T23:08:42.799 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.799 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.799 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.799 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.799 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.799 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.799 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.799 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.799 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.799 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.799 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.799 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.800 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.800 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.800 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.800 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.800 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.800 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.800 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.800 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.800 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.800 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.800 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.800 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.800 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.800 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.801 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.801 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.801 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.801 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.801 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.801 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.801 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.801 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.801 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.801 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.801 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.801 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.801 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.801 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.802 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.802 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.802 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.802 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.802 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.802 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.802 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.802 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.802 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.802 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.802 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.802 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.802 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.802 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.803 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.803 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.803 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.803 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.803 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.803 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.803 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.803 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.803 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.803 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.803 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.803 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.803 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.803 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.804 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-04-26T23:08:42.804 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-04-26T23:08:42.804 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.804 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.804 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.804 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.804 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.804 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.804 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.804 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.804 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.804 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.804 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.805 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T23:08:42.805 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T23:08:42.805 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.805 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.805 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.805 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-04-26T23:08:42.805 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.805 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.805 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.805 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.805 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.805 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.805 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.805 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-04-26T23:08:42.806 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-04-26T23:08:42.806 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.806 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.806 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.806 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-04-26T23:08:42.806 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.806 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.806 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.806 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.806 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.806 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.806 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.806 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T23:08:42.807 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T23:08:42.807 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.807 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.807 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.807 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-04-26T23:08:42.807 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.807 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.807 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.807 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T23:08:42.807 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.807 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.807 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.807 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T23:08:42.807 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T23:08:42.808 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.808 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.808 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.808 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-04-26T23:08:42.808 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.808 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.808 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.808 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T23:08:42.808 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.808 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.808 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.808 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-04-26T23:08:42.808 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-04-26T23:08:42.808 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.809 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.809 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.809 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-04-26T23:08:42.809 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.809 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.809 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.809 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.809 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.809 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.809 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.809 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-04-26T23:08:42.809 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-04-26T23:08:42.809 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.809 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.810 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.810 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-04-26T23:08:42.810 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.810 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.810 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.810 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-04-26T23:08:42.810 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-04-26T23:08:42.810 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.810 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.810 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.810 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.810 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.810 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-04-26T23:08:42.811 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-04-26T23:08:42.811 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.811 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.811 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.811 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-04-26T23:08:42.811 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-04-26T23:08:42.811 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-04-26T23:08:42.811 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.811 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.811 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.811 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.811 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.811 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.811 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.812 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-04-26T23:08:42.812 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.812 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.812 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.812 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.812 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.812 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.812 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.812 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.812 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.812 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.812 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.812 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.812 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.813 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.813 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.813 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.813 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.813 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.813 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.813 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.813 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.813 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.813 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.813 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.813 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.813 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.813 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.814 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.814 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.814 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.814 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.814 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.814 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.814 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.814 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.814 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.814 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.814 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.814 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.814 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.814 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.815 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.815 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.815 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.815 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.815 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.815 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.815 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.815 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.815 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.815 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.815 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.815 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.815 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.815 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.816 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.816 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.816 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.816 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.816 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.816 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.816 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.816 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.816 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.816 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.816 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.816 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.816 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.817 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.817 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-04-26T23:08:42.817 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-04-26T23:08:42.817 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.817 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.817 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.817 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.817 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.817 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.817 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.818 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.818 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.818 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.818 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.818 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-04-26T23:08:42.818 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T23:08:42.818 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.818 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.818 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.818 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.819 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.819 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.819 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.819 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.819 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.819 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.819 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.819 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-04-26T23:08:42.819 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-04-26T23:08:42.819 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.819 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.819 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.820 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.820 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.820 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.820 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.820 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.820 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.820 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.820 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.820 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.820 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.820 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-04-26T23:08:42.820 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.821 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.821 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.821 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-04-26T23:08:42.821 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-04-26T23:08:42.821 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.821 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.821 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.821 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.821 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.821 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.821 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.821 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.821 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.821 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.822 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.822 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-04-26T23:08:42.822 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-04-26T23:08:42.822 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.822 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.822 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.822 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.822 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.822 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.822 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.822 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.822 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.822 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.823 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.823 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.823 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.823 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.823 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.823 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.823 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.823 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.823 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.823 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.823 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.823 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.823 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.823 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.824 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.824 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.824 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.824 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.824 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.824 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.824 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.824 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.824 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.824 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.824 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.824 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.824 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.825 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.825 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.825 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.825 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.825 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.825 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.825 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.825 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.825 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.825 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.825 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.825 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.825 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.825 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.826 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.826 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.826 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.826 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.826 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.826 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.826 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.826 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.826 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.826 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.826 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.826 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.826 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.826 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.827 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.827 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.827 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.827 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.827 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.827 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.827 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.827 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.827 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.827 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.827 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.827 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T23:08:42.827 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T23:08:42.827 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.827 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.828 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.828 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.828 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.828 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.828 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.828 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.828 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.828 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.828 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.828 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T23:08:42.828 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T23:08:42.828 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.828 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.828 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.829 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.829 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.829 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.829 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.829 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.829 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.829 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.829 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.829 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.829 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.829 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-04-26T23:08:42.829 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.829 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.830 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.830 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.830 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.830 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.830 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.830 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.830 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.830 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.830 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.830 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.830 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.830 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.830 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.830 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.830 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.831 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.831 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.831 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.831 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.831 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.831 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.831 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.831 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.831 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.831 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.831 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.831 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.831 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.832 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.832 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.832 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.832 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.832 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.832 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.832 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.832 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.832 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.832 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.832 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.832 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.832 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.833 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.833 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.833 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.833 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.833 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.833 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.833 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.833 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.833 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.833 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.833 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.833 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.833 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.833 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.834 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.834 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.834 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.834 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.834 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.834 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.834 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.834 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.834 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.834 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.834 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.834 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.835 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-04-26T23:08:42.835 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.835 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.835 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.835 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.835 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.835 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.835 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.835 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.835 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.835 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.836 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.836 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.836 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.836 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.836 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.836 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.836 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.836 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.836 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.836 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.836 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.836 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.836 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.837 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.837 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.837 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.837 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.837 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.837 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.837 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.837 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.837 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.837 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.837 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.837 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.837 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.837 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.837 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.838 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.838 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.838 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.838 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.838 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.838 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.838 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.838 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.838 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.838 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.838 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.838 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.838 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.838 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.840 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.840 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.840 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-04-26T23:08:42.840 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-04-26T23:08:42.840 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.840 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.840 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.840 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-04-26T23:08:42.840 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.840 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.840 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.840 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T23:08:42.840 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.840 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.841 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.841 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.841 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.841 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-04-26T23:08:42.841 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.841 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.841 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.841 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.841 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.841 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.841 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.841 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.841 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.842 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.842 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.842 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.842 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.842 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.842 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.842 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.842 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.842 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.842 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.842 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.842 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.842 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.842 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.843 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.843 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.843 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.843 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.843 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.843 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.843 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.843 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.843 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.843 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.843 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.843 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.843 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.843 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.843 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.844 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.844 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.844 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.844 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.844 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.844 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.844 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.844 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.844 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.844 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.844 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.844 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.844 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.844 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.845 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.845 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.845 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.845 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.845 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.845 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.845 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.845 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.845 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.845 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.845 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.845 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.845 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.846 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.846 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.846 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.846 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.846 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-04-26T23:08:42.846 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-04-26T23:08:42.846 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.846 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.846 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.846 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.846 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.846 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.846 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.847 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.847 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.847 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.847 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.847 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-04-26T23:08:42.847 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-04-26T23:08:42.847 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.847 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.847 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.847 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-04-26T23:08:42.847 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.847 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.847 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.847 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.847 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.848 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.848 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.848 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-04-26T23:08:42.848 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-04-26T23:08:42.848 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.848 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.848 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.848 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.848 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.848 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.848 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.848 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.848 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.849 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.849 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.849 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-04-26T23:08:42.849 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-04-26T23:08:42.849 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.849 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.849 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.849 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.849 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.849 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.849 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.849 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.849 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.849 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.850 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.850 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-04-26T23:08:42.850 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-04-26T23:08:42.850 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.850 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.850 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.850 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.850 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.850 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.850 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.850 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.850 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.850 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.850 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.851 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-04-26T23:08:42.851 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-04-26T23:08:42.851 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.851 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.851 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.851 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.851 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.851 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.851 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.851 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.851 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.851 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.851 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.852 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-04-26T23:08:42.852 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-04-26T23:08:42.852 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.852 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.852 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.852 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.852 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.852 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.852 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.852 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.852 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.852 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.852 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.852 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-04-26T23:08:42.852 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-04-26T23:08:42.853 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.853 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.853 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.853 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.853 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.853 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.853 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.853 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.853 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.853 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.853 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.853 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-04-26T23:08:42.853 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-04-26T23:08:42.854 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.854 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.854 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.854 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.854 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T23:08:42.854 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-04-26T23:08:42.854 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.854 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.854 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.854 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.854 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.854 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-04-26T23:08:42.854 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-04-26T23:08:42.854 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.855 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.855 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.855 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.855 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.855 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.855 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.855 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.855 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.855 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.855 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.855 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-04-26T23:08:42.855 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-04-26T23:08:42.855 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.855 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.856 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.856 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.856 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.856 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.856 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.856 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.856 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.856 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.856 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.856 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-04-26T23:08:42.856 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-04-26T23:08:42.856 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.856 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.856 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.857 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.857 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.857 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.857 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.857 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.857 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.857 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.857 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.857 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.857 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.857 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-04-26T23:08:42.857 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.858 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.858 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.858 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-04-26T23:08:42.858 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-04-26T23:08:42.858 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.858 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.858 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.858 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.858 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.858 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.858 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.858 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T23:08:42.858 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.858 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.859 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.859 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-04-26T23:08:42.859 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-04-26T23:08:42.859 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.859 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.859 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.859 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.859 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.859 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.859 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.859 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-04-26T23:08:42.859 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.859 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.859 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.860 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.860 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.860 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.860 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.860 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.860 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.860 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.860 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.860 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.860 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.860 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.860 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.860 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.861 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.861 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.861 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.861 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.861 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.861 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.861 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.861 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.861 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.861 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.861 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.861 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.861 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.861 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.862 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.862 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.862 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.862 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.862 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.862 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.862 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.862 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.862 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.862 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.862 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.862 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.862 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.863 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.863 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.863 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.863 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.863 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.863 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.863 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.863 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.863 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.863 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.863 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.863 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.863 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.863 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.864 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.864 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.864 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.864 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.864 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.864 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.864 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.864 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.864 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.864 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.864 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.864 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.864 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.864 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.865 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-04-26T23:08:42.865 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.865 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.865 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.865 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.865 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.865 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.865 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.865 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.865 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.865 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.865 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.865 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.866 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.866 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.866 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.866 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.866 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.866 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.866 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.866 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.866 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.866 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.866 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.866 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.866 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.867 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.867 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.867 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.867 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.867 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.867 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.867 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.867 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.867 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.867 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.867 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.867 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.867 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.867 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.868 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.868 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.868 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.868 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.868 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.868 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.868 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.868 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.868 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.868 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.868 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.868 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.868 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.868 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.868 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.869 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.869 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.869 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.869 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.869 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.869 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.869 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.869 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.869 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.869 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.869 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.869 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.869 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.869 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.869 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.870 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.870 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-04-26T23:08:42.870 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.870 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.870 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.870 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.870 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.870 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.870 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.870 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.870 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.870 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.871 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.871 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.871 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.871 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.871 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.871 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.871 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.871 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.871 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.871 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.871 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.871 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.871 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.872 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.872 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.872 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.872 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.872 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.872 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.872 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.872 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.872 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.872 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.872 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.872 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.872 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.872 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-04-26T23:08:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-04-26T23:08:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-04-26T23:08:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T23:08:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T23:08:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T23:08:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.877 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-04-26T23:08:42.877 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.877 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.877 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.877 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.877 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.877 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.877 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.877 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.877 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.877 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.877 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.877 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.877 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.878 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.878 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.878 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.878 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.878 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.878 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.878 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.878 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.878 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.878 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.878 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.878 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.878 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.878 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.879 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.879 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.879 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.879 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.879 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.879 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.879 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.879 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.879 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.879 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.879 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.879 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.879 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.879 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.880 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.880 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.880 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.880 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.880 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.880 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.880 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.880 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.880 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.880 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.880 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.880 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.880 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.880 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.880 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.881 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.881 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.881 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.881 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.881 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.881 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.881 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.881 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.881 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.881 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.881 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.881 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.881 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.881 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-04-26T23:08:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-04-26T23:08:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-04-26T23:08:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T23:08:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-04-26T23:08:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-04-26T23:08:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T23:08:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-04-26T23:08:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-04-26T23:08:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T23:08:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-04-26T23:08:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-04-26T23:08:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.886 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.886 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.886 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T23:08:42.886 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.886 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.886 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.886 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-04-26T23:08:42.886 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-04-26T23:08:42.886 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.886 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.886 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.886 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.886 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.887 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.887 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.887 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T23:08:42.887 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.887 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.887 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.887 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-04-26T23:08:42.887 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-04-26T23:08:42.887 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.887 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.887 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.887 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.887 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.888 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.888 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.888 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.888 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.888 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.888 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.888 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-04-26T23:08:42.888 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-04-26T23:08:42.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.888 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.888 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.888 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.888 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.888 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.889 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.889 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.889 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.889 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.889 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.889 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-04-26T23:08:42.889 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-04-26T23:08:42.889 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.889 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.889 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.889 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T23:08:42.889 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.889 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.889 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.890 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.890 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.890 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.890 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.890 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-04-26T23:08:42.890 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-04-26T23:08:42.890 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.890 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.890 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.890 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.890 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.890 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.890 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.891 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.891 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.891 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.891 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.893 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T23:08:42.893 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T23:08:42.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.893 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.893 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.893 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-04-26T23:08:42.893 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-04-26T23:08:42.893 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.894 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.894 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.894 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.894 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.894 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.894 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-04-26T23:08:42.894 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-04-26T23:08:42.894 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.894 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.894 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.894 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-04-26T23:08:42.894 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.894 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.895 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.895 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.895 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.895 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.895 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.895 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-04-26T23:08:42.895 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-04-26T23:08:42.895 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.895 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.895 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.895 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.895 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.895 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.895 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.896 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.896 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.896 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.896 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.896 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-04-26T23:08:42.896 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-04-26T23:08:42.896 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.896 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.896 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.896 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.896 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.896 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.896 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.896 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.897 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.897 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.897 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.897 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.897 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.897 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.897 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.897 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.897 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.897 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.897 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.897 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.897 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.897 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.898 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.898 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.898 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.898 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.898 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.898 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.898 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.898 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.898 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.898 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.898 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.898 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.899 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.899 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.899 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.899 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.899 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.899 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.899 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.899 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.899 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.899 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.899 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.899 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.899 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.899 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.900 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.900 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.900 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.900 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.900 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.900 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.900 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.900 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.900 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.900 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.900 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.900 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.900 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.900 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.900 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.901 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.901 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.901 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.901 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.901 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.901 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.901 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.901 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.901 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.901 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.901 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.901 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.901 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.902 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-04-26T23:08:42.902 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-04-26T23:08:42.902 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.902 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.902 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.902 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.902 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.902 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.902 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.902 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.902 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.903 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.903 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.903 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-04-26T23:08:42.903 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-04-26T23:08:42.903 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.903 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.903 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.903 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.903 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.903 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.904 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.904 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.904 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.904 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.904 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.904 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-04-26T23:08:42.904 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-04-26T23:08:42.904 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.904 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.904 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.905 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T23:08:42.905 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.905 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.905 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.905 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.905 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.905 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.905 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.905 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.905 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.905 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-04-26T23:08:42.905 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.906 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.906 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.906 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.906 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.906 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.906 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.906 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.906 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.906 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.906 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.906 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.906 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.906 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.906 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.907 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.907 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.907 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.907 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.907 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.907 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.907 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.907 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.907 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.907 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.907 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.907 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.907 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.907 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.908 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.908 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.908 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.908 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.908 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.908 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.908 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.908 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.908 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.908 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.908 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.908 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.908 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.908 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.909 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.909 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.909 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.909 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.909 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.909 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.909 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.909 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.909 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.909 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.909 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.909 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.909 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.910 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.910 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.910 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.910 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.910 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.910 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.910 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.910 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.910 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.910 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.910 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.910 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.910 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.910 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.911 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-04-26T23:08:42.911 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.911 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.911 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.911 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.911 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.911 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.911 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.911 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.911 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.911 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.911 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.911 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.912 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.912 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.912 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.912 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.912 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.912 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.912 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.912 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.912 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.912 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.912 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.912 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.912 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.912 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.913 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.913 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.913 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.913 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.913 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.913 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.913 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.913 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.913 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.913 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.913 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.913 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.913 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.913 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.914 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.914 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.914 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.914 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.914 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.914 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.914 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.914 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.914 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.914 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.914 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.914 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.914 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.914 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.914 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.915 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.915 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.915 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.915 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.915 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.915 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.915 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.915 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.915 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.915 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.915 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.915 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.915 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.916 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.916 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-04-26T23:08:42.916 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-04-26T23:08:42.916 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.916 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.916 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.916 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-04-26T23:08:42.916 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.916 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.916 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.916 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T23:08:42.916 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.916 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.916 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.917 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-04-26T23:08:42.917 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-04-26T23:08:42.917 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.917 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.917 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.917 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.917 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.917 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.917 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.917 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-04-26T23:08:42.917 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.917 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.917 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.918 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-04-26T23:08:42.918 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-04-26T23:08:42.918 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.918 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.918 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.918 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T23:08:42.918 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.918 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.918 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.918 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T23:08:42.918 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.918 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.918 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.919 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-04-26T23:08:42.919 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T23:08:42.919 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.919 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.919 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.919 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.919 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.919 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.919 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.919 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T23:08:42.919 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.919 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.919 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.919 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.920 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.920 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-04-26T23:08:42.920 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.920 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.920 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.920 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-04-26T23:08:42.920 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-04-26T23:08:42.920 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T23:08:42.920 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.920 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.921 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-04-26T23:08:42.921 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.921 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.921 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.921 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.921 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.921 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.921 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.921 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-04-26T23:08:42.921 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-04-26T23:08:42.921 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.922 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.922 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.922 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.922 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.922 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.922 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.922 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.922 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.922 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.922 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.922 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T23:08:42.922 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T23:08:42.922 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.923 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.923 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.923 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T23:08:42.923 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.923 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.923 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.923 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.923 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.923 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.923 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.923 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.923 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.923 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.923 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.924 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.924 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.924 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.924 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.924 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.924 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.924 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.924 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.924 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.924 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.924 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.924 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.924 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.924 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.924 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.925 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.925 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.925 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.925 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.925 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.925 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.925 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.925 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.925 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.925 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.925 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.926 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.926 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.926 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.926 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.926 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.926 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.926 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.926 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.926 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.927 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.927 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.927 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.927 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.927 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.927 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.927 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.927 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.927 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.927 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.927 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.928 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.928 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.928 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.928 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.928 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.928 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.928 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.928 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.928 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.928 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.928 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.928 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.929 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.929 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.929 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.929 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.929 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-04-26T23:08:42.929 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-04-26T23:08:42.929 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.929 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.929 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.929 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.929 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.929 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.929 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.929 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-04-26T23:08:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-04-26T23:08:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-04-26T23:08:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.931 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.931 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.931 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.931 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-04-26T23:08:42.931 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-04-26T23:08:42.931 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.931 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.931 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.931 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T23:08:42.931 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.931 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.931 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.931 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.932 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.932 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.932 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.932 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.932 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.932 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.932 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T23:08:42.932 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.932 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.932 INFO:teuthology.orchestra.run.smithi027.stdout: "modules": [ 2024-04-26T23:08:42.932 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-04-26T23:08:42.932 INFO:teuthology.orchestra.run.smithi027.stdout: "dashboard", 2024-04-26T23:08:42.932 INFO:teuthology.orchestra.run.smithi027.stdout: "iostat", 2024-04-26T23:08:42.932 INFO:teuthology.orchestra.run.smithi027.stdout: "nfs", 2024-04-26T23:08:42.932 INFO:teuthology.orchestra.run.smithi027.stdout: "restful", 2024-04-26T23:08:42.932 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_schedule" 2024-04-26T23:08:42.933 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.933 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-04-26T23:08:42.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.933 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-04-26T23:08:42.933 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.933 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.933 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.933 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T23:08:42.933 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T23:08:42.933 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.933 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.933 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.933 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T23:08:42.933 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.934 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.934 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.934 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T23:08:42.934 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.934 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.934 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.934 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.934 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.934 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.934 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.934 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.934 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.934 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.934 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.935 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.935 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.935 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.935 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.935 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.935 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.935 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.935 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.935 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.935 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.935 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.935 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.935 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.935 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.936 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.936 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.936 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.936 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.936 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.936 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.936 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.936 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.936 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.936 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.936 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.936 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.936 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-04-26T23:08:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-04-26T23:08:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to", 2024-04-26T23:08:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.940 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.940 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-04-26T23:08:42.940 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-04-26T23:08:42.940 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.940 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.940 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.940 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-04-26T23:08:42.940 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.940 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.940 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.940 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-04-26T23:08:42.940 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.940 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.940 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.941 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-04-26T23:08:42.941 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-04-26T23:08:42.941 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.941 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.941 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.941 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.941 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.941 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.941 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.941 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-04-26T23:08:42.941 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.941 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.941 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.942 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-04-26T23:08:42.942 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-04-26T23:08:42.942 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.942 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.942 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.942 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.942 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.942 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.942 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.942 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-04-26T23:08:42.942 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.942 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.942 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.943 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-04-26T23:08:42.943 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-04-26T23:08:42.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.943 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.943 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.943 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-04-26T23:08:42.943 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.943 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.943 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.943 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-04-26T23:08:42.943 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.943 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.943 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.943 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-04-26T23:08:42.944 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-04-26T23:08:42.944 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.944 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.944 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.944 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.944 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.944 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.944 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.944 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-04-26T23:08:42.944 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.944 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.944 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.944 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-04-26T23:08:42.945 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-04-26T23:08:42.945 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.945 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.945 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.945 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.945 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.945 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.945 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.945 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T23:08:42.945 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.945 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.945 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.945 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-04-26T23:08:42.945 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-04-26T23:08:42.946 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.946 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.946 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.946 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.946 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.946 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.946 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.946 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-04-26T23:08:42.946 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.946 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.946 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.946 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.946 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.947 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-04-26T23:08:42.947 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.947 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.947 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.947 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-04-26T23:08:42.947 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-04-26T23:08:42.947 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.947 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.947 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.947 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.947 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.947 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.947 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.947 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T23:08:42.948 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.948 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.948 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.948 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-04-26T23:08:42.948 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-04-26T23:08:42.948 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.948 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.948 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.948 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-04-26T23:08:42.948 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.948 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.948 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.948 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T23:08:42.948 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T23:08:42.949 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.949 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.949 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-04-26T23:08:42.949 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-04-26T23:08:42.949 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T23:08:42.949 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.949 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.949 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T23:08:42.949 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-04-26T23:08:42.949 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-04-26T23:08:42.949 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.949 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T23:08:42.949 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T23:08:42.949 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.950 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.950 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-04-26T23:08:42.950 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-04-26T23:08:42.950 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T23:08:42.950 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.950 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.950 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-04-26T23:08:42.950 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T23:08:42.950 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-04-26T23:08:42.950 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.950 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T23:08:42.950 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.950 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.951 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.951 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-04-26T23:08:42.951 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-04-26T23:08:42.951 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.951 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.951 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.951 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T23:08:42.951 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.951 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.951 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.951 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T23:08:42.951 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T23:08:42.951 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-04-26T23:08:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-04-26T23:08:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-04-26T23:08:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-04-26T23:08:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-04-26T23:08:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-04-26T23:08:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T23:08:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.953 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-04-26T23:08:42.953 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-04-26T23:08:42.953 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.953 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.953 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.953 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-04-26T23:08:42.953 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.953 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.953 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.953 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T23:08:42.953 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T23:08:42.953 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.953 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.954 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-04-26T23:08:42.954 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-04-26T23:08:42.954 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T23:08:42.954 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.954 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.954 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T23:08:42.954 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-04-26T23:08:42.954 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-04-26T23:08:42.954 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.954 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T23:08:42.954 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T23:08:42.954 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.954 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.955 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:42.955 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:42.955 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.955 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.955 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.955 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.955 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.955 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.955 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.955 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.955 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.955 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.955 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.955 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.955 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.955 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.956 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.956 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.956 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.956 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.956 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:42.956 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:42.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.956 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.956 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.956 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.956 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.956 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.956 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.957 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.957 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.957 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.957 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.957 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:42.957 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:42.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.957 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.957 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.957 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:42.957 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.957 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.957 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.958 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:42.958 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:42.958 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:42.958 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:42.958 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:42.958 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:42.958 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.958 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.958 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.958 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.958 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.958 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:42.958 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:42.958 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.959 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.959 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.959 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.959 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.959 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.959 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.959 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:42.959 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.959 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.959 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.959 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-04-26T23:08:42.959 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-04-26T23:08:42.959 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.960 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.960 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.960 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T23:08:42.960 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.960 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.960 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.960 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T23:08:42.960 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.960 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.960 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.960 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-04-26T23:08:42.960 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-04-26T23:08:42.960 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.960 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.961 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.961 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-04-26T23:08:42.961 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.961 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.961 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.961 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-04-26T23:08:42.961 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-04-26T23:08:42.961 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-04-26T23:08:42.961 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-04-26T23:08:42.961 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-04-26T23:08:42.961 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.961 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-04-26T23:08:42.961 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.961 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.961 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.962 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-04-26T23:08:42.962 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-04-26T23:08:42.962 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.962 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.962 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.962 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.962 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.962 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.962 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.962 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T23:08:42.962 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.962 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.962 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.963 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T23:08:42.963 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T23:08:42.963 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.963 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.963 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.963 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T23:08:42.963 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.963 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.963 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.963 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T23:08:42.963 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.963 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.963 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.964 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-04-26T23:08:42.964 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-04-26T23:08:42.964 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.964 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.964 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.964 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T23:08:42.964 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T23:08:42.964 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.964 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.964 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T23:08:42.964 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T23:08:42.964 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.964 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.965 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-04-26T23:08:42.965 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-04-26T23:08:42.965 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T23:08:42.965 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.965 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.965 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:42.965 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.965 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.965 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.965 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T23:08:42.965 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.965 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.965 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.965 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.965 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:42.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:42.966 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-04-26T23:08:42.966 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:42.966 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:42.966 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:42.966 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-04-26T23:08:42.966 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-04-26T23:08:42.966 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.966 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.966 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.966 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-04-26T23:08:42.966 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.966 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.967 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.967 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T23:08:42.967 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.967 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.967 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.967 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-04-26T23:08:42.967 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-04-26T23:08:42.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.967 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.967 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.967 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-04-26T23:08:42.967 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.967 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.967 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.968 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T23:08:42.968 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.968 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.968 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.968 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-04-26T23:08:42.968 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-04-26T23:08:42.968 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.968 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.968 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.968 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-04-26T23:08:42.968 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.968 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.968 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.969 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T23:08:42.969 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.969 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.969 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.969 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-04-26T23:08:42.969 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-04-26T23:08:42.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.969 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.969 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.969 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.969 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.969 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.969 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.969 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T23:08:42.970 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T23:08:42.970 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.970 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.970 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-04-26T23:08:42.970 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-04-26T23:08:42.970 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.970 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.970 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.970 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T23:08:42.970 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.970 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.970 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.970 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T23:08:42.971 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.971 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.971 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.971 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-04-26T23:08:42.971 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T23:08:42.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:42.971 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.971 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.971 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-04-26T23:08:42.971 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.971 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.971 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.972 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T23:08:42.972 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.972 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.972 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.972 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-04-26T23:08:42.972 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-04-26T23:08:42.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.972 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.972 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.973 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:42.973 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.973 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.973 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:42.973 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-04-26T23:08:42.973 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-04-26T23:08:42.973 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-04-26T23:08:42.973 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:42.973 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T23:08:42.973 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.973 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.973 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.974 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-04-26T23:08:42.974 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-04-26T23:08:42.974 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.974 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.974 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.974 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.974 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.974 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.974 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.974 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T23:08:42.974 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.974 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.975 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.975 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-04-26T23:08:42.975 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-04-26T23:08:42.975 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.975 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.975 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.975 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:42.975 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.975 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.975 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.975 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T23:08:42.976 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.976 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.976 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.976 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-04-26T23:08:42.976 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-04-26T23:08:42.976 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.976 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.976 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.976 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.976 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.976 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.976 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.976 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T23:08:42.977 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.977 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.977 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.977 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-04-26T23:08:42.977 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-04-26T23:08:42.977 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.977 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.977 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.978 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T23:08:42.978 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.978 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.978 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.978 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T23:08:42.978 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.978 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.978 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.978 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-04-26T23:08:42.979 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-04-26T23:08:42.979 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.979 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.979 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:42.979 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T23:08:42.979 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.979 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.979 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.979 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-04-26T23:08:42.979 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.979 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.980 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.980 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-04-26T23:08:42.980 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-04-26T23:08:42.980 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.980 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.980 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.980 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T23:08:42.981 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.981 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.981 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.981 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-04-26T23:08:42.981 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.981 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.981 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.981 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-04-26T23:08:42.981 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-04-26T23:08:42.981 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.981 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.981 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.981 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T23:08:42.982 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.982 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.982 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.982 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T23:08:42.982 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.982 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.982 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.982 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-04-26T23:08:42.982 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-04-26T23:08:42.982 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.982 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.982 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.982 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T23:08:42.983 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.983 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.983 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.983 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-04-26T23:08:42.983 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.983 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.983 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.983 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-04-26T23:08:42.983 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-04-26T23:08:42.983 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.983 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.983 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.984 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T23:08:42.984 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.984 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.984 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.984 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-04-26T23:08:42.984 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.984 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.984 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.984 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-04-26T23:08:42.984 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-04-26T23:08:42.984 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.984 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.985 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.985 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T23:08:42.985 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.985 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.985 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.985 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-04-26T23:08:42.985 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.985 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.985 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.986 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-04-26T23:08:42.986 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-04-26T23:08:42.986 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.986 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.986 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.986 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T23:08:42.986 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.986 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.986 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.986 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-04-26T23:08:42.986 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.986 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.987 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.987 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-04-26T23:08:42.987 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-04-26T23:08:42.987 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.987 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.987 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.987 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T23:08:42.987 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.987 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.987 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.987 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-04-26T23:08:42.987 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.988 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.988 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.988 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-04-26T23:08:42.988 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-04-26T23:08:42.988 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.988 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.988 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.988 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T23:08:42.988 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.988 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.988 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.988 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-04-26T23:08:42.989 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.989 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.989 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.989 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-04-26T23:08:42.989 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-04-26T23:08:42.989 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.989 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.989 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.989 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T23:08:42.989 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.989 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.990 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.990 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T23:08:42.990 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.990 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.990 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.990 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-04-26T23:08:42.990 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-04-26T23:08:42.990 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.990 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.990 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.990 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T23:08:42.990 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.990 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.991 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.991 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-04-26T23:08:42.991 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.991 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.991 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.991 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-04-26T23:08:42.991 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-04-26T23:08:42.991 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.991 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.991 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.992 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T23:08:42.992 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.992 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.992 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.992 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T23:08:42.992 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.992 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.992 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.992 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-04-26T23:08:42.992 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-04-26T23:08:42.992 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.992 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.993 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.993 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T23:08:42.993 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.993 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.993 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.993 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-04-26T23:08:42.993 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.993 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.993 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.993 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-04-26T23:08:42.993 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-04-26T23:08:42.993 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.994 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.994 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.994 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T23:08:42.994 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.994 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.994 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.994 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-04-26T23:08:42.994 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.994 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.994 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.995 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-04-26T23:08:42.995 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-04-26T23:08:42.995 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.995 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.995 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.995 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T23:08:42.995 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.995 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.995 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.995 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-04-26T23:08:42.995 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.995 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.995 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.996 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-04-26T23:08:42.996 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-04-26T23:08:42.996 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:42.996 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.996 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.996 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:42.996 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.996 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.996 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.996 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T23:08:42.996 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.996 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.997 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.997 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-04-26T23:08:42.997 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-04-26T23:08:42.997 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:42.997 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.997 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.997 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T23:08:42.997 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.997 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.997 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.997 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T23:08:42.997 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.997 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.997 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.998 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-04-26T23:08:42.998 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T23:08:42.998 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:42.998 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.998 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.998 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-04-26T23:08:42.998 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.998 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.998 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.998 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T23:08:42.998 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.998 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.998 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:42.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:42.999 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-04-26T23:08:42.999 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-04-26T23:08:42.999 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:42.999 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:42.999 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:42.999 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-04-26T23:08:42.999 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:42.999 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:42.999 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:42.999 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T23:08:42.999 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:42.999 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:42.999 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.000 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-04-26T23:08:43.000 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-04-26T23:08:43.000 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:43.000 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.000 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.000 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-04-26T23:08:43.000 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.000 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.000 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.000 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-04-26T23:08:43.001 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.001 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.001 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.001 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-04-26T23:08:43.001 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-04-26T23:08:43.001 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.001 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.001 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.001 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.001 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.001 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.001 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.002 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T23:08:43.002 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.002 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.002 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.002 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-04-26T23:08:43.002 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-04-26T23:08:43.002 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:43.002 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.002 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.002 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T23:08:43.002 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.002 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.002 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.003 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-04-26T23:08:43.003 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.003 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.003 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.003 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-04-26T23:08:43.003 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-04-26T23:08:43.003 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.003 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.003 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.003 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T23:08:43.003 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.003 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.003 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.004 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T23:08:43.004 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.004 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.004 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.004 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-04-26T23:08:43.004 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-04-26T23:08:43.004 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:43.004 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.004 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.004 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T23:08:43.005 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.005 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.005 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.005 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-04-26T23:08:43.005 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.005 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.005 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.005 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-04-26T23:08:43.005 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-04-26T23:08:43.005 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.005 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.006 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.006 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.006 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.006 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.006 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.006 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T23:08:43.006 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.006 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.006 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.006 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-04-26T23:08:43.006 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-04-26T23:08:43.006 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.006 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.007 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.007 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.007 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.007 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.007 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.007 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T23:08:43.007 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.007 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.007 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.007 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.007 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.007 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.007 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.008 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.008 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.008 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.008 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.008 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.008 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.008 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.008 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.008 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.008 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.008 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.008 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.008 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.009 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.009 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.009 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.009 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-04-26T23:08:43.009 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-04-26T23:08:43.009 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.009 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.009 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.009 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.009 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.009 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.010 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.010 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T23:08:43.010 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.010 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.010 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.010 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.010 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.010 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.010 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.010 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.010 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.010 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.011 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.011 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.011 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T23:08:43.011 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.011 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.011 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.011 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.011 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.011 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.011 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.011 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.012 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.012 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.012 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.012 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.012 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.012 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.012 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.012 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.012 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.012 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.012 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.012 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.012 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.013 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.013 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.013 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.013 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.013 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.013 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.013 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.013 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.013 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.013 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.013 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.013 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.013 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.014 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.014 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.014 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T23:08:43.014 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T23:08:43.014 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.014 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.014 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.014 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.014 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.014 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.015 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.015 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T23:08:43.015 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.015 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.015 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.015 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T23:08:43.015 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T23:08:43.015 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.015 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.015 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.015 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-04-26T23:08:43.015 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.015 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.015 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.016 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T23:08:43.016 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.016 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.016 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.016 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-04-26T23:08:43.016 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-04-26T23:08:43.016 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.016 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.016 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.016 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:43.016 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.016 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.016 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.017 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-04-26T23:08:43.017 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.017 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.017 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.017 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-04-26T23:08:43.017 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-04-26T23:08:43.017 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.017 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.017 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.017 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:43.018 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.018 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.018 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.018 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T23:08:43.018 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.018 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.018 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.018 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-04-26T23:08:43.018 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-04-26T23:08:43.018 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.018 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.018 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.018 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-04-26T23:08:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-04-26T23:08:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-04-26T23:08:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-04-26T23:08:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-04-26T23:08:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-04-26T23:08:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T23:08:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-04-26T23:08:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-04-26T23:08:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-04-26T23:08:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T23:08:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-04-26T23:08:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-04-26T23:08:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.022 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.022 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T23:08:43.022 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.022 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.022 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.022 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T23:08:43.022 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.022 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.022 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.022 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-04-26T23:08:43.022 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-04-26T23:08:43.022 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.023 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.023 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.023 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.023 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.023 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.023 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.023 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T23:08:43.023 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.023 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.023 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.023 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-04-26T23:08:43.023 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-04-26T23:08:43.023 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.023 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.024 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.024 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.024 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.024 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.024 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.024 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T23:08:43.024 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.024 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.024 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.024 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-04-26T23:08:43.024 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-04-26T23:08:43.024 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.025 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.025 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.025 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.025 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.025 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.025 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.025 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T23:08:43.025 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.025 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.025 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.025 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-04-26T23:08:43.025 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-04-26T23:08:43.025 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.026 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.026 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.026 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.026 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.026 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.026 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.026 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T23:08:43.026 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.026 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.026 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.027 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-04-26T23:08:43.027 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-04-26T23:08:43.027 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.027 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.027 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.027 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.027 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.027 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.027 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.027 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T23:08:43.027 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.027 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.027 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.028 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-04-26T23:08:43.028 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-04-26T23:08:43.028 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.028 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.028 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.028 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-04-26T23:08:43.028 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.028 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.028 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.028 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-04-26T23:08:43.028 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.028 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.028 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.028 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-04-26T23:08:43.029 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-04-26T23:08:43.029 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.029 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.029 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.029 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.029 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.029 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.029 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.029 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T23:08:43.029 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.029 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.029 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.029 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-04-26T23:08:43.030 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-04-26T23:08:43.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.030 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.030 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.030 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.030 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.030 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.030 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T23:08:43.030 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.030 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.030 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.030 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-04-26T23:08:43.031 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-04-26T23:08:43.031 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.031 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.031 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.031 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.031 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.031 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.031 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.031 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T23:08:43.031 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.031 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.031 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.031 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-04-26T23:08:43.031 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-04-26T23:08:43.032 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.032 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.032 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.032 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.032 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.032 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.032 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.032 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T23:08:43.032 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.032 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.032 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.032 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-04-26T23:08:43.032 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-04-26T23:08:43.033 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.033 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.033 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.033 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.033 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.033 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.033 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.033 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T23:08:43.033 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.033 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.033 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.033 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-04-26T23:08:43.033 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-04-26T23:08:43.034 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.034 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.034 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.034 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.034 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.034 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.034 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.034 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T23:08:43.034 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.034 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.034 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.034 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.034 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.035 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-04-26T23:08:43.035 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.035 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.035 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.035 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.035 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.035 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.035 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.035 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.035 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.035 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.035 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.035 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.035 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.036 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.036 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.036 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.036 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.036 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.036 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.036 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.036 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.036 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.036 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.036 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.036 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.036 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.037 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.037 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.037 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.037 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.037 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.037 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.037 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.037 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.037 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.038 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.038 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.038 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.038 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.038 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.038 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.038 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.038 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.038 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.038 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.038 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.038 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.039 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.039 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.039 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.039 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.039 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.039 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.039 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.039 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.039 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.039 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.039 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.039 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.039 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.039 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.040 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.040 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.040 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.040 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.040 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.040 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.040 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.040 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.040 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-04-26T23:08:43.040 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-04-26T23:08:43.040 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:43.040 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.040 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.041 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-04-26T23:08:43.041 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.041 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.041 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.041 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T23:08:43.041 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.041 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.041 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.041 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-04-26T23:08:43.041 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-04-26T23:08:43.041 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:43.041 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.041 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.042 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-04-26T23:08:43.042 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.042 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.042 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.042 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T23:08:43.042 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.042 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.042 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.042 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.042 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.042 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-04-26T23:08:43.042 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.043 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.043 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.043 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T23:08:43.043 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T23:08:43.043 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.043 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.043 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.043 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:43.043 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.043 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.043 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.043 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.043 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.043 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.043 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.044 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T23:08:43.044 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T23:08:43.044 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.044 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.044 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.044 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.044 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.044 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.044 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.044 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.044 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.044 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.044 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T23:08:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T23:08:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.045 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-04-26T23:08:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T23:08:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.046 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T23:08:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T23:08:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T23:08:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T23:08:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.048 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T23:08:43.049 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T23:08:43.049 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.049 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.049 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.049 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.049 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.049 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.049 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.049 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.049 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.049 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.049 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.049 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T23:08:43.049 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T23:08:43.049 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.050 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.050 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.050 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.050 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.050 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.050 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.050 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.050 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.050 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.050 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.050 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T23:08:43.050 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T23:08:43.050 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.051 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.051 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.051 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.051 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.051 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.051 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.051 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.051 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.051 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.051 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.051 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T23:08:43.051 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T23:08:43.052 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.052 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.052 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.052 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.052 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.052 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.052 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.052 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.052 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.052 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.052 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.053 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T23:08:43.053 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T23:08:43.053 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.053 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.053 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.053 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.053 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.053 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.053 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.053 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.054 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.054 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.054 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.054 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T23:08:43.054 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T23:08:43.054 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.054 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.054 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.054 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.054 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.054 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.055 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.055 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.055 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.055 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.055 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.055 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T23:08:43.055 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T23:08:43.055 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.055 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.055 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.055 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.055 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.055 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.055 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.056 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.056 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.056 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.056 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.056 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T23:08:43.056 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T23:08:43.056 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.056 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.056 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.056 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.056 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.056 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.056 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.057 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.057 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.057 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.057 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.057 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T23:08:43.057 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T23:08:43.057 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.057 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.057 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.057 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T23:08:43.057 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.057 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.057 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.058 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.058 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.058 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.058 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.058 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T23:08:43.058 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T23:08:43.058 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.058 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.058 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.058 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.058 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.058 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.058 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.059 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.059 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.059 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.059 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.059 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-04-26T23:08:43.059 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-04-26T23:08:43.059 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.059 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.059 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.059 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.059 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.059 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.059 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.060 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.060 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.060 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.060 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.060 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T23:08:43.060 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T23:08:43.060 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.060 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.060 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.060 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T23:08:43.060 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.060 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.060 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.060 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.061 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.061 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.061 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.061 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T23:08:43.061 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T23:08:43.061 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.061 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.061 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.061 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.061 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.061 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.061 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.061 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.062 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.062 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.062 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.062 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T23:08:43.062 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T23:08:43.062 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.062 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.062 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.062 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.062 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.062 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.062 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T23:08:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T23:08:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T23:08:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T23:08:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.065 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.065 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.065 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T23:08:43.065 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T23:08:43.065 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.065 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.065 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.065 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-04-26T23:08:43.065 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.065 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.065 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.065 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.065 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.065 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.066 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.066 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T23:08:43.066 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T23:08:43.066 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.066 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.066 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.066 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-04-26T23:08:43.066 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.066 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.066 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.066 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.066 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.067 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.067 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.067 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T23:08:43.067 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T23:08:43.067 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.067 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.067 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.067 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.067 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.067 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.067 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.067 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.067 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.067 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.068 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.068 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T23:08:43.068 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T23:08:43.068 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.068 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.068 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.068 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.068 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.068 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.068 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.068 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.068 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.069 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.069 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.069 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T23:08:43.069 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T23:08:43.069 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.069 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.069 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.069 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.069 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.069 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.069 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.069 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.070 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.070 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.070 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.070 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T23:08:43.070 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T23:08:43.070 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.070 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.070 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.070 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.070 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.070 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.070 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.070 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.071 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.071 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.071 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.071 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T23:08:43.071 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T23:08:43.071 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.071 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.071 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.071 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.071 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.071 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.071 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.071 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.072 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.072 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.072 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.072 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T23:08:43.072 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T23:08:43.072 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.072 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.072 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.072 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.072 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.072 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.072 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.072 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.072 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T23:08:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T23:08:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.074 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.074 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.074 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T23:08:43.074 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T23:08:43.074 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.074 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.074 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.074 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.074 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.074 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.074 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.074 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T23:08:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T23:08:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.076 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.076 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.076 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T23:08:43.076 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T23:08:43.076 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.076 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.076 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.076 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.076 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.076 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.076 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.076 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.076 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.076 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.077 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.077 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T23:08:43.077 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T23:08:43.077 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.077 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.077 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.077 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T23:08:43.077 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.077 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.077 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.077 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.077 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.077 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.077 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.078 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T23:08:43.078 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T23:08:43.078 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.078 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.078 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.078 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:43.078 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.078 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.078 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.078 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.078 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.078 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.078 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.079 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T23:08:43.079 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T23:08:43.079 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.079 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.079 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.079 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-04-26T23:08:43.079 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.079 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.079 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.079 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.079 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.079 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.079 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.080 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T23:08:43.080 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T23:08:43.080 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.080 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.080 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.080 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-04-26T23:08:43.080 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.080 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.080 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.080 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.080 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.080 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.080 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.081 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T23:08:43.081 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T23:08:43.081 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.081 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.081 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.081 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.081 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.081 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.081 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.081 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.081 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.081 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.081 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.082 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T23:08:43.082 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T23:08:43.082 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.082 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.082 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.082 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T23:08:43.082 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.082 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.082 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.082 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.082 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.082 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.083 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.083 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T23:08:43.083 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T23:08:43.083 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.083 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.083 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.083 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.083 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.083 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.083 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.083 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.083 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.083 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.084 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.084 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T23:08:43.084 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T23:08:43.084 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.084 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.084 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.084 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.084 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.084 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.084 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.084 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.084 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.085 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.085 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.085 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T23:08:43.085 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T23:08:43.085 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.085 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.085 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.085 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.085 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.085 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.086 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.086 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.086 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.086 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.086 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.086 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T23:08:43.086 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T23:08:43.086 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.086 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.087 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.087 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T23:08:43.087 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.087 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.087 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.087 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.087 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.087 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.087 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.087 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T23:08:43.087 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T23:08:43.088 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.088 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.088 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.088 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:43.088 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.088 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.088 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.088 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.088 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.088 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.089 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.089 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T23:08:43.089 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T23:08:43.089 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.089 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.089 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.089 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T23:08:43.089 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.089 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.089 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.090 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.090 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.090 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.090 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.090 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-04-26T23:08:43.090 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-04-26T23:08:43.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.090 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.090 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.091 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.091 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.091 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.091 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.091 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.091 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.091 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.091 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.091 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-04-26T23:08:43.091 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-04-26T23:08:43.092 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.092 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.092 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.092 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.092 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.092 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.092 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.093 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.093 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.093 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.093 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.093 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-04-26T23:08:43.093 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-04-26T23:08:43.093 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.094 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.094 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.094 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.094 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.094 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.094 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.094 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-04-26T23:08:43.095 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.095 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.095 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.095 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-04-26T23:08:43.095 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-04-26T23:08:43.095 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.095 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.095 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.095 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-04-26T23:08:43.095 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.095 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.095 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.096 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.096 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.096 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.097 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.098 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-04-26T23:08:43.098 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-04-26T23:08:43.098 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.098 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.098 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.098 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.098 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.098 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.098 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.098 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.098 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.098 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.098 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.099 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.099 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.099 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.099 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.099 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.099 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.099 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.099 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.099 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.100 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.100 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.100 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.100 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.100 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.100 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.100 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.100 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.101 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.101 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.101 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.101 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.101 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.101 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.102 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.102 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.102 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.102 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.102 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.102 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.103 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.103 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.103 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.103 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.103 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.103 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.103 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.104 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.104 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.104 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.104 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.104 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.104 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.104 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.105 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.105 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.105 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.105 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.105 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.105 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.105 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.105 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.106 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.106 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.106 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.106 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.106 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.106 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.106 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.107 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.107 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.107 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.107 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.107 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.107 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.107 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.107 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-04-26T23:08:43.107 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-04-26T23:08:43.107 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.107 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.107 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.108 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.108 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.108 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.108 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.108 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-04-26T23:08:43.108 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.108 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.108 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.108 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-04-26T23:08:43.108 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T23:08:43.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.109 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.109 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.109 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.109 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.109 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.109 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.109 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.109 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.109 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.109 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.109 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T23:08:43.110 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T23:08:43.110 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.110 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.110 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.110 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-04-26T23:08:43.110 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.110 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.110 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.110 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.110 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.111 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.111 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.111 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T23:08:43.111 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T23:08:43.111 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.111 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.111 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.111 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-04-26T23:08:43.111 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.111 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.111 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.112 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.112 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.112 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.112 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.112 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-04-26T23:08:43.112 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-04-26T23:08:43.112 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.112 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.112 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.112 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.112 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.112 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.112 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.113 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.113 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.113 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.113 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.113 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-04-26T23:08:43.113 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-04-26T23:08:43.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.113 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.113 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-04-26T23:08:43.113 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.113 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.114 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.114 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.114 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.114 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.114 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.114 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-04-26T23:08:43.114 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-04-26T23:08:43.114 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.114 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.114 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.114 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-04-26T23:08:43.114 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.115 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.115 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.115 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.115 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-04-26T23:08:43.115 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.115 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.115 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.115 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.115 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.115 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-04-26T23:08:43.115 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-04-26T23:08:43.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.115 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.116 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.116 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-04-26T23:08:43.116 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-04-26T23:08:43.116 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-04-26T23:08:43.116 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.116 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.116 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.116 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.116 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.116 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-04-26T23:08:43.116 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-04-26T23:08:43.116 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.117 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.117 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.117 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.117 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.117 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.117 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.117 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.117 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.117 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.117 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.117 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.117 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.117 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-04-26T23:08:43.118 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.118 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.118 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.118 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-04-26T23:08:43.118 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-04-26T23:08:43.118 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.118 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.118 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.118 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.118 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.118 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.118 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.118 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-04-26T23:08:43.118 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.119 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.119 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.119 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.119 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.119 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.119 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.119 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.119 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.119 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.119 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.119 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.119 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.120 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.120 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.120 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.120 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.120 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.120 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.120 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.120 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.120 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.120 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.120 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.120 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.120 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.120 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.121 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.121 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.121 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.121 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.121 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.121 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.121 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.121 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.121 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.121 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.121 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.121 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.122 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.122 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.122 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.122 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.122 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.122 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.122 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.122 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.122 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.122 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.122 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.122 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.122 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.122 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.123 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.123 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.123 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.123 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.123 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.123 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.123 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.123 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.123 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.123 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.123 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.124 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.124 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.124 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.124 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-04-26T23:08:43.124 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-04-26T23:08:43.124 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:43.124 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.124 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.125 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-04-26T23:08:43.125 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.125 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.125 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.125 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T23:08:43.125 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.125 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.125 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.125 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-04-26T23:08:43.125 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-04-26T23:08:43.125 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.126 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.126 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.126 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-04-26T23:08:43.126 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.126 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.126 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.126 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T23:08:43.126 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.126 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.126 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.126 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-04-26T23:08:43.127 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-04-26T23:08:43.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:43.127 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.127 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.127 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-04-26T23:08:43.127 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.127 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.128 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.128 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-04-26T23:08:43.128 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.128 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.128 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.128 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-04-26T23:08:43.128 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-04-26T23:08:43.128 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:43.129 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.129 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.129 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-04-26T23:08:43.129 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.129 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.129 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.129 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T23:08:43.129 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.129 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.130 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.130 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-04-26T23:08:43.130 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-04-26T23:08:43.130 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.130 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.130 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.130 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.130 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.131 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.131 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.131 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T23:08:43.131 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.131 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.131 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.132 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T23:08:43.132 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T23:08:43.132 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:43.132 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.132 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.132 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T23:08:43.132 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.132 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.132 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.132 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T23:08:43.132 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.132 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.133 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.133 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-04-26T23:08:43.133 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-04-26T23:08:43.133 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:43.133 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.133 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.133 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-04-26T23:08:43.133 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.134 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.134 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.134 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T23:08:43.134 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.134 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.134 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.135 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-04-26T23:08:43.135 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.135 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.135 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.135 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.135 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.136 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.136 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.136 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.136 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.136 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.136 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.136 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.136 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.137 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.137 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.137 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.137 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.137 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.137 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.137 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.138 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.138 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.138 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.138 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.138 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.138 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.138 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.138 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.138 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.138 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.139 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.139 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.139 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.139 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.139 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.139 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.139 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.139 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.139 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.139 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.139 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.139 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.140 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.140 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.140 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.140 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.140 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.140 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.140 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.140 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.140 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.140 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.140 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.140 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.141 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.141 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.141 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.141 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.141 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.141 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.141 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.142 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.142 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.142 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.142 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.142 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.142 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.142 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.142 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.142 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-04-26T23:08:43.142 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-04-26T23:08:43.142 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.143 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.143 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.143 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-04-26T23:08:43.143 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.143 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.143 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.143 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.143 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.143 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.143 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.143 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-04-26T23:08:43.143 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-04-26T23:08:43.143 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.143 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.144 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.144 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-04-26T23:08:43.144 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.144 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.144 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.144 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.144 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.144 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.144 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.144 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T23:08:43.144 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T23:08:43.144 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.145 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.145 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.145 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T23:08:43.145 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.145 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.145 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.145 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.145 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.145 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.145 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.145 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.146 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.146 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-04-26T23:08:43.146 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-04-26T23:08:43.146 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-04-26T23:08:43.146 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.146 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-04-26T23:08:43.146 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-04-26T23:08:43.146 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.146 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.146 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.146 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-04-26T23:08:43.147 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.147 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.147 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.147 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T23:08:43.147 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.147 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.147 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.147 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-04-26T23:08:43.147 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-04-26T23:08:43.147 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.147 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.147 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.148 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-04-26T23:08:43.148 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.148 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.148 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.148 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T23:08:43.148 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.148 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.148 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.148 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-04-26T23:08:43.149 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-04-26T23:08:43.149 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.149 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.149 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.149 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.149 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.149 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.149 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.149 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-04-26T23:08:43.149 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.150 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.150 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.150 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T23:08:43.150 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T23:08:43.150 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:43.150 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.150 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.150 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-04-26T23:08:43.151 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-04-26T23:08:43.151 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.151 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.151 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T23:08:43.151 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.151 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.151 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.152 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.152 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.152 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.152 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.152 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.152 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.152 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.152 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.152 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.152 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.152 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.153 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.153 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.153 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.153 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.153 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.153 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.153 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.153 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.153 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.153 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.153 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.154 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.154 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.154 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.154 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.154 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.154 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.154 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.154 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.154 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.154 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.154 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.155 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.155 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.155 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.155 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.155 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.155 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.155 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.155 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.155 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.155 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.155 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.156 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.156 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.156 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.156 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.156 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.156 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.156 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.156 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.156 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.156 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.156 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.157 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.157 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.157 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.157 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.157 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.157 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.157 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.157 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.157 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.157 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.157 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-04-26T23:08:43.158 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-04-26T23:08:43.158 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.158 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.158 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.158 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.158 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.158 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.158 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.158 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-04-26T23:08:43.159 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.159 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.159 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.159 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-04-26T23:08:43.159 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-04-26T23:08:43.159 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.159 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.159 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-04-26T23:08:43.159 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.160 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.160 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.160 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-04-26T23:08:43.160 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.160 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.160 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.160 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-04-26T23:08:43.160 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-04-26T23:08:43.160 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.160 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.160 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.160 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-04-26T23:08:43.161 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.161 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.161 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.161 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T23:08:43.161 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.161 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.161 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.161 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-04-26T23:08:43.161 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-04-26T23:08:43.161 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.161 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.161 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.162 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T23:08:43.162 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T23:08:43.162 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-04-26T23:08:43.162 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.162 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T23:08:43.162 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.162 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.162 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.162 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-04-26T23:08:43.162 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-04-26T23:08:43.163 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.163 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.163 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.163 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.163 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.163 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.163 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.163 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-04-26T23:08:43.163 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.163 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.163 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.164 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-04-26T23:08:43.164 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-04-26T23:08:43.164 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.164 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.164 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.164 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-04-26T23:08:43.164 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.164 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.164 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.164 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T23:08:43.164 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.165 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.165 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.165 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.165 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.165 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-04-26T23:08:43.165 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.165 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.165 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.165 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.165 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.166 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.166 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.166 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.166 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.166 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.166 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.166 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.166 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.166 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.166 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.167 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.167 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.167 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.167 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.167 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.167 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.167 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.168 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.168 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.168 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.168 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.168 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.168 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.168 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.168 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.169 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.169 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.169 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.169 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.169 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.169 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.169 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.169 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.169 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.169 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.169 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.169 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.169 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.170 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.170 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.170 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.170 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.170 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.170 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.170 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.170 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.170 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.170 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.170 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.170 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.171 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.171 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.171 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.171 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.171 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.171 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.171 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.171 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.171 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.172 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.172 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.172 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.172 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.172 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.172 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.172 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-04-26T23:08:43.172 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.173 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.173 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.173 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.173 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.173 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.173 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.173 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.173 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.173 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.173 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.173 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.174 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.174 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.174 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.174 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.174 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.174 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.174 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.174 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.174 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.174 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.174 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.175 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.175 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.175 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.175 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.175 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.175 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.175 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.175 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.175 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.176 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.176 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.176 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.176 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.176 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.176 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.176 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.176 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.176 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.177 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.177 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.177 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.177 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.177 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.177 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.177 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.178 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.178 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.178 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.178 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.178 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.178 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.178 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.179 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.179 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.179 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.179 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.179 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.179 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.179 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.180 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.180 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.180 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.180 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.180 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.180 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.180 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.181 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.181 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-04-26T23:08:43.181 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.181 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.181 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.181 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-04-26T23:08:43.182 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-04-26T23:08:43.182 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.182 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.182 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.182 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-04-26T23:08:43.182 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.182 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.182 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.183 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T23:08:43.183 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.183 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.183 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.183 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-04-26T23:08:43.183 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-04-26T23:08:43.183 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.183 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.183 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.183 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:43.184 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-04-26T23:08:43.184 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.184 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.184 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T23:08:43.184 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.184 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.184 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.184 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.184 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.185 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.185 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.185 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.185 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.185 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.185 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.185 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.185 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.185 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.185 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.185 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.185 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.185 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.186 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.186 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.186 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.186 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.186 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.186 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.186 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.186 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.186 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.187 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.187 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.187 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.187 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.187 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.187 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.187 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.187 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.187 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.187 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.187 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.188 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.188 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.188 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.188 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.188 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.188 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.188 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.188 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.188 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.188 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.188 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.188 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.189 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.189 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.189 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.189 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.189 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.189 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.189 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.189 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.189 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.189 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.189 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.189 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.190 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.190 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.190 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.190 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.190 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.190 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.190 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.190 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.190 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.190 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-04-26T23:08:43.191 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.191 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.191 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.191 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-04-26T23:08:43.191 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-04-26T23:08:43.191 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.191 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.191 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.191 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-04-26T23:08:43.191 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.191 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.191 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.191 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-04-26T23:08:43.191 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T23:08:43.192 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.192 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.192 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.192 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.192 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.192 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.192 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.192 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.192 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.192 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.192 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.192 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.192 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.193 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.193 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.193 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.193 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.193 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.193 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.193 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.193 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.193 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.193 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.193 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.193 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.193 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.194 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.194 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.194 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.194 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.194 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.194 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.194 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.194 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.194 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.194 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.194 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.195 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.195 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.195 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.195 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.195 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.195 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.195 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.195 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.195 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.195 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.195 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.195 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.195 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.196 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.196 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.196 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.196 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.196 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.196 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.196 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.196 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.196 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.196 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.196 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-04-26T23:08:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-04-26T23:08:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-04-26T23:08:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T23:08:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-04-26T23:08:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-04-26T23:08:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-04-26T23:08:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-04-26T23:08:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-04-26T23:08:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-04-26T23:08:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-04-26T23:08:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.200 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.200 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.200 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T23:08:43.200 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.200 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.200 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.200 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-04-26T23:08:43.200 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-04-26T23:08:43.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.200 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.200 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.200 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.201 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.201 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.201 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.201 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-04-26T23:08:43.201 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.201 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.201 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.201 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-04-26T23:08:43.201 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-04-26T23:08:43.201 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.201 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.201 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.201 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-04-26T23:08:43.202 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.202 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.202 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.202 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T23:08:43.202 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T23:08:43.202 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.202 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.202 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.202 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.203 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-04-26T23:08:43.203 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.203 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.203 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.203 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.203 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.203 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.203 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.203 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.203 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.203 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.204 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.204 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.204 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.204 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.204 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.204 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.204 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.204 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.204 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.204 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.204 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.204 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.205 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.205 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.205 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.205 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.205 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.205 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.205 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.205 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.206 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.206 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.206 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.206 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.206 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.206 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.206 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.206 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.206 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.207 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.207 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.207 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.207 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.207 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.207 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.207 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.207 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.207 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.207 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.207 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.208 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.208 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.208 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.208 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.208 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.208 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.208 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.209 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.209 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.209 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.209 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.209 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.209 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.209 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.209 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.209 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.210 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-04-26T23:08:43.210 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.210 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.210 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.210 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.210 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.210 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.211 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.211 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.211 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.211 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.211 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.211 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.211 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.211 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.211 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.211 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.211 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.212 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.212 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.212 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.212 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.212 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.212 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.212 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.212 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.212 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.212 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.213 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.213 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.213 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.213 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.213 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.213 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.213 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.213 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.214 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.214 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.214 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.214 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.214 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.214 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.214 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.214 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.214 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.214 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.214 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.214 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.215 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.215 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.215 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.215 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.215 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.215 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.215 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.215 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.215 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.215 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.216 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.216 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.216 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.216 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.216 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.216 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.216 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.216 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.216 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.216 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.217 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.217 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.217 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.217 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-04-26T23:08:43.217 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.217 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.217 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.217 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.217 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.217 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.218 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.218 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.218 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.218 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.218 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.218 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.218 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.218 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.218 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.218 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.218 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.218 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.219 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.219 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.219 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.219 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.219 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.219 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.219 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.220 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.220 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.220 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.220 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.220 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.220 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.220 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.220 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.221 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.221 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.221 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.221 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.221 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.221 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.221 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.221 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.221 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.221 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.221 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.221 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.222 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.222 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.222 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.222 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.222 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.222 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.222 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.222 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.222 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.222 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.222 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.222 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.223 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.223 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.223 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.223 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.223 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.223 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.223 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.223 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.223 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.223 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.223 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.223 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.223 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.224 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-04-26T23:08:43.224 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.224 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.224 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.224 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-04-26T23:08:43.224 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-04-26T23:08:43.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.225 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.225 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.225 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.225 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.225 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.225 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.225 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T23:08:43.225 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.225 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.226 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.226 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.226 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.226 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.226 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.226 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.226 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.226 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.226 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.226 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.226 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.226 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.227 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.227 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.227 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.227 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.227 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.227 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.227 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.227 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.227 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.227 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.227 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.228 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.228 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.228 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.228 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.228 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.228 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.228 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.228 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.228 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.228 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.228 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.228 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.229 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.229 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.229 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.229 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.229 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.229 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.229 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.229 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.229 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.229 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.229 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.229 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.230 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.230 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.230 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.230 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.230 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.230 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.230 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.230 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.231 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.231 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.231 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.231 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.231 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.231 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.231 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.231 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.231 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.231 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.231 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-04-26T23:08:43.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-04-26T23:08:43.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.232 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.232 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.232 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.232 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.232 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.232 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.232 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-04-26T23:08:43.232 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-04-26T23:08:43.232 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-04-26T23:08:43.233 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.233 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-04-26T23:08:43.233 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.233 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.233 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.233 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.233 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.234 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-04-26T23:08:43.234 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.234 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.234 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.234 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.234 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.234 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.235 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.235 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.235 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.235 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.235 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.235 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.235 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.235 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.235 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.235 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.235 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.236 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.236 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.236 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.236 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.236 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.236 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.236 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.236 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.236 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.236 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.236 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.237 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.237 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.237 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.237 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.237 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.237 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.237 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.238 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.238 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.238 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.238 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.238 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.238 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.238 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.239 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.239 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.239 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.239 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.239 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.239 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.239 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.239 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.240 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.240 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.240 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.240 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.240 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.240 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.240 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.240 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.240 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.241 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.241 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.241 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.241 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.241 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.241 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.241 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.241 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.241 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.242 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-04-26T23:08:43.242 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.242 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.242 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.242 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.242 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.242 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.243 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.243 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.244 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.244 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.244 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.244 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.244 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.245 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.245 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.246 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.247 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.247 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.247 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.247 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.247 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.247 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.248 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.248 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.248 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.248 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.248 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.248 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.248 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.248 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.248 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.248 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.248 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.248 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.248 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.249 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.249 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.249 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.249 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.249 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.249 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.249 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.249 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.249 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.249 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.249 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.249 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.249 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.249 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.250 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.250 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.250 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.250 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.250 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.250 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.250 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.250 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.250 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.250 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.250 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.250 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.251 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.251 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.251 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.251 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.251 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.251 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.251 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.251 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.251 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-04-26T23:08:43.251 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.251 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.251 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.251 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.252 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.252 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.252 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.252 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.252 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.252 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.252 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.252 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.252 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.252 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.252 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.252 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.252 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.252 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.253 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.253 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.253 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.253 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.253 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.253 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.253 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.253 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.253 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.253 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.253 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.253 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.254 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.254 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.254 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.254 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.254 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.254 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.254 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.254 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.254 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.254 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.254 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.254 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.254 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.255 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.255 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.255 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.255 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.255 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.255 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.255 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.255 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.255 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.255 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.255 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.255 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.256 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.256 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.256 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.256 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.256 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.256 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.256 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.256 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.256 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T23:08:43.256 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T23:08:43.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:43.256 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.256 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T23:08:43.257 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.257 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.257 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.257 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.257 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.257 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.257 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.257 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-04-26T23:08:43.257 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-04-26T23:08:43.257 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:43.257 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.257 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.257 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-04-26T23:08:43.257 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-04-26T23:08:43.258 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.258 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.258 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-04-26T23:08:43.258 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T23:08:43.258 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.258 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.258 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.258 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.258 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-04-26T23:08:43.258 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.258 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.259 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.259 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-04-26T23:08:43.259 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-04-26T23:08:43.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.259 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.259 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.259 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.259 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.259 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.259 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.259 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T23:08:43.259 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.259 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.259 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.260 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-04-26T23:08:43.260 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-04-26T23:08:43.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.260 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.260 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.260 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.260 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.260 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.260 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.260 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.260 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.260 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.261 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.261 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.261 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.261 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.261 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.261 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.261 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.261 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.261 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.261 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.261 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.261 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.261 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.262 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.262 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.262 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.262 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.262 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.262 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.262 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.262 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.262 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.262 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.262 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.263 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.263 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.263 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.263 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.263 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.263 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.263 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.263 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.263 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.263 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.263 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.263 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.263 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.264 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.264 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.264 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.264 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.264 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.264 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.264 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.264 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.264 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.264 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.264 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.264 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.264 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.265 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.265 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.265 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.265 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.265 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.265 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.265 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.265 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.265 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.265 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.265 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.265 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-04-26T23:08:43.265 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-04-26T23:08:43.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.266 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.266 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.266 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-04-26T23:08:43.266 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.266 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.266 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.266 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-04-26T23:08:43.266 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.266 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.266 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.266 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T23:08:43.266 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T23:08:43.267 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:43.267 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.267 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.267 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T23:08:43.267 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.267 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.267 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.267 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-04-26T23:08:43.267 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.267 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.267 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.267 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.267 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.268 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-04-26T23:08:43.268 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.268 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.268 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.268 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-04-26T23:08:43.268 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-04-26T23:08:43.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.268 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.268 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.268 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.268 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.268 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.268 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.269 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.269 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.269 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.269 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-04-26T23:08:43.269 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-04-26T23:08:43.269 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.269 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.269 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.269 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.269 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.269 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.269 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.269 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T23:08:43.269 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T23:08:43.270 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.270 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.270 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.270 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.270 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.270 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.270 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.270 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.270 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.270 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.270 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.270 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.270 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.271 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.271 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.271 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.271 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.271 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.271 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.271 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.271 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.271 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.271 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.271 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.271 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.271 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.272 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.272 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.272 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.272 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.272 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.272 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.272 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.272 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.272 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.272 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.272 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.272 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.272 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.273 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.273 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.273 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.273 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.273 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.273 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.273 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.273 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.273 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.273 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.273 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.274 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.274 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.274 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.274 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.274 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.274 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.274 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.274 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.274 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.275 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.275 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.275 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.275 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.275 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.275 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.275 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.275 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-04-26T23:08:43.275 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-04-26T23:08:43.275 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.275 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.275 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.275 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.275 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.276 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.276 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.276 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.276 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.276 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.276 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.276 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T23:08:43.276 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T23:08:43.276 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.276 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.276 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.276 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-04-26T23:08:43.277 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.277 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.277 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.277 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.277 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.277 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.277 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.277 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-04-26T23:08:43.277 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-04-26T23:08:43.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:43.277 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.277 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-04-26T23:08:43.277 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.278 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.278 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.278 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.278 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.278 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.278 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T23:08:43.278 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T23:08:43.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.278 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.278 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-04-26T23:08:43.278 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.279 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.279 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.279 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T23:08:43.279 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.279 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.279 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.279 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T23:08:43.279 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T23:08:43.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.279 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.279 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-04-26T23:08:43.280 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.280 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.280 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.280 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T23:08:43.280 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.280 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.280 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.280 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-04-26T23:08:43.280 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-04-26T23:08:43.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.280 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.280 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.280 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-04-26T23:08:43.280 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.281 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.281 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.281 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.281 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.281 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.281 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.281 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-04-26T23:08:43.281 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-04-26T23:08:43.281 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.281 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.281 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.281 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-04-26T23:08:43.281 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.282 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.282 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.282 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-04-26T23:08:43.282 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-04-26T23:08:43.282 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.282 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.282 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.282 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.282 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.282 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-04-26T23:08:43.282 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-04-26T23:08:43.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.283 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.283 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-04-26T23:08:43.283 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-04-26T23:08:43.283 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-04-26T23:08:43.283 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.283 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.283 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.283 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.283 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.283 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.283 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-04-26T23:08:43.284 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.284 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.284 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.284 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.284 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.284 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.284 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.284 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.284 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.284 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.284 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.284 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.284 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.284 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.284 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.285 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.285 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.285 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.285 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.285 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.285 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.285 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.285 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.285 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.285 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.285 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.285 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.285 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.285 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.286 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.286 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.286 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.286 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.286 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.286 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.286 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.286 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.286 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.286 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.286 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.286 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.286 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.286 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.287 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.287 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.287 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.287 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.287 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.287 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.287 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.287 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.287 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.287 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.287 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.287 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.287 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-04-26T23:08:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-04-26T23:08:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.289 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.289 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T23:08:43.289 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.289 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.289 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.289 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.289 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.289 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.289 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.289 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-04-26T23:08:43.289 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T23:08:43.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.289 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.290 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.290 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.290 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.290 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.290 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.290 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.290 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.290 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.290 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.290 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-04-26T23:08:43.290 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-04-26T23:08:43.290 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.290 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.291 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.291 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.291 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.291 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.291 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.291 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.291 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.291 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.291 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.291 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.291 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.291 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-04-26T23:08:43.292 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.292 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.292 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.292 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-04-26T23:08:43.292 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-04-26T23:08:43.292 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.292 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.292 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.292 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.292 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.292 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.292 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.292 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.292 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.293 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.293 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-04-26T23:08:43.293 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-04-26T23:08:43.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.293 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.293 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.293 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.293 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.293 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.293 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.293 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.293 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.293 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.293 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.294 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.294 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.294 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.294 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.294 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.294 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.294 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.294 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.294 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.294 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.294 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.294 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.294 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.295 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.295 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.295 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.295 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.295 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.295 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.295 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.295 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.295 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.295 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.295 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.295 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.295 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.295 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.296 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.296 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.296 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.296 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.296 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.296 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.296 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.296 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.296 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.296 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.296 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.296 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.296 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.297 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.297 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.297 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.297 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.297 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.297 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.297 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.297 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.297 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.297 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.297 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.297 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.297 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.297 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.298 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.298 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.298 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.298 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.298 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.298 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.298 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.298 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.298 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.298 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.298 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.298 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.298 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T23:08:43.298 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T23:08:43.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.299 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.299 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.299 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.299 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.299 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.299 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.299 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.299 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.299 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.299 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T23:08:43.299 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T23:08:43.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.300 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.300 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.300 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.300 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.300 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.300 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.300 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.300 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.300 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.300 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.300 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.300 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.301 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-04-26T23:08:43.301 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.301 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.301 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.301 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.301 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.301 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.301 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.301 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.301 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.301 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.301 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.301 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.301 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.302 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.302 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.302 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.302 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.302 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.302 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.302 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.302 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.302 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.302 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.302 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.302 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.302 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.303 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.303 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.303 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.303 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.303 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.303 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.303 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.303 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.303 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.303 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.303 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.303 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.304 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.304 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.304 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.304 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.304 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.304 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.304 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.304 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.304 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.304 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.304 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.304 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.305 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.305 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.305 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.305 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.305 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.305 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.305 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.305 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.305 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.306 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.306 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.306 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.306 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.306 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.306 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.306 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.306 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.306 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-04-26T23:08:43.306 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.306 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.306 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.307 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.307 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.307 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.307 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.307 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.307 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.307 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.307 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.307 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.307 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.307 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.307 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.307 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.307 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.307 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.308 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.308 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.308 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.308 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.308 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.308 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.308 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.308 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.308 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.308 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.308 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.308 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.308 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.309 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.309 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.309 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.309 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.309 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.309 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.309 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.309 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.309 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.309 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.309 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.309 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.309 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.310 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.310 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.310 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.310 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.310 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.310 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.310 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.310 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.310 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.310 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.310 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.310 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.310 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.311 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.311 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.311 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.311 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.311 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.311 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.311 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.311 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.311 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.311 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.311 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-04-26T23:08:43.311 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-04-26T23:08:43.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.312 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.312 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.312 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-04-26T23:08:43.312 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.312 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.312 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.312 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T23:08:43.312 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.312 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.312 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.313 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.313 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-04-26T23:08:43.313 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.313 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.313 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.313 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.313 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.313 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.313 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.313 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.313 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.313 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.314 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.314 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.314 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.314 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.314 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.314 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.314 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.314 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.314 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.314 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.314 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.314 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.314 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.315 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.315 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.315 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.315 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.315 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.315 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.315 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.315 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.315 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.315 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.315 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.315 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.315 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.315 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.316 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.316 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.316 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.316 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.316 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.316 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.316 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.316 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.316 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.316 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.316 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.316 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.316 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.316 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.316 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.317 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.317 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.317 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.317 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.317 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.317 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.317 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.317 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.317 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.317 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.317 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.317 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.317 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.318 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.318 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.318 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.318 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.318 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-04-26T23:08:43.318 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-04-26T23:08:43.318 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.318 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.318 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.318 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.318 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.318 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.318 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.318 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.319 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.319 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.319 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.319 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-04-26T23:08:43.319 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-04-26T23:08:43.319 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.319 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.319 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.319 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-04-26T23:08:43.319 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.319 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.319 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.319 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.319 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.320 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.320 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.320 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-04-26T23:08:43.320 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-04-26T23:08:43.320 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.320 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.320 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.320 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.320 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.320 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.320 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.320 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.320 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.320 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.321 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.321 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-04-26T23:08:43.321 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-04-26T23:08:43.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.321 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.321 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.321 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.321 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.321 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.321 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.321 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.321 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.321 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.321 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.322 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-04-26T23:08:43.322 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-04-26T23:08:43.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T23:08:43.322 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.322 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.322 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.322 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.322 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.322 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.322 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.322 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.323 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.323 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-04-26T23:08:43.323 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-04-26T23:08:43.323 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.323 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.323 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.323 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.323 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.323 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.323 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.323 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.323 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.323 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.324 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-04-26T23:08:43.324 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-04-26T23:08:43.324 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.324 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.324 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.324 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.324 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.324 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.324 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.324 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.324 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.324 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.324 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.325 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-04-26T23:08:43.325 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-04-26T23:08:43.325 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.325 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.325 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.325 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.325 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.325 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.325 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.325 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.325 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.325 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.325 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.326 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-04-26T23:08:43.326 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-04-26T23:08:43.326 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.326 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.326 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.326 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.326 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T23:08:43.326 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-04-26T23:08:43.326 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.326 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.326 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.326 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.326 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.326 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-04-26T23:08:43.327 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-04-26T23:08:43.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.327 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.327 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.327 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.327 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.327 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.327 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.327 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.327 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.327 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.327 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.328 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-04-26T23:08:43.328 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-04-26T23:08:43.328 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.328 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.328 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.328 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.328 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.328 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.328 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.328 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.328 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.328 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.328 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.329 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-04-26T23:08:43.329 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-04-26T23:08:43.329 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.329 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.329 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.329 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.329 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.329 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.329 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.329 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.329 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.329 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.329 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.330 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.330 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.330 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-04-26T23:08:43.330 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.330 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.330 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.330 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-04-26T23:08:43.330 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-04-26T23:08:43.330 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.330 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.330 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.330 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.330 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.331 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.331 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.331 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T23:08:43.331 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.331 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.331 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.331 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-04-26T23:08:43.331 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-04-26T23:08:43.331 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.331 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.331 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.331 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.331 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.332 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.332 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.332 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-04-26T23:08:43.332 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.332 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.332 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.332 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.332 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.332 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.332 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.333 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.333 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.333 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.333 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.333 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.333 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.333 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.333 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.333 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.333 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.333 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.333 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.333 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.333 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.334 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.334 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.334 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.334 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.334 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.334 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.334 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.334 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.334 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.334 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.334 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.334 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.334 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.335 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.335 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.335 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.335 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.335 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.335 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.335 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.335 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.335 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.335 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.335 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.335 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.335 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.336 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.336 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.336 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.336 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.336 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.336 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.336 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.336 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.336 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.336 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.336 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.336 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.336 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.336 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.337 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.337 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.337 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.337 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.337 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.337 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.337 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.337 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.337 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-04-26T23:08:43.337 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.337 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.337 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.337 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.338 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.338 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.338 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.338 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.338 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.338 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.338 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.338 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.338 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.338 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.338 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.338 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.338 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.338 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.339 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.339 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.339 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.339 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.339 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.339 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.339 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.339 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.339 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.339 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.339 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.340 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.340 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.340 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.340 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.340 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.340 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.340 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.340 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.340 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.340 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.340 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.340 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.340 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.340 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.341 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.341 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.341 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.341 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.341 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.341 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.341 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.341 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.341 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.341 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.341 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.341 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.341 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.342 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.342 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.342 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.342 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.342 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.342 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.342 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.342 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.342 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.342 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.342 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.343 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-04-26T23:08:43.343 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.343 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.343 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.343 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.343 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.343 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.343 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.343 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.343 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.343 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.343 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.343 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.343 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.344 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.344 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.344 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.344 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.344 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.344 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.344 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.344 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.344 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.344 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.344 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.344 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.344 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.345 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.345 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.345 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.345 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.345 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.345 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.345 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.345 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.345 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.345 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.345 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.345 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.346 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.346 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.346 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.346 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.346 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.346 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.346 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.346 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.346 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.346 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.346 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.346 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.346 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.346 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.346 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.347 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.347 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.347 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.347 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.347 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.347 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.347 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.347 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.347 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.347 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.347 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.348 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.348 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.348 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.348 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.348 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-04-26T23:08:43.348 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.348 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.348 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.348 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-04-26T23:08:43.348 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-04-26T23:08:43.348 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.348 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.349 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.349 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T23:08:43.349 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.349 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.349 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.349 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.349 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.349 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.349 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.349 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T23:08:43.349 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T23:08:43.349 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:43.349 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.350 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.350 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-04-26T23:08:43.350 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.350 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.350 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.350 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.350 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.350 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.350 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.350 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.350 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.350 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.350 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.351 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.351 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.351 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.351 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.351 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.351 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.351 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.351 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.351 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.351 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.351 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.351 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.351 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.351 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.351 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.352 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.352 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.352 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.352 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.352 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.352 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.352 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.352 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.352 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.352 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.352 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.352 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.352 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.353 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.353 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.353 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.353 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.353 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.353 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.353 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.353 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.353 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.353 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.353 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.353 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.354 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.354 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.354 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.354 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.354 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.354 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.354 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.354 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.354 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.354 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.354 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.354 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.354 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.355 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.355 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.355 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.355 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.355 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.355 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.355 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.355 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.355 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.355 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-04-26T23:08:43.355 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.355 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-04-26T23:08:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-04-26T23:08:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T23:08:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-04-26T23:08:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-04-26T23:08:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T23:08:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-04-26T23:08:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-04-26T23:08:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T23:08:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-04-26T23:08:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-04-26T23:08:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T23:08:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-04-26T23:08:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-04-26T23:08:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T23:08:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-04-26T23:08:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-04-26T23:08:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-04-26T23:08:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-04-26T23:08:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-04-26T23:08:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-04-26T23:08:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T23:08:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-04-26T23:08:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-04-26T23:08:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T23:08:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T23:08:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-04-26T23:08:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-04-26T23:08:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-04-26T23:08:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-04-26T23:08:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-04-26T23:08:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-04-26T23:08:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-04-26T23:08:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-04-26T23:08:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-04-26T23:08:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-04-26T23:08:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-04-26T23:08:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-04-26T23:08:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-04-26T23:08:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-04-26T23:08:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-04-26T23:08:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T23:08:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-04-26T23:08:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-04-26T23:08:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-04-26T23:08:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-04-26T23:08:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-04-26T23:08:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T23:08:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-04-26T23:08:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-04-26T23:08:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-04-26T23:08:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-04-26T23:08:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-04-26T23:08:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T23:08:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T23:08:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-04-26T23:08:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T23:08:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T23:08:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T23:08:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-04-26T23:08:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T23:08:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T23:08:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T23:08:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-04-26T23:08:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-04-26T23:08:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T23:08:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-04-26T23:08:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-04-26T23:08:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-04-26T23:08:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T23:08:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T23:08:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T23:08:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T23:08:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T23:08:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T23:08:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T23:08:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T23:08:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T23:08:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T23:08:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T23:08:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T23:08:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T23:08:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T23:08:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T23:08:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T23:08:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T23:08:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T23:08:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T23:08:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T23:08:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T23:08:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T23:08:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T23:08:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-04-26T23:08:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-04-26T23:08:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T23:08:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-04-26T23:08:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-04-26T23:08:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T23:08:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-04-26T23:08:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-04-26T23:08:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-04-26T23:08:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T23:08:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T23:08:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T23:08:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T23:08:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T23:08:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T23:08:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T23:08:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T23:08:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T23:08:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T23:08:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T23:08:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "services": { 2024-04-26T23:08:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "dashboard": "https://172.21.15.27:8443/" 2024-04-26T23:08:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "always_on_modules": { 2024-04-26T23:08:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "octopus": [ 2024-04-26T23:08:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-04-26T23:08:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-04-26T23:08:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-04-26T23:08:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-04-26T23:08:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-04-26T23:08:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-04-26T23:08:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-04-26T23:08:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-04-26T23:08:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-04-26T23:08:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-04-26T23:08:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "pacific": [ 2024-04-26T23:08:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-04-26T23:08:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-04-26T23:08:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-04-26T23:08:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-04-26T23:08:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-04-26T23:08:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-04-26T23:08:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-04-26T23:08:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-04-26T23:08:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-04-26T23:08:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-04-26T23:08:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "quincy": [ 2024-04-26T23:08:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-04-26T23:08:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-04-26T23:08:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-04-26T23:08:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-04-26T23:08:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-04-26T23:08:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-04-26T23:08:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-04-26T23:08:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-04-26T23:08:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-04-26T23:08:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-04-26T23:08:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "reef": [ 2024-04-26T23:08:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-04-26T23:08:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-04-26T23:08:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-04-26T23:08:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-04-26T23:08:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-04-26T23:08:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-04-26T23:08:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-04-26T23:08:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-04-26T23:08:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-04-26T23:08:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-04-26T23:08:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T23:08:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "squid": [ 2024-04-26T23:08:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-04-26T23:08:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-04-26T23:08:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-04-26T23:08:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-04-26T23:08:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-04-26T23:08:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-04-26T23:08:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-04-26T23:08:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-04-26T23:08:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-04-26T23:08:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-04-26T23:08:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T23:08:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "last_failure_osd_epoch": 95, 2024-04-26T23:08:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "active_clients": [ 2024-04-26T23:08:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-04-26T23:08:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-04-26T23:08:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-04-26T23:08:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-04-26T23:08:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2700279863 2024-04-26T23:08:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T23:08:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "libcephsqlite", 2024-04-26T23:08:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-04-26T23:08:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-04-26T23:08:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-04-26T23:08:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1006243127 2024-04-26T23:08:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T23:08:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-04-26T23:08:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-04-26T23:08:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-04-26T23:08:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-04-26T23:08:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 784949520 2024-04-26T23:08:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T23:08:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-04-26T23:08:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-04-26T23:08:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-04-26T23:08:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-04-26T23:08:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2462684438 2024-04-26T23:08:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T23:08:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-04-26T23:08:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-04-26T23:08:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-04-26T23:08:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-04-26T23:08:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3328059543 2024-04-26T23:08:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T23:08:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-04-26T23:08:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-04-26T23:08:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T23:08:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-04-26T23:08:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-04-26T23:08:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3098695798 2024-04-26T23:08:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T23:08:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T23:08:43.411 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T23:08:43.413 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:42 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2926173439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:08:43.413 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:42 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1689222691' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T23:08:43.413 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:42 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2926173439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:08:43.413 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:42 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1689222691' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T23:08:43.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.968+0000 7fe5d6600640 1 -- 172.21.15.27:0/1689222691 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe5b80a3d90 msgr2=0x7fe5b80a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:43.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.968+0000 7fe5d6600640 1 --2- 172.21.15.27:0/1689222691 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe5b80a3d90 0x7fe5b80a6250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc02a9e0 tx=0x7fe5bc005a20 comp rx=0 tx=0).stop 2024-04-26T23:08:43.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.968+0000 7fe5d6600640 1 -- 172.21.15.27:0/1689222691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5d01090e0 msgr2=0x7fe5d019f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:08:43.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.968+0000 7fe5d6600640 1 --2- 172.21.15.27:0/1689222691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5d01090e0 0x7fe5d019f670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400ea60 tx=0x7fe5c400ef20 comp rx=0 tx=0).stop 2024-04-26T23:08:43.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.968+0000 7fe5d6600640 1 -- 172.21.15.27:0/1689222691 shutdown_connections 2024-04-26T23:08:43.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.968+0000 7fe5d6600640 1 --2- 172.21.15.27:0/1689222691 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fe5b80a3d90 0x7fe5b80a6250 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:43.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.968+0000 7fe5d6600640 1 --2- 172.21.15.27:0/1689222691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5d01a3ee0 0x7fe5d01a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:43.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.968+0000 7fe5d6600640 1 --2- 172.21.15.27:0/1689222691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d010e160 0x7fe5d019fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:43.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.968+0000 7fe5d6600640 1 --2- 172.21.15.27:0/1689222691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5d01090e0 0x7fe5d019f670 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:08:43.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.968+0000 7fe5d6600640 1 -- 172.21.15.27:0/1689222691 >> 172.21.15.27:0/1689222691 conn(0x7fe5d0078700 msgr2=0x7fe5d007a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:08:43.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.968+0000 7fe5d6600640 1 -- 172.21.15.27:0/1689222691 shutdown_connections 2024-04-26T23:08:43.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:08:41.968+0000 7fe5d6600640 1 -- 172.21.15.27:0/1689222691 wait complete. 2024-04-26T23:08:43.420 DEBUG:tasks.check_counter:Getting stats from a 2024-04-26T23:08:43.420 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-26T23:08:43.420 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:42 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2926173439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:08:43.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:42 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1689222691' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T23:08:43.505 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:43 smithi027 ceph-mon[28034]: pgmap v312: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T23:08:43.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:43 smithi080 ceph-mon[32324]: pgmap v312: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T23:08:43.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:43 smithi094 ceph-mon[32889]: pgmap v312: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T23:08:43.894 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:08:44.297 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:44 smithi027 ceph-mon[28034]: pgmap v313: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T23:08:44.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:44 smithi080 ceph-mon[32324]: pgmap v313: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T23:08:44.534 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:44 smithi094 ceph-mon[32889]: pgmap v313: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T23:08:46.666 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T23:08:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T23:08:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 8068, 2024-04-26T23:08:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 8097, 2024-04-26T23:08:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 2368985, 2024-04-26T23:08:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 44389824, 2024-04-26T23:08:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 14, 2024-04-26T23:08:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 4, 2024-04-26T23:08:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 1.708339417, 2024-04-26T23:08:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.651006227, 2024-04-26T23:08:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.320181921, 2024-04-26T23:08:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.572127532, 2024-04-26T23:08:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:08:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 8096, 2024-04-26T23:08:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 2.313107310, 2024-04-26T23:08:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000285709 2024-04-26T23:08:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:08:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 82, 2024-04-26T23:08:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000029761, 2024-04-26T23:08:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000362 2024-04-26T23:08:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 10048, 2024-04-26T23:08:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 15856 2024-04-26T23:08:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T23:08:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 28735, 2024-04-26T23:08:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 64785, 2024-04-26T23:08:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 9270274, 2024-04-26T23:08:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 72818536, 2024-04-26T23:08:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 14, 2024-04-26T23:08:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 6, 2024-04-26T23:08:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 10.411125447, 2024-04-26T23:08:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 6.690181845, 2024-04-26T23:08:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 5.490940596, 2024-04-26T23:08:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.411719531, 2024-04-26T23:08:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:08:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 64782, 2024-04-26T23:08:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 62.171938793, 2024-04-26T23:08:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000959710 2024-04-26T23:08:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:08:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 56581, 2024-04-26T23:08:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.034504289, 2024-04-26T23:08:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000609 2024-04-26T23:08:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T23:08:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 453920 2024-04-26T23:08:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T23:08:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 43249, 2024-04-26T23:08:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 59771, 2024-04-26T23:08:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 33212686, 2024-04-26T23:08:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 38306506, 2024-04-26T23:08:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 7, 2024-04-26T23:08:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 6, 2024-04-26T23:08:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 10.217326882, 2024-04-26T23:08:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 5.099927346, 2024-04-26T23:08:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 6.287598267, 2024-04-26T23:08:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.376752849, 2024-04-26T23:08:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:08:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 59770, 2024-04-26T23:08:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 7.151850395, 2024-04-26T23:08:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000119656 2024-04-26T23:08:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:08:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 90573, 2024-04-26T23:08:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.042933379, 2024-04-26T23:08:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000474 2024-04-26T23:08:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1676400, 2024-04-26T23:08:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 142560 2024-04-26T23:08:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-04-26T23:08:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 1, 2024-04-26T23:08:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-04-26T23:08:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-MDSRank": { 2024-04-26T23:08:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-04-26T23:08:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-04-26T23:08:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2771, 2024-04-26T23:08:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 26.452837200, 2024-04-26T23:08:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.009546314 2024-04-26T23:08:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-PurgeQueue": { 2024-04-26T23:08:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-04-26T23:08:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-04-26T23:08:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 12855, 2024-04-26T23:08:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 7.415123886, 2024-04-26T23:08:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000576827 2024-04-26T23:08:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2024-04-26T23:08:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "request": 16480, 2024-04-26T23:08:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "reply": 16405, 2024-04-26T23:08:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "reply_latency": { 2024-04-26T23:08:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 16405, 2024-04-26T23:08:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 65.822318567, 2024-04-26T23:08:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.004012332 2024-04-26T23:08:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_reply": 0, 2024-04-26T23:08:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "forward": 71, 2024-04-26T23:08:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_complete": 57, 2024-04-26T23:08:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_keys": 0, 2024-04-26T23:08:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_commit": 61, 2024-04-26T23:08:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_split": 6, 2024-04-26T23:08:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_merge": 0, 2024-04-26T23:08:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes": 23601, 2024-04-26T23:08:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_top": 1875, 2024-04-26T23:08:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_bottom": 20168, 2024-04-26T23:08:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pin_tail": 1558, 2024-04-26T23:08:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pinned": 20920, 2024-04-26T23:08:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_expired": 7089, 2024-04-26T23:08:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_with_caps": 1, 2024-04-26T23:08:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": 1, 2024-04-26T23:08:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "subtrees": 4, 2024-04-26T23:08:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse": 25611, 2024-04-26T23:08:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_hit": 22470, 2024-04-26T23:08:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_forward": 51, 2024-04-26T23:08:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_discover": 832, 2024-04-26T23:08:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_dir_fetch": 0, 2024-04-26T23:08:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_remote_ino": 0, 2024-04-26T23:08:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_lock": 14, 2024-04-26T23:08:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "load_cent": 0, 2024-04-26T23:08:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "q": 0, 2024-04-26T23:08:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "exported": 42, 2024-04-26T23:08:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "exported_inodes": 5243, 2024-04-26T23:08:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "imported": 52, 2024-04-26T23:08:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "imported_inodes": 17290, 2024-04-26T23:08:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_dir_fetch": 0, 2024-04-26T23:08:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_backtrace_fetch": 0, 2024-04-26T23:08:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_peer_discover": 2, 2024-04-26T23:08:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rfiles": 10835, 2024-04-26T23:08:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rbytes": 248684371, 2024-04-26T23:08:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rsnaps": 4, 2024-04-26T23:08:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T23:08:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_set_tag": 0, 2024-04-26T23:08:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T23:08:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_inotable_repaired": 0, 2024-04-26T23:08:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_inodes": 0, 2024-04-26T23:08:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T23:08:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T23:08:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_file_inodes": 0, 2024-04-26T23:08:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_inode_file_caps": 26, 2024-04-26T23:08:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_revoke": 2118, 2024-04-26T23:08:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_grant": 11564, 2024-04-26T23:08:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T23:08:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T23:08:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T23:08:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps": 5804, 2024-04-26T23:08:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps_dirty": 3286, 2024-04-26T23:08:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_cap_release": 93, 2024-04-26T23:08:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "process_request_cap_release": 11434 2024-04-26T23:08:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache": { 2024-04-26T23:08:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update": 1790, 2024-04-26T23:08:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update_receipt": 2381, 2024-04-26T23:08:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_try_discover": 1589, 2024-04-26T23:08:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_send_discover": 1594, 2024-04-26T23:08:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_handle_discover": 3444, 2024-04-26T23:08:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays": 1024, 2024-04-26T23:08:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_delayed": 0, 2024-04-26T23:08:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_enqueuing": 0, 2024-04-26T23:08:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_created": 8932, 2024-04-26T23:08:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_enqueued": 7901, 2024-04-26T23:08:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_reintegrated": 38, 2024-04-26T23:08:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_migrated": 0, 2024-04-26T23:08:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_processing": 0, 2024-04-26T23:08:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_enqueued": 0, 2024-04-26T23:08:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_prioritized": 0, 2024-04-26T23:08:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_started": 0, 2024-04-26T23:08:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_completed": 0, 2024-04-26T23:08:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_path": 0, 2024-04-26T23:08:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_inode": 0, 2024-04-26T23:08:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T23:08:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_exportdir": 42, 2024-04-26T23:08:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_flush": 0, 2024-04-26T23:08:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragmentdir": 6, 2024-04-26T23:08:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragstats": 0, 2024-04-26T23:08:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_inodestats": 0 2024-04-26T23:08:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log": { 2024-04-26T23:08:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "evlrg": 3, 2024-04-26T23:08:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "evadd": 28027, 2024-04-26T23:08:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "evex": 0, 2024-04-26T23:08:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "evtrm": 0, 2024-04-26T23:08:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "ev": 28027, 2024-04-26T23:08:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "evexg": 0, 2024-04-26T23:08:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "evexd": 0, 2024-04-26T23:08:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "segadd": 42, 2024-04-26T23:08:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "segex": 0, 2024-04-26T23:08:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "segtrm": 0, 2024-04-26T23:08:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "seg": 42, 2024-04-26T23:08:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "segmjr": 4, 2024-04-26T23:08:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "segexg": 0, 2024-04-26T23:08:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "segexd": 0, 2024-04-26T23:08:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "expos": 4194304, 2024-04-26T23:08:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "wrpos": 94236401, 2024-04-26T23:08:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "rdpos": 0, 2024-04-26T23:08:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "jlat": { 2024-04-26T23:08:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1549, 2024-04-26T23:08:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 53.746230316, 2024-04-26T23:08:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.034697372 2024-04-26T23:08:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "replayed": 0 2024-04-26T23:08:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mem": { 2024-04-26T23:08:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 13018, 2024-04-26T23:08:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "ino+": 26760, 2024-04-26T23:08:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "ino-": 13742, 2024-04-26T23:08:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "dir": 1143, 2024-04-26T23:08:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "dir+": 2481, 2024-04-26T23:08:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "dir-": 1338, 2024-04-26T23:08:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "dn": 23601, 2024-04-26T23:08:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "dn+": 38620, 2024-04-26T23:08:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "dn-": 15019, 2024-04-26T23:08:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "cap": 1, 2024-04-26T23:08:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "cap+": 13402, 2024-04-26T23:08:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "cap-": 13401, 2024-04-26T23:08:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "rss": 182712, 2024-04-26T23:08:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "heap": 223608 2024-04-26T23:08:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server": { 2024-04-26T23:08:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_client_request": 24752, 2024-04-26T23:08:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_server_request": 570, 2024-04-26T23:08:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_request": 16480, 2024-04-26T23:08:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_session": 50, 2024-04-26T23:08:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_peer_request": 1320, 2024-04-26T23:08:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "req_create_latency": { 2024-04-26T23:08:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 812, 2024-04-26T23:08:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 1.373563647, 2024-04-26T23:08:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.001691580 2024-04-26T23:08:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getattr_latency": { 2024-04-26T23:08:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 19, 2024-04-26T23:08:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.166555381, 2024-04-26T23:08:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.008766072 2024-04-26T23:08:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getfilelock_latency": { 2024-04-26T23:08:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "req_link_latency": { 2024-04-26T23:08:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookup_latency": { 2024-04-26T23:08:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 5316, 2024-04-26T23:08:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 5.428054572, 2024-04-26T23:08:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.001021078 2024-04-26T23:08:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookuphash_latency": { 2024-04-26T23:08:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupino_latency": { 2024-04-26T23:08:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupname_latency": { 2024-04-26T23:08:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupparent_latency": { 2024-04-26T23:08:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupsnap_latency": { 2024-04-26T23:08:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lssnap_latency": { 2024-04-26T23:08:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mkdir_latency": { 2024-04-26T23:08:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 185, 2024-04-26T23:08:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.878698485, 2024-04-26T23:08:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.004749721 2024-04-26T23:08:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mknod_latency": { 2024-04-26T23:08:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mksnap_latency": { 2024-04-26T23:08:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "req_open_latency": { 2024-04-26T23:08:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "req_readdir_latency": { 2024-04-26T23:08:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1063, 2024-04-26T23:08:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 1.757874692, 2024-04-26T23:08:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.001653692 2024-04-26T23:08:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rename_latency": { 2024-04-26T23:08:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 39, 2024-04-26T23:08:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 1.271671084, 2024-04-26T23:08:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.032606950 2024-04-26T23:08:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "req_renamesnap_latency": { 2024-04-26T23:08:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "req_snapdiff_latency": { 2024-04-26T23:08:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmdir_latency": { 2024-04-26T23:08:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 964, 2024-04-26T23:08:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 29.630924682, 2024-04-26T23:08:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.030737473 2024-04-26T23:08:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmsnap_latency": { 2024-04-26T23:08:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmxattr_latency": { 2024-04-26T23:08:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setattr_latency": { 2024-04-26T23:08:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1, 2024-04-26T23:08:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.003241160, 2024-04-26T23:08:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.003241160 2024-04-26T23:08:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setdirlayout_latency": { 2024-04-26T23:08:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setfilelock_latency": { 2024-04-26T23:08:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setlayout_latency": { 2024-04-26T23:08:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setxattr_latency": { 2024-04-26T23:08:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "req_symlink_latency": { 2024-04-26T23:08:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "req_unlink_latency": { 2024-04-26T23:08:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 8006, 2024-04-26T23:08:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 25.311734864, 2024-04-26T23:08:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.003161595 2024-04-26T23:08:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_revoke_eviction": 0, 2024-04-26T23:08:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_acquisition_throttle": 0, 2024-04-26T23:08:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getvxattr_latency": { 2024-04-26T23:08:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessions": { 2024-04-26T23:08:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "session_count": 2, 2024-04-26T23:08:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "session_add": 4, 2024-04-26T23:08:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "session_remove": 2, 2024-04-26T23:08:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_open": 2, 2024-04-26T23:08:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_stale": 0, 2024-04-26T23:08:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "total_load": 8805, 2024-04-26T23:08:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "average_load": 4402, 2024-04-26T23:08:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "avg_session_uptime": 598, 2024-04-26T23:08:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "mdthresh_evicted": 0 2024-04-26T23:08:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-04-26T23:08:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-04-26T23:08:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-04-26T23:08:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T23:08:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-04-26T23:08:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T23:08:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-04-26T23:08:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T23:08:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T23:08:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T23:08:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T23:08:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T23:08:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-04-26T23:08:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T23:08:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T23:08:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-04-26T23:08:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-04-26T23:08:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-04-26T23:08:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-04-26T23:08:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T23:08:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T23:08:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T23:08:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T23:08:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T23:08:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-04-26T23:08:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-04-26T23:08:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-04-26T23:08:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T23:08:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T23:08:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-04-26T23:08:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-04-26T23:08:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-04-26T23:08:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-04-26T23:08:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T23:08:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-04-26T23:08:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T23:08:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-04-26T23:08:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 8996013, 2024-04-26T23:08:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 15004, 2024-04-26T23:08:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-04-26T23:08:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-04-26T23:08:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-04-26T23:08:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-04-26T23:08:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-04-26T23:08:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-04-26T23:08:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-04-26T23:08:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-04-26T23:08:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 10232, 2024-04-26T23:08:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 215, 2024-04-26T23:08:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T23:08:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-04-26T23:08:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-04-26T23:08:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-04-26T23:08:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 50931563, 2024-04-26T23:08:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 1015788, 2024-04-26T23:08:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-04-26T23:08:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-04-26T23:08:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-04-26T23:08:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-04-26T23:08:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter": { 2024-04-26T23:08:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "op_active": 0, 2024-04-26T23:08:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "op_laggy": 0, 2024-04-26T23:08:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send": 18092, 2024-04-26T23:08:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send_bytes": 92790455, 2024-04-26T23:08:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "op_resend": 33, 2024-04-26T23:08:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "op_reply": 18092, 2024-04-26T23:08:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "op_latency": { 2024-04-26T23:08:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 18092, 2024-04-26T23:08:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 156.825961122, 2024-04-26T23:08:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.008668249 2024-04-26T23:08:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "op_inflight": 0, 2024-04-26T23:08:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "oplen_avg": { 2024-04-26T23:08:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 18092, 2024-04-26T23:08:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 18447 2024-04-26T23:08:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "op": 18092, 2024-04-26T23:08:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "op_r": 254, 2024-04-26T23:08:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "op_w": 17838, 2024-04-26T23:08:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "op_rmw": 0, 2024-04-26T23:08:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "op_pg": 0, 2024-04-26T23:08:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_stat": 0, 2024-04-26T23:08:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_create": 10, 2024-04-26T23:08:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_read": 197, 2024-04-26T23:08:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_write": 1746, 2024-04-26T23:08:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writefull": 84, 2024-04-26T23:08:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writesame": 0, 2024-04-26T23:08:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_append": 0, 2024-04-26T23:08:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_zero": 28, 2024-04-26T23:08:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_truncate": 6, 2024-04-26T23:08:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_delete": 15797, 2024-04-26T23:08:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_mapext": 0, 2024-04-26T23:08:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_sparse_read": 0, 2024-04-26T23:08:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_clonerange": 0, 2024-04-26T23:08:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_getxattr": 49, 2024-04-26T23:08:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_setxattr": 20, 2024-04-26T23:08:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_cmpxattr": 0, 2024-04-26T23:08:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_rmxattr": 0, 2024-04-26T23:08:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_resetxattrs": 0, 2024-04-26T23:08:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_call": 0, 2024-04-26T23:08:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_watch": 0, 2024-04-26T23:08:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_notify": 0, 2024-04-26T23:08:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T23:08:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls": 0, 2024-04-26T23:08:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls_filter": 0, 2024-04-26T23:08:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_other": 0, 2024-04-26T23:08:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_active": 0, 2024-04-26T23:08:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_send": 0, 2024-04-26T23:08:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_resend": 0, 2024-04-26T23:08:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_ping": 0, 2024-04-26T23:08:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_active": 0, 2024-04-26T23:08:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_send": 0, 2024-04-26T23:08:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_resend": 0, 2024-04-26T23:08:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_active": 0, 2024-04-26T23:08:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_send": 0, 2024-04-26T23:08:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_resend": 0, 2024-04-26T23:08:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_active": 0, 2024-04-26T23:08:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_send": 0, 2024-04-26T23:08:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_resend": 0, 2024-04-26T23:08:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "command_active": 0, 2024-04-26T23:08:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "command_send": 0, 2024-04-26T23:08:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "command_resend": 0, 2024-04-26T23:08:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "map_epoch": 101, 2024-04-26T23:08:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "map_full": 0, 2024-04-26T23:08:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "map_inc": 11, 2024-04-26T23:08:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_sessions": 12, 2024-04-26T23:08:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_open": 12, 2024-04-26T23:08:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_close": 0, 2024-04-26T23:08:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_laggy": 0, 2024-04-26T23:08:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_wr": 313, 2024-04-26T23:08:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_rd": 114, 2024-04-26T23:08:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_del": 83 2024-04-26T23:08:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "oft": { 2024-04-26T23:08:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_objs": 1, 2024-04-26T23:08:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_kv_pairs": 0, 2024-04-26T23:08:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_updates": 0, 2024-04-26T23:08:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_removes": 0 2024-04-26T23:08:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "osd-slow-ops": { 2024-04-26T23:08:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_ops_count": 0 2024-04-26T23:08:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "purge_queue": { 2024-04-26T23:08:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops": 0, 2024-04-26T23:08:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops_high_water": 22, 2024-04-26T23:08:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing": 0, 2024-04-26T23:08:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_high_water": 11, 2024-04-26T23:08:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed_ops": 15792, 2024-04-26T23:08:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed": 7901, 2024-04-26T23:08:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_item_in_journal": 0 2024-04-26T23:08:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T23:08:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T23:08:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-04-26T23:08:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T23:08:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 80048, 2024-04-26T23:08:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 37628831, 2024-04-26T23:08:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T23:08:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 80048, 2024-04-26T23:08:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-04-26T23:08:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-04-26T23:08:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 80048, 2024-04-26T23:08:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 37628831, 2024-04-26T23:08:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T23:08:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_bytes": { 2024-04-26T23:08:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T23:08:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-04-26T23:08:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T23:08:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-04-26T23:08:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-04-26T23:08:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T23:08:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-04-26T23:08:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 18092, 2024-04-26T23:08:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 93789832, 2024-04-26T23:08:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 18092, 2024-04-26T23:08:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 93789832, 2024-04-26T23:08:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T23:08:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_ops": { 2024-04-26T23:08:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T23:08:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 1024, 2024-04-26T23:08:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T23:08:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-04-26T23:08:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-04-26T23:08:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T23:08:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-04-26T23:08:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 18092, 2024-04-26T23:08:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 18092, 2024-04-26T23:08:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 18092, 2024-04-26T23:08:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 18092, 2024-04-26T23:08:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T23:08:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle": { 2024-04-26T23:08:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T23:08:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-04-26T23:08:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T23:08:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 7901, 2024-04-26T23:08:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1000289, 2024-04-26T23:08:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T23:08:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 7901, 2024-04-26T23:08:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-04-26T23:08:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-04-26T23:08:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 197, 2024-04-26T23:08:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1000289, 2024-04-26T23:08:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T23:08:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle-0x555ddfe4a460": { 2024-04-26T23:08:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T23:08:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-04-26T23:08:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T23:08:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 28027, 2024-04-26T23:08:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 90042097, 2024-04-26T23:08:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T23:08:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 28027, 2024-04-26T23:08:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-04-26T23:08:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-04-26T23:08:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 1549, 2024-04-26T23:08:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 90042097, 2024-04-26T23:08:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T23:08:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:46.711 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T23:08:47.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:46 smithi027 ceph-mon[28034]: pgmap v314: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T23:08:47.274 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:46 smithi080 ceph-mon[32324]: pgmap v314: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T23:08:47.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:46 smithi094 ceph-mon[32889]: pgmap v314: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T23:08:47.852 INFO:tasks.check_counter:Daemon mds.a mds.exported=42 2024-04-26T23:08:47.852 INFO:tasks.check_counter:Daemon mds.a mds.imported=52 2024-04-26T23:08:47.852 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=1790 2024-04-26T23:08:47.852 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=2381 2024-04-26T23:08:47.852 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=4 2024-04-26T23:08:47.853 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-26T23:08:47.853 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T23:08:47.853 DEBUG:tasks.check_counter:Getting stats from d 2024-04-26T23:08:47.853 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-26T23:08:48.329 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:08:49.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:48 smithi027 ceph-mon[28034]: pgmap v315: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T23:08:49.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:48 smithi080 ceph-mon[32324]: pgmap v315: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T23:08:49.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:48 smithi094 ceph-mon[32889]: pgmap v315: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T23:08:50.818 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T23:08:50.818 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T23:08:50.818 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 0, 2024-04-26T23:08:50.818 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 2, 2024-04-26T23:08:50.819 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 0, 2024-04-26T23:08:50.819 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 0, 2024-04-26T23:08:50.819 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-04-26T23:08:50.819 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-04-26T23:08:50.819 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.004280321, 2024-04-26T23:08:50.819 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T23:08:50.819 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.000767654, 2024-04-26T23:08:50.819 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T23:08:50.819 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:08:50.819 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:50.819 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:50.820 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:50.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:50.820 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:08:50.820 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:50.820 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:50.820 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:50.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:50.820 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T23:08:50.820 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T23:08:50.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:50.820 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T23:08:50.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 239, 2024-04-26T23:08:50.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 226, 2024-04-26T23:08:50.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1596384, 2024-04-26T23:08:50.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 138592, 2024-04-26T23:08:50.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-04-26T23:08:50.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-04-26T23:08:50.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.116500746, 2024-04-26T23:08:50.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.024150731, 2024-04-26T23:08:50.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.127359273, 2024-04-26T23:08:50.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.011017849, 2024-04-26T23:08:50.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:08:50.821 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 226, 2024-04-26T23:08:50.821 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.022868785, 2024-04-26T23:08:50.822 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000101189 2024-04-26T23:08:50.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:50.822 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:08:50.822 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:50.822 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:50.822 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:50.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:50.822 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1596384, 2024-04-26T23:08:50.822 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 138592 2024-04-26T23:08:50.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:50.822 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T23:08:50.822 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 2, 2024-04-26T23:08:50.823 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 171, 2024-04-26T23:08:50.823 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 320, 2024-04-26T23:08:50.823 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 187200, 2024-04-26T23:08:50.823 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 5, 2024-04-26T23:08:50.823 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-04-26T23:08:50.823 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.039834059, 2024-04-26T23:08:50.823 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.026900680, 2024-04-26T23:08:50.823 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.006834070, 2024-04-26T23:08:50.823 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T23:08:50.823 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:08:50.823 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 168, 2024-04-26T23:08:50.823 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.038158189, 2024-04-26T23:08:50.823 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000227132 2024-04-26T23:08:50.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:50.824 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:08:50.824 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:50.824 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:50.824 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:50.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:50.824 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T23:08:50.824 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 187200 2024-04-26T23:08:50.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:50.824 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-04-26T23:08:50.824 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-04-26T23:08:50.824 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-04-26T23:08:50.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:50.825 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-04-26T23:08:50.825 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-04-26T23:08:50.825 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-04-26T23:08:50.825 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T23:08:50.825 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-04-26T23:08:50.825 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T23:08:50.825 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-04-26T23:08:50.825 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T23:08:50.825 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T23:08:50.825 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T23:08:50.825 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T23:08:50.825 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T23:08:50.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-04-26T23:08:50.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T23:08:50.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T23:08:50.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-04-26T23:08:50.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-04-26T23:08:50.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-04-26T23:08:50.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-04-26T23:08:50.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T23:08:50.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T23:08:50.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T23:08:50.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T23:08:50.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T23:08:50.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-04-26T23:08:50.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-04-26T23:08:50.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-04-26T23:08:50.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T23:08:50.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T23:08:50.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-04-26T23:08:50.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-04-26T23:08:50.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-04-26T23:08:50.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-04-26T23:08:50.828 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T23:08:50.828 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-04-26T23:08:50.828 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T23:08:50.828 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-04-26T23:08:50.828 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 149745, 2024-04-26T23:08:50.828 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 31, 2024-04-26T23:08:50.828 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-04-26T23:08:50.828 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-04-26T23:08:50.828 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-04-26T23:08:50.828 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-04-26T23:08:50.828 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-04-26T23:08:50.828 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-04-26T23:08:50.829 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-04-26T23:08:50.829 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-04-26T23:08:50.829 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-04-26T23:08:50.829 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-04-26T23:08:50.829 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T23:08:50.829 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-04-26T23:08:50.829 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-04-26T23:08:50.829 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-04-26T23:08:50.829 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 928, 2024-04-26T23:08:50.829 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-04-26T23:08:50.829 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-04-26T23:08:50.829 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-04-26T23:08:50.829 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-04-26T23:08:50.830 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-04-26T23:08:50.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:50.830 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T23:08:50.830 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T23:08:50.830 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-04-26T23:08:50.830 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T23:08:50.830 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 237, 2024-04-26T23:08:50.830 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1560263, 2024-04-26T23:08:50.830 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T23:08:50.830 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 237, 2024-04-26T23:08:50.831 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-04-26T23:08:50.831 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-04-26T23:08:50.831 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 237, 2024-04-26T23:08:50.831 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1560263, 2024-04-26T23:08:50.831 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T23:08:50.831 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:50.831 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:50.831 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:50.831 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:50.831 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:50.831 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T23:08:51.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:50 smithi027 ceph-mon[28034]: pgmap v316: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T23:08:51.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:50 smithi080 ceph-mon[32324]: pgmap v316: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T23:08:51.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:50 smithi094 ceph-mon[32889]: pgmap v316: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T23:08:51.824 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-26T23:08:51.824 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-26T23:08:51.824 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-26T23:08:51.824 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-26T23:08:51.824 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-26T23:08:51.825 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-26T23:08:51.825 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-26T23:08:51.825 DEBUG:tasks.check_counter:Getting stats from g 2024-04-26T23:08:51.825 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-26T23:08:52.302 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:08:53.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:52 smithi027 ceph-mon[28034]: pgmap v317: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T23:08:53.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:52 smithi080 ceph-mon[32324]: pgmap v317: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T23:08:53.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:52 smithi094 ceph-mon[32889]: pgmap v317: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T23:08:54.933 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T23:08:54.933 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T23:08:54.934 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 49538, 2024-04-26T23:08:54.934 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 27025, 2024-04-26T23:08:54.934 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 29450612, 2024-04-26T23:08:54.934 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 44857741, 2024-04-26T23:08:54.934 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 10, 2024-04-26T23:08:54.934 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-04-26T23:08:54.934 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 8.140464831, 2024-04-26T23:08:54.934 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 3.907076749, 2024-04-26T23:08:54.934 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 6.054973286, 2024-04-26T23:08:54.934 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.272974571, 2024-04-26T23:08:54.934 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:08:54.934 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 27027, 2024-04-26T23:08:54.934 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 3.329846989, 2024-04-26T23:08:54.935 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000123204 2024-04-26T23:08:54.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.935 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:08:54.935 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 72790, 2024-04-26T23:08:54.935 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.024893206, 2024-04-26T23:08:54.935 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000341 2024-04-26T23:08:54.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.935 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T23:08:54.935 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T23:08:54.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.935 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T23:08:54.935 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 878, 2024-04-26T23:08:54.935 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 993, 2024-04-26T23:08:54.935 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1829018, 2024-04-26T23:08:54.936 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 9662703, 2024-04-26T23:08:54.936 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 12, 2024-04-26T23:08:54.936 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 7, 2024-04-26T23:08:54.936 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.292306897, 2024-04-26T23:08:54.936 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.118798506, 2024-04-26T23:08:54.936 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.169811169, 2024-04-26T23:08:54.936 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.046565272, 2024-04-26T23:08:54.936 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:08:54.936 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 990, 2024-04-26T23:08:54.936 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.258770975, 2024-04-26T23:08:54.936 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000261384 2024-04-26T23:08:54.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.936 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:08:54.936 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 33, 2024-04-26T23:08:54.937 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000014960, 2024-04-26T23:08:54.937 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000453 2024-04-26T23:08:54.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.937 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1682608, 2024-04-26T23:08:54.937 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 459392 2024-04-26T23:08:54.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.937 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T23:08:54.937 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 30253, 2024-04-26T23:08:54.937 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 65734, 2024-04-26T23:08:54.937 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 10858200, 2024-04-26T23:08:54.937 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 55479734, 2024-04-26T23:08:54.938 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 12, 2024-04-26T23:08:54.938 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 7, 2024-04-26T23:08:54.938 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 10.796681075, 2024-04-26T23:08:54.938 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 6.914700317, 2024-04-26T23:08:54.938 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 14.526813271, 2024-04-26T23:08:54.938 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.151479454, 2024-04-26T23:08:54.938 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:08:54.938 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 65730, 2024-04-26T23:08:54.938 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 11.336226010, 2024-04-26T23:08:54.938 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000172466 2024-04-26T23:08:54.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.938 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:08:54.938 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 75080, 2024-04-26T23:08:54.939 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.065410211, 2024-04-26T23:08:54.939 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000871 2024-04-26T23:08:54.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.939 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 4080, 2024-04-26T23:08:54.939 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 145280 2024-04-26T23:08:54.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.939 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-04-26T23:08:54.939 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 1, 2024-04-26T23:08:54.939 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-04-26T23:08:54.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.939 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-MDSRank": { 2024-04-26T23:08:54.939 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-04-26T23:08:54.939 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-04-26T23:08:54.939 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1808, 2024-04-26T23:08:54.939 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 31.390073159, 2024-04-26T23:08:54.940 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.017361766 2024-04-26T23:08:54.940 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:54.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.940 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-PurgeQueue": { 2024-04-26T23:08:54.940 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-04-26T23:08:54.940 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-04-26T23:08:54.940 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2298, 2024-04-26T23:08:54.940 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.806761930, 2024-04-26T23:08:54.940 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000351071 2024-04-26T23:08:54.940 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:54.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.940 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2024-04-26T23:08:54.940 INFO:teuthology.orchestra.run.smithi027.stdout: "request": 12722, 2024-04-26T23:08:54.940 INFO:teuthology.orchestra.run.smithi027.stdout: "reply": 12580, 2024-04-26T23:08:54.940 INFO:teuthology.orchestra.run.smithi027.stdout: "reply_latency": { 2024-04-26T23:08:54.941 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 12580, 2024-04-26T23:08:54.941 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 54.429509835, 2024-04-26T23:08:54.941 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.004326670 2024-04-26T23:08:54.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.941 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_reply": 0, 2024-04-26T23:08:54.941 INFO:teuthology.orchestra.run.smithi027.stdout: "forward": 140, 2024-04-26T23:08:54.941 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_complete": 0, 2024-04-26T23:08:54.941 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_keys": 0, 2024-04-26T23:08:54.941 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_commit": 83, 2024-04-26T23:08:54.941 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_split": 8, 2024-04-26T23:08:54.941 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_merge": 0, 2024-04-26T23:08:54.941 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes": 1377, 2024-04-26T23:08:54.941 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_top": 2, 2024-04-26T23:08:54.941 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_bottom": 5, 2024-04-26T23:08:54.941 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pin_tail": 1370, 2024-04-26T23:08:54.942 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pinned": 1373, 2024-04-26T23:08:54.942 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_expired": 19834, 2024-04-26T23:08:54.942 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_with_caps": 2, 2024-04-26T23:08:54.942 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": 2, 2024-04-26T23:08:54.942 INFO:teuthology.orchestra.run.smithi027.stdout: "subtrees": 4, 2024-04-26T23:08:54.942 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse": 17800, 2024-04-26T23:08:54.942 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_hit": 15397, 2024-04-26T23:08:54.942 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_forward": 131, 2024-04-26T23:08:54.942 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_discover": 1554, 2024-04-26T23:08:54.942 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_dir_fetch": 0, 2024-04-26T23:08:54.942 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_remote_ino": 0, 2024-04-26T23:08:54.942 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_lock": 12, 2024-04-26T23:08:54.942 INFO:teuthology.orchestra.run.smithi027.stdout: "load_cent": 0, 2024-04-26T23:08:54.942 INFO:teuthology.orchestra.run.smithi027.stdout: "q": 0, 2024-04-26T23:08:54.943 INFO:teuthology.orchestra.run.smithi027.stdout: "exported": 40, 2024-04-26T23:08:54.943 INFO:teuthology.orchestra.run.smithi027.stdout: "exported_inodes": 16229, 2024-04-26T23:08:54.943 INFO:teuthology.orchestra.run.smithi027.stdout: "imported": 30, 2024-04-26T23:08:54.943 INFO:teuthology.orchestra.run.smithi027.stdout: "imported_inodes": 5111, 2024-04-26T23:08:54.943 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_dir_fetch": 0, 2024-04-26T23:08:54.943 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_backtrace_fetch": 0, 2024-04-26T23:08:54.943 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_peer_discover": 0, 2024-04-26T23:08:54.943 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rfiles": 10835, 2024-04-26T23:08:54.943 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rbytes": 248684371, 2024-04-26T23:08:54.943 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rsnaps": 4, 2024-04-26T23:08:54.943 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T23:08:54.943 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_set_tag": 0, 2024-04-26T23:08:54.943 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T23:08:54.943 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_inotable_repaired": 0, 2024-04-26T23:08:54.944 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_inodes": 0, 2024-04-26T23:08:54.944 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T23:08:54.944 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T23:08:54.944 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_file_inodes": 0, 2024-04-26T23:08:54.944 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_inode_file_caps": 4, 2024-04-26T23:08:54.944 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_revoke": 1656, 2024-04-26T23:08:54.944 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_grant": 10517, 2024-04-26T23:08:54.944 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_trunc": 32, 2024-04-26T23:08:54.944 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T23:08:54.944 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T23:08:54.944 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps": 18340, 2024-04-26T23:08:54.944 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps_dirty": 16601, 2024-04-26T23:08:54.944 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_cap_release": 24, 2024-04-26T23:08:54.944 INFO:teuthology.orchestra.run.smithi027.stdout: "process_request_cap_release": 3164 2024-04-26T23:08:54.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.945 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache": { 2024-04-26T23:08:54.945 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update": 244, 2024-04-26T23:08:54.945 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update_receipt": 4441, 2024-04-26T23:08:54.945 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_try_discover": 2987, 2024-04-26T23:08:54.945 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_send_discover": 2863, 2024-04-26T23:08:54.945 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_handle_discover": 522, 2024-04-26T23:08:54.945 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays": 138, 2024-04-26T23:08:54.945 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_delayed": 0, 2024-04-26T23:08:54.945 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_enqueuing": 0, 2024-04-26T23:08:54.945 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_created": 1376, 2024-04-26T23:08:54.945 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_enqueued": 1204, 2024-04-26T23:08:54.945 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_reintegrated": 47, 2024-04-26T23:08:54.945 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_migrated": 1, 2024-04-26T23:08:54.945 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_processing": 0, 2024-04-26T23:08:54.945 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_enqueued": 0, 2024-04-26T23:08:54.946 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_prioritized": 0, 2024-04-26T23:08:54.946 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_started": 0, 2024-04-26T23:08:54.946 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_completed": 0, 2024-04-26T23:08:54.946 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_path": 0, 2024-04-26T23:08:54.946 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_inode": 0, 2024-04-26T23:08:54.946 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T23:08:54.946 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_exportdir": 40, 2024-04-26T23:08:54.946 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_flush": 0, 2024-04-26T23:08:54.946 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragmentdir": 8, 2024-04-26T23:08:54.946 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragstats": 0, 2024-04-26T23:08:54.946 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_inodestats": 0 2024-04-26T23:08:54.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.946 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log": { 2024-04-26T23:08:54.946 INFO:teuthology.orchestra.run.smithi027.stdout: "evlrg": 2, 2024-04-26T23:08:54.946 INFO:teuthology.orchestra.run.smithi027.stdout: "evadd": 25244, 2024-04-26T23:08:54.947 INFO:teuthology.orchestra.run.smithi027.stdout: "evex": 0, 2024-04-26T23:08:54.947 INFO:teuthology.orchestra.run.smithi027.stdout: "evtrm": 0, 2024-04-26T23:08:54.947 INFO:teuthology.orchestra.run.smithi027.stdout: "ev": 25244, 2024-04-26T23:08:54.947 INFO:teuthology.orchestra.run.smithi027.stdout: "evexg": 0, 2024-04-26T23:08:54.947 INFO:teuthology.orchestra.run.smithi027.stdout: "evexd": 0, 2024-04-26T23:08:54.947 INFO:teuthology.orchestra.run.smithi027.stdout: "segadd": 32, 2024-04-26T23:08:54.947 INFO:teuthology.orchestra.run.smithi027.stdout: "segex": 0, 2024-04-26T23:08:54.947 INFO:teuthology.orchestra.run.smithi027.stdout: "segtrm": 0, 2024-04-26T23:08:54.947 INFO:teuthology.orchestra.run.smithi027.stdout: "seg": 32, 2024-04-26T23:08:54.947 INFO:teuthology.orchestra.run.smithi027.stdout: "segmjr": 4, 2024-04-26T23:08:54.947 INFO:teuthology.orchestra.run.smithi027.stdout: "segexg": 0, 2024-04-26T23:08:54.947 INFO:teuthology.orchestra.run.smithi027.stdout: "segexd": 0, 2024-04-26T23:08:54.947 INFO:teuthology.orchestra.run.smithi027.stdout: "expos": 4194304, 2024-04-26T23:08:54.947 INFO:teuthology.orchestra.run.smithi027.stdout: "wrpos": 68916576, 2024-04-26T23:08:54.948 INFO:teuthology.orchestra.run.smithi027.stdout: "rdpos": 0, 2024-04-26T23:08:54.948 INFO:teuthology.orchestra.run.smithi027.stdout: "jlat": { 2024-04-26T23:08:54.948 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1265, 2024-04-26T23:08:54.948 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 119.679055651, 2024-04-26T23:08:54.948 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.094607949 2024-04-26T23:08:54.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.948 INFO:teuthology.orchestra.run.smithi027.stdout: "replayed": 0 2024-04-26T23:08:54.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.948 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mem": { 2024-04-26T23:08:54.948 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 538, 2024-04-26T23:08:54.948 INFO:teuthology.orchestra.run.smithi027.stdout: "ino+": 18935, 2024-04-26T23:08:54.948 INFO:teuthology.orchestra.run.smithi027.stdout: "ino-": 18397, 2024-04-26T23:08:54.948 INFO:teuthology.orchestra.run.smithi027.stdout: "dir": 129, 2024-04-26T23:08:54.948 INFO:teuthology.orchestra.run.smithi027.stdout: "dir+": 3116, 2024-04-26T23:08:54.948 INFO:teuthology.orchestra.run.smithi027.stdout: "dir-": 2987, 2024-04-26T23:08:54.949 INFO:teuthology.orchestra.run.smithi027.stdout: "dn": 1377, 2024-04-26T23:08:54.949 INFO:teuthology.orchestra.run.smithi027.stdout: "dn+": 23020, 2024-04-26T23:08:54.949 INFO:teuthology.orchestra.run.smithi027.stdout: "dn-": 21643, 2024-04-26T23:08:54.949 INFO:teuthology.orchestra.run.smithi027.stdout: "cap": 2, 2024-04-26T23:08:54.949 INFO:teuthology.orchestra.run.smithi027.stdout: "cap+": 14034, 2024-04-26T23:08:54.949 INFO:teuthology.orchestra.run.smithi027.stdout: "cap-": 14032, 2024-04-26T23:08:54.949 INFO:teuthology.orchestra.run.smithi027.stdout: "rss": 132648, 2024-04-26T23:08:54.949 INFO:teuthology.orchestra.run.smithi027.stdout: "heap": 223608 2024-04-26T23:08:54.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.949 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server": { 2024-04-26T23:08:54.949 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_client_request": 14549, 2024-04-26T23:08:54.949 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_server_request": 274, 2024-04-26T23:08:54.949 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_request": 12722, 2024-04-26T23:08:54.949 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_session": 101, 2024-04-26T23:08:54.949 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_peer_request": 902, 2024-04-26T23:08:54.950 INFO:teuthology.orchestra.run.smithi027.stdout: "req_create_latency": { 2024-04-26T23:08:54.950 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 8625, 2024-04-26T23:08:54.950 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 34.276578929, 2024-04-26T23:08:54.950 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.003974096 2024-04-26T23:08:54.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.950 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getattr_latency": { 2024-04-26T23:08:54.950 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 4, 2024-04-26T23:08:54.950 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.020919622, 2024-04-26T23:08:54.950 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.005229905 2024-04-26T23:08:54.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.950 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getfilelock_latency": { 2024-04-26T23:08:54.950 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.950 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.950 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.951 INFO:teuthology.orchestra.run.smithi027.stdout: "req_link_latency": { 2024-04-26T23:08:54.951 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 32, 2024-04-26T23:08:54.951 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.127408367, 2024-04-26T23:08:54.951 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.003981511 2024-04-26T23:08:54.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.951 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookup_latency": { 2024-04-26T23:08:54.951 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1033, 2024-04-26T23:08:54.951 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 2.172033191, 2024-04-26T23:08:54.951 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.002102645 2024-04-26T23:08:54.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.951 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookuphash_latency": { 2024-04-26T23:08:54.951 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.951 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.951 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.952 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupino_latency": { 2024-04-26T23:08:54.952 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2, 2024-04-26T23:08:54.952 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000208434, 2024-04-26T23:08:54.952 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000104217 2024-04-26T23:08:54.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.952 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupname_latency": { 2024-04-26T23:08:54.952 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.952 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.952 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.952 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupparent_latency": { 2024-04-26T23:08:54.952 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.952 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.952 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.953 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupsnap_latency": { 2024-04-26T23:08:54.953 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.953 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.953 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.953 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lssnap_latency": { 2024-04-26T23:08:54.954 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.954 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.954 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.954 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mkdir_latency": { 2024-04-26T23:08:54.954 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 956, 2024-04-26T23:08:54.954 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 5.258238172, 2024-04-26T23:08:54.954 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.005500249 2024-04-26T23:08:54.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.954 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mknod_latency": { 2024-04-26T23:08:54.954 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 63, 2024-04-26T23:08:54.954 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.177932585, 2024-04-26T23:08:54.954 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.002824326 2024-04-26T23:08:54.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.955 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mksnap_latency": { 2024-04-26T23:08:54.955 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.955 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.955 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.955 INFO:teuthology.orchestra.run.smithi027.stdout: "req_open_latency": { 2024-04-26T23:08:54.955 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 41, 2024-04-26T23:08:54.955 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.022801153, 2024-04-26T23:08:54.955 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000556125 2024-04-26T23:08:54.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.955 INFO:teuthology.orchestra.run.smithi027.stdout: "req_readdir_latency": { 2024-04-26T23:08:54.955 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 198, 2024-04-26T23:08:54.955 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.633653104, 2024-04-26T23:08:54.955 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.003200268 2024-04-26T23:08:54.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.956 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rename_latency": { 2024-04-26T23:08:54.956 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 99, 2024-04-26T23:08:54.956 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 1.557404832, 2024-04-26T23:08:54.956 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.015731361 2024-04-26T23:08:54.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.956 INFO:teuthology.orchestra.run.smithi027.stdout: "req_renamesnap_latency": { 2024-04-26T23:08:54.957 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.957 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.957 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.957 INFO:teuthology.orchestra.run.smithi027.stdout: "req_snapdiff_latency": { 2024-04-26T23:08:54.957 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.957 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.957 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.957 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmdir_latency": { 2024-04-26T23:08:54.957 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 234, 2024-04-26T23:08:54.957 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 5.178009925, 2024-04-26T23:08:54.957 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.022128247 2024-04-26T23:08:54.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.958 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmsnap_latency": { 2024-04-26T23:08:54.958 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.958 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.958 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.958 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmxattr_latency": { 2024-04-26T23:08:54.958 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.958 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.958 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.958 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setattr_latency": { 2024-04-26T23:08:54.958 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 33, 2024-04-26T23:08:54.958 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.332603992, 2024-04-26T23:08:54.958 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.010078908 2024-04-26T23:08:54.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.959 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setdirlayout_latency": { 2024-04-26T23:08:54.959 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.959 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.959 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.959 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setfilelock_latency": { 2024-04-26T23:08:54.959 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.959 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.959 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.959 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setlayout_latency": { 2024-04-26T23:08:54.959 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.959 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.959 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.960 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setxattr_latency": { 2024-04-26T23:08:54.960 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.960 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.960 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.960 INFO:teuthology.orchestra.run.smithi027.stdout: "req_symlink_latency": { 2024-04-26T23:08:54.960 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 46, 2024-04-26T23:08:54.960 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.090079464, 2024-04-26T23:08:54.960 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.001958249 2024-04-26T23:08:54.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.960 INFO:teuthology.orchestra.run.smithi027.stdout: "req_unlink_latency": { 2024-04-26T23:08:54.960 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1214, 2024-04-26T23:08:54.960 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 4.581638065, 2024-04-26T23:08:54.961 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.003774001 2024-04-26T23:08:54.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.961 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_revoke_eviction": 0, 2024-04-26T23:08:54.961 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_acquisition_throttle": 0, 2024-04-26T23:08:54.961 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getvxattr_latency": { 2024-04-26T23:08:54.961 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.961 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.961 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.961 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:54.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.961 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessions": { 2024-04-26T23:08:54.961 INFO:teuthology.orchestra.run.smithi027.stdout: "session_count": 2, 2024-04-26T23:08:54.961 INFO:teuthology.orchestra.run.smithi027.stdout: "session_add": 3, 2024-04-26T23:08:54.961 INFO:teuthology.orchestra.run.smithi027.stdout: "session_remove": 1, 2024-04-26T23:08:54.961 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_open": 2, 2024-04-26T23:08:54.962 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_stale": 0, 2024-04-26T23:08:54.962 INFO:teuthology.orchestra.run.smithi027.stdout: "total_load": 2603, 2024-04-26T23:08:54.962 INFO:teuthology.orchestra.run.smithi027.stdout: "average_load": 1301, 2024-04-26T23:08:54.962 INFO:teuthology.orchestra.run.smithi027.stdout: "avg_session_uptime": 602, 2024-04-26T23:08:54.962 INFO:teuthology.orchestra.run.smithi027.stdout: "mdthresh_evicted": 0 2024-04-26T23:08:54.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.962 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-04-26T23:08:54.962 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-04-26T23:08:54.962 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-04-26T23:08:54.962 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T23:08:54.962 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-04-26T23:08:54.962 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T23:08:54.962 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-04-26T23:08:54.962 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T23:08:54.963 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T23:08:54.963 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T23:08:54.963 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T23:08:54.963 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T23:08:54.963 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-04-26T23:08:54.963 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T23:08:54.963 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T23:08:54.963 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-04-26T23:08:54.963 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-04-26T23:08:54.963 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-04-26T23:08:54.963 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-04-26T23:08:54.963 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T23:08:54.963 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T23:08:54.963 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T23:08:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T23:08:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T23:08:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-04-26T23:08:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-04-26T23:08:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-04-26T23:08:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T23:08:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T23:08:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-04-26T23:08:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-04-26T23:08:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-04-26T23:08:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-04-26T23:08:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T23:08:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-04-26T23:08:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T23:08:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-04-26T23:08:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 1418384, 2024-04-26T23:08:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 930, 2024-04-26T23:08:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 1496, 2024-04-26T23:08:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 17, 2024-04-26T23:08:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-04-26T23:08:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-04-26T23:08:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-04-26T23:08:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-04-26T23:08:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-04-26T23:08:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-04-26T23:08:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 10232, 2024-04-26T23:08:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 215, 2024-04-26T23:08:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T23:08:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-04-26T23:08:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-04-26T23:08:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-04-26T23:08:54.966 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 2762226, 2024-04-26T23:08:54.966 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 197501, 2024-04-26T23:08:54.966 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-04-26T23:08:54.966 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-04-26T23:08:54.966 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-04-26T23:08:54.966 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-04-26T23:08:54.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.966 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter": { 2024-04-26T23:08:54.966 INFO:teuthology.orchestra.run.smithi027.stdout: "op_active": 0, 2024-04-26T23:08:54.966 INFO:teuthology.orchestra.run.smithi027.stdout: "op_laggy": 0, 2024-04-26T23:08:54.966 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send": 3999, 2024-04-26T23:08:54.966 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send_bytes": 67448497, 2024-04-26T23:08:54.966 INFO:teuthology.orchestra.run.smithi027.stdout: "op_resend": 31, 2024-04-26T23:08:54.966 INFO:teuthology.orchestra.run.smithi027.stdout: "op_reply": 3999, 2024-04-26T23:08:54.967 INFO:teuthology.orchestra.run.smithi027.stdout: "op_latency": { 2024-04-26T23:08:54.967 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3999, 2024-04-26T23:08:54.967 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 32.299167940, 2024-04-26T23:08:54.967 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.008076811 2024-04-26T23:08:54.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.967 INFO:teuthology.orchestra.run.smithi027.stdout: "op_inflight": 0, 2024-04-26T23:08:54.967 INFO:teuthology.orchestra.run.smithi027.stdout: "oplen_avg": { 2024-04-26T23:08:54.967 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3999, 2024-04-26T23:08:54.967 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 4242 2024-04-26T23:08:54.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.967 INFO:teuthology.orchestra.run.smithi027.stdout: "op": 3999, 2024-04-26T23:08:54.967 INFO:teuthology.orchestra.run.smithi027.stdout: "op_r": 54, 2024-04-26T23:08:54.967 INFO:teuthology.orchestra.run.smithi027.stdout: "op_w": 3945, 2024-04-26T23:08:54.967 INFO:teuthology.orchestra.run.smithi027.stdout: "op_rmw": 0, 2024-04-26T23:08:54.968 INFO:teuthology.orchestra.run.smithi027.stdout: "op_pg": 0, 2024-04-26T23:08:54.968 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_stat": 0, 2024-04-26T23:08:54.968 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_create": 10, 2024-04-26T23:08:54.968 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_read": 54, 2024-04-26T23:08:54.968 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_write": 1319, 2024-04-26T23:08:54.968 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writefull": 52, 2024-04-26T23:08:54.968 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writesame": 0, 2024-04-26T23:08:54.968 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_append": 0, 2024-04-26T23:08:54.968 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_zero": 13, 2024-04-26T23:08:54.968 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_truncate": 8, 2024-04-26T23:08:54.968 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_delete": 2319, 2024-04-26T23:08:54.968 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_mapext": 0, 2024-04-26T23:08:54.968 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_sparse_read": 0, 2024-04-26T23:08:54.968 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_clonerange": 0, 2024-04-26T23:08:54.968 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_getxattr": 0, 2024-04-26T23:08:54.969 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_setxattr": 20, 2024-04-26T23:08:54.969 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_cmpxattr": 0, 2024-04-26T23:08:54.969 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_rmxattr": 0, 2024-04-26T23:08:54.969 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_resetxattrs": 0, 2024-04-26T23:08:54.969 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_call": 0, 2024-04-26T23:08:54.969 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_watch": 0, 2024-04-26T23:08:54.969 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_notify": 0, 2024-04-26T23:08:54.969 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T23:08:54.969 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls": 0, 2024-04-26T23:08:54.969 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls_filter": 0, 2024-04-26T23:08:54.969 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_other": 38, 2024-04-26T23:08:54.969 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_active": 0, 2024-04-26T23:08:54.969 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_send": 0, 2024-04-26T23:08:54.969 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_resend": 0, 2024-04-26T23:08:54.969 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_ping": 0, 2024-04-26T23:08:54.970 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_active": 0, 2024-04-26T23:08:54.970 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_send": 0, 2024-04-26T23:08:54.970 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_resend": 0, 2024-04-26T23:08:54.970 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_active": 0, 2024-04-26T23:08:54.970 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_send": 0, 2024-04-26T23:08:54.970 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_resend": 0, 2024-04-26T23:08:54.970 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_active": 0, 2024-04-26T23:08:54.970 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_send": 0, 2024-04-26T23:08:54.970 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_resend": 0, 2024-04-26T23:08:54.970 INFO:teuthology.orchestra.run.smithi027.stdout: "command_active": 0, 2024-04-26T23:08:54.970 INFO:teuthology.orchestra.run.smithi027.stdout: "command_send": 0, 2024-04-26T23:08:54.970 INFO:teuthology.orchestra.run.smithi027.stdout: "command_resend": 0, 2024-04-26T23:08:54.970 INFO:teuthology.orchestra.run.smithi027.stdout: "map_epoch": 101, 2024-04-26T23:08:54.970 INFO:teuthology.orchestra.run.smithi027.stdout: "map_full": 0, 2024-04-26T23:08:54.971 INFO:teuthology.orchestra.run.smithi027.stdout: "map_inc": 11, 2024-04-26T23:08:54.971 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_sessions": 12, 2024-04-26T23:08:54.971 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_open": 12, 2024-04-26T23:08:54.971 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_close": 0, 2024-04-26T23:08:54.971 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_laggy": 0, 2024-04-26T23:08:54.971 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_wr": 344, 2024-04-26T23:08:54.971 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_rd": 0, 2024-04-26T23:08:54.971 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_del": 65 2024-04-26T23:08:54.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.971 INFO:teuthology.orchestra.run.smithi027.stdout: "oft": { 2024-04-26T23:08:54.971 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_objs": 1, 2024-04-26T23:08:54.971 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_kv_pairs": 0, 2024-04-26T23:08:54.971 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_updates": 0, 2024-04-26T23:08:54.971 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_removes": 0 2024-04-26T23:08:54.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.972 INFO:teuthology.orchestra.run.smithi027.stdout: "osd-slow-ops": { 2024-04-26T23:08:54.972 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_ops_count": 0 2024-04-26T23:08:54.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.972 INFO:teuthology.orchestra.run.smithi027.stdout: "purge_queue": { 2024-04-26T23:08:54.972 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops": 0, 2024-04-26T23:08:54.972 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops_high_water": 22, 2024-04-26T23:08:54.972 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing": 0, 2024-04-26T23:08:54.972 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_high_water": 16, 2024-04-26T23:08:54.972 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed_ops": 2305, 2024-04-26T23:08:54.972 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed": 1204, 2024-04-26T23:08:54.972 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_item_in_journal": 0 2024-04-26T23:08:54.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.972 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T23:08:54.972 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T23:08:54.973 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-04-26T23:08:54.973 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T23:08:54.973 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 80665, 2024-04-26T23:08:54.973 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 34859756, 2024-04-26T23:08:54.973 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T23:08:54.973 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 80665, 2024-04-26T23:08:54.973 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-04-26T23:08:54.973 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-04-26T23:08:54.973 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 80665, 2024-04-26T23:08:54.973 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 34859756, 2024-04-26T23:08:54.973 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T23:08:54.973 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.973 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.973 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.973 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:54.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.974 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_bytes": { 2024-04-26T23:08:54.974 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T23:08:54.974 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-04-26T23:08:54.974 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T23:08:54.974 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-04-26T23:08:54.974 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-04-26T23:08:54.974 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T23:08:54.974 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-04-26T23:08:54.974 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 3999, 2024-04-26T23:08:54.974 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 67584901, 2024-04-26T23:08:54.974 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 3999, 2024-04-26T23:08:54.974 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 67584901, 2024-04-26T23:08:54.974 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T23:08:54.974 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.975 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.975 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.975 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:54.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.975 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_ops": { 2024-04-26T23:08:54.975 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T23:08:54.975 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 1024, 2024-04-26T23:08:54.975 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T23:08:54.975 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-04-26T23:08:54.975 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-04-26T23:08:54.975 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T23:08:54.975 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-04-26T23:08:54.975 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 3999, 2024-04-26T23:08:54.975 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 3999, 2024-04-26T23:08:54.975 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 3999, 2024-04-26T23:08:54.976 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 3999, 2024-04-26T23:08:54.976 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T23:08:54.976 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.976 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.976 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.976 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:54.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.976 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle": { 2024-04-26T23:08:54.976 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T23:08:54.976 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-04-26T23:08:54.976 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T23:08:54.976 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 1204, 2024-04-26T23:08:54.976 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 136404, 2024-04-26T23:08:54.976 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T23:08:54.976 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 1204, 2024-04-26T23:08:54.977 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-04-26T23:08:54.977 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-04-26T23:08:54.977 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 54, 2024-04-26T23:08:54.977 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 136404, 2024-04-26T23:08:54.977 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T23:08:54.977 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.977 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.977 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.977 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:54.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:54.977 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle-0x5627493ea460": { 2024-04-26T23:08:54.977 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T23:08:54.977 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-04-26T23:08:54.977 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T23:08:54.977 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 25244, 2024-04-26T23:08:54.978 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 64722272, 2024-04-26T23:08:54.978 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T23:08:54.978 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 25244, 2024-04-26T23:08:54.978 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-04-26T23:08:54.978 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-04-26T23:08:54.978 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 1265, 2024-04-26T23:08:54.978 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 64722272, 2024-04-26T23:08:54.978 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T23:08:54.978 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:54.978 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:54.978 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:54.978 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:54.978 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:54.978 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T23:08:55.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:54 smithi027 ceph-mon[28034]: pgmap v318: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:08:55.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:54 smithi080 ceph-mon[32324]: pgmap v318: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:08:55.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:54 smithi094 ceph-mon[32889]: pgmap v318: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:08:55.943 INFO:tasks.check_counter:Daemon mds.g mds.exported=40 2024-04-26T23:08:55.944 INFO:tasks.check_counter:Daemon mds.g mds.imported=30 2024-04-26T23:08:55.944 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=244 2024-04-26T23:08:55.944 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=4441 2024-04-26T23:08:55.944 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=4 2024-04-26T23:08:55.944 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-26T23:08:55.944 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T23:08:55.944 DEBUG:tasks.check_counter:Getting stats from j 2024-04-26T23:08:55.944 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-26T23:08:56.412 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:08:57.215 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:56 smithi027 ceph-mon[28034]: pgmap v319: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:08:57.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:56 smithi080 ceph-mon[32324]: pgmap v319: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:08:57.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:56 smithi094 ceph-mon[32889]: pgmap v319: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:08:59.024 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T23:08:59.025 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T23:08:59.025 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 0, 2024-04-26T23:08:59.025 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 2, 2024-04-26T23:08:59.025 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 0, 2024-04-26T23:08:59.025 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 0, 2024-04-26T23:08:59.025 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-04-26T23:08:59.025 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-04-26T23:08:59.025 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.003992128, 2024-04-26T23:08:59.025 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T23:08:59.025 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.000685572, 2024-04-26T23:08:59.025 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T23:08:59.025 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:08:59.026 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:59.026 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:59.026 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:59.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:59.026 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:08:59.026 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:59.026 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:59.026 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:59.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:59.026 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T23:08:59.026 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T23:08:59.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:59.026 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T23:08:59.027 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 242, 2024-04-26T23:08:59.027 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 229, 2024-04-26T23:08:59.027 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1597200, 2024-04-26T23:08:59.027 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 140512, 2024-04-26T23:08:59.027 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-04-26T23:08:59.027 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-04-26T23:08:59.027 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.109488905, 2024-04-26T23:08:59.027 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.038097358, 2024-04-26T23:08:59.027 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.085544047, 2024-04-26T23:08:59.027 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.010845810, 2024-04-26T23:08:59.027 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:08:59.027 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 229, 2024-04-26T23:08:59.027 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.024340481, 2024-04-26T23:08:59.027 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000106290 2024-04-26T23:08:59.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:59.028 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:08:59.028 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:59.028 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:59.028 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:59.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:59.028 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1597200, 2024-04-26T23:08:59.028 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 140512 2024-04-26T23:08:59.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:59.028 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T23:08:59.028 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 2, 2024-04-26T23:08:59.028 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 174, 2024-04-26T23:08:59.028 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 320, 2024-04-26T23:08:59.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 187728, 2024-04-26T23:08:59.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 5, 2024-04-26T23:08:59.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-04-26T23:08:59.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.035856380, 2024-04-26T23:08:59.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.026890815, 2024-04-26T23:08:59.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.003245110, 2024-04-26T23:08:59.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T23:08:59.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:08:59.029 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 171, 2024-04-26T23:08:59.029 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.032074214, 2024-04-26T23:08:59.029 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000187568 2024-04-26T23:08:59.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:59.030 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:08:59.030 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:59.030 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:59.030 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:59.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:59.030 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T23:08:59.030 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 187728 2024-04-26T23:08:59.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:59.030 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-04-26T23:08:59.030 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-04-26T23:08:59.031 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-04-26T23:08:59.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:59.031 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-04-26T23:08:59.031 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-04-26T23:08:59.031 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-04-26T23:08:59.031 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T23:08:59.031 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-04-26T23:08:59.031 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T23:08:59.031 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-04-26T23:08:59.031 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T23:08:59.031 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T23:08:59.032 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T23:08:59.032 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T23:08:59.032 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T23:08:59.032 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-04-26T23:08:59.032 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T23:08:59.032 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T23:08:59.032 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-04-26T23:08:59.032 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-04-26T23:08:59.032 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-04-26T23:08:59.032 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-04-26T23:08:59.032 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T23:08:59.032 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T23:08:59.033 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T23:08:59.033 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T23:08:59.033 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T23:08:59.033 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-04-26T23:08:59.033 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-04-26T23:08:59.033 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-04-26T23:08:59.033 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T23:08:59.033 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T23:08:59.033 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-04-26T23:08:59.033 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-04-26T23:08:59.033 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-04-26T23:08:59.034 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-04-26T23:08:59.034 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T23:08:59.034 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-04-26T23:08:59.034 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T23:08:59.034 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-04-26T23:08:59.034 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 148714, 2024-04-26T23:08:59.034 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 30, 2024-04-26T23:08:59.034 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-04-26T23:08:59.034 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-04-26T23:08:59.034 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-04-26T23:08:59.034 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-04-26T23:08:59.035 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-04-26T23:08:59.035 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-04-26T23:08:59.035 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-04-26T23:08:59.035 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-04-26T23:08:59.035 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-04-26T23:08:59.035 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-04-26T23:08:59.035 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T23:08:59.035 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-04-26T23:08:59.035 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-04-26T23:08:59.035 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-04-26T23:08:59.035 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 928, 2024-04-26T23:08:59.036 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-04-26T23:08:59.036 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-04-26T23:08:59.036 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-04-26T23:08:59.036 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-04-26T23:08:59.036 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-04-26T23:08:59.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T23:08:59.036 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T23:08:59.036 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T23:08:59.036 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-04-26T23:08:59.036 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T23:08:59.036 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 240, 2024-04-26T23:08:59.036 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1560653, 2024-04-26T23:08:59.037 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T23:08:59.037 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 240, 2024-04-26T23:08:59.037 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-04-26T23:08:59.037 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-04-26T23:08:59.037 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 240, 2024-04-26T23:08:59.037 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1560653, 2024-04-26T23:08:59.037 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T23:08:59.037 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T23:08:59.037 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T23:08:59.037 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T23:08:59.037 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:59.038 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T23:08:59.038 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T23:08:59.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:08:58 smithi027 ceph-mon[28034]: pgmap v320: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:08:59.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:08:58 smithi080 ceph-mon[32324]: pgmap v320: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:08:59.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:08:58 smithi094 ceph-mon[32889]: pgmap v320: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:09:00.100 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-26T23:09:00.100 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-26T23:09:00.100 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-26T23:09:00.100 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-26T23:09:00.100 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-26T23:09:00.100 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-26T23:09:00.100 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-26T23:09:00.100 DEBUG:tasks.check_counter:Getting stats from b 2024-04-26T23:09:00.100 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-26T23:09:00.362 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T23:09:01.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:00 smithi080 ceph-mon[32324]: pgmap v321: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:09:01.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:00 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:09:01.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:00 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:01.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:00 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:01.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:00 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:01.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:00 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:01.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:00 smithi094 ceph-mon[32889]: pgmap v321: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:09:01.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:00 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:09:01.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:00 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:01.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:00 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:01.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:00 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:01.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:00 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:01.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:00 smithi027 ceph-mon[28034]: pgmap v321: 129 pgs: 129 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:09:01.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:00 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:09:01.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:00 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:01.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:00 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:01.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:00 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:01.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:00 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:02.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:02 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:02.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:02 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:02.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:02 smithi094 ceph-mon[32889]: pgmap v322: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T23:09:02.323 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-26T23:09:02.323 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T23:09:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 9964, 2024-04-26T23:09:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 9660, 2024-04-26T23:09:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 3026287, 2024-04-26T23:09:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 56505564, 2024-04-26T23:09:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 55, 2024-04-26T23:09:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 7, 2024-04-26T23:09:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.821621421, 2024-04-26T23:09:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.333226056, 2024-04-26T23:09:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.161377485, 2024-04-26T23:09:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.272996423, 2024-04-26T23:09:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 9655, 2024-04-26T23:09:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 7.038178275, 2024-04-26T23:09:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000728967 2024-04-26T23:09:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 495, 2024-04-26T23:09:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000079484, 2024-04-26T23:09:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000160 2024-04-26T23:09:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 170912, 2024-04-26T23:09:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 34384 2024-04-26T23:09:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T23:09:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 38325, 2024-04-26T23:09:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 36148, 2024-04-26T23:09:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 21334840, 2024-04-26T23:09:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 36308894, 2024-04-26T23:09:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 11, 2024-04-26T23:09:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 6, 2024-04-26T23:09:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 2.091172167, 2024-04-26T23:09:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.779712527, 2024-04-26T23:09:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 1.228894504, 2024-04-26T23:09:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.325176186, 2024-04-26T23:09:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 36149, 2024-04-26T23:09:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 8.761514265, 2024-04-26T23:09:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000242372 2024-04-26T23:09:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 47474, 2024-04-26T23:09:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.007124732, 2024-04-26T23:09:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000150 2024-04-26T23:09:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-26T23:09:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 470272 2024-04-26T23:09:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T23:09:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 41929, 2024-04-26T23:09:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 80480, 2024-04-26T23:09:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 22259825, 2024-04-26T23:09:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 53439671, 2024-04-26T23:09:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 17, 2024-04-26T23:09:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 7, 2024-04-26T23:09:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 2.870045785, 2024-04-26T23:09:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 1.240352507, 2024-04-26T23:09:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 4.141998069, 2024-04-26T23:09:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.356666291, 2024-04-26T23:09:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 80479, 2024-04-26T23:09:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 7.793113864, 2024-04-26T23:09:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000096834 2024-04-26T23:09:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 75938, 2024-04-26T23:09:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.015917068, 2024-04-26T23:09:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000209 2024-04-26T23:09:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 2013136, 2024-04-26T23:09:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 245696 2024-04-26T23:09:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "cct": { 2024-04-26T23:09:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "total_workers": 1, 2024-04-26T23:09:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "unhealthy_workers": 0 2024-04-26T23:09:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "finisher-MDSRank": { 2024-04-26T23:09:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "queue_len": 0, 2024-04-26T23:09:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "complete_latency": { 2024-04-26T23:09:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 5759, 2024-04-26T23:09:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 15.353192182, 2024-04-26T23:09:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.002665947 2024-04-26T23:09:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "finisher-PurgeQueue": { 2024-04-26T23:09:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "queue_len": 0, 2024-04-26T23:09:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "complete_latency": { 2024-04-26T23:09:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 7291, 2024-04-26T23:09:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.742049853, 2024-04-26T23:09:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000101776 2024-04-26T23:09:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": { 2024-04-26T23:09:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "request": 12023, 2024-04-26T23:09:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "reply": 11893, 2024-04-26T23:09:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "reply_latency": { 2024-04-26T23:09:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 11893, 2024-04-26T23:09:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 40.843833098, 2024-04-26T23:09:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.003434275 2024-04-26T23:09:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "slow_reply": 0, 2024-04-26T23:09:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "forward": 130, 2024-04-26T23:09:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_fetch_complete": 28, 2024-04-26T23:09:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_fetch_keys": 0, 2024-04-26T23:09:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_commit": 1756, 2024-04-26T23:09:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_split": 11, 2024-04-26T23:09:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_merge": 0, 2024-04-26T23:09:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes": 8666, 2024-04-26T23:09:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_top": 4951, 2024-04-26T23:09:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_bottom": 2961, 2024-04-26T23:09:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_pin_tail": 754, 2024-04-26T23:09:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_pinned": 1592, 2024-04-26T23:09:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_expired": 13968, 2024-04-26T23:09:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_with_caps": 2, 2024-04-26T23:09:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": 2, 2024-04-26T23:09:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "subtrees": 4, 2024-04-26T23:09:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse": 17860, 2024-04-26T23:09:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_hit": 15498, 2024-04-26T23:09:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_forward": 15, 2024-04-26T23:09:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_discover": 1191, 2024-04-26T23:09:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_dir_fetch": 0, 2024-04-26T23:09:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_remote_ino": 0, 2024-04-26T23:09:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_lock": 12, 2024-04-26T23:09:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "load_cent": 0, 2024-04-26T23:09:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "q": 0, 2024-04-26T23:09:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "exported": 24, 2024-04-26T23:09:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "exported_inodes": 11161, 2024-04-26T23:09:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "imported": 24, 2024-04-26T23:09:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "imported_inodes": 10232, 2024-04-26T23:09:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "openino_dir_fetch": 0, 2024-04-26T23:09:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "openino_backtrace_fetch": 0, 2024-04-26T23:09:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "openino_peer_discover": 0, 2024-04-26T23:09:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "root_rfiles": 4, 2024-04-26T23:09:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "root_rbytes": 234, 2024-04-26T23:09:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "root_rsnaps": 7, 2024-04-26T23:09:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T23:09:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_set_tag": 0, 2024-04-26T23:09:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T23:09:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_inotable_repaired": 0, 2024-04-26T23:09:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_dir_inodes": 0, 2024-04-26T23:09:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T23:09:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T23:09:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_file_inodes": 0, 2024-04-26T23:09:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_inode_file_caps": 16, 2024-04-26T23:09:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_revoke": 3103, 2024-04-26T23:09:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_grant": 8973, 2024-04-26T23:09:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_trunc": 207, 2024-04-26T23:09:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T23:09:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T23:09:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_caps": 11522, 2024-04-26T23:09:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_caps_dirty": 7910, 2024-04-26T23:09:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_cap_release": 44, 2024-04-26T23:09:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "process_request_cap_release": 7153 2024-04-26T23:09:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_cache": { 2024-04-26T23:09:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_update": 1396, 2024-04-26T23:09:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_update_receipt": 3459, 2024-04-26T23:09:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_try_discover": 2282, 2024-04-26T23:09:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_send_discover": 2279, 2024-04-26T23:09:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_handle_discover": 2777, 2024-04-26T23:09:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "num_strays": 576, 2024-04-26T23:09:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "num_strays_delayed": 0, 2024-04-26T23:09:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "num_strays_enqueuing": 0, 2024-04-26T23:09:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_created": 4321, 2024-04-26T23:09:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_enqueued": 3703, 2024-04-26T23:09:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_reintegrated": 52, 2024-04-26T23:09:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_migrated": 0, 2024-04-26T23:09:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "num_recovering_processing": 0, 2024-04-26T23:09:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "num_recovering_enqueued": 0, 2024-04-26T23:09:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "num_recovering_prioritized": 0, 2024-04-26T23:09:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_started": 2, 2024-04-26T23:09:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_completed": 2, 2024-04-26T23:09:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_quiesce_path": 0, 2024-04-26T23:09:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_quiesce_inode": 0, 2024-04-26T23:09:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T23:09:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_exportdir": 24, 2024-04-26T23:09:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_flush": 0, 2024-04-26T23:09:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_fragmentdir": 11, 2024-04-26T23:09:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_fragstats": 0, 2024-04-26T23:09:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_inodestats": 0 2024-04-26T23:09:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_log": { 2024-04-26T23:09:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "evlrg": 3, 2024-04-26T23:09:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "evadd": 24393, 2024-04-26T23:09:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "evex": 22934, 2024-04-26T23:09:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "evtrm": 22934, 2024-04-26T23:09:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "ev": 1459, 2024-04-26T23:09:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "evexg": 0, 2024-04-26T23:09:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "evexd": 5012, 2024-04-26T23:09:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "segadd": 56, 2024-04-26T23:09:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "segex": 53, 2024-04-26T23:09:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "segtrm": 53, 2024-04-26T23:09:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "seg": 3, 2024-04-26T23:09:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "segmjr": 1, 2024-04-26T23:09:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "segexg": 0, 2024-04-26T23:09:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "segexd": 8, 2024-04-26T23:09:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "expos": 72931436, 2024-04-26T23:09:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "wrpos": 76273593, 2024-04-26T23:09:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "rdpos": 0, 2024-04-26T23:09:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "jlat": { 2024-04-26T23:09:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 2859, 2024-04-26T23:09:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 50.011981607, 2024-04-26T23:09:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.017492823 2024-04-26T23:09:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "replayed": 0 2024-04-26T23:09:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_mem": { 2024-04-26T23:09:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "ino": 4351, 2024-04-26T23:09:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "ino+": 19657, 2024-04-26T23:09:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "ino-": 15306, 2024-04-26T23:09:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "dir": 626, 2024-04-26T23:09:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "dir+": 2948, 2024-04-26T23:09:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "dir-": 2322, 2024-04-26T23:09:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "dn": 8666, 2024-04-26T23:09:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "dn+": 26744, 2024-04-26T23:09:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "dn-": 18078, 2024-04-26T23:09:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "cap": 2, 2024-04-26T23:09:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "cap+": 11731, 2024-04-26T23:09:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "cap-": 11729, 2024-04-26T23:09:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "rss": 155180, 2024-04-26T23:09:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "heap": 231800 2024-04-26T23:09:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_server": { 2024-04-26T23:09:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "dispatch_client_request": 16675, 2024-04-26T23:09:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "dispatch_server_request": 54, 2024-04-26T23:09:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_request": 12023, 2024-04-26T23:09:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_session": 263, 2024-04-26T23:09:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_peer_request": 378, 2024-04-26T23:09:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "req_create_latency": { 2024-04-26T23:09:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 2244, 2024-04-26T23:09:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 5.751990789, 2024-04-26T23:09:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.002563275 2024-04-26T23:09:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "req_getattr_latency": { 2024-04-26T23:09:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 183, 2024-04-26T23:09:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.254116248, 2024-04-26T23:09:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.001388613 2024-04-26T23:09:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "req_getfilelock_latency": { 2024-04-26T23:09:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "req_link_latency": { 2024-04-26T23:09:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 235, 2024-04-26T23:09:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.931147357, 2024-04-26T23:09:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.003962329 2024-04-26T23:09:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookup_latency": { 2024-04-26T23:09:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 1513, 2024-04-26T23:09:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 1.002241404, 2024-04-26T23:09:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000662419 2024-04-26T23:09:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookuphash_latency": { 2024-04-26T23:09:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupino_latency": { 2024-04-26T23:09:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupname_latency": { 2024-04-26T23:09:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupparent_latency": { 2024-04-26T23:09:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupsnap_latency": { 2024-04-26T23:09:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lssnap_latency": { 2024-04-26T23:09:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 11, 2024-04-26T23:09:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.002510415, 2024-04-26T23:09:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000228219 2024-04-26T23:09:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: "req_mkdir_latency": { 2024-04-26T23:09:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 709, 2024-04-26T23:09:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 1.964606888, 2024-04-26T23:09:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.002770954 2024-04-26T23:09:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "req_mknod_latency": { 2024-04-26T23:09:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 436, 2024-04-26T23:09:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 1.358688235, 2024-04-26T23:09:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.003116257 2024-04-26T23:09:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "req_mksnap_latency": { 2024-04-26T23:09:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 11, 2024-04-26T23:09:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 1.009275983, 2024-04-26T23:09:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.091752362 2024-04-26T23:09:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "req_open_latency": { 2024-04-26T23:09:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 5, 2024-04-26T23:09:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.007418092, 2024-04-26T23:09:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.001483618 2024-04-26T23:09:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "req_readdir_latency": { 2024-04-26T23:09:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 865, 2024-04-26T23:09:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 3.473132859, 2024-04-26T23:09:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.004015182 2024-04-26T23:09:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rename_latency": { 2024-04-26T23:09:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 482, 2024-04-26T23:09:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 3.160524436, 2024-04-26T23:09:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.006557104 2024-04-26T23:09:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "req_renamesnap_latency": { 2024-04-26T23:09:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "req_snapdiff_latency": { 2024-04-26T23:09:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rmdir_latency": { 2024-04-26T23:09:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 848, 2024-04-26T23:09:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 11.755334427, 2024-04-26T23:09:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.013862422 2024-04-26T23:09:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rmsnap_latency": { 2024-04-26T23:09:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 4, 2024-04-26T23:09:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.022307541, 2024-04-26T23:09:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.005576885 2024-04-26T23:09:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rmxattr_latency": { 2024-04-26T23:09:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setattr_latency": { 2024-04-26T23:09:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 212, 2024-04-26T23:09:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 2.338777959, 2024-04-26T23:09:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.011031971 2024-04-26T23:09:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setdirlayout_latency": { 2024-04-26T23:09:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setfilelock_latency": { 2024-04-26T23:09:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setlayout_latency": { 2024-04-26T23:09:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setxattr_latency": { 2024-04-26T23:09:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 8, 2024-04-26T23:09:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.021154103, 2024-04-26T23:09:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.002644262 2024-04-26T23:09:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "req_symlink_latency": { 2024-04-26T23:09:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 429, 2024-04-26T23:09:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.913705197, 2024-04-26T23:09:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.002129848 2024-04-26T23:09:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "req_unlink_latency": { 2024-04-26T23:09:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 3698, 2024-04-26T23:09:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 6.876901165, 2024-04-26T23:09:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.001859627 2024-04-26T23:09:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "cap_revoke_eviction": 0, 2024-04-26T23:09:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "cap_acquisition_throttle": 0, 2024-04-26T23:09:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "req_getvxattr_latency": { 2024-04-26T23:09:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_sessions": { 2024-04-26T23:09:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "session_count": 2, 2024-04-26T23:09:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "session_add": 6, 2024-04-26T23:09:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "session_remove": 4, 2024-04-26T23:09:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "sessions_open": 2, 2024-04-26T23:09:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "sessions_stale": 0, 2024-04-26T23:09:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "total_load": 159, 2024-04-26T23:09:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "average_load": 79, 2024-04-26T23:09:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "avg_session_uptime": 611, 2024-04-26T23:09:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "mdthresh_evicted": 0 2024-04-26T23:09:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "mempool": { 2024-04-26T23:09:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_bytes": 720, 2024-04-26T23:09:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_items": 720, 2024-04-26T23:09:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T23:09:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_items": 0, 2024-04-26T23:09:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T23:09:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_items": 0, 2024-04-26T23:09:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T23:09:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T23:09:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T23:09:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T23:09:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T23:09:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_items": 0, 2024-04-26T23:09:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T23:09:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T23:09:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_bytes": 0, 2024-04-26T23:09:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_items": 0, 2024-04-26T23:09:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_bytes": 0, 2024-04-26T23:09:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_items": 0, 2024-04-26T23:09:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T23:09:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T23:09:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T23:09:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T23:09:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T23:09:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_items": 0, 2024-04-26T23:09:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_bytes": 0, 2024-04-26T23:09:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_items": 0, 2024-04-26T23:09:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T23:09:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T23:09:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_bytes": 0, 2024-04-26T23:09:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_items": 0, 2024-04-26T23:09:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_bytes": 0, 2024-04-26T23:09:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_items": 0, 2024-04-26T23:09:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T23:09:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_items": 0, 2024-04-26T23:09:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T23:09:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_items": 0, 2024-04-26T23:09:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_bytes": 4814508, 2024-04-26T23:09:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_items": 5693, 2024-04-26T23:09:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_bytes": 1320, 2024-04-26T23:09:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_items": 15, 2024-04-26T23:09:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_bytes": 0, 2024-04-26T23:09:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_items": 0, 2024-04-26T23:09:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_bytes": 0, 2024-04-26T23:09:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_items": 0, 2024-04-26T23:09:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_bytes": 0, 2024-04-26T23:09:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_items": 0, 2024-04-26T23:09:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_bytes": 10232, 2024-04-26T23:09:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_items": 215, 2024-04-26T23:09:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T23:09:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_items": 0, 2024-04-26T23:09:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_bytes": 0, 2024-04-26T23:09:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_items": 0, 2024-04-26T23:09:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_bytes": 18144812, 2024-04-26T23:09:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_items": 346553, 2024-04-26T23:09:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_bytes": 0, 2024-04-26T23:09:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_items": 0, 2024-04-26T23:09:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_bytes": 0, 2024-04-26T23:09:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_items": 0 2024-04-26T23:09:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "objecter": { 2024-04-26T23:09:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "op_active": 0, 2024-04-26T23:09:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "op_laggy": 0, 2024-04-26T23:09:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "op_send": 27518, 2024-04-26T23:09:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "op_send_bytes": 93075014, 2024-04-26T23:09:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "op_resend": 34, 2024-04-26T23:09:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "op_reply": 27518, 2024-04-26T23:09:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "op_latency": { 2024-04-26T23:09:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 27518, 2024-04-26T23:09:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 5618.493595076, 2024-04-26T23:09:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.204175216 2024-04-26T23:09:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "op_inflight": 0, 2024-04-26T23:09:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "oplen_avg": { 2024-04-26T23:09:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 27518, 2024-04-26T23:09:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 53585 2024-04-26T23:09:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "op": 27518, 2024-04-26T23:09:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "op_r": 117, 2024-04-26T23:09:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "op_w": 27401, 2024-04-26T23:09:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "op_rmw": 0, 2024-04-26T23:09:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "op_pg": 0, 2024-04-26T23:09:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_stat": 617, 2024-04-26T23:09:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_create": 14328, 2024-04-26T23:09:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_read": 87, 2024-04-26T23:09:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_write": 2946, 2024-04-26T23:09:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_writefull": 141, 2024-04-26T23:09:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_writesame": 0, 2024-04-26T23:09:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_append": 0, 2024-04-26T23:09:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_zero": 10, 2024-04-26T23:09:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_truncate": 3, 2024-04-26T23:09:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_delete": 7824, 2024-04-26T23:09:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_mapext": 0, 2024-04-26T23:09:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_sparse_read": 0, 2024-04-26T23:09:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_clonerange": 0, 2024-04-26T23:09:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_getxattr": 23, 2024-04-26T23:09:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_setxattr": 22928, 2024-04-26T23:09:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_cmpxattr": 0, 2024-04-26T23:09:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_rmxattr": 0, 2024-04-26T23:09:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_resetxattrs": 0, 2024-04-26T23:09:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_call": 0, 2024-04-26T23:09:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_watch": 0, 2024-04-26T23:09:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_notify": 0, 2024-04-26T23:09:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T23:09:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_pgls": 0, 2024-04-26T23:09:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_pgls_filter": 0, 2024-04-26T23:09:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_other": 257, 2024-04-26T23:09:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_active": 0, 2024-04-26T23:09:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_send": 0, 2024-04-26T23:09:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_resend": 0, 2024-04-26T23:09:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_ping": 0, 2024-04-26T23:09:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "poolop_active": 0, 2024-04-26T23:09:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "poolop_send": 0, 2024-04-26T23:09:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "poolop_resend": 0, 2024-04-26T23:09:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "poolstat_active": 0, 2024-04-26T23:09:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "poolstat_send": 0, 2024-04-26T23:09:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "poolstat_resend": 0, 2024-04-26T23:09:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "statfs_active": 0, 2024-04-26T23:09:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "statfs_send": 0, 2024-04-26T23:09:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "statfs_resend": 0, 2024-04-26T23:09:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "command_active": 0, 2024-04-26T23:09:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "command_send": 0, 2024-04-26T23:09:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "command_resend": 0, 2024-04-26T23:09:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "map_epoch": 101, 2024-04-26T23:09:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "map_full": 0, 2024-04-26T23:09:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "map_inc": 17, 2024-04-26T23:09:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_sessions": 12, 2024-04-26T23:09:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_session_open": 12, 2024-04-26T23:09:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_session_close": 0, 2024-04-26T23:09:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_laggy": 0, 2024-04-26T23:09:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_wr": 3594, 2024-04-26T23:09:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_rd": 56, 2024-04-26T23:09:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_del": 771 2024-04-26T23:09:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "oft": { 2024-04-26T23:09:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_objs": 1, 2024-04-26T23:09:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_kv_pairs": 0, 2024-04-26T23:09:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_updates": 0, 2024-04-26T23:09:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_removes": 0 2024-04-26T23:09:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "osd-slow-ops": { 2024-04-26T23:09:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "slow_ops_count": 0 2024-04-26T23:09:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "purge_queue": { 2024-04-26T23:09:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing_ops": 0, 2024-04-26T23:09:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing_ops_high_water": 23, 2024-04-26T23:09:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing": 0, 2024-04-26T23:09:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing_high_water": 15, 2024-04-26T23:09:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executed_ops": 7281, 2024-04-26T23:09:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executed": 3703, 2024-04-26T23:09:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_item_in_journal": 0 2024-04-26T23:09:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T23:09:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-04-26T23:09:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-04-26T23:09:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-04-26T23:09:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 90214, 2024-04-26T23:09:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 38431979, 2024-04-26T23:09:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 90214, 2024-04-26T23:09:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-04-26T23:09:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-04-26T23:09:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 90214, 2024-04-26T23:09:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 38431979, 2024-04-26T23:09:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-04-26T23:09:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-objecter_bytes": { 2024-04-26T23:09:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-04-26T23:09:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-04-26T23:09:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-04-26T23:09:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 0, 2024-04-26T23:09:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 0, 2024-04-26T23:09:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 0, 2024-04-26T23:09:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 27518, 2024-04-26T23:09:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 93524929, 2024-04-26T23:09:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 27518, 2024-04-26T23:09:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 93524929, 2024-04-26T23:09:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-04-26T23:09:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-objecter_ops": { 2024-04-26T23:09:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-04-26T23:09:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 1024, 2024-04-26T23:09:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-04-26T23:09:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 0, 2024-04-26T23:09:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 0, 2024-04-26T23:09:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 0, 2024-04-26T23:09:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 27518, 2024-04-26T23:09:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 27518, 2024-04-26T23:09:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 27518, 2024-04-26T23:09:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 27518, 2024-04-26T23:09:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-04-26T23:09:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-write_buf_throttle": { 2024-04-26T23:09:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-04-26T23:09:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 3758096384, 2024-04-26T23:09:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-04-26T23:09:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 3703, 2024-04-26T23:09:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 450363, 2024-04-26T23:09:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 3703, 2024-04-26T23:09:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-04-26T23:09:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-04-26T23:09:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 87, 2024-04-26T23:09:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 450363, 2024-04-26T23:09:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-04-26T23:09:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-write_buf_throttle-0x5612e7324460": { 2024-04-26T23:09:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 84, 2024-04-26T23:09:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 3758096384, 2024-04-26T23:09:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-04-26T23:09:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 24393, 2024-04-26T23:09:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 72079289, 2024-04-26T23:09:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 24393, 2024-04-26T23:09:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-04-26T23:09:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-04-26T23:09:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 2859, 2024-04-26T23:09:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 72079205, 2024-04-26T23:09:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-04-26T23:09:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:02.369 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-26T23:09:02.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:02 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:02.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:02 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:02.521 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:02 smithi080 ceph-mon[32324]: pgmap v322: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T23:09:02.559 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:02 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:02.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:02 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:02.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:02 smithi027 ceph-mon[28034]: pgmap v322: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T23:09:03.116 INFO:tasks.check_counter:Daemon mds.b mds.exported=24 2024-04-26T23:09:03.116 INFO:tasks.check_counter:Daemon mds.b mds.imported=24 2024-04-26T23:09:03.116 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=1396 2024-04-26T23:09:03.116 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=3459 2024-04-26T23:09:03.116 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-04-26T23:09:03.116 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-04-26T23:09:03.116 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-26T23:09:03.116 DEBUG:tasks.check_counter:Getting stats from e 2024-04-26T23:09:03.116 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-26T23:09:03.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:03 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:03.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:03 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:03.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:03 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:03.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:03 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:03.771 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:03 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:03.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:03 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:03.833 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T23:09:03.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:03 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:03.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:03 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:03.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:03 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:03.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:03 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:03.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:03 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:03.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:03 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:04.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:03 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:04.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:03 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:04.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:03 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:04.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:03 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:04.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:03 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:04.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:03 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:04.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:04 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-26T23:09:04.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:04 smithi027 ceph-mon[28034]: Unable to set osd_memory_target on smithi094 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:09:04.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:04 smithi027 ceph-mon[28034]: pgmap v323: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T23:09:04.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:04 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:04.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:04 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:04.981 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:04 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-26T23:09:04.981 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:04 smithi080 ceph-mon[32324]: Unable to set osd_memory_target on smithi094 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:09:04.981 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:04 smithi080 ceph-mon[32324]: pgmap v323: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T23:09:04.981 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:04 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:04.981 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:04 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:05.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:04 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-26T23:09:05.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:04 smithi094 ceph-mon[32889]: Unable to set osd_memory_target on smithi094 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:09:05.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:04 smithi094 ceph-mon[32889]: pgmap v323: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T23:09:05.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:04 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:05.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:04 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:05.584 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-26T23:09:05.584 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T23:09:05.584 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 242, 2024-04-26T23:09:05.584 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 229, 2024-04-26T23:09:05.584 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 1597200, 2024-04-26T23:09:05.584 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 140512, 2024-04-26T23:09:05.584 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 4, 2024-04-26T23:09:05.584 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 0, 2024-04-26T23:09:05.584 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.046695388, 2024-04-26T23:09:05.584 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.014386668, 2024-04-26T23:09:05.584 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.034515108, 2024-04-26T23:09:05.584 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.005623515, 2024-04-26T23:09:05.585 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:05.585 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 229, 2024-04-26T23:09:05.585 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.012165877, 2024-04-26T23:09:05.585 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000053126 2024-04-26T23:09:05.585 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:05.585 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:05.585 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:05.585 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:05.585 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:05.585 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:05.585 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 1597200, 2024-04-26T23:09:05.585 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 140512 2024-04-26T23:09:05.585 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:05.585 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T23:09:05.585 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 2, 2024-04-26T23:09:05.585 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 174, 2024-04-26T23:09:05.586 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 320, 2024-04-26T23:09:05.586 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 187728, 2024-04-26T23:09:05.586 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 5, 2024-04-26T23:09:05.586 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 3, 2024-04-26T23:09:05.586 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.010302008, 2024-04-26T23:09:05.586 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.006862409, 2024-04-26T23:09:05.586 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.001306733, 2024-04-26T23:09:05.586 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T23:09:05.586 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:05.586 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 171, 2024-04-26T23:09:05.586 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.033604490, 2024-04-26T23:09:05.586 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000196517 2024-04-26T23:09:05.586 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:05.587 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:05.587 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:05.587 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:05.587 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:05.587 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:05.587 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T23:09:05.587 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 187728 2024-04-26T23:09:05.587 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:05.587 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T23:09:05.587 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 0, 2024-04-26T23:09:05.587 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 2, 2024-04-26T23:09:05.587 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 0, 2024-04-26T23:09:05.587 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 0, 2024-04-26T23:09:05.587 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 2, 2024-04-26T23:09:05.588 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 1, 2024-04-26T23:09:05.588 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.000188913, 2024-04-26T23:09:05.588 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T23:09:05.588 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.000124983, 2024-04-26T23:09:05.588 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T23:09:05.588 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:05.588 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:05.588 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:05.588 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:05.588 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:05.588 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:05.588 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:05.588 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:05.588 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:05.588 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:05.588 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T23:09:05.589 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T23:09:05.589 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:05.589 INFO:teuthology.orchestra.run.smithi080.stdout: "cct": { 2024-04-26T23:09:05.589 INFO:teuthology.orchestra.run.smithi080.stdout: "total_workers": 0, 2024-04-26T23:09:05.589 INFO:teuthology.orchestra.run.smithi080.stdout: "unhealthy_workers": 0 2024-04-26T23:09:05.589 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:05.589 INFO:teuthology.orchestra.run.smithi080.stdout: "mempool": { 2024-04-26T23:09:05.589 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_bytes": 0, 2024-04-26T23:09:05.589 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_items": 0, 2024-04-26T23:09:05.589 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T23:09:05.589 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_items": 0, 2024-04-26T23:09:05.589 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T23:09:05.589 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_items": 0, 2024-04-26T23:09:05.589 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T23:09:05.589 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T23:09:05.590 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T23:09:05.590 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T23:09:05.590 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T23:09:05.590 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_items": 0, 2024-04-26T23:09:05.590 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T23:09:05.590 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T23:09:05.590 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_bytes": 0, 2024-04-26T23:09:05.590 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_items": 0, 2024-04-26T23:09:05.590 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_bytes": 0, 2024-04-26T23:09:05.590 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_items": 0, 2024-04-26T23:09:05.590 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T23:09:05.590 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T23:09:05.590 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T23:09:05.590 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T23:09:05.591 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T23:09:05.591 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_items": 0, 2024-04-26T23:09:05.591 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_bytes": 0, 2024-04-26T23:09:05.591 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_items": 0, 2024-04-26T23:09:05.591 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T23:09:05.591 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T23:09:05.591 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_bytes": 0, 2024-04-26T23:09:05.591 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_items": 0, 2024-04-26T23:09:05.591 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_bytes": 0, 2024-04-26T23:09:05.591 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_items": 0, 2024-04-26T23:09:05.591 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T23:09:05.591 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_items": 0, 2024-04-26T23:09:05.591 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T23:09:05.591 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_items": 0, 2024-04-26T23:09:05.591 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_bytes": 148692, 2024-04-26T23:09:05.592 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_items": 30, 2024-04-26T23:09:05.592 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_bytes": 0, 2024-04-26T23:09:05.592 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_items": 0, 2024-04-26T23:09:05.592 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_bytes": 0, 2024-04-26T23:09:05.592 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_items": 0, 2024-04-26T23:09:05.592 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_bytes": 0, 2024-04-26T23:09:05.592 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_items": 0, 2024-04-26T23:09:05.592 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_bytes": 0, 2024-04-26T23:09:05.592 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_items": 0, 2024-04-26T23:09:05.592 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_bytes": 0, 2024-04-26T23:09:05.592 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_items": 0, 2024-04-26T23:09:05.592 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T23:09:05.592 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_items": 0, 2024-04-26T23:09:05.592 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_bytes": 0, 2024-04-26T23:09:05.592 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_items": 0, 2024-04-26T23:09:05.593 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_bytes": 928, 2024-04-26T23:09:05.593 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_items": 2, 2024-04-26T23:09:05.593 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_bytes": 0, 2024-04-26T23:09:05.593 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_items": 0, 2024-04-26T23:09:05.593 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_bytes": 0, 2024-04-26T23:09:05.593 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_items": 0 2024-04-26T23:09:05.593 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:05.593 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T23:09:05.593 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-04-26T23:09:05.593 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-04-26T23:09:05.593 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-04-26T23:09:05.593 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 240, 2024-04-26T23:09:05.593 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 1560653, 2024-04-26T23:09:05.593 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:05.593 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 240, 2024-04-26T23:09:05.594 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-04-26T23:09:05.594 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-04-26T23:09:05.594 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 240, 2024-04-26T23:09:05.594 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 1560653, 2024-04-26T23:09:05.594 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-04-26T23:09:05.594 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:05.594 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:05.594 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:05.594 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:05.594 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:05.594 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-26T23:09:05.981 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:05 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:05.981 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:05 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:05.981 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:05 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:05.981 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:05 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:06.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:05 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:06.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:05 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:06.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:05 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:06.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:05 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:06.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:05 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:06.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:05 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:06.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:05 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:06.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:05 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:06.415 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-04-26T23:09:06.415 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-04-26T23:09:06.415 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-04-26T23:09:06.415 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-04-26T23:09:06.415 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-26T23:09:06.415 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-26T23:09:06.415 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-26T23:09:06.415 DEBUG:tasks.check_counter:Getting stats from h 2024-04-26T23:09:06.416 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-26T23:09:06.692 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T23:09:07.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:06 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-04-26T23:09:07.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:06 smithi094 ceph-mon[32889]: Unable to set osd_memory_target on smithi027 to 185039769: error parsing value: Value '185039769' is below minimum 939524096 2024-04-26T23:09:07.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:06 smithi094 ceph-mon[32889]: pgmap v324: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T23:09:07.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:06 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:07.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:06 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:07.101 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:06 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-04-26T23:09:07.101 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:06 smithi080 ceph-mon[32324]: Unable to set osd_memory_target on smithi027 to 185039769: error parsing value: Value '185039769' is below minimum 939524096 2024-04-26T23:09:07.101 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:06 smithi080 ceph-mon[32324]: pgmap v324: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T23:09:07.101 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:06 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:07.101 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:06 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:07.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:06 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-04-26T23:09:07.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:06 smithi027 ceph-mon[28034]: Unable to set osd_memory_target on smithi027 to 185039769: error parsing value: Value '185039769' is below minimum 939524096 2024-04-26T23:09:07.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:06 smithi027 ceph-mon[28034]: pgmap v324: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T23:09:07.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:06 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:07.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:06 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:08.440 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-26T23:09:08.440 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T23:09:08.440 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 1139, 2024-04-26T23:09:08.440 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 1139, 2024-04-26T23:09:08.440 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 27426609, 2024-04-26T23:09:08.440 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 361442, 2024-04-26T23:09:08.440 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 7, 2024-04-26T23:09:08.440 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 3, 2024-04-26T23:09:08.440 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.115304406, 2024-04-26T23:09:08.441 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.028437504, 2024-04-26T23:09:08.441 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.042822394, 2024-04-26T23:09:08.441 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.035568799, 2024-04-26T23:09:08.441 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:08.441 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 1138, 2024-04-26T23:09:08.441 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.058370054, 2024-04-26T23:09:08.441 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000051291 2024-04-26T23:09:08.441 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.441 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:08.441 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.441 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.441 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.441 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.441 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-26T23:09:08.442 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T23:09:08.442 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.442 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T23:09:08.442 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 353, 2024-04-26T23:09:08.442 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 525, 2024-04-26T23:09:08.442 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 37837466, 2024-04-26T23:09:08.442 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 651913, 2024-04-26T23:09:08.442 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 9, 2024-04-26T23:09:08.442 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 6, 2024-04-26T23:09:08.442 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.059737612, 2024-04-26T23:09:08.442 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.014009044, 2024-04-26T23:09:08.442 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.031803852, 2024-04-26T23:09:08.442 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.009711640, 2024-04-26T23:09:08.442 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:08.443 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 522, 2024-04-26T23:09:08.443 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.043024237, 2024-04-26T23:09:08.443 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000082421 2024-04-26T23:09:08.443 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.443 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:08.443 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.443 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.443 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.443 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.443 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-26T23:09:08.443 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 540608 2024-04-26T23:09:08.443 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.443 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T23:09:08.443 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 1176, 2024-04-26T23:09:08.444 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 1153, 2024-04-26T23:09:08.444 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 27130658, 2024-04-26T23:09:08.444 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 431890, 2024-04-26T23:09:08.444 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 5, 2024-04-26T23:09:08.444 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 4, 2024-04-26T23:09:08.444 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.159996662, 2024-04-26T23:09:08.444 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.045033765, 2024-04-26T23:09:08.444 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.074632857, 2024-04-26T23:09:08.444 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.040160426, 2024-04-26T23:09:08.444 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:08.444 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 1152, 2024-04-26T23:09:08.444 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.061431524, 2024-04-26T23:09:08.444 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000053325 2024-04-26T23:09:08.444 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.444 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:08.445 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.445 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.445 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.445 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.445 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 1668560, 2024-04-26T23:09:08.445 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 142192 2024-04-26T23:09:08.445 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.445 INFO:teuthology.orchestra.run.smithi080.stdout: "cct": { 2024-04-26T23:09:08.445 INFO:teuthology.orchestra.run.smithi080.stdout: "total_workers": 1, 2024-04-26T23:09:08.445 INFO:teuthology.orchestra.run.smithi080.stdout: "unhealthy_workers": 0 2024-04-26T23:09:08.445 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.445 INFO:teuthology.orchestra.run.smithi080.stdout: "finisher-MDSRank": { 2024-04-26T23:09:08.445 INFO:teuthology.orchestra.run.smithi080.stdout: "queue_len": 0, 2024-04-26T23:09:08.445 INFO:teuthology.orchestra.run.smithi080.stdout: "complete_latency": { 2024-04-26T23:09:08.446 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 4073, 2024-04-26T23:09:08.446 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.115771469, 2024-04-26T23:09:08.446 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000028424 2024-04-26T23:09:08.446 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:08.446 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.446 INFO:teuthology.orchestra.run.smithi080.stdout: "finisher-PurgeQueue": { 2024-04-26T23:09:08.446 INFO:teuthology.orchestra.run.smithi080.stdout: "queue_len": 0, 2024-04-26T23:09:08.446 INFO:teuthology.orchestra.run.smithi080.stdout: "complete_latency": { 2024-04-26T23:09:08.446 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.446 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.446 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.446 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:08.446 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.446 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": { 2024-04-26T23:09:08.447 INFO:teuthology.orchestra.run.smithi080.stdout: "request": 0, 2024-04-26T23:09:08.447 INFO:teuthology.orchestra.run.smithi080.stdout: "reply": 0, 2024-04-26T23:09:08.447 INFO:teuthology.orchestra.run.smithi080.stdout: "reply_latency": { 2024-04-26T23:09:08.447 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.447 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.447 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.447 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.447 INFO:teuthology.orchestra.run.smithi080.stdout: "slow_reply": 0, 2024-04-26T23:09:08.447 INFO:teuthology.orchestra.run.smithi080.stdout: "forward": 0, 2024-04-26T23:09:08.447 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_fetch_complete": 0, 2024-04-26T23:09:08.447 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_fetch_keys": 0, 2024-04-26T23:09:08.447 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_commit": 0, 2024-04-26T23:09:08.447 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_split": 0, 2024-04-26T23:09:08.447 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_merge": 0, 2024-04-26T23:09:08.448 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes": 31687, 2024-04-26T23:09:08.448 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_top": 1959, 2024-04-26T23:09:08.448 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_bottom": 29728, 2024-04-26T23:09:08.448 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_pin_tail": 0, 2024-04-26T23:09:08.448 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_pinned": 28887, 2024-04-26T23:09:08.448 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_expired": 0, 2024-04-26T23:09:08.448 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_with_caps": 0, 2024-04-26T23:09:08.448 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": 0, 2024-04-26T23:09:08.448 INFO:teuthology.orchestra.run.smithi080.stdout: "subtrees": 1, 2024-04-26T23:09:08.448 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse": 0, 2024-04-26T23:09:08.448 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_hit": 0, 2024-04-26T23:09:08.448 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_forward": 0, 2024-04-26T23:09:08.448 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_discover": 0, 2024-04-26T23:09:08.448 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_dir_fetch": 0, 2024-04-26T23:09:08.449 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_remote_ino": 0, 2024-04-26T23:09:08.449 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_lock": 0, 2024-04-26T23:09:08.449 INFO:teuthology.orchestra.run.smithi080.stdout: "load_cent": 0, 2024-04-26T23:09:08.449 INFO:teuthology.orchestra.run.smithi080.stdout: "q": 0, 2024-04-26T23:09:08.449 INFO:teuthology.orchestra.run.smithi080.stdout: "exported": 0, 2024-04-26T23:09:08.449 INFO:teuthology.orchestra.run.smithi080.stdout: "exported_inodes": 0, 2024-04-26T23:09:08.449 INFO:teuthology.orchestra.run.smithi080.stdout: "imported": 0, 2024-04-26T23:09:08.449 INFO:teuthology.orchestra.run.smithi080.stdout: "imported_inodes": 0, 2024-04-26T23:09:08.449 INFO:teuthology.orchestra.run.smithi080.stdout: "openino_dir_fetch": 0, 2024-04-26T23:09:08.449 INFO:teuthology.orchestra.run.smithi080.stdout: "openino_backtrace_fetch": 0, 2024-04-26T23:09:08.449 INFO:teuthology.orchestra.run.smithi080.stdout: "openino_peer_discover": 0, 2024-04-26T23:09:08.449 INFO:teuthology.orchestra.run.smithi080.stdout: "root_rfiles": 0, 2024-04-26T23:09:08.449 INFO:teuthology.orchestra.run.smithi080.stdout: "root_rbytes": 0, 2024-04-26T23:09:08.449 INFO:teuthology.orchestra.run.smithi080.stdout: "root_rsnaps": 0, 2024-04-26T23:09:08.449 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T23:09:08.449 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_set_tag": 0, 2024-04-26T23:09:08.450 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T23:09:08.450 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_inotable_repaired": 0, 2024-04-26T23:09:08.450 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_dir_inodes": 0, 2024-04-26T23:09:08.450 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T23:09:08.450 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T23:09:08.450 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_file_inodes": 0, 2024-04-26T23:09:08.450 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_inode_file_caps": 0, 2024-04-26T23:09:08.450 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T23:09:08.450 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_grant": 0, 2024-04-26T23:09:08.450 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T23:09:08.450 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T23:09:08.450 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T23:09:08.450 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_caps": 0, 2024-04-26T23:09:08.450 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_caps_dirty": 0, 2024-04-26T23:09:08.451 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_cap_release": 0, 2024-04-26T23:09:08.451 INFO:teuthology.orchestra.run.smithi080.stdout: "process_request_cap_release": 0 2024-04-26T23:09:08.451 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.451 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_cache": { 2024-04-26T23:09:08.451 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_update": 0, 2024-04-26T23:09:08.451 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_update_receipt": 0, 2024-04-26T23:09:08.451 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_try_discover": 0, 2024-04-26T23:09:08.451 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_send_discover": 0, 2024-04-26T23:09:08.451 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_handle_discover": 0, 2024-04-26T23:09:08.451 INFO:teuthology.orchestra.run.smithi080.stdout: "num_strays": 0, 2024-04-26T23:09:08.451 INFO:teuthology.orchestra.run.smithi080.stdout: "num_strays_delayed": 0, 2024-04-26T23:09:08.451 INFO:teuthology.orchestra.run.smithi080.stdout: "num_strays_enqueuing": 0, 2024-04-26T23:09:08.451 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_created": 0, 2024-04-26T23:09:08.451 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_enqueued": 0, 2024-04-26T23:09:08.452 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_reintegrated": 0, 2024-04-26T23:09:08.452 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_migrated": 0, 2024-04-26T23:09:08.452 INFO:teuthology.orchestra.run.smithi080.stdout: "num_recovering_processing": 0, 2024-04-26T23:09:08.452 INFO:teuthology.orchestra.run.smithi080.stdout: "num_recovering_enqueued": 0, 2024-04-26T23:09:08.452 INFO:teuthology.orchestra.run.smithi080.stdout: "num_recovering_prioritized": 0, 2024-04-26T23:09:08.452 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_started": 0, 2024-04-26T23:09:08.452 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_completed": 0, 2024-04-26T23:09:08.452 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_quiesce_path": 0, 2024-04-26T23:09:08.452 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_quiesce_inode": 0, 2024-04-26T23:09:08.452 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T23:09:08.452 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_exportdir": 0, 2024-04-26T23:09:08.452 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_flush": 0, 2024-04-26T23:09:08.452 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_fragmentdir": 0, 2024-04-26T23:09:08.452 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_fragstats": 0, 2024-04-26T23:09:08.452 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_inodestats": 0 2024-04-26T23:09:08.453 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.453 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_log": { 2024-04-26T23:09:08.453 INFO:teuthology.orchestra.run.smithi080.stdout: "evlrg": 0, 2024-04-26T23:09:08.453 INFO:teuthology.orchestra.run.smithi080.stdout: "evadd": 0, 2024-04-26T23:09:08.453 INFO:teuthology.orchestra.run.smithi080.stdout: "evex": 0, 2024-04-26T23:09:08.453 INFO:teuthology.orchestra.run.smithi080.stdout: "evtrm": 0, 2024-04-26T23:09:08.453 INFO:teuthology.orchestra.run.smithi080.stdout: "ev": 28027, 2024-04-26T23:09:08.453 INFO:teuthology.orchestra.run.smithi080.stdout: "evexg": 0, 2024-04-26T23:09:08.453 INFO:teuthology.orchestra.run.smithi080.stdout: "evexd": 0, 2024-04-26T23:09:08.453 INFO:teuthology.orchestra.run.smithi080.stdout: "segadd": 0, 2024-04-26T23:09:08.453 INFO:teuthology.orchestra.run.smithi080.stdout: "segex": 0, 2024-04-26T23:09:08.453 INFO:teuthology.orchestra.run.smithi080.stdout: "segtrm": 0, 2024-04-26T23:09:08.453 INFO:teuthology.orchestra.run.smithi080.stdout: "seg": 42, 2024-04-26T23:09:08.453 INFO:teuthology.orchestra.run.smithi080.stdout: "segmjr": 4, 2024-04-26T23:09:08.454 INFO:teuthology.orchestra.run.smithi080.stdout: "segexg": 0, 2024-04-26T23:09:08.454 INFO:teuthology.orchestra.run.smithi080.stdout: "segexd": 0, 2024-04-26T23:09:08.454 INFO:teuthology.orchestra.run.smithi080.stdout: "expos": 4194304, 2024-04-26T23:09:08.454 INFO:teuthology.orchestra.run.smithi080.stdout: "wrpos": 94236401, 2024-04-26T23:09:08.454 INFO:teuthology.orchestra.run.smithi080.stdout: "rdpos": 94234313, 2024-04-26T23:09:08.454 INFO:teuthology.orchestra.run.smithi080.stdout: "jlat": { 2024-04-26T23:09:08.454 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.454 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.454 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.454 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.454 INFO:teuthology.orchestra.run.smithi080.stdout: "replayed": 28027 2024-04-26T23:09:08.454 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.454 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_mem": { 2024-04-26T23:09:08.454 INFO:teuthology.orchestra.run.smithi080.stdout: "ino": 13173, 2024-04-26T23:09:08.455 INFO:teuthology.orchestra.run.smithi080.stdout: "ino+": 25898, 2024-04-26T23:09:08.455 INFO:teuthology.orchestra.run.smithi080.stdout: "ino-": 12725, 2024-04-26T23:09:08.455 INFO:teuthology.orchestra.run.smithi080.stdout: "dir": 1122, 2024-04-26T23:09:08.455 INFO:teuthology.orchestra.run.smithi080.stdout: "dir+": 1669, 2024-04-26T23:09:08.455 INFO:teuthology.orchestra.run.smithi080.stdout: "dir-": 547, 2024-04-26T23:09:08.455 INFO:teuthology.orchestra.run.smithi080.stdout: "dn": 31687, 2024-04-26T23:09:08.455 INFO:teuthology.orchestra.run.smithi080.stdout: "dn+": 36955, 2024-04-26T23:09:08.455 INFO:teuthology.orchestra.run.smithi080.stdout: "dn-": 5268, 2024-04-26T23:09:08.455 INFO:teuthology.orchestra.run.smithi080.stdout: "cap": 0, 2024-04-26T23:09:08.455 INFO:teuthology.orchestra.run.smithi080.stdout: "cap+": 0, 2024-04-26T23:09:08.455 INFO:teuthology.orchestra.run.smithi080.stdout: "cap-": 0, 2024-04-26T23:09:08.455 INFO:teuthology.orchestra.run.smithi080.stdout: "rss": 130944, 2024-04-26T23:09:08.455 INFO:teuthology.orchestra.run.smithi080.stdout: "heap": 215416 2024-04-26T23:09:08.455 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.455 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_server": { 2024-04-26T23:09:08.456 INFO:teuthology.orchestra.run.smithi080.stdout: "dispatch_client_request": 0, 2024-04-26T23:09:08.456 INFO:teuthology.orchestra.run.smithi080.stdout: "dispatch_server_request": 0, 2024-04-26T23:09:08.456 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_request": 0, 2024-04-26T23:09:08.456 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_session": 0, 2024-04-26T23:09:08.456 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_peer_request": 0, 2024-04-26T23:09:08.456 INFO:teuthology.orchestra.run.smithi080.stdout: "req_create_latency": { 2024-04-26T23:09:08.456 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.456 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.456 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.456 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.456 INFO:teuthology.orchestra.run.smithi080.stdout: "req_getattr_latency": { 2024-04-26T23:09:08.456 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.456 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.456 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.456 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.457 INFO:teuthology.orchestra.run.smithi080.stdout: "req_getfilelock_latency": { 2024-04-26T23:09:08.457 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.457 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.457 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.457 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.457 INFO:teuthology.orchestra.run.smithi080.stdout: "req_link_latency": { 2024-04-26T23:09:08.457 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.457 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.457 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.457 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.457 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookup_latency": { 2024-04-26T23:09:08.457 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.457 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.457 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.457 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.458 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookuphash_latency": { 2024-04-26T23:09:08.458 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.458 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.458 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.458 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.458 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupino_latency": { 2024-04-26T23:09:08.458 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.458 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.458 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.458 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.458 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupname_latency": { 2024-04-26T23:09:08.458 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.458 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.458 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.459 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.459 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupparent_latency": { 2024-04-26T23:09:08.459 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.459 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.459 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.459 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.459 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupsnap_latency": { 2024-04-26T23:09:08.459 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.459 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.459 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.459 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.459 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lssnap_latency": { 2024-04-26T23:09:08.459 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.459 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.459 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.460 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.460 INFO:teuthology.orchestra.run.smithi080.stdout: "req_mkdir_latency": { 2024-04-26T23:09:08.460 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.460 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.460 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.460 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.461 INFO:teuthology.orchestra.run.smithi080.stdout: "req_mknod_latency": { 2024-04-26T23:09:08.461 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.461 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.461 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.461 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.461 INFO:teuthology.orchestra.run.smithi080.stdout: "req_mksnap_latency": { 2024-04-26T23:09:08.461 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.461 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.461 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.461 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.461 INFO:teuthology.orchestra.run.smithi080.stdout: "req_open_latency": { 2024-04-26T23:09:08.461 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.461 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.461 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.461 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.462 INFO:teuthology.orchestra.run.smithi080.stdout: "req_readdir_latency": { 2024-04-26T23:09:08.462 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.462 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.462 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.462 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.462 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rename_latency": { 2024-04-26T23:09:08.462 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.462 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.462 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.462 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.462 INFO:teuthology.orchestra.run.smithi080.stdout: "req_renamesnap_latency": { 2024-04-26T23:09:08.462 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.462 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.462 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.463 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.463 INFO:teuthology.orchestra.run.smithi080.stdout: "req_snapdiff_latency": { 2024-04-26T23:09:08.463 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.463 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.463 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.463 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.463 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rmdir_latency": { 2024-04-26T23:09:08.463 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.463 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.463 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.463 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.463 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rmsnap_latency": { 2024-04-26T23:09:08.463 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.463 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.463 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.463 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.464 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rmxattr_latency": { 2024-04-26T23:09:08.464 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.464 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.464 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.464 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.464 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setattr_latency": { 2024-04-26T23:09:08.464 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.464 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.464 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.464 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.464 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setdirlayout_latency": { 2024-04-26T23:09:08.464 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.464 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.464 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.464 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.465 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setfilelock_latency": { 2024-04-26T23:09:08.465 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.465 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.465 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.465 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.465 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setlayout_latency": { 2024-04-26T23:09:08.465 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.465 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.465 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.465 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.465 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setxattr_latency": { 2024-04-26T23:09:08.465 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.465 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.465 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.466 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.466 INFO:teuthology.orchestra.run.smithi080.stdout: "req_symlink_latency": { 2024-04-26T23:09:08.466 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.466 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.466 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.466 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.466 INFO:teuthology.orchestra.run.smithi080.stdout: "req_unlink_latency": { 2024-04-26T23:09:08.466 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.466 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.466 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.466 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.466 INFO:teuthology.orchestra.run.smithi080.stdout: "cap_revoke_eviction": 0, 2024-04-26T23:09:08.466 INFO:teuthology.orchestra.run.smithi080.stdout: "cap_acquisition_throttle": 0, 2024-04-26T23:09:08.466 INFO:teuthology.orchestra.run.smithi080.stdout: "req_getvxattr_latency": { 2024-04-26T23:09:08.467 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.467 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.467 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.467 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:08.467 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.467 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_sessions": { 2024-04-26T23:09:08.467 INFO:teuthology.orchestra.run.smithi080.stdout: "session_count": 2, 2024-04-26T23:09:08.467 INFO:teuthology.orchestra.run.smithi080.stdout: "session_add": 4, 2024-04-26T23:09:08.467 INFO:teuthology.orchestra.run.smithi080.stdout: "session_remove": 2, 2024-04-26T23:09:08.467 INFO:teuthology.orchestra.run.smithi080.stdout: "sessions_open": 3, 2024-04-26T23:09:08.467 INFO:teuthology.orchestra.run.smithi080.stdout: "sessions_stale": 0, 2024-04-26T23:09:08.467 INFO:teuthology.orchestra.run.smithi080.stdout: "total_load": 0, 2024-04-26T23:09:08.467 INFO:teuthology.orchestra.run.smithi080.stdout: "average_load": 0, 2024-04-26T23:09:08.467 INFO:teuthology.orchestra.run.smithi080.stdout: "avg_session_uptime": 5565, 2024-04-26T23:09:08.467 INFO:teuthology.orchestra.run.smithi080.stdout: "mdthresh_evicted": 0 2024-04-26T23:09:08.468 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.468 INFO:teuthology.orchestra.run.smithi080.stdout: "mempool": { 2024-04-26T23:09:08.468 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_bytes": 0, 2024-04-26T23:09:08.468 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_items": 0, 2024-04-26T23:09:08.468 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T23:09:08.468 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_items": 0, 2024-04-26T23:09:08.468 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T23:09:08.468 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_items": 0, 2024-04-26T23:09:08.468 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T23:09:08.468 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T23:09:08.468 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T23:09:08.468 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T23:09:08.468 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T23:09:08.468 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_items": 0, 2024-04-26T23:09:08.469 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T23:09:08.469 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T23:09:08.469 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_bytes": 0, 2024-04-26T23:09:08.469 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_items": 0, 2024-04-26T23:09:08.469 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_bytes": 0, 2024-04-26T23:09:08.469 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_items": 0, 2024-04-26T23:09:08.469 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T23:09:08.469 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T23:09:08.469 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T23:09:08.469 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T23:09:08.469 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T23:09:08.469 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_items": 0, 2024-04-26T23:09:08.469 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_bytes": 0, 2024-04-26T23:09:08.469 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_items": 0, 2024-04-26T23:09:08.469 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T23:09:08.470 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T23:09:08.470 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_bytes": 0, 2024-04-26T23:09:08.470 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_items": 0, 2024-04-26T23:09:08.470 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_bytes": 0, 2024-04-26T23:09:08.470 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_items": 0, 2024-04-26T23:09:08.470 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T23:09:08.470 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_items": 0, 2024-04-26T23:09:08.470 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T23:09:08.470 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_items": 0, 2024-04-26T23:09:08.470 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_bytes": 769157, 2024-04-26T23:09:08.470 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_items": 14825, 2024-04-26T23:09:08.470 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_bytes": 0, 2024-04-26T23:09:08.470 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_items": 0, 2024-04-26T23:09:08.470 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_bytes": 0, 2024-04-26T23:09:08.471 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_items": 0, 2024-04-26T23:09:08.471 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_bytes": 0, 2024-04-26T23:09:08.471 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_items": 0, 2024-04-26T23:09:08.471 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_bytes": 0, 2024-04-26T23:09:08.471 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_items": 0, 2024-04-26T23:09:08.471 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_bytes": 10232, 2024-04-26T23:09:08.471 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_items": 215, 2024-04-26T23:09:08.471 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T23:09:08.471 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_items": 0, 2024-04-26T23:09:08.471 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_bytes": 0, 2024-04-26T23:09:08.471 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_items": 0, 2024-04-26T23:09:08.471 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_bytes": 56292620, 2024-04-26T23:09:08.471 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_items": 968624, 2024-04-26T23:09:08.471 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_bytes": 0, 2024-04-26T23:09:08.471 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_items": 0, 2024-04-26T23:09:08.472 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_bytes": 0, 2024-04-26T23:09:08.472 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_items": 0 2024-04-26T23:09:08.472 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.472 INFO:teuthology.orchestra.run.smithi080.stdout: "objecter": { 2024-04-26T23:09:08.472 INFO:teuthology.orchestra.run.smithi080.stdout: "op_active": 0, 2024-04-26T23:09:08.472 INFO:teuthology.orchestra.run.smithi080.stdout: "op_laggy": 0, 2024-04-26T23:09:08.472 INFO:teuthology.orchestra.run.smithi080.stdout: "op_send": 2396, 2024-04-26T23:09:08.472 INFO:teuthology.orchestra.run.smithi080.stdout: "op_send_bytes": 21, 2024-04-26T23:09:08.472 INFO:teuthology.orchestra.run.smithi080.stdout: "op_resend": 3, 2024-04-26T23:09:08.472 INFO:teuthology.orchestra.run.smithi080.stdout: "op_reply": 2396, 2024-04-26T23:09:08.472 INFO:teuthology.orchestra.run.smithi080.stdout: "op_latency": { 2024-04-26T23:09:08.472 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 2396, 2024-04-26T23:09:08.472 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 3.053928552, 2024-04-26T23:09:08.472 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.001274594 2024-04-26T23:09:08.472 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.473 INFO:teuthology.orchestra.run.smithi080.stdout: "op_inflight": 0, 2024-04-26T23:09:08.473 INFO:teuthology.orchestra.run.smithi080.stdout: "oplen_avg": { 2024-04-26T23:09:08.473 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 2396, 2024-04-26T23:09:08.473 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 2397 2024-04-26T23:09:08.473 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.473 INFO:teuthology.orchestra.run.smithi080.stdout: "op": 2396, 2024-04-26T23:09:08.473 INFO:teuthology.orchestra.run.smithi080.stdout: "op_r": 2396, 2024-04-26T23:09:08.473 INFO:teuthology.orchestra.run.smithi080.stdout: "op_w": 0, 2024-04-26T23:09:08.473 INFO:teuthology.orchestra.run.smithi080.stdout: "op_rmw": 0, 2024-04-26T23:09:08.473 INFO:teuthology.orchestra.run.smithi080.stdout: "op_pg": 0, 2024-04-26T23:09:08.473 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_stat": 1470, 2024-04-26T23:09:08.473 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_create": 0, 2024-04-26T23:09:08.473 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_read": 924, 2024-04-26T23:09:08.473 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_write": 0, 2024-04-26T23:09:08.474 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_writefull": 0, 2024-04-26T23:09:08.474 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_writesame": 0, 2024-04-26T23:09:08.474 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_append": 0, 2024-04-26T23:09:08.474 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_zero": 0, 2024-04-26T23:09:08.474 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_truncate": 0, 2024-04-26T23:09:08.474 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_delete": 0, 2024-04-26T23:09:08.474 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_mapext": 0, 2024-04-26T23:09:08.474 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_sparse_read": 0, 2024-04-26T23:09:08.474 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_clonerange": 0, 2024-04-26T23:09:08.474 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_getxattr": 1, 2024-04-26T23:09:08.474 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_setxattr": 0, 2024-04-26T23:09:08.474 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_cmpxattr": 0, 2024-04-26T23:09:08.474 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_rmxattr": 0, 2024-04-26T23:09:08.474 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_resetxattrs": 0, 2024-04-26T23:09:08.475 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_call": 0, 2024-04-26T23:09:08.475 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_watch": 0, 2024-04-26T23:09:08.475 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_notify": 0, 2024-04-26T23:09:08.475 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T23:09:08.475 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_pgls": 0, 2024-04-26T23:09:08.475 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_pgls_filter": 0, 2024-04-26T23:09:08.475 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_other": 0, 2024-04-26T23:09:08.475 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_active": 0, 2024-04-26T23:09:08.475 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_send": 0, 2024-04-26T23:09:08.475 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_resend": 0, 2024-04-26T23:09:08.475 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_ping": 0, 2024-04-26T23:09:08.475 INFO:teuthology.orchestra.run.smithi080.stdout: "poolop_active": 0, 2024-04-26T23:09:08.475 INFO:teuthology.orchestra.run.smithi080.stdout: "poolop_send": 0, 2024-04-26T23:09:08.475 INFO:teuthology.orchestra.run.smithi080.stdout: "poolop_resend": 0, 2024-04-26T23:09:08.475 INFO:teuthology.orchestra.run.smithi080.stdout: "poolstat_active": 0, 2024-04-26T23:09:08.476 INFO:teuthology.orchestra.run.smithi080.stdout: "poolstat_send": 0, 2024-04-26T23:09:08.476 INFO:teuthology.orchestra.run.smithi080.stdout: "poolstat_resend": 0, 2024-04-26T23:09:08.476 INFO:teuthology.orchestra.run.smithi080.stdout: "statfs_active": 0, 2024-04-26T23:09:08.476 INFO:teuthology.orchestra.run.smithi080.stdout: "statfs_send": 0, 2024-04-26T23:09:08.476 INFO:teuthology.orchestra.run.smithi080.stdout: "statfs_resend": 0, 2024-04-26T23:09:08.476 INFO:teuthology.orchestra.run.smithi080.stdout: "command_active": 0, 2024-04-26T23:09:08.476 INFO:teuthology.orchestra.run.smithi080.stdout: "command_send": 0, 2024-04-26T23:09:08.476 INFO:teuthology.orchestra.run.smithi080.stdout: "command_resend": 0, 2024-04-26T23:09:08.476 INFO:teuthology.orchestra.run.smithi080.stdout: "map_epoch": 101, 2024-04-26T23:09:08.476 INFO:teuthology.orchestra.run.smithi080.stdout: "map_full": 0, 2024-04-26T23:09:08.476 INFO:teuthology.orchestra.run.smithi080.stdout: "map_inc": 11, 2024-04-26T23:09:08.476 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_sessions": 9, 2024-04-26T23:09:08.476 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_session_open": 9, 2024-04-26T23:09:08.476 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_session_close": 0, 2024-04-26T23:09:08.476 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_laggy": 0, 2024-04-26T23:09:08.476 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_wr": 0, 2024-04-26T23:09:08.477 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_rd": 2, 2024-04-26T23:09:08.477 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_del": 0 2024-04-26T23:09:08.477 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.477 INFO:teuthology.orchestra.run.smithi080.stdout: "oft": { 2024-04-26T23:09:08.477 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_objs": 0, 2024-04-26T23:09:08.477 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_kv_pairs": 0, 2024-04-26T23:09:08.477 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_updates": 0, 2024-04-26T23:09:08.477 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_removes": 0 2024-04-26T23:09:08.477 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.477 INFO:teuthology.orchestra.run.smithi080.stdout: "osd-slow-ops": { 2024-04-26T23:09:08.477 INFO:teuthology.orchestra.run.smithi080.stdout: "slow_ops_count": 0 2024-04-26T23:09:08.477 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.477 INFO:teuthology.orchestra.run.smithi080.stdout: "purge_queue": { 2024-04-26T23:09:08.477 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing_ops": 0, 2024-04-26T23:09:08.478 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T23:09:08.478 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing": 0, 2024-04-26T23:09:08.478 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing_high_water": 0, 2024-04-26T23:09:08.478 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executed_ops": 0, 2024-04-26T23:09:08.478 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executed": 0, 2024-04-26T23:09:08.478 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_item_in_journal": 0 2024-04-26T23:09:08.478 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.478 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T23:09:08.478 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-04-26T23:09:08.478 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-04-26T23:09:08.478 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-04-26T23:09:08.478 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 2664, 2024-04-26T23:09:08.478 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 92138432, 2024-04-26T23:09:08.478 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:08.478 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 2664, 2024-04-26T23:09:08.479 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-04-26T23:09:08.479 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-04-26T23:09:08.479 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 2664, 2024-04-26T23:09:08.479 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 92138432, 2024-04-26T23:09:08.479 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-04-26T23:09:08.479 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.479 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.479 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.479 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:08.479 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.479 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-objecter_bytes": { 2024-04-26T23:09:08.479 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-04-26T23:09:08.479 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-04-26T23:09:08.479 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-04-26T23:09:08.479 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 0, 2024-04-26T23:09:08.480 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 0, 2024-04-26T23:09:08.480 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:08.480 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 0, 2024-04-26T23:09:08.480 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 2396, 2024-04-26T23:09:08.480 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 90046717, 2024-04-26T23:09:08.480 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 2396, 2024-04-26T23:09:08.480 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 90046717, 2024-04-26T23:09:08.480 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-04-26T23:09:08.480 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.480 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.480 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.480 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:08.480 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.480 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-objecter_ops": { 2024-04-26T23:09:08.481 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-04-26T23:09:08.481 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 1024, 2024-04-26T23:09:08.481 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-04-26T23:09:08.481 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 0, 2024-04-26T23:09:08.481 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 0, 2024-04-26T23:09:08.481 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:08.481 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 0, 2024-04-26T23:09:08.481 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 2396, 2024-04-26T23:09:08.481 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 2396, 2024-04-26T23:09:08.481 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 2396, 2024-04-26T23:09:08.481 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 2396, 2024-04-26T23:09:08.481 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-04-26T23:09:08.481 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.481 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.481 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.482 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:08.482 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.482 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-write_buf_throttle": { 2024-04-26T23:09:08.482 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-04-26T23:09:08.482 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 3758096384, 2024-04-26T23:09:08.482 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-04-26T23:09:08.482 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 0, 2024-04-26T23:09:08.482 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 0, 2024-04-26T23:09:08.482 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:08.482 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 0, 2024-04-26T23:09:08.482 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-04-26T23:09:08.482 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-04-26T23:09:08.482 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 0, 2024-04-26T23:09:08.482 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 0, 2024-04-26T23:09:08.483 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-04-26T23:09:08.483 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.483 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.483 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.483 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:08.483 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:08.483 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-write_buf_throttle-0x55a4b712e6e0": { 2024-04-26T23:09:08.483 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-04-26T23:09:08.483 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 3758096384, 2024-04-26T23:09:08.483 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-04-26T23:09:08.483 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 0, 2024-04-26T23:09:08.483 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 0, 2024-04-26T23:09:08.483 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:08.483 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 0, 2024-04-26T23:09:08.483 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-04-26T23:09:08.484 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-04-26T23:09:08.484 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 0, 2024-04-26T23:09:08.484 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 0, 2024-04-26T23:09:08.484 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-04-26T23:09:08.484 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:08.484 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:08.484 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:08.484 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:08.484 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:08.484 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-26T23:09:09.214 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:08 smithi080 ceph-mon[32324]: pgmap v325: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T23:09:09.214 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:08 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:09.214 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:08 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:09.214 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:08 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:09.215 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:08 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:09.215 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:08 smithi080 ceph-mon[32324]: Adjusting osd_memory_target on smithi080 to 176.2M 2024-04-26T23:09:09.215 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:08 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:09:09.215 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:08 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:09:09.215 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:08 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:09.261 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-26T23:09:09.261 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-26T23:09:09.261 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-26T23:09:09.261 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-04-26T23:09:09.261 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-04-26T23:09:09.262 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-26T23:09:09.262 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T23:09:09.262 DEBUG:tasks.check_counter:Getting stats from k 2024-04-26T23:09:09.262 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-26T23:09:09.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:08 smithi094 ceph-mon[32889]: pgmap v325: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T23:09:09.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:08 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:09.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:08 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:09.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:08 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:09.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:08 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:09.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:08 smithi094 ceph-mon[32889]: Adjusting osd_memory_target on smithi080 to 176.2M 2024-04-26T23:09:09.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:08 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:09:09.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:08 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:09:09.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:08 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:09.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:08 smithi027 ceph-mon[28034]: pgmap v325: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T23:09:09.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:08 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:09.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:08 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:09.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:08 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:09.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:08 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:09:09.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:08 smithi027 ceph-mon[28034]: Adjusting osd_memory_target on smithi080 to 176.2M 2024-04-26T23:09:09.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:08 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:09:09.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:08 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:09:09.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:08 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' 2024-04-26T23:09:09.692 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T23:09:10.115 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:09 smithi080 ceph-mon[32324]: Unable to set osd_memory_target on smithi080 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:09:10.115 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:09 smithi080 ceph-mon[32324]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T23:09:10.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:09 smithi094 ceph-mon[32889]: Unable to set osd_memory_target on smithi080 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:09:10.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:09 smithi094 ceph-mon[32889]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T23:09:10.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:09 smithi027 ceph-mon[28034]: Unable to set osd_memory_target on smithi080 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:09:10.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:09 smithi027 ceph-mon[28034]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T23:09:11.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:10 smithi080 ceph-mon[32324]: pgmap v327: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-26T23:09:11.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:10 smithi094 ceph-mon[32889]: pgmap v327: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-26T23:09:11.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:10 smithi027 ceph-mon[28034]: pgmap v327: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-26T23:09:12.238 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-26T23:09:12.238 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T23:09:12.238 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 0, 2024-04-26T23:09:12.238 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 2, 2024-04-26T23:09:12.238 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 0, 2024-04-26T23:09:12.238 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 0, 2024-04-26T23:09:12.238 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 4, 2024-04-26T23:09:12.238 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 0, 2024-04-26T23:09:12.239 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.000962884, 2024-04-26T23:09:12.239 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T23:09:12.239 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.000154242, 2024-04-26T23:09:12.239 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T23:09:12.239 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:12.239 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:12.239 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:12.239 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:12.239 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:12.239 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:12.239 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:12.239 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:12.239 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:12.239 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:12.240 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T23:09:12.240 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T23:09:12.240 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:12.240 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T23:09:12.240 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 6, 2024-04-26T23:09:12.240 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 180, 2024-04-26T23:09:12.240 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 3312, 2024-04-26T23:09:12.240 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 188880, 2024-04-26T23:09:12.240 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 5, 2024-04-26T23:09:12.240 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 3, 2024-04-26T23:09:12.240 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.011254436, 2024-04-26T23:09:12.240 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.007568338, 2024-04-26T23:09:12.240 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.001502608, 2024-04-26T23:09:12.240 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T23:09:12.241 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:12.241 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 178, 2024-04-26T23:09:12.241 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.035129014, 2024-04-26T23:09:12.241 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000197354 2024-04-26T23:09:12.241 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:12.241 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:12.241 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:12.241 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:12.241 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:12.241 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:12.241 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-26T23:09:12.241 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 188880 2024-04-26T23:09:12.241 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:12.241 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T23:09:12.241 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 243, 2024-04-26T23:09:12.242 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 232, 2024-04-26T23:09:12.242 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 1595568, 2024-04-26T23:09:12.242 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 143264, 2024-04-26T23:09:12.242 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 2, 2024-04-26T23:09:12.242 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 1, 2024-04-26T23:09:12.242 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.046038725, 2024-04-26T23:09:12.242 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.014396456, 2024-04-26T23:09:12.242 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.033721933, 2024-04-26T23:09:12.242 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.005809190, 2024-04-26T23:09:12.242 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:12.242 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 231, 2024-04-26T23:09:12.242 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.011035563, 2024-04-26T23:09:12.242 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000047773 2024-04-26T23:09:12.242 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:12.243 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:12.243 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:12.243 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:12.243 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:12.243 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:12.243 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 1595568, 2024-04-26T23:09:12.243 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 143264 2024-04-26T23:09:12.243 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:12.243 INFO:teuthology.orchestra.run.smithi080.stdout: "cct": { 2024-04-26T23:09:12.243 INFO:teuthology.orchestra.run.smithi080.stdout: "total_workers": 0, 2024-04-26T23:09:12.243 INFO:teuthology.orchestra.run.smithi080.stdout: "unhealthy_workers": 0 2024-04-26T23:09:12.243 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:12.243 INFO:teuthology.orchestra.run.smithi080.stdout: "mempool": { 2024-04-26T23:09:12.243 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_bytes": 0, 2024-04-26T23:09:12.243 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_items": 0, 2024-04-26T23:09:12.244 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T23:09:12.244 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_items": 0, 2024-04-26T23:09:12.244 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T23:09:12.244 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_items": 0, 2024-04-26T23:09:12.244 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T23:09:12.244 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T23:09:12.244 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T23:09:12.244 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T23:09:12.244 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T23:09:12.244 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_items": 0, 2024-04-26T23:09:12.244 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T23:09:12.244 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T23:09:12.244 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_bytes": 0, 2024-04-26T23:09:12.244 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_items": 0, 2024-04-26T23:09:12.244 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_bytes": 0, 2024-04-26T23:09:12.245 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_items": 0, 2024-04-26T23:09:12.245 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T23:09:12.245 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T23:09:12.245 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T23:09:12.245 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T23:09:12.245 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T23:09:12.245 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_items": 0, 2024-04-26T23:09:12.245 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_bytes": 0, 2024-04-26T23:09:12.245 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_items": 0, 2024-04-26T23:09:12.245 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T23:09:12.245 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T23:09:12.245 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_bytes": 0, 2024-04-26T23:09:12.245 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_items": 0, 2024-04-26T23:09:12.245 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_bytes": 0, 2024-04-26T23:09:12.245 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_items": 0, 2024-04-26T23:09:12.246 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T23:09:12.246 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_items": 0, 2024-04-26T23:09:12.246 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T23:09:12.246 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_items": 0, 2024-04-26T23:09:12.246 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_bytes": 148714, 2024-04-26T23:09:12.246 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_items": 30, 2024-04-26T23:09:12.246 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_bytes": 0, 2024-04-26T23:09:12.246 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_items": 0, 2024-04-26T23:09:12.246 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_bytes": 0, 2024-04-26T23:09:12.246 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_items": 0, 2024-04-26T23:09:12.246 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_bytes": 0, 2024-04-26T23:09:12.246 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_items": 0, 2024-04-26T23:09:12.246 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_bytes": 0, 2024-04-26T23:09:12.246 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_items": 0, 2024-04-26T23:09:12.247 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_bytes": 0, 2024-04-26T23:09:12.247 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_items": 0, 2024-04-26T23:09:12.247 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T23:09:12.247 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_items": 0, 2024-04-26T23:09:12.247 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_bytes": 0, 2024-04-26T23:09:12.247 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_items": 0, 2024-04-26T23:09:12.247 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_bytes": 928, 2024-04-26T23:09:12.247 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_items": 2, 2024-04-26T23:09:12.247 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_bytes": 0, 2024-04-26T23:09:12.247 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_items": 0, 2024-04-26T23:09:12.247 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_bytes": 0, 2024-04-26T23:09:12.247 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_items": 0 2024-04-26T23:09:12.247 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-26T23:09:12.247 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T23:09:12.248 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-04-26T23:09:12.248 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-04-26T23:09:12.248 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-04-26T23:09:12.248 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 245, 2024-04-26T23:09:12.248 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 1561303, 2024-04-26T23:09:12.248 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:12.248 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 245, 2024-04-26T23:09:12.248 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-04-26T23:09:12.248 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-04-26T23:09:12.248 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 245, 2024-04-26T23:09:12.248 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 1561303, 2024-04-26T23:09:12.248 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-04-26T23:09:12.248 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-26T23:09:12.248 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-26T23:09:12.249 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-26T23:09:12.249 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:12.249 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-26T23:09:12.249 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-26T23:09:13.134 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:12 smithi080 ceph-mon[32324]: pgmap v328: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 165 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:09:13.135 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-26T23:09:13.135 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-26T23:09:13.135 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-26T23:09:13.135 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-26T23:09:13.135 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-26T23:09:13.136 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-26T23:09:13.136 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-26T23:09:13.136 DEBUG:tasks.check_counter:Getting stats from c 2024-04-26T23:09:13.136 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-26T23:09:13.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:12 smithi094 ceph-mon[32889]: pgmap v328: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 165 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:09:13.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:12 smithi027 ceph-mon[28034]: pgmap v328: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 165 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:09:13.385 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T23:09:15.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:14 smithi094 ceph-mon[32889]: pgmap v329: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 165 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:09:15.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:14 smithi080 ceph-mon[32324]: pgmap v329: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 165 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:09:15.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:14 smithi027 ceph-mon[28034]: pgmap v329: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 165 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:09:15.669 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T23:09:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T23:09:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1803, 2024-04-26T23:09:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 1803, 2024-04-26T23:09:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 34083626, 2024-04-26T23:09:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 572537, 2024-04-26T23:09:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 8, 2024-04-26T23:09:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 4, 2024-04-26T23:09:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.201385119, 2024-04-26T23:09:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.054812095, 2024-04-26T23:09:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.066891862, 2024-04-26T23:09:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.064191303, 2024-04-26T23:09:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1802, 2024-04-26T23:09:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.084367263, 2024-04-26T23:09:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000046818 2024-04-26T23:09:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-26T23:09:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T23:09:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T23:09:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 834, 2024-04-26T23:09:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 811, 2024-04-26T23:09:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 24603668, 2024-04-26T23:09:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 323698, 2024-04-26T23:09:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 5, 2024-04-26T23:09:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 4, 2024-04-26T23:09:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.119579466, 2024-04-26T23:09:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.030868703, 2024-04-26T23:09:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.070148629, 2024-04-26T23:09:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.023980265, 2024-04-26T23:09:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 810, 2024-04-26T23:09:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.035436640, 2024-04-26T23:09:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000043748 2024-04-26T23:09:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1670416, 2024-04-26T23:09:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 143648 2024-04-26T23:09:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T23:09:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 40, 2024-04-26T23:09:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 213, 2024-04-26T23:09:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 8398940, 2024-04-26T23:09:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 552374, 2024-04-26T23:09:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 9, 2024-04-26T23:09:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 6, 2024-04-26T23:09:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.021252125, 2024-04-26T23:09:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.008399824, 2024-04-26T23:09:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.009655047, 2024-04-26T23:09:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000863472, 2024-04-26T23:09:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 209, 2024-04-26T23:09:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.038423151, 2024-04-26T23:09:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000183842 2024-04-26T23:09:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-26T23:09:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 540608 2024-04-26T23:09:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-26T23:09:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-04-26T23:09:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-26T23:09:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-04-26T23:09:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-26T23:09:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-26T23:09:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 4085, 2024-04-26T23:09:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.120398448, 2024-04-26T23:09:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000029473 2024-04-26T23:09:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-04-26T23:09:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-26T23:09:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-26T23:09:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-26T23:09:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 0, 2024-04-26T23:09:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 0, 2024-04-26T23:09:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-04-26T23:09:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-04-26T23:09:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 0, 2024-04-26T23:09:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-04-26T23:09:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-04-26T23:09:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 0, 2024-04-26T23:09:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-04-26T23:09:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-04-26T23:09:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 2588, 2024-04-26T23:09:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 21, 2024-04-26T23:09:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 2567, 2024-04-26T23:09:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-04-26T23:09:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 2557, 2024-04-26T23:09:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 0, 2024-04-26T23:09:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 0, 2024-04-26T23:09:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 0, 2024-04-26T23:09:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 1, 2024-04-26T23:09:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 0, 2024-04-26T23:09:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 0, 2024-04-26T23:09:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 0, 2024-04-26T23:09:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 0, 2024-04-26T23:09:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-04-26T23:09:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-04-26T23:09:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-04-26T23:09:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-04-26T23:09:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-04-26T23:09:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-04-26T23:09:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-04-26T23:09:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-04-26T23:09:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-04-26T23:09:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-04-26T23:09:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-04-26T23:09:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-04-26T23:09:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 0, 2024-04-26T23:09:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 0, 2024-04-26T23:09:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 0, 2024-04-26T23:09:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T23:09:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-04-26T23:09:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T23:09:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-04-26T23:09:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-04-26T23:09:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T23:09:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T23:09:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-04-26T23:09:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-04-26T23:09:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T23:09:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 0, 2024-04-26T23:09:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T23:09:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T23:09:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T23:09:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 0, 2024-04-26T23:09:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 0, 2024-04-26T23:09:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 0, 2024-04-26T23:09:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 0 2024-04-26T23:09:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-04-26T23:09:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-04-26T23:09:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 0, 2024-04-26T23:09:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 0, 2024-04-26T23:09:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 0, 2024-04-26T23:09:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 0, 2024-04-26T23:09:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-04-26T23:09:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-04-26T23:09:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-04-26T23:09:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-04-26T23:09:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-04-26T23:09:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-04-26T23:09:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-04-26T23:09:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-04-26T23:09:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-04-26T23:09:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-04-26T23:09:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-04-26T23:09:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-04-26T23:09:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-04-26T23:09:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-04-26T23:09:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T23:09:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-04-26T23:09:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-04-26T23:09:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-04-26T23:09:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-04-26T23:09:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-04-26T23:09:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-04-26T23:09:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-04-26T23:09:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 0, 2024-04-26T23:09:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-04-26T23:09:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-04-26T23:09:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 25244, 2024-04-26T23:09:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-04-26T23:09:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-04-26T23:09:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 0, 2024-04-26T23:09:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-04-26T23:09:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-04-26T23:09:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 32, 2024-04-26T23:09:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 4, 2024-04-26T23:09:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-04-26T23:09:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-04-26T23:09:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4194304, 2024-04-26T23:09:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 68916576, 2024-04-26T23:09:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 68914015, 2024-04-26T23:09:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-04-26T23:09:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 25244 2024-04-26T23:09:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-04-26T23:09:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 525, 2024-04-26T23:09:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 17597, 2024-04-26T23:09:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 17072, 2024-04-26T23:09:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 115, 2024-04-26T23:09:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 1561, 2024-04-26T23:09:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 1446, 2024-04-26T23:09:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 2588, 2024-04-26T23:09:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 19490, 2024-04-26T23:09:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 16902, 2024-04-26T23:09:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 0, 2024-04-26T23:09:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 0, 2024-04-26T23:09:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 0, 2024-04-26T23:09:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 74028, 2024-04-26T23:09:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 215416 2024-04-26T23:09:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-04-26T23:09:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 0, 2024-04-26T23:09:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 0, 2024-04-26T23:09:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 0, 2024-04-26T23:09:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 0, 2024-04-26T23:09:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 0, 2024-04-26T23:09:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-04-26T23:09:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-04-26T23:09:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-04-26T23:09:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-04-26T23:09:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-04-26T23:09:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-04-26T23:09:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-04-26T23:09:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-04-26T23:09:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-04-26T23:09:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-04-26T23:09:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-04-26T23:09:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-04-26T23:09:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-04-26T23:09:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-04-26T23:09:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-04-26T23:09:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-04-26T23:09:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-04-26T23:09:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-04-26T23:09:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-04-26T23:09:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-04-26T23:09:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-04-26T23:09:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-04-26T23:09:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-04-26T23:09:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-04-26T23:09:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-04-26T23:09:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-04-26T23:09:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-04-26T23:09:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-04-26T23:09:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-04-26T23:09:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-04-26T23:09:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-04-26T23:09:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-04-26T23:09:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-04-26T23:09:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 2, 2024-04-26T23:09:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 3, 2024-04-26T23:09:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 1, 2024-04-26T23:09:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 3, 2024-04-26T23:09:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-04-26T23:09:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 0, 2024-04-26T23:09:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 0, 2024-04-26T23:09:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 3245, 2024-04-26T23:09:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-04-26T23:09:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-26T23:09:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-26T23:09:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-26T23:09:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T23:09:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-26T23:09:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T23:09:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-26T23:09:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T23:09:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T23:09:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T23:09:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T23:09:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T23:09:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-26T23:09:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T23:09:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T23:09:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-26T23:09:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-26T23:09:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-26T23:09:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-26T23:09:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T23:09:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T23:09:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T23:09:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T23:09:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T23:09:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-26T23:09:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-26T23:09:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-26T23:09:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T23:09:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T23:09:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-26T23:09:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-26T23:09:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-26T23:09:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-26T23:09:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T23:09:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-26T23:09:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T23:09:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-26T23:09:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 255044, 2024-04-26T23:09:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 715, 2024-04-26T23:09:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-26T23:09:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-26T23:09:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-26T23:09:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-26T23:09:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-26T23:09:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-26T23:09:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-26T23:09:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-26T23:09:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 10472, 2024-04-26T23:09:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 221, 2024-04-26T23:09:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T23:09:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-26T23:09:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-26T23:09:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-26T23:09:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 3496671, 2024-04-26T23:09:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 194974, 2024-04-26T23:09:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-26T23:09:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-26T23:09:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-26T23:09:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-26T23:09:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-04-26T23:09:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-04-26T23:09:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-04-26T23:09:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 2402, 2024-04-26T23:09:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 21, 2024-04-26T23:09:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 3, 2024-04-26T23:09:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 2402, 2024-04-26T23:09:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-04-26T23:09:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2402, 2024-04-26T23:09:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 2.067967832, 2024-04-26T23:09:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000860935 2024-04-26T23:09:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-04-26T23:09:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-04-26T23:09:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2402, 2024-04-26T23:09:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 2403 2024-04-26T23:09:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 2402, 2024-04-26T23:09:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 2402, 2024-04-26T23:09:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 0, 2024-04-26T23:09:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-04-26T23:09:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-04-26T23:09:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 1479, 2024-04-26T23:09:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 0, 2024-04-26T23:09:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 921, 2024-04-26T23:09:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 0, 2024-04-26T23:09:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 0, 2024-04-26T23:09:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-04-26T23:09:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-04-26T23:09:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-04-26T23:09:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-04-26T23:09:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 0, 2024-04-26T23:09:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-04-26T23:09:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-04-26T23:09:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-04-26T23:09:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 1, 2024-04-26T23:09:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 0, 2024-04-26T23:09:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-04-26T23:09:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-04-26T23:09:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-04-26T23:09:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-04-26T23:09:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-04-26T23:09:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-04-26T23:09:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T23:09:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-04-26T23:09:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-04-26T23:09:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-04-26T23:09:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-04-26T23:09:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-04-26T23:09:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-04-26T23:09:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-04-26T23:09:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-04-26T23:09:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-04-26T23:09:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-04-26T23:09:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-04-26T23:09:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-04-26T23:09:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-04-26T23:09:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-04-26T23:09:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-04-26T23:09:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-04-26T23:09:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-04-26T23:09:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-04-26T23:09:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-04-26T23:09:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 102, 2024-04-26T23:09:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-04-26T23:09:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 12, 2024-04-26T23:09:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 10, 2024-04-26T23:09:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 10, 2024-04-26T23:09:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 0, 2024-04-26T23:09:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-04-26T23:09:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 0, 2024-04-26T23:09:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 2, 2024-04-26T23:09:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 0 2024-04-26T23:09:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-04-26T23:09:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 0, 2024-04-26T23:09:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-04-26T23:09:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-04-26T23:09:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-04-26T23:09:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-04-26T23:09:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-04-26T23:09:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-04-26T23:09:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-04-26T23:09:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T23:09:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-04-26T23:09:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-04-26T23:09:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-04-26T23:09:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-04-26T23:09:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-04-26T23:09:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T23:09:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T23:09:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-26T23:09:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T23:09:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 2673, 2024-04-26T23:09:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 66828969, 2024-04-26T23:09:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 2673, 2024-04-26T23:09:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T23:09:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T23:09:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2673, 2024-04-26T23:09:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 66828969, 2024-04-26T23:09:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T23:09:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-04-26T23:09:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T23:09:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-26T23:09:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T23:09:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T23:09:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T23:09:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T23:09:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 2402, 2024-04-26T23:09:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 64734172, 2024-04-26T23:09:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2402, 2024-04-26T23:09:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 64734172, 2024-04-26T23:09:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T23:09:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-04-26T23:09:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T23:09:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-04-26T23:09:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T23:09:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T23:09:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T23:09:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T23:09:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 2402, 2024-04-26T23:09:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 2402, 2024-04-26T23:09:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2402, 2024-04-26T23:09:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 2402, 2024-04-26T23:09:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T23:09:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-04-26T23:09:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T23:09:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-26T23:09:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T23:09:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T23:09:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T23:09:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T23:09:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T23:09:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T23:09:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-26T23:09:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-26T23:09:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T23:09:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x5621eb1686e0": { 2024-04-26T23:09:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T23:09:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-26T23:09:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T23:09:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T23:09:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T23:09:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T23:09:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T23:09:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T23:09:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-26T23:09:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-26T23:09:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T23:09:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:15.714 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T23:09:16.466 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-26T23:09:16.466 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-26T23:09:16.466 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-26T23:09:16.467 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-26T23:09:16.467 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-26T23:09:16.467 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-26T23:09:16.467 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T23:09:16.467 DEBUG:tasks.check_counter:Getting stats from f 2024-04-26T23:09:16.467 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-26T23:09:16.714 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T23:09:17.035 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:16 smithi094 ceph-mon[32889]: pgmap v330: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 165 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:09:17.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:16 smithi080 ceph-mon[32324]: pgmap v330: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 165 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:09:17.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:16 smithi027 ceph-mon[28034]: pgmap v330: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 165 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:09:18.743 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T23:09:18.743 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T23:09:18.743 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 4, 2024-04-26T23:09:18.743 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 4, 2024-04-26T23:09:18.743 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 2992, 2024-04-26T23:09:18.743 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 448, 2024-04-26T23:09:18.743 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-04-26T23:09:18.743 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-04-26T23:09:18.743 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.001259501, 2024-04-26T23:09:18.743 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000040432, 2024-04-26T23:09:18.744 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.000447019, 2024-04-26T23:09:18.744 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T23:09:18.744 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:18.744 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3, 2024-04-26T23:09:18.744 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000680251, 2024-04-26T23:09:18.744 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000226750 2024-04-26T23:09:18.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:18.744 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:18.744 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:18.744 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:18.744 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:18.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:18.744 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-26T23:09:18.744 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T23:09:18.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:18.745 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T23:09:18.745 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 241, 2024-04-26T23:09:18.745 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 230, 2024-04-26T23:09:18.745 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1595024, 2024-04-26T23:09:18.745 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 141984, 2024-04-26T23:09:18.745 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-04-26T23:09:18.745 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-04-26T23:09:18.745 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.048034785, 2024-04-26T23:09:18.745 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.015159099, 2024-04-26T23:09:18.745 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.036600180, 2024-04-26T23:09:18.745 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.005764746, 2024-04-26T23:09:18.745 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:18.745 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 229, 2024-04-26T23:09:18.746 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.012125768, 2024-04-26T23:09:18.746 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000052950 2024-04-26T23:09:18.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:18.746 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:18.746 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:18.746 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:18.746 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:18.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:18.746 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1595024, 2024-04-26T23:09:18.746 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 141984 2024-04-26T23:09:18.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:18.746 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T23:09:18.746 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 2, 2024-04-26T23:09:18.746 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 176, 2024-04-26T23:09:18.746 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 320, 2024-04-26T23:09:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 188080, 2024-04-26T23:09:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 5, 2024-04-26T23:09:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-04-26T23:09:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.011816767, 2024-04-26T23:09:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.008056284, 2024-04-26T23:09:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.001444409, 2024-04-26T23:09:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T23:09:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 173, 2024-04-26T23:09:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.028542068, 2024-04-26T23:09:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000164983 2024-04-26T23:09:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:18.748 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:18.748 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:18.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:18.748 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T23:09:18.748 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 188080 2024-04-26T23:09:18.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:18.748 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-26T23:09:18.748 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-04-26T23:09:18.748 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-26T23:09:18.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:18.748 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-26T23:09:18.748 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-26T23:09:18.748 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-26T23:09:18.748 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T23:09:18.748 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-26T23:09:18.749 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T23:09:18.749 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-26T23:09:18.749 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T23:09:18.749 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T23:09:18.749 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T23:09:18.749 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T23:09:18.749 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T23:09:18.749 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-26T23:09:18.749 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T23:09:18.749 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T23:09:18.749 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-26T23:09:18.749 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-26T23:09:18.749 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-26T23:09:18.749 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-26T23:09:18.750 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T23:09:18.750 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T23:09:18.750 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T23:09:18.750 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T23:09:18.750 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T23:09:18.750 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-26T23:09:18.750 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-26T23:09:18.750 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-26T23:09:18.750 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T23:09:18.750 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T23:09:18.750 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-26T23:09:18.750 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-26T23:09:18.750 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-26T23:09:18.750 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-26T23:09:18.751 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T23:09:18.751 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-26T23:09:18.751 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T23:09:18.751 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-26T23:09:18.751 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 148732, 2024-04-26T23:09:18.751 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 31, 2024-04-26T23:09:18.751 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-26T23:09:18.751 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-26T23:09:18.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-26T23:09:18.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-26T23:09:18.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-26T23:09:18.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-26T23:09:18.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-26T23:09:18.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-26T23:09:18.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-04-26T23:09:18.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-04-26T23:09:18.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T23:09:18.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-26T23:09:18.752 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-26T23:09:18.752 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-26T23:09:18.752 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-04-26T23:09:18.752 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-04-26T23:09:18.752 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-26T23:09:18.752 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-26T23:09:18.752 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-26T23:09:18.752 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-26T23:09:18.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:18.752 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T23:09:18.752 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T23:09:18.753 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-26T23:09:18.753 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T23:09:18.753 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 243, 2024-04-26T23:09:18.753 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1561043, 2024-04-26T23:09:18.753 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:18.753 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 243, 2024-04-26T23:09:18.753 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T23:09:18.753 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T23:09:18.753 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 243, 2024-04-26T23:09:18.753 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1561043, 2024-04-26T23:09:18.753 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T23:09:18.753 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:18.753 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:18.753 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:18.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:18.754 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:18.754 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T23:09:19.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:18 smithi094 ceph-mon[32889]: pgmap v331: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:09:19.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:18 smithi080 ceph-mon[32324]: pgmap v331: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:09:19.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:18 smithi027 ceph-mon[28034]: pgmap v331: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T23:09:20.815 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-04-26T23:09:20.816 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-04-26T23:09:20.816 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.f 2024-04-26T23:09:20.816 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.f 2024-04-26T23:09:20.816 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-04-26T23:09:20.816 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-04-26T23:09:20.816 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-04-26T23:09:20.816 DEBUG:tasks.check_counter:Getting stats from i 2024-04-26T23:09:20.817 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-26T23:09:21.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:20 smithi094 ceph-mon[32889]: pgmap v332: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 190 B/s wr, 5 op/s 2024-04-26T23:09:21.061 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T23:09:21.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:20 smithi080 ceph-mon[32324]: pgmap v332: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 190 B/s wr, 5 op/s 2024-04-26T23:09:21.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:20 smithi027 ceph-mon[28034]: pgmap v332: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 190 B/s wr, 5 op/s 2024-04-26T23:09:22.840 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:22 smithi094 ceph-mon[32889]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T23:09:22.841 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:22 smithi094 ceph-mon[32889]: pgmap v334: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:09:22.855 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T23:09:22.855 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T23:09:22.855 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1634, 2024-04-26T23:09:22.855 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 1634, 2024-04-26T23:09:22.855 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 25651260, 2024-04-26T23:09:22.855 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 518798, 2024-04-26T23:09:22.855 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 8, 2024-04-26T23:09:22.855 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 4, 2024-04-26T23:09:22.855 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.174221254, 2024-04-26T23:09:22.855 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.049365410, 2024-04-26T23:09:22.855 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.055453350, 2024-04-26T23:09:22.855 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.054519509, 2024-04-26T23:09:22.855 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:22.856 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1633, 2024-04-26T23:09:22.856 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.064788344, 2024-04-26T23:09:22.856 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000039674 2024-04-26T23:09:22.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.856 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:22.856 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.856 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.856 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.856 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-26T23:09:22.856 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T23:09:22.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.856 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T23:09:22.857 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 214, 2024-04-26T23:09:22.857 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 388, 2024-04-26T23:09:22.857 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 25223478, 2024-04-26T23:09:22.857 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 611356, 2024-04-26T23:09:22.857 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 9, 2024-04-26T23:09:22.857 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 6, 2024-04-26T23:09:22.857 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.041492381, 2024-04-26T23:09:22.857 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.011699880, 2024-04-26T23:09:22.857 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.021201004, 2024-04-26T23:09:22.857 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.005408417, 2024-04-26T23:09:22.857 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:22.857 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 385, 2024-04-26T23:09:22.857 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.042074015, 2024-04-26T23:09:22.857 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000109283 2024-04-26T23:09:22.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.858 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:22.858 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.858 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.858 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.858 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-26T23:09:22.858 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 544192 2024-04-26T23:09:22.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.858 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T23:09:22.858 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1074, 2024-04-26T23:09:22.858 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 1051, 2024-04-26T23:09:22.858 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 23623414, 2024-04-26T23:09:22.858 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 399979, 2024-04-26T23:09:22.859 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 5, 2024-04-26T23:09:22.859 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 4, 2024-04-26T23:09:22.859 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.132319674, 2024-04-26T23:09:22.859 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.044143667, 2024-04-26T23:09:22.859 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.062797669, 2024-04-26T23:09:22.859 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.027562439, 2024-04-26T23:09:22.859 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:22.859 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1050, 2024-04-26T23:09:22.859 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.044910339, 2024-04-26T23:09:22.859 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000042771 2024-04-26T23:09:22.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.859 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:22.859 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.859 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.860 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.860 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1671360, 2024-04-26T23:09:22.860 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 144624 2024-04-26T23:09:22.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.860 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-26T23:09:22.860 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-04-26T23:09:22.860 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-26T23:09:22.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.860 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-04-26T23:09:22.860 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-26T23:09:22.860 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-26T23:09:22.861 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 4376, 2024-04-26T23:09:22.861 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.244249714, 2024-04-26T23:09:22.861 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000055815 2024-04-26T23:09:22.861 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:22.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.861 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-04-26T23:09:22.861 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-26T23:09:22.861 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-26T23:09:22.861 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.861 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.861 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.861 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:22.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.861 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-26T23:09:22.862 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 0, 2024-04-26T23:09:22.862 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 0, 2024-04-26T23:09:22.862 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-04-26T23:09:22.862 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.862 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.862 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.862 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-04-26T23:09:22.862 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 0, 2024-04-26T23:09:22.862 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-04-26T23:09:22.862 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-04-26T23:09:22.862 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 0, 2024-04-26T23:09:22.862 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-04-26T23:09:22.863 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-04-26T23:09:22.863 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 12532, 2024-04-26T23:09:22.863 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 7393, 2024-04-26T23:09:22.863 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 5139, 2024-04-26T23:09:22.863 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-04-26T23:09:22.863 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 1971, 2024-04-26T23:09:22.863 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 0, 2024-04-26T23:09:22.863 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 0, 2024-04-26T23:09:22.863 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 0, 2024-04-26T23:09:22.863 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 2, 2024-04-26T23:09:22.863 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 0, 2024-04-26T23:09:22.864 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 0, 2024-04-26T23:09:22.864 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 0, 2024-04-26T23:09:22.864 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 0, 2024-04-26T23:09:22.864 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-04-26T23:09:22.864 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-04-26T23:09:22.864 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-04-26T23:09:22.864 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-04-26T23:09:22.864 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-04-26T23:09:22.864 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-04-26T23:09:22.864 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-04-26T23:09:22.864 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-04-26T23:09:22.865 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-04-26T23:09:22.865 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-04-26T23:09:22.865 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-04-26T23:09:22.865 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-04-26T23:09:22.865 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 4, 2024-04-26T23:09:22.865 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 234, 2024-04-26T23:09:22.865 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 7, 2024-04-26T23:09:22.865 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T23:09:22.865 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-04-26T23:09:22.865 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T23:09:22.865 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-04-26T23:09:22.865 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-04-26T23:09:22.865 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T23:09:22.865 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T23:09:22.865 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-04-26T23:09:22.866 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-04-26T23:09:22.866 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T23:09:22.866 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 0, 2024-04-26T23:09:22.866 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T23:09:22.866 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T23:09:22.866 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T23:09:22.866 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 0, 2024-04-26T23:09:22.866 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 0, 2024-04-26T23:09:22.866 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 0, 2024-04-26T23:09:22.866 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 0 2024-04-26T23:09:22.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.866 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-04-26T23:09:22.866 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-04-26T23:09:22.866 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 0, 2024-04-26T23:09:22.867 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 0, 2024-04-26T23:09:22.867 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 0, 2024-04-26T23:09:22.867 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 0, 2024-04-26T23:09:22.867 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-04-26T23:09:22.867 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-04-26T23:09:22.867 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-04-26T23:09:22.867 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-04-26T23:09:22.867 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-04-26T23:09:22.867 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-04-26T23:09:22.867 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-04-26T23:09:22.867 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-04-26T23:09:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-04-26T23:09:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-04-26T23:09:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-04-26T23:09:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-04-26T23:09:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-04-26T23:09:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-04-26T23:09:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T23:09:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-04-26T23:09:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-04-26T23:09:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-04-26T23:09:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-04-26T23:09:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-04-26T23:09:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-04-26T23:09:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-04-26T23:09:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 0, 2024-04-26T23:09:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-04-26T23:09:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-04-26T23:09:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 24395, 2024-04-26T23:09:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-04-26T23:09:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-04-26T23:09:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 0, 2024-04-26T23:09:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-04-26T23:09:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-04-26T23:09:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 3, 2024-04-26T23:09:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 24, 2024-04-26T23:09:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-04-26T23:09:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-04-26T23:09:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 72931436, 2024-04-26T23:09:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 76273789, 2024-04-26T23:09:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 76273653, 2024-04-26T23:09:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-04-26T23:09:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 24395 2024-04-26T23:09:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-04-26T23:09:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 4456, 2024-04-26T23:09:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 18534, 2024-04-26T23:09:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 14078, 2024-04-26T23:09:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 604, 2024-04-26T23:09:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 1678, 2024-04-26T23:09:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 1074, 2024-04-26T23:09:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 12532, 2024-04-26T23:09:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 24468, 2024-04-26T23:09:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 11936, 2024-04-26T23:09:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 0, 2024-04-26T23:09:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 0, 2024-04-26T23:09:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 0, 2024-04-26T23:09:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 87840, 2024-04-26T23:09:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 215416 2024-04-26T23:09:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-04-26T23:09:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 0, 2024-04-26T23:09:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 0, 2024-04-26T23:09:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 0, 2024-04-26T23:09:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 0, 2024-04-26T23:09:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 0, 2024-04-26T23:09:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-04-26T23:09:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-04-26T23:09:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-04-26T23:09:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-04-26T23:09:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-04-26T23:09:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-04-26T23:09:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-04-26T23:09:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-04-26T23:09:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-04-26T23:09:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-04-26T23:09:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-04-26T23:09:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.877 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:22 smithi027 ceph-mon[28034]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T23:09:22.877 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:22 smithi027 ceph-mon[28034]: pgmap v334: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:09:22.877 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-04-26T23:09:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-04-26T23:09:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-04-26T23:09:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-04-26T23:09:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-04-26T23:09:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-04-26T23:09:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-04-26T23:09:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-04-26T23:09:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-04-26T23:09:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-04-26T23:09:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-04-26T23:09:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-04-26T23:09:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-04-26T23:09:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-04-26T23:09:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-04-26T23:09:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-04-26T23:09:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-04-26T23:09:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-04-26T23:09:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-04-26T23:09:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-04-26T23:09:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-04-26T23:09:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-04-26T23:09:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 2, 2024-04-26T23:09:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 6, 2024-04-26T23:09:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 4, 2024-04-26T23:09:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 4, 2024-04-26T23:09:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-04-26T23:09:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 0, 2024-04-26T23:09:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 0, 2024-04-26T23:09:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 7287, 2024-04-26T23:09:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-04-26T23:09:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-26T23:09:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-26T23:09:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-26T23:09:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T23:09:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-26T23:09:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T23:09:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-26T23:09:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T23:09:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T23:09:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T23:09:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T23:09:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T23:09:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-26T23:09:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T23:09:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T23:09:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-26T23:09:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-26T23:09:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-26T23:09:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-26T23:09:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T23:09:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T23:09:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T23:09:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T23:09:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T23:09:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-26T23:09:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-26T23:09:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-26T23:09:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T23:09:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T23:09:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-26T23:09:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-26T23:09:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-26T23:09:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-26T23:09:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T23:09:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-26T23:09:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T23:09:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-26T23:09:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 421540, 2024-04-26T23:09:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 5215, 2024-04-26T23:09:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-26T23:09:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-26T23:09:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-26T23:09:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-26T23:09:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-26T23:09:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-26T23:09:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-26T23:09:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-26T23:09:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 10232, 2024-04-26T23:09:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 215, 2024-04-26T23:09:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T23:09:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-26T23:09:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-26T23:09:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-26T23:09:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 20877237, 2024-04-26T23:09:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 397530, 2024-04-26T23:09:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-26T23:09:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-26T23:09:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-26T23:09:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-26T23:09:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-04-26T23:09:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-04-26T23:09:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-04-26T23:09:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 2644, 2024-04-26T23:09:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 26, 2024-04-26T23:09:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 4, 2024-04-26T23:09:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 2644, 2024-04-26T23:09:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-04-26T23:09:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2644, 2024-04-26T23:09:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 4.412931337, 2024-04-26T23:09:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.001669036 2024-04-26T23:09:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-04-26T23:09:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-04-26T23:09:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2644, 2024-04-26T23:09:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 2645 2024-04-26T23:09:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 2644, 2024-04-26T23:09:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 2644, 2024-04-26T23:09:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 0, 2024-04-26T23:09:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-04-26T23:09:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-04-26T23:09:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 1651, 2024-04-26T23:09:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 0, 2024-04-26T23:09:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 990, 2024-04-26T23:09:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 0, 2024-04-26T23:09:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 0, 2024-04-26T23:09:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-04-26T23:09:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-04-26T23:09:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-04-26T23:09:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-04-26T23:09:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 0, 2024-04-26T23:09:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-04-26T23:09:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-04-26T23:09:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-04-26T23:09:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 2, 2024-04-26T23:09:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 0, 2024-04-26T23:09:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-04-26T23:09:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-04-26T23:09:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-04-26T23:09:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-04-26T23:09:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-04-26T23:09:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-04-26T23:09:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T23:09:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-04-26T23:09:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-04-26T23:09:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-04-26T23:09:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-04-26T23:09:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-04-26T23:09:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-04-26T23:09:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-04-26T23:09:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-04-26T23:09:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-04-26T23:09:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-04-26T23:09:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-04-26T23:09:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-04-26T23:09:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-04-26T23:09:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-04-26T23:09:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-04-26T23:09:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-04-26T23:09:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-04-26T23:09:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-04-26T23:09:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-04-26T23:09:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 103, 2024-04-26T23:09:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-04-26T23:09:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 13, 2024-04-26T23:09:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 10, 2024-04-26T23:09:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 10, 2024-04-26T23:09:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 0, 2024-04-26T23:09:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-04-26T23:09:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 0, 2024-04-26T23:09:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 2, 2024-04-26T23:09:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 0 2024-04-26T23:09:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-04-26T23:09:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 0, 2024-04-26T23:09:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-04-26T23:09:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-04-26T23:09:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-04-26T23:09:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-04-26T23:09:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-04-26T23:09:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-04-26T23:09:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-04-26T23:09:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T23:09:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-04-26T23:09:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-04-26T23:09:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-04-26T23:09:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-04-26T23:09:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-04-26T23:09:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T23:09:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T23:09:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-26T23:09:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T23:09:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 2918, 2024-04-26T23:09:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 74218693, 2024-04-26T23:09:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 2918, 2024-04-26T23:09:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T23:09:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T23:09:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2918, 2024-04-26T23:09:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 74218693, 2024-04-26T23:09:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T23:09:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-04-26T23:09:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T23:09:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-26T23:09:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T23:09:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T23:09:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T23:09:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T23:09:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 2644, 2024-04-26T23:09:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 72084588, 2024-04-26T23:09:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2644, 2024-04-26T23:09:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 72084588, 2024-04-26T23:09:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T23:09:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-04-26T23:09:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T23:09:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-04-26T23:09:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T23:09:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T23:09:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T23:09:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T23:09:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 2644, 2024-04-26T23:09:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 2644, 2024-04-26T23:09:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2644, 2024-04-26T23:09:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 2644, 2024-04-26T23:09:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T23:09:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-04-26T23:09:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T23:09:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-26T23:09:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T23:09:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T23:09:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T23:09:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T23:09:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T23:09:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T23:09:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-26T23:09:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-26T23:09:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T23:09:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x55dacb9c06e0": { 2024-04-26T23:09:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T23:09:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-26T23:09:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T23:09:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T23:09:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T23:09:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T23:09:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T23:09:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T23:09:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-26T23:09:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-26T23:09:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T23:09:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:22.906 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:22.906 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:22.906 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T23:09:23.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:22 smithi080 ceph-mon[32324]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T23:09:23.022 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:22 smithi080 ceph-mon[32324]: pgmap v334: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:09:23.667 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-26T23:09:23.668 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-26T23:09:23.668 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-26T23:09:23.668 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-04-26T23:09:23.668 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-04-26T23:09:23.668 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-26T23:09:23.668 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T23:09:23.668 DEBUG:tasks.check_counter:Getting stats from l 2024-04-26T23:09:23.668 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-26T23:09:23.902 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T23:09:25.034 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:24 smithi094 ceph-mon[32889]: pgmap v335: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:09:25.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:24 smithi080 ceph-mon[32324]: pgmap v335: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:09:25.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:24 smithi027 ceph-mon[28034]: pgmap v335: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:09:25.637 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T23:09:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T23:09:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 0, 2024-04-26T23:09:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2, 2024-04-26T23:09:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 0, 2024-04-26T23:09:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 0, 2024-04-26T23:09:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-04-26T23:09:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-04-26T23:09:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.000930597, 2024-04-26T23:09:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T23:09:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.000131194, 2024-04-26T23:09:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T23:09:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T23:09:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T23:09:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T23:09:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 2, 2024-04-26T23:09:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 181, 2024-04-26T23:09:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 320, 2024-04-26T23:09:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 267776, 2024-04-26T23:09:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 6, 2024-04-26T23:09:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-04-26T23:09:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.011899171, 2024-04-26T23:09:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.008162995, 2024-04-26T23:09:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.001430050, 2024-04-26T23:09:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T23:09:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 178, 2024-04-26T23:09:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.031536844, 2024-04-26T23:09:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000177173 2024-04-26T23:09:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T23:09:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 267776 2024-04-26T23:09:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T23:09:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 251, 2024-04-26T23:09:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 238, 2024-04-26T23:09:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1599648, 2024-04-26T23:09:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 146272, 2024-04-26T23:09:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-04-26T23:09:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-04-26T23:09:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.048512513, 2024-04-26T23:09:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.014651439, 2024-04-26T23:09:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.037193316, 2024-04-26T23:09:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.005660380, 2024-04-26T23:09:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T23:09:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 238, 2024-04-26T23:09:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.014802114, 2024-04-26T23:09:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000062193 2024-04-26T23:09:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T23:09:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:25.642 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:25.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:25.642 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1599648, 2024-04-26T23:09:25.642 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 146272 2024-04-26T23:09:25.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:25.642 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-26T23:09:25.642 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-04-26T23:09:25.642 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-26T23:09:25.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:25.642 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-26T23:09:25.642 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-26T23:09:25.642 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-26T23:09:25.642 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T23:09:25.642 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-26T23:09:25.643 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T23:09:25.643 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-26T23:09:25.643 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T23:09:25.643 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T23:09:25.643 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T23:09:25.643 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T23:09:25.643 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T23:09:25.643 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-26T23:09:25.643 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T23:09:25.643 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T23:09:25.643 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-26T23:09:25.643 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-26T23:09:25.643 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-26T23:09:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-26T23:09:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T23:09:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T23:09:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T23:09:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T23:09:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T23:09:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-26T23:09:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-26T23:09:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-26T23:09:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T23:09:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T23:09:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-26T23:09:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-26T23:09:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-26T23:09:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-26T23:09:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T23:09:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-26T23:09:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T23:09:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-26T23:09:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 148670, 2024-04-26T23:09:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 30, 2024-04-26T23:09:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-26T23:09:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-26T23:09:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-26T23:09:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-26T23:09:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-26T23:09:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-26T23:09:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-26T23:09:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-26T23:09:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-04-26T23:09:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-04-26T23:09:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T23:09:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-26T23:09:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-26T23:09:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-26T23:09:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-04-26T23:09:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-04-26T23:09:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-26T23:09:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-26T23:09:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-26T23:09:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-26T23:09:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T23:09:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T23:09:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T23:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-26T23:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T23:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 249, 2024-04-26T23:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1561823, 2024-04-26T23:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T23:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 249, 2024-04-26T23:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T23:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T23:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 249, 2024-04-26T23:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1561823, 2024-04-26T23:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T23:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T23:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T23:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T23:09:25.648 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:25.648 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T23:09:25.648 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T23:09:26.344 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-26T23:09:26.344 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-26T23:09:26.345 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-26T23:09:26.345 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-26T23:09:26.345 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-26T23:09:26.345 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-26T23:09:26.345 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-26T23:09:26.345 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T23:09:26.362 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T23:09:26.362 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T23:09:26.362 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:09:26.427 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T23:09:26.427 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:09:26.485 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:09:26.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.713+0000 7fdcc0a00640 1 -- 172.21.15.27:0/3435522628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcbc1096d0 msgr2=0x7fdcbc115be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:26.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.713+0000 7fdcc0a00640 1 --2- 172.21.15.27:0/3435522628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcbc1096d0 0x7fdcbc115be0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009930 tx=0x7fdcb402a420 comp rx=0 tx=0).stop 2024-04-26T23:09:26.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.714+0000 7fdcc0a00640 1 -- 172.21.15.27:0/3435522628 shutdown_connections 2024-04-26T23:09:26.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.714+0000 7fdcc0a00640 1 --2- 172.21.15.27:0/3435522628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcbc1096d0 0x7fdcbc115be0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:26.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.714+0000 7fdcc0a00640 1 --2- 172.21.15.27:0/3435522628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcbc108d30 0x7fdcbc109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:26.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.714+0000 7fdcc0a00640 1 --2- 172.21.15.27:0/3435522628 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcbc10edb0 0x7fdcbc10f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:26.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.714+0000 7fdcc0a00640 1 -- 172.21.15.27:0/3435522628 >> 172.21.15.27:0/3435522628 conn(0x7fdcbc078490 msgr2=0x7fdcbc07a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:26.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.714+0000 7fdcc0a00640 1 -- 172.21.15.27:0/3435522628 shutdown_connections 2024-04-26T23:09:26.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.714+0000 7fdcc0a00640 1 -- 172.21.15.27:0/3435522628 wait complete. 2024-04-26T23:09:26.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.715+0000 7fdcc0a00640 1 Processor -- start 2024-04-26T23:09:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.715+0000 7fdcc0a00640 1 -- start start 2024-04-26T23:09:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.715+0000 7fdcc0a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcbc108d30 0x7fdcbc19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.715+0000 7fdcc0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcbc1096d0 0x7fdcbc19f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.715+0000 7fdcc0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcbc10edb0 0x7fdcbc1a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.715+0000 7fdcc0a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc118fe0 con 0x7fdcbc1096d0 2024-04-26T23:09:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.715+0000 7fdcc0a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc118e60 con 0x7fdcbc108d30 2024-04-26T23:09:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.715+0000 7fdcc0a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc119160 con 0x7fdcbc10edb0 2024-04-26T23:09:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.716+0000 7fdcb9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcbc1096d0 0x7fdcbc19f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.716+0000 7fdcba800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcbc108d30 0x7fdcbc19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.716+0000 7fdcba800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcbc108d30 0x7fdcbc19f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:48544/0 (socket says 172.21.15.27:48544) 2024-04-26T23:09:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.716+0000 7fdcbb200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcbc10edb0 0x7fdcbc1a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.716+0000 7fdcba800640 1 -- 172.21.15.27:0/4185510123 learned_addr learned my addr 172.21.15.27:0/4185510123 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:09:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.716+0000 7fdcba800640 1 -- 172.21.15.27:0/4185510123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcbc10edb0 msgr2=0x7fdcbc1a3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.716+0000 7fdcba800640 1 --2- 172.21.15.27:0/4185510123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcbc10edb0 0x7fdcbc1a3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.716+0000 7fdcba800640 1 -- 172.21.15.27:0/4185510123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcbc1096d0 msgr2=0x7fdcbc19f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.716+0000 7fdcba800640 1 --2- 172.21.15.27:0/4185510123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcbc1096d0 0x7fdcbc19f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.716+0000 7fdcba800640 1 -- 172.21.15.27:0/4185510123 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcbc1a43d0 con 0x7fdcbc108d30 2024-04-26T23:09:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.717+0000 7fdcba800640 1 --2- 172.21.15.27:0/4185510123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcbc108d30 0x7fdcbc19f3e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00e9f0 tx=0x7fdcac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:26.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.717+0000 7fdcb3400640 1 -- 172.21.15.27:0/4185510123 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac00cd80 con 0x7fdcbc108d30 2024-04-26T23:09:26.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.717+0000 7fdcb3400640 1 -- 172.21.15.27:0/4185510123 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcac0108e0 con 0x7fdcbc108d30 2024-04-26T23:09:26.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.717+0000 7fdcc0a00640 1 -- 172.21.15.27:0/4185510123 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcbc1a46c0 con 0x7fdcbc108d30 2024-04-26T23:09:26.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.717+0000 7fdcb3400640 1 -- 172.21.15.27:0/4185510123 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac018980 con 0x7fdcbc108d30 2024-04-26T23:09:26.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.717+0000 7fdcc0a00640 1 -- 172.21.15.27:0/4185510123 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcbc1abf50 con 0x7fdcbc108d30 2024-04-26T23:09:26.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.718+0000 7fdcc0a00640 1 -- 172.21.15.27:0/4185510123 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcbc0506c0 con 0x7fdcbc108d30 2024-04-26T23:09:26.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.721+0000 7fdcb3400640 1 -- 172.21.15.27:0/4185510123 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdcac010430 con 0x7fdcbc108d30 2024-04-26T23:09:26.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.722+0000 7fdcb3400640 1 --2- 172.21.15.27:0/4185510123 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fdca00a3dc0 0x7fdca00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:26.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.722+0000 7fdcb3400640 1 -- 172.21.15.27:0/4185510123 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9700+0+0 (secure 0 0 0) 0x7fdcac0d1410 con 0x7fdcbc108d30 2024-04-26T23:09:26.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.722+0000 7fdcb9e00640 1 --2- 172.21.15.27:0/4185510123 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fdca00a3dc0 0x7fdca00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:26.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.723+0000 7fdcb9e00640 1 --2- 172.21.15.27:0/4185510123 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fdca00a3dc0 0x7fdca00a6280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc10e3d0 tx=0x7fdca4009760 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:26.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.724+0000 7fdcb3400640 1 -- 172.21.15.27:0/4185510123 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdcac09cf60 con 0x7fdcbc108d30 2024-04-26T23:09:26.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.936+0000 7fdcc0a00640 1 -- 172.21.15.27:0/4185510123 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdcbc1ac200 con 0x7fdcbc108d30 2024-04-26T23:09:26.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.939+0000 7fdcb3400640 1 -- 172.21.15.27:0/4185510123 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+25528 (secure 0 0 0) 0x7fdcac0a1260 con 0x7fdcbc108d30 2024-04-26T23:09:26.939 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T23:09:26.940 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":103,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T23:09:21.551256+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4},{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/4202829010":"2024-04-27T22:58:35.543340+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3746496955":"2024-04-27T22:58:35.543340+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/1199904816":"2024-04-27T22:58:35.543340+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3276968774":"2024-04-27T22:58:35.543340+0000","172.21.15.27:0/3449139270":"2024-04-27T22:58:35.543340+0000","172.21.15.27:6801/3636526714":"2024-04-27T22:58:35.543340+0000","172.21.15.27:6800/3636526714":"2024-04-27T22:58:35.543340+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:09:26.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.941+0000 7fdcc0a00640 1 -- 172.21.15.27:0/4185510123 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fdca00a3dc0 msgr2=0x7fdca00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:26.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.941+0000 7fdcc0a00640 1 --2- 172.21.15.27:0/4185510123 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fdca00a3dc0 0x7fdca00a6280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc10e3d0 tx=0x7fdca4009760 comp rx=0 tx=0).stop 2024-04-26T23:09:26.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.941+0000 7fdcc0a00640 1 -- 172.21.15.27:0/4185510123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcbc108d30 msgr2=0x7fdcbc19f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:26.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.941+0000 7fdcc0a00640 1 --2- 172.21.15.27:0/4185510123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcbc108d30 0x7fdcbc19f3e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00e9f0 tx=0x7fdcac00eeb0 comp rx=0 tx=0).stop 2024-04-26T23:09:26.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.942+0000 7fdcc0a00640 1 -- 172.21.15.27:0/4185510123 shutdown_connections 2024-04-26T23:09:26.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.942+0000 7fdcc0a00640 1 --2- 172.21.15.27:0/4185510123 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fdca00a3dc0 0x7fdca00a6280 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:26.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.942+0000 7fdcc0a00640 1 --2- 172.21.15.27:0/4185510123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcbc10edb0 0x7fdcbc1a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:26.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.942+0000 7fdcc0a00640 1 --2- 172.21.15.27:0/4185510123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcbc1096d0 0x7fdcbc19f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:26.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.942+0000 7fdcc0a00640 1 --2- 172.21.15.27:0/4185510123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcbc108d30 0x7fdcbc19f3e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:26.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.942+0000 7fdcc0a00640 1 -- 172.21.15.27:0/4185510123 >> 172.21.15.27:0/4185510123 conn(0x7fdcbc078490 msgr2=0x7fdcbc079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:26.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.942+0000 7fdcc0a00640 1 -- 172.21.15.27:0/4185510123 shutdown_connections 2024-04-26T23:09:26.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:26.942+0000 7fdcc0a00640 1 -- 172.21.15.27:0/4185510123 wait complete. 2024-04-26T23:09:26.961 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:49:54.482679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:55:22.899174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:55:24.218295+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T22:55:28.394547+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:09:26.962 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:09:27.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.183+0000 7f5f1e400640 1 -- 172.21.15.27:0/1674809238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f1810a960 msgr2=0x7f5f1810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:27.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.183+0000 7f5f1e400640 1 --2- 172.21.15.27:0/1674809238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f1810a960 0x7f5f1810ad40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009930 tx=0x7f5f0c02a420 comp rx=0 tx=0).stop 2024-04-26T23:09:27.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.184+0000 7f5f1e400640 1 -- 172.21.15.27:0/1674809238 shutdown_connections 2024-04-26T23:09:27.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.184+0000 7f5f1e400640 1 --2- 172.21.15.27:0/1674809238 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f18105310 0x7f5f18111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.184+0000 7f5f1e400640 1 --2- 172.21.15.27:0/1674809238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f18104950 0x7f5f18104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.184+0000 7f5f1e400640 1 --2- 172.21.15.27:0/1674809238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f1810a960 0x7f5f1810ad40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.184+0000 7f5f1e400640 1 -- 172.21.15.27:0/1674809238 >> 172.21.15.27:0/1674809238 conn(0x7f5f18100420 msgr2=0x7f5f18102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:27.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.184+0000 7f5f1e400640 1 -- 172.21.15.27:0/1674809238 shutdown_connections 2024-04-26T23:09:27.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.184+0000 7f5f1e400640 1 -- 172.21.15.27:0/1674809238 wait complete. 2024-04-26T23:09:27.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.185+0000 7f5f1e400640 1 Processor -- start 2024-04-26T23:09:27.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.185+0000 7f5f1e400640 1 -- start start 2024-04-26T23:09:27.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.185+0000 7f5f1e400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f18104950 0x7f5f1819b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:27.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.185+0000 7f5f1e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f18105310 0x7f5f1819b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:27.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.185+0000 7f5f1e400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f1810a960 0x7f5f1819fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:27.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.185+0000 7f5f1e400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18114c40 con 0x7f5f18104950 2024-04-26T23:09:27.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.185+0000 7f5f1e400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18114ac0 con 0x7f5f1810a960 2024-04-26T23:09:27.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.185+0000 7f5f1e400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18114dc0 con 0x7f5f18105310 2024-04-26T23:09:27.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.186+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f18105310 0x7f5f1819b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:27.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.186+0000 7f5f1d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f18104950 0x7f5f1819b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:27.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.186+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f18105310 0x7f5f1819b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:40788/0 (socket says 172.21.15.27:40788) 2024-04-26T23:09:27.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.186+0000 7f5f17e00640 1 -- 172.21.15.27:0/3494797968 learned_addr learned my addr 172.21.15.27:0/3494797968 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:09:27.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.186+0000 7f5f1da00640 1 --2- 172.21.15.27:0/3494797968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f1810a960 0x7f5f1819fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:27.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.186+0000 7f5f17e00640 1 -- 172.21.15.27:0/3494797968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f1810a960 msgr2=0x7f5f1819fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:27.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.186+0000 7f5f17e00640 1 --2- 172.21.15.27:0/3494797968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f1810a960 0x7f5f1819fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.186+0000 7f5f17e00640 1 -- 172.21.15.27:0/3494797968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f18104950 msgr2=0x7f5f1819b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:27.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.186+0000 7f5f17e00640 1 --2- 172.21.15.27:0/3494797968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f18104950 0x7f5f1819b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.186+0000 7f5f17e00640 1 -- 172.21.15.27:0/3494797968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f181a0190 con 0x7f5f18105310 2024-04-26T23:09:27.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.186+0000 7f5f17e00640 1 --2- 172.21.15.27:0/3494797968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f18105310 0x7f5f1819b6e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400e9f0 tx=0x7f5f0400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:27.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.187+0000 7f5f15600640 1 -- 172.21.15.27:0/3494797968 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0400cd80 con 0x7f5f18105310 2024-04-26T23:09:27.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.187+0000 7f5f1e400640 1 -- 172.21.15.27:0/3494797968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f181a0480 con 0x7f5f18105310 2024-04-26T23:09:27.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.187+0000 7f5f15600640 1 -- 172.21.15.27:0/3494797968 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f040108e0 con 0x7f5f18105310 2024-04-26T23:09:27.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.187+0000 7f5f15600640 1 -- 172.21.15.27:0/3494797968 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04018980 con 0x7f5f18105310 2024-04-26T23:09:27.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.187+0000 7f5f1e400640 1 -- 172.21.15.27:0/3494797968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f181a7d10 con 0x7f5f18105310 2024-04-26T23:09:27.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.188+0000 7f5f1e400640 1 -- 172.21.15.27:0/3494797968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f180506c0 con 0x7f5f18105310 2024-04-26T23:09:27.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.191+0000 7f5f15600640 1 -- 172.21.15.27:0/3494797968 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5f04010430 con 0x7f5f18105310 2024-04-26T23:09:27.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.192+0000 7f5f15600640 1 --2- 172.21.15.27:0/3494797968 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f5f000a3da0 0x7f5f000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:27.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.193+0000 7f5f15600640 1 -- 172.21.15.27:0/3494797968 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9700+0+0 (secure 0 0 0) 0x7f5f040d1410 con 0x7f5f18105310 2024-04-26T23:09:27.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.193+0000 7f5f1d000640 1 --2- 172.21.15.27:0/3494797968 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f5f000a3da0 0x7f5f000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:27.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.193+0000 7f5f1d000640 1 --2- 172.21.15.27:0/3494797968 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f5f000a3da0 0x7f5f000a6260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c031440 tx=0x7f5f0c005a20 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:27.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.196+0000 7f5f15600640 1 -- 172.21.15.27:0/3494797968 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5f0409cf60 con 0x7f5f18105310 2024-04-26T23:09:27.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:26 smithi080 ceph-mon[32324]: pgmap v336: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:09:27.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:26 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/4185510123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:09:27.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:26 smithi094 ceph-mon[32889]: pgmap v336: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:09:27.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:26 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/4185510123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:09:27.300 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:26 smithi027 ceph-mon[28034]: pgmap v336: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T23:09:27.300 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:26 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/4185510123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:09:27.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.411+0000 7f5f1e400640 1 -- 172.21.15.27:0/3494797968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5f1819c4f0 con 0x7f5f18105310 2024-04-26T23:09:27.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.412+0000 7f5f15600640 1 -- 172.21.15.27:0/3494797968 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7f5f040a1260 con 0x7f5f18105310 2024-04-26T23:09:27.413 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T23:09:27.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.415+0000 7f5f1e400640 1 -- 172.21.15.27:0/3494797968 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f5f000a3da0 msgr2=0x7f5f000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:27.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.415+0000 7f5f1e400640 1 --2- 172.21.15.27:0/3494797968 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f5f000a3da0 0x7f5f000a6260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c031440 tx=0x7f5f0c005a20 comp rx=0 tx=0).stop 2024-04-26T23:09:27.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.415+0000 7f5f1e400640 1 -- 172.21.15.27:0/3494797968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f18105310 msgr2=0x7f5f1819b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:27.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.415+0000 7f5f1e400640 1 --2- 172.21.15.27:0/3494797968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f18105310 0x7f5f1819b6e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400e9f0 tx=0x7f5f0400eeb0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.415+0000 7f5f1e400640 1 -- 172.21.15.27:0/3494797968 shutdown_connections 2024-04-26T23:09:27.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.415+0000 7f5f1e400640 1 --2- 172.21.15.27:0/3494797968 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f5f000a3da0 0x7f5f000a6260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.415+0000 7f5f1e400640 1 --2- 172.21.15.27:0/3494797968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f1810a960 0x7f5f1819fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.415+0000 7f5f1e400640 1 --2- 172.21.15.27:0/3494797968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f18105310 0x7f5f1819b6e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.415+0000 7f5f1e400640 1 --2- 172.21.15.27:0/3494797968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f18104950 0x7f5f1819b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.415+0000 7f5f1e400640 1 -- 172.21.15.27:0/3494797968 >> 172.21.15.27:0/3494797968 conn(0x7f5f18100420 msgr2=0x7f5f18100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:27.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.416+0000 7f5f1e400640 1 -- 172.21.15.27:0/3494797968 shutdown_connections 2024-04-26T23:09:27.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.416+0000 7f5f1e400640 1 -- 172.21.15.27:0/3494797968 wait complete. 2024-04-26T23:09:27.435 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:09:27.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.660+0000 7f09cd600640 1 -- 172.21.15.27:0/3935307952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c8102720 msgr2=0x7f09c8102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:27.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.660+0000 7f09cd600640 1 --2- 172.21.15.27:0/3935307952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c8102720 0x7f09c8102ba0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009930 tx=0x7f09b402a420 comp rx=0 tx=0).stop 2024-04-26T23:09:27.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.660+0000 7f09cd600640 1 -- 172.21.15.27:0/3935307952 shutdown_connections 2024-04-26T23:09:27.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.660+0000 7f09cd600640 1 --2- 172.21.15.27:0/3935307952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c81030e0 0x7f09c810f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.660+0000 7f09cd600640 1 --2- 172.21.15.27:0/3935307952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c8102720 0x7f09c8102ba0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.660+0000 7f09cd600640 1 --2- 172.21.15.27:0/3935307952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c8108730 0x7f09c8108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.660+0000 7f09cd600640 1 -- 172.21.15.27:0/3935307952 >> 172.21.15.27:0/3935307952 conn(0x7f09c80fe250 msgr2=0x7f09c8100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:27.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.660+0000 7f09cd600640 1 -- 172.21.15.27:0/3935307952 shutdown_connections 2024-04-26T23:09:27.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.660+0000 7f09cd600640 1 -- 172.21.15.27:0/3935307952 wait complete. 2024-04-26T23:09:27.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.661+0000 7f09cd600640 1 Processor -- start 2024-04-26T23:09:27.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.662+0000 7f09cd600640 1 -- start start 2024-04-26T23:09:27.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.662+0000 7f09cd600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c8102720 0x7f09c8076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:27.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.662+0000 7f09cd600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c81030e0 0x7f09c8074c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:27.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.662+0000 7f09cd600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c8108730 0x7f09c8075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:27.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.662+0000 7f09cd600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f09c8112a40 con 0x7f09c8108730 2024-04-26T23:09:27.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.662+0000 7f09cd600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f09c81128c0 con 0x7f09c8102720 2024-04-26T23:09:27.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.662+0000 7f09cd600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f09c8112bc0 con 0x7f09c81030e0 2024-04-26T23:09:27.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.662+0000 7f09c7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c81030e0 0x7f09c8074c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:27.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.662+0000 7f09c7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c81030e0 0x7f09c8074c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:40816/0 (socket says 172.21.15.27:40816) 2024-04-26T23:09:27.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.662+0000 7f09c7400640 1 -- 172.21.15.27:0/1542750322 learned_addr learned my addr 172.21.15.27:0/1542750322 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:09:27.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.662+0000 7f09ccc00640 1 --2- 172.21.15.27:0/1542750322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c8108730 0x7f09c8075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:27.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.662+0000 7f09c7e00640 1 --2- 172.21.15.27:0/1542750322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c8102720 0x7f09c8076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:27.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.663+0000 7f09c7400640 1 -- 172.21.15.27:0/1542750322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c8102720 msgr2=0x7f09c8076830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:27.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.663+0000 7f09c7400640 1 --2- 172.21.15.27:0/1542750322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c8102720 0x7f09c8076830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.663+0000 7f09c7400640 1 -- 172.21.15.27:0/1542750322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c8108730 msgr2=0x7f09c8075270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:27.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.663+0000 7f09c7400640 1 --2- 172.21.15.27:0/1542750322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c8108730 0x7f09c8075270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.663+0000 7f09c7400640 1 -- 172.21.15.27:0/1542750322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c81ab980 con 0x7f09c81030e0 2024-04-26T23:09:27.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.663+0000 7f09c7e00640 1 --2- 172.21.15.27:0/1542750322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c8102720 0x7f09c8076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:09:27.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.663+0000 7f09c7400640 1 --2- 172.21.15.27:0/1542750322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c81030e0 0x7f09c8074c30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005ab0 tx=0x7f09b4003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:27.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.663+0000 7f09c4c00640 1 -- 172.21.15.27:0/1542750322 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b403f070 con 0x7f09c81030e0 2024-04-26T23:09:27.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.663+0000 7f09c4c00640 1 -- 172.21.15.27:0/1542750322 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09b4031b50 con 0x7f09c81030e0 2024-04-26T23:09:27.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.663+0000 7f09c4c00640 1 -- 172.21.15.27:0/1542750322 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4035030 con 0x7f09c81030e0 2024-04-26T23:09:27.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.663+0000 7f09cd600640 1 -- 172.21.15.27:0/1542750322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c81abb50 con 0x7f09c81030e0 2024-04-26T23:09:27.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.664+0000 7f09cd600640 1 -- 172.21.15.27:0/1542750322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c81ac040 con 0x7f09c81030e0 2024-04-26T23:09:27.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.665+0000 7f09cd600640 1 -- 172.21.15.27:0/1542750322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c8103e60 con 0x7f09c81030e0 2024-04-26T23:09:27.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.667+0000 7f09c4c00640 1 -- 172.21.15.27:0/1542750322 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f09b4042070 con 0x7f09c81030e0 2024-04-26T23:09:27.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.668+0000 7f09c4c00640 1 --2- 172.21.15.27:0/1542750322 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f09ac0a3da0 0x7f09ac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:27.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.668+0000 7f09c4c00640 1 -- 172.21.15.27:0/1542750322 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9700+0+0 (secure 0 0 0) 0x7f09b40f1280 con 0x7f09c81030e0 2024-04-26T23:09:27.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.668+0000 7f09c7e00640 1 --2- 172.21.15.27:0/1542750322 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f09ac0a3da0 0x7f09ac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:27.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.669+0000 7f09c7e00640 1 --2- 172.21.15.27:0/1542750322 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f09ac0a3da0 0x7f09ac0a6260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009ba0 tx=0x7f09b8005f90 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:27.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.672+0000 7f09c4c00640 1 -- 172.21.15.27:0/1542750322 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f09b40bd420 con 0x7f09c81030e0 2024-04-26T23:09:27.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.880+0000 7f09cd600640 1 -- 172.21.15.27:0/1542750322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f09c802dad0 con 0x7f09c81030e0 2024-04-26T23:09:27.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.881+0000 7f09c4c00640 1 -- 172.21.15.27:0/1542750322 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7f09b40bcd20 con 0x7f09c81030e0 2024-04-26T23:09:27.881 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T23:09:27.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.883+0000 7f09cd600640 1 -- 172.21.15.27:0/1542750322 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f09ac0a3da0 msgr2=0x7f09ac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:27.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.883+0000 7f09cd600640 1 --2- 172.21.15.27:0/1542750322 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f09ac0a3da0 0x7f09ac0a6260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009ba0 tx=0x7f09b8005f90 comp rx=0 tx=0).stop 2024-04-26T23:09:27.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.883+0000 7f09cd600640 1 -- 172.21.15.27:0/1542750322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c81030e0 msgr2=0x7f09c8074c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.883+0000 7f09cd600640 1 --2- 172.21.15.27:0/1542750322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c81030e0 0x7f09c8074c30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005ab0 tx=0x7f09b4003b90 comp rx=0 tx=0).stop 2024-04-26T23:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.884+0000 7f09cd600640 1 -- 172.21.15.27:0/1542750322 shutdown_connections 2024-04-26T23:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.884+0000 7f09cd600640 1 --2- 172.21.15.27:0/1542750322 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f09ac0a3da0 0x7f09ac0a6260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.884+0000 7f09cd600640 1 --2- 172.21.15.27:0/1542750322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c8108730 0x7f09c8075270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.884+0000 7f09cd600640 1 --2- 172.21.15.27:0/1542750322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c81030e0 0x7f09c8074c30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.884+0000 7f09cd600640 1 --2- 172.21.15.27:0/1542750322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c8102720 0x7f09c8076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.884+0000 7f09cd600640 1 -- 172.21.15.27:0/1542750322 >> 172.21.15.27:0/1542750322 conn(0x7f09c80fe250 msgr2=0x7f09c80fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.884+0000 7f09cd600640 1 -- 172.21.15.27:0/1542750322 shutdown_connections 2024-04-26T23:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:27.884+0000 7f09cd600640 1 -- 172.21.15.27:0/1542750322 wait complete. 2024-04-26T23:09:27.904 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:09:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.113+0000 7f14e2800640 1 -- 172.21.15.27:0/541561570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14dc106960 msgr2=0x7f14dc10e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.113+0000 7f14e2800640 1 --2- 172.21.15.27:0/541561570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14dc106960 0x7f14dc10e530 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009ca0 tx=0x7f14d402a8c0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.114+0000 7f14e2800640 1 -- 172.21.15.27:0/541561570 shutdown_connections 2024-04-26T23:09:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.114+0000 7f14e2800640 1 --2- 172.21.15.27:0/541561570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14dc102d50 0x7f14dc106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.114+0000 7f14e2800640 1 --2- 172.21.15.27:0/541561570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14dc106960 0x7f14dc10e530 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.114+0000 7f14e2800640 1 --2- 172.21.15.27:0/541561570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14dc1023a0 0x7f14dc102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.114+0000 7f14e2800640 1 -- 172.21.15.27:0/541561570 >> 172.21.15.27:0/541561570 conn(0x7f14dc076930 msgr2=0x7f14dc076d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.114+0000 7f14e2800640 1 -- 172.21.15.27:0/541561570 shutdown_connections 2024-04-26T23:09:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.114+0000 7f14e2800640 1 -- 172.21.15.27:0/541561570 wait complete. 2024-04-26T23:09:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.115+0000 7f14e2800640 1 Processor -- start 2024-04-26T23:09:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.115+0000 7f14e2800640 1 -- start start 2024-04-26T23:09:28.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.116+0000 7f14e2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14dc1023a0 0x7f14dc198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:28.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.116+0000 7f14e2800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14dc106960 0x7f14dc199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:28.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.116+0000 7f14e2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14dc19d780 0x7f14dc19dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:28.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.116+0000 7f14e2800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc111910 con 0x7f14dc19d780 2024-04-26T23:09:28.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.116+0000 7f14e2800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc111790 con 0x7f14dc106960 2024-04-26T23:09:28.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.116+0000 7f14e2800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc111a90 con 0x7f14dc1023a0 2024-04-26T23:09:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.116+0000 7f14e0a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14dc106960 0x7f14dc199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.116+0000 7f14e1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14dc1023a0 0x7f14dc198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.116+0000 7f14e1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14dc19d780 0x7f14dc19dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.116+0000 7f14e0a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14dc106960 0x7f14dc199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:47680/0 (socket says 172.21.15.27:47680) 2024-04-26T23:09:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.116+0000 7f14e1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14dc1023a0 0x7f14dc198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:47204/0 (socket says 172.21.15.27:47204) 2024-04-26T23:09:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.116+0000 7f14e0a00640 1 -- 172.21.15.27:0/117394574 learned_addr learned my addr 172.21.15.27:0/117394574 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:09:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.117+0000 7f14e1400640 1 -- 172.21.15.27:0/117394574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14dc106960 msgr2=0x7f14dc199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.117+0000 7f14e1400640 1 --2- 172.21.15.27:0/117394574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14dc106960 0x7f14dc199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.117+0000 7f14e1400640 1 -- 172.21.15.27:0/117394574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14dc19d780 msgr2=0x7f14dc19dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.117+0000 7f14e1400640 1 --2- 172.21.15.27:0/117394574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14dc19d780 0x7f14dc19dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.117+0000 7f14e1400640 1 -- 172.21.15.27:0/117394574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc19e270 con 0x7f14dc1023a0 2024-04-26T23:09:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.117+0000 7f14e1400640 1 --2- 172.21.15.27:0/117394574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14dc1023a0 0x7f14dc198ec0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14d000e9f0 tx=0x7f14d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.117+0000 7f14da000640 1 -- 172.21.15.27:0/117394574 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d000ccc0 con 0x7f14dc1023a0 2024-04-26T23:09:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.117+0000 7f14da000640 1 -- 172.21.15.27:0/117394574 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d00108e0 con 0x7f14dc1023a0 2024-04-26T23:09:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.117+0000 7f14da000640 1 -- 172.21.15.27:0/117394574 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0018880 con 0x7f14dc1023a0 2024-04-26T23:09:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.117+0000 7f14e2800640 1 -- 172.21.15.27:0/117394574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14dc19e560 con 0x7f14dc1023a0 2024-04-26T23:09:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.118+0000 7f14e2800640 1 -- 172.21.15.27:0/117394574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14dc1a5d00 con 0x7f14dc1023a0 2024-04-26T23:09:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.119+0000 7f14e2800640 1 -- 172.21.15.27:0/117394574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14dc0506c0 con 0x7f14dc1023a0 2024-04-26T23:09:28.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.121+0000 7f14da000640 1 -- 172.21.15.27:0/117394574 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f14d0010430 con 0x7f14dc1023a0 2024-04-26T23:09:28.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.123+0000 7f14da000640 1 --2- 172.21.15.27:0/117394574 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f14c40a3e60 0x7f14c40a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:28.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.123+0000 7f14da000640 1 -- 172.21.15.27:0/117394574 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9700+0+0 (secure 0 0 0) 0x7f14d00d0f10 con 0x7f14dc1023a0 2024-04-26T23:09:28.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.123+0000 7f14e0a00640 1 --2- 172.21.15.27:0/117394574 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f14c40a3e60 0x7f14c40a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:28.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.124+0000 7f14e0a00640 1 --2- 172.21.15.27:0/117394574 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f14c40a3e60 0x7f14c40a6320 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f14dc107ee0 tx=0x7f14c8005f90 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.127+0000 7f14da000640 1 -- 172.21.15.27:0/117394574 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f14d009ca60 con 0x7f14dc1023a0 2024-04-26T23:09:28.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:27 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3494797968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:09:28.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:27 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1542750322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:09:28.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:27 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3494797968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:09:28.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:27 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1542750322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:09:28.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.334+0000 7f14e2800640 1 -- 172.21.15.27:0/117394574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f14dc102d50 con 0x7f14dc1023a0 2024-04-26T23:09:28.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.335+0000 7f14da000640 1 -- 172.21.15.27:0/117394574 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7f14d00a0d60 con 0x7f14dc1023a0 2024-04-26T23:09:28.336 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-26T23:09:28.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.338+0000 7f14e2800640 1 -- 172.21.15.27:0/117394574 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f14c40a3e60 msgr2=0x7f14c40a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:28.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.338+0000 7f14e2800640 1 --2- 172.21.15.27:0/117394574 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f14c40a3e60 0x7f14c40a6320 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f14dc107ee0 tx=0x7f14c8005f90 comp rx=0 tx=0).stop 2024-04-26T23:09:28.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.338+0000 7f14e2800640 1 -- 172.21.15.27:0/117394574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14dc1023a0 msgr2=0x7f14dc198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:28.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.338+0000 7f14e2800640 1 --2- 172.21.15.27:0/117394574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14dc1023a0 0x7f14dc198ec0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14d000e9f0 tx=0x7f14d000eeb0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.338+0000 7f14e2800640 1 -- 172.21.15.27:0/117394574 shutdown_connections 2024-04-26T23:09:28.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.338+0000 7f14e2800640 1 --2- 172.21.15.27:0/117394574 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f14c40a3e60 0x7f14c40a6320 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.338+0000 7f14e2800640 1 --2- 172.21.15.27:0/117394574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14dc106960 0x7f14dc199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.338+0000 7f14e2800640 1 --2- 172.21.15.27:0/117394574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14dc19d780 0x7f14dc19dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.338+0000 7f14e2800640 1 --2- 172.21.15.27:0/117394574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14dc1023a0 0x7f14dc198ec0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.338+0000 7f14e2800640 1 -- 172.21.15.27:0/117394574 >> 172.21.15.27:0/117394574 conn(0x7f14dc076930 msgr2=0x7f14dc1056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:28.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.338+0000 7f14e2800640 1 -- 172.21.15.27:0/117394574 shutdown_connections 2024-04-26T23:09:28.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.338+0000 7f14e2800640 1 -- 172.21.15.27:0/117394574 wait complete. 2024-04-26T23:09:28.355 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:27 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3494797968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:09:28.355 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:27 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1542750322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:09:28.359 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:09:28.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.579+0000 7fd0a9400640 1 -- 172.21.15.27:0/261736486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a41015e0 msgr2=0x7fd0a410f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:28.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.579+0000 7fd0a9400640 1 --2- 172.21.15.27:0/261736486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a41015e0 0x7fd0a410f5a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009ca0 tx=0x7fd09c02a8c0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.579+0000 7fd0a9400640 1 -- 172.21.15.27:0/261736486 shutdown_connections 2024-04-26T23:09:28.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.579+0000 7fd0a9400640 1 --2- 172.21.15.27:0/261736486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a4100c20 0x7fd0a41010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.579+0000 7fd0a9400640 1 --2- 172.21.15.27:0/261736486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a41015e0 0x7fd0a410f5a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.579+0000 7fd0a9400640 1 --2- 172.21.15.27:0/261736486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a4108720 0x7fd0a4108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.579+0000 7fd0a9400640 1 -- 172.21.15.27:0/261736486 >> 172.21.15.27:0/261736486 conn(0x7fd0a4077b70 msgr2=0x7fd0a40fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:28.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.579+0000 7fd0a9400640 1 -- 172.21.15.27:0/261736486 shutdown_connections 2024-04-26T23:09:28.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.579+0000 7fd0a9400640 1 -- 172.21.15.27:0/261736486 wait complete. 2024-04-26T23:09:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.580+0000 7fd0a9400640 1 Processor -- start 2024-04-26T23:09:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.580+0000 7fd0a9400640 1 -- start start 2024-04-26T23:09:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a4100c20 0x7fd0a4198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a41015e0 0x7fd0a4199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a9400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a4108720 0x7fd0a419d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a9400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41129c0 con 0x7fd0a4100c20 2024-04-26T23:09:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a9400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4112840 con 0x7fd0a4108720 2024-04-26T23:09:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a9400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4112b40 con 0x7fd0a41015e0 2024-04-26T23:09:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a4100c20 0x7fd0a4198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a4100c20 0x7fd0a4198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51324/0 (socket says 172.21.15.27:51324) 2024-04-26T23:09:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a3e00640 1 -- 172.21.15.27:0/693669781 learned_addr learned my addr 172.21.15.27:0/693669781 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:09:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a3400640 1 --2- 172.21.15.27:0/693669781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a41015e0 0x7fd0a4199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a8a00640 1 --2- 172.21.15.27:0/693669781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a4108720 0x7fd0a419d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a3e00640 1 -- 172.21.15.27:0/693669781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a41015e0 msgr2=0x7fd0a4199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a3e00640 1 --2- 172.21.15.27:0/693669781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a41015e0 0x7fd0a4199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a3e00640 1 -- 172.21.15.27:0/693669781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a4108720 msgr2=0x7fd0a419d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a3e00640 1 --2- 172.21.15.27:0/693669781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a4108720 0x7fd0a419d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a3e00640 1 -- 172.21.15.27:0/693669781 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a419df30 con 0x7fd0a4100c20 2024-04-26T23:09:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.581+0000 7fd0a3400640 1 --2- 172.21.15.27:0/693669781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a41015e0 0x7fd0a4199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:09:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.582+0000 7fd0a3e00640 1 --2- 172.21.15.27:0/693669781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a4100c20 0x7fd0a4198f40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd09400e9f0 tx=0x7fd09400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.582+0000 7fd0a0c00640 1 -- 172.21.15.27:0/693669781 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09400cd80 con 0x7fd0a4100c20 2024-04-26T23:09:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.582+0000 7fd0a9400640 1 -- 172.21.15.27:0/693669781 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a419e220 con 0x7fd0a4100c20 2024-04-26T23:09:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.582+0000 7fd0a0c00640 1 -- 172.21.15.27:0/693669781 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0940054d0 con 0x7fd0a4100c20 2024-04-26T23:09:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.582+0000 7fd0a0c00640 1 -- 172.21.15.27:0/693669781 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094010690 con 0x7fd0a4100c20 2024-04-26T23:09:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.582+0000 7fd0a9400640 1 -- 172.21.15.27:0/693669781 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a41a5ab0 con 0x7fd0a4100c20 2024-04-26T23:09:28.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.583+0000 7fd0a9400640 1 -- 172.21.15.27:0/693669781 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a40506c0 con 0x7fd0a4100c20 2024-04-26T23:09:28.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.585+0000 7fd0a0c00640 1 -- 172.21.15.27:0/693669781 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd094004e90 con 0x7fd0a4100c20 2024-04-26T23:09:28.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.587+0000 7fd0a0c00640 1 --2- 172.21.15.27:0/693669781 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd0880a3da0 0x7fd0880a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:28.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.587+0000 7fd0a0c00640 1 -- 172.21.15.27:0/693669781 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9700+0+0 (secure 0 0 0) 0x7fd0940d0fe0 con 0x7fd0a4100c20 2024-04-26T23:09:28.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.587+0000 7fd0a3400640 1 --2- 172.21.15.27:0/693669781 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd0880a3da0 0x7fd0880a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:28.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.588+0000 7fd0a3400640 1 --2- 172.21.15.27:0/693669781 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd0880a3da0 0x7fd0880a6260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd0900095f0 tx=0x7fd090005f90 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.591+0000 7fd0a0c00640 1 -- 172.21.15.27:0/693669781 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd09409d1c0 con 0x7fd0a4100c20 2024-04-26T23:09:28.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.799+0000 7fd0a9400640 1 -- 172.21.15.27:0/693669781 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd0a402dad0 con 0x7fd0a4100c20 2024-04-26T23:09:28.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.800+0000 7fd0a0c00640 1 -- 172.21.15.27:0/693669781 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v103) ==== 103+0+11 (secure 0 0 0) 0x7fd09409cac0 con 0x7fd0a4100c20 2024-04-26T23:09:28.801 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T23:09:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.803+0000 7fd0a9400640 1 -- 172.21.15.27:0/693669781 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd0880a3da0 msgr2=0x7fd0880a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.803+0000 7fd0a9400640 1 --2- 172.21.15.27:0/693669781 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd0880a3da0 0x7fd0880a6260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd0900095f0 tx=0x7fd090005f90 comp rx=0 tx=0).stop 2024-04-26T23:09:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.803+0000 7fd0a9400640 1 -- 172.21.15.27:0/693669781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a4100c20 msgr2=0x7fd0a4198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.803+0000 7fd0a9400640 1 --2- 172.21.15.27:0/693669781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a4100c20 0x7fd0a4198f40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd09400e9f0 tx=0x7fd09400edf0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.803+0000 7fd0a9400640 1 -- 172.21.15.27:0/693669781 shutdown_connections 2024-04-26T23:09:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.803+0000 7fd0a9400640 1 --2- 172.21.15.27:0/693669781 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd0880a3da0 0x7fd0880a6260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.803+0000 7fd0a9400640 1 --2- 172.21.15.27:0/693669781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a4108720 0x7fd0a419d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.803+0000 7fd0a9400640 1 --2- 172.21.15.27:0/693669781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0a41015e0 0x7fd0a4199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.803+0000 7fd0a9400640 1 --2- 172.21.15.27:0/693669781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a4100c20 0x7fd0a4198f40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.803+0000 7fd0a9400640 1 -- 172.21.15.27:0/693669781 >> 172.21.15.27:0/693669781 conn(0x7fd0a4077b70 msgr2=0x7fd0a40ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.804+0000 7fd0a9400640 1 -- 172.21.15.27:0/693669781 shutdown_connections 2024-04-26T23:09:28.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:28.804+0000 7fd0a9400640 1 -- 172.21.15.27:0/693669781 wait complete. 2024-04-26T23:09:28.823 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:09:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.038+0000 7fc30a000640 1 -- 172.21.15.27:0/952777004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3041026f0 msgr2=0x7fc304102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.038+0000 7fc30a000640 1 --2- 172.21.15.27:0/952777004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3041026f0 0x7fc304102b70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009900 tx=0x7fc2ec02a410 comp rx=0 tx=0).stop 2024-04-26T23:09:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.038+0000 7fc30a000640 1 -- 172.21.15.27:0/952777004 shutdown_connections 2024-04-26T23:09:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.038+0000 7fc30a000640 1 --2- 172.21.15.27:0/952777004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3041026f0 0x7fc304102b70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.038+0000 7fc30a000640 1 --2- 172.21.15.27:0/952777004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3041030b0 0x7fc30410f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.038+0000 7fc30a000640 1 --2- 172.21.15.27:0/952777004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc304108700 0x7fc304108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.038+0000 7fc30a000640 1 -- 172.21.15.27:0/952777004 >> 172.21.15.27:0/952777004 conn(0x7fc3040fe200 msgr2=0x7fc304100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.038+0000 7fc30a000640 1 -- 172.21.15.27:0/952777004 shutdown_connections 2024-04-26T23:09:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.039+0000 7fc30a000640 1 -- 172.21.15.27:0/952777004 wait complete. 2024-04-26T23:09:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.039+0000 7fc30a000640 1 Processor -- start 2024-04-26T23:09:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.040+0000 7fc30a000640 1 -- start start 2024-04-26T23:09:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.040+0000 7fc30a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3041026f0 0x7fc30419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.040+0000 7fc30a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3041030b0 0x7fc30419b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.040+0000 7fc30a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc304108700 0x7fc30419f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.040+0000 7fc30a000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc304112a10 con 0x7fc3041030b0 2024-04-26T23:09:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.040+0000 7fc30a000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc304112890 con 0x7fc304108700 2024-04-26T23:09:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.040+0000 7fc30a000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc304112b90 con 0x7fc3041026f0 2024-04-26T23:09:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.040+0000 7fc303e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3041030b0 0x7fc30419b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.040+0000 7fc308c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3041026f0 0x7fc30419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.040+0000 7fc303e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3041030b0 0x7fc30419b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51340/0 (socket says 172.21.15.27:51340) 2024-04-26T23:09:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.040+0000 7fc308c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3041026f0 0x7fc30419b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:47240/0 (socket says 172.21.15.27:47240) 2024-04-26T23:09:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.040+0000 7fc303e00640 1 -- 172.21.15.27:0/1141141128 learned_addr learned my addr 172.21.15.27:0/1141141128 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:09:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.040+0000 7fc309600640 1 --2- 172.21.15.27:0/1141141128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc304108700 0x7fc30419f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.041+0000 7fc308c00640 1 -- 172.21.15.27:0/1141141128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc304108700 msgr2=0x7fc30419f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.041+0000 7fc308c00640 1 --2- 172.21.15.27:0/1141141128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc304108700 0x7fc30419f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.041+0000 7fc308c00640 1 -- 172.21.15.27:0/1141141128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3041030b0 msgr2=0x7fc30419b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.041+0000 7fc308c00640 1 --2- 172.21.15.27:0/1141141128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3041030b0 0x7fc30419b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.041+0000 7fc308c00640 1 -- 172.21.15.27:0/1141141128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3041a0100 con 0x7fc3041026f0 2024-04-26T23:09:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.041+0000 7fc308c00640 1 --2- 172.21.15.27:0/1141141128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3041026f0 0x7fc30419b110 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400eb00 tx=0x7fc2f400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.041+0000 7fc301600640 1 -- 172.21.15.27:0/1141141128 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4010430 con 0x7fc3041026f0 2024-04-26T23:09:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.041+0000 7fc30a000640 1 -- 172.21.15.27:0/1141141128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3041a03f0 con 0x7fc3041026f0 2024-04-26T23:09:29.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.041+0000 7fc301600640 1 -- 172.21.15.27:0/1141141128 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f4010a80 con 0x7fc3041026f0 2024-04-26T23:09:29.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.041+0000 7fc301600640 1 -- 172.21.15.27:0/1141141128 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4017a40 con 0x7fc3041026f0 2024-04-26T23:09:29.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.042+0000 7fc30a000640 1 -- 172.21.15.27:0/1141141128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3041a7c80 con 0x7fc3041026f0 2024-04-26T23:09:29.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.043+0000 7fc30a000640 1 -- 172.21.15.27:0/1141141128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3040506c0 con 0x7fc3041026f0 2024-04-26T23:09:29.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.045+0000 7fc301600640 1 -- 172.21.15.27:0/1141141128 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc2f4017c60 con 0x7fc3041026f0 2024-04-26T23:09:29.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.046+0000 7fc301600640 1 --2- 172.21.15.27:0/1141141128 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc2e80a3d10 0x7fc2e80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:29.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.046+0000 7fc301600640 1 -- 172.21.15.27:0/1141141128 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9700+0+0 (secure 0 0 0) 0x7fc2f40d1650 con 0x7fc3041026f0 2024-04-26T23:09:29.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.046+0000 7fc303e00640 1 --2- 172.21.15.27:0/1141141128 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc2e80a3d10 0x7fc2e80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:29.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.047+0000 7fc303e00640 1 --2- 172.21.15.27:0/1141141128 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc2e80a3d10 0x7fc2e80a61d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0096c0 tx=0x7fc2ec0046c0 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:29.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.050+0000 7fc301600640 1 -- 172.21.15.27:0/1141141128 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc2f409d850 con 0x7fc3041026f0 2024-04-26T23:09:29.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:28 smithi027 ceph-mon[28034]: pgmap v337: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:09:29.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:28 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/117394574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:09:29.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:28 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/693669781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:09:29.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.259+0000 7fc30a000640 1 -- 172.21.15.27:0/1141141128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc30419c460 con 0x7fc3041026f0 2024-04-26T23:09:29.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.260+0000 7fc301600640 1 -- 172.21.15.27:0/1141141128 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v103) ==== 81+0+1003 (secure 0 0 0) 0x7fc30419c460 con 0x7fc3041026f0 2024-04-26T23:09:29.260 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/4202829010 2024-04-27T22:58:35.543340+0000 2024-04-26T23:09:29.261 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3814309846 2024-04-27T22:47:04.292223+0000 2024-04-26T23:09:29.261 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2767985348 2024-04-27T22:46:28.107236+0000 2024-04-26T23:09:29.261 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1524274936 2024-04-27T22:46:28.107236+0000 2024-04-26T23:09:29.261 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3451701535 2024-04-27T22:47:04.292223+0000 2024-04-26T23:09:29.261 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/542026788 2024-04-27T22:46:28.107236+0000 2024-04-26T23:09:29.261 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/1521074971 2024-04-27T22:46:28.107236+0000 2024-04-26T23:09:29.261 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3746496955 2024-04-27T22:58:35.543340+0000 2024-04-26T23:09:29.261 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3136245138 2024-04-27T22:47:04.292223+0000 2024-04-26T23:09:29.261 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/1661629783 2024-04-27T22:47:04.292223+0000 2024-04-26T23:09:29.261 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1199904816 2024-04-27T22:58:35.543340+0000 2024-04-26T23:09:29.262 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/1521074971 2024-04-27T22:46:28.107236+0000 2024-04-26T23:09:29.262 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/1661629783 2024-04-27T22:47:04.292223+0000 2024-04-26T23:09:29.262 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3276968774 2024-04-27T22:58:35.543340+0000 2024-04-26T23:09:29.262 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3449139270 2024-04-27T22:58:35.543340+0000 2024-04-26T23:09:29.262 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/3636526714 2024-04-27T22:58:35.543340+0000 2024-04-26T23:09:29.262 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/3636526714 2024-04-27T22:58:35.543340+0000 2024-04-26T23:09:29.263 INFO:teuthology.orchestra.run.smithi027.stderr:listed 17 entries 2024-04-26T23:09:29.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.262+0000 7fc30a000640 1 -- 172.21.15.27:0/1141141128 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc2e80a3d10 msgr2=0x7fc2e80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:29.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.262+0000 7fc30a000640 1 --2- 172.21.15.27:0/1141141128 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc2e80a3d10 0x7fc2e80a61d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0096c0 tx=0x7fc2ec0046c0 comp rx=0 tx=0).stop 2024-04-26T23:09:29.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.262+0000 7fc30a000640 1 -- 172.21.15.27:0/1141141128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3041026f0 msgr2=0x7fc30419b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:29.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.262+0000 7fc30a000640 1 --2- 172.21.15.27:0/1141141128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3041026f0 0x7fc30419b110 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400eb00 tx=0x7fc2f400c2d0 comp rx=0 tx=0).stop 2024-04-26T23:09:29.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.262+0000 7fc30a000640 1 -- 172.21.15.27:0/1141141128 shutdown_connections 2024-04-26T23:09:29.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.262+0000 7fc30a000640 1 --2- 172.21.15.27:0/1141141128 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fc2e80a3d10 0x7fc2e80a61d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:29.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.262+0000 7fc30a000640 1 --2- 172.21.15.27:0/1141141128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc304108700 0x7fc30419f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:29.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.262+0000 7fc30a000640 1 --2- 172.21.15.27:0/1141141128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3041030b0 0x7fc30419b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:29.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.262+0000 7fc30a000640 1 --2- 172.21.15.27:0/1141141128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3041026f0 0x7fc30419b110 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:29.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.263+0000 7fc30a000640 1 -- 172.21.15.27:0/1141141128 >> 172.21.15.27:0/1141141128 conn(0x7fc3040fe200 msgr2=0x7fc30410d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:29.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.263+0000 7fc30a000640 1 -- 172.21.15.27:0/1141141128 shutdown_connections 2024-04-26T23:09:29.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.263+0000 7fc30a000640 1 -- 172.21.15.27:0/1141141128 wait complete. 2024-04-26T23:09:29.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:28 smithi080 ceph-mon[32324]: pgmap v337: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:09:29.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:28 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/117394574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:09:29.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:28 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/693669781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:09:29.281 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:09:29.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:09:29.281 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:09:29.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:28 smithi094 ceph-mon[32889]: pgmap v337: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:09:29.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:28 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/117394574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:09:29.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:28 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/693669781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:09:29.486 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T23:09:29.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:09:29.486 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:09:29.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:09:29.565 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-04-26T23:09:29.623 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:09:29.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:09:29.623 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:09:29.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:09:29.686 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-04-26T23:09:29.790 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-26T23:09:29.802 INFO:tasks.cephadm:Teardown begin 2024-04-26T23:09:29.803 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:09:29.803 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:09:29.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.979+0000 7faa95e00640 1 -- 172.21.15.27:0/195526695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa90075620 msgr2=0x7faa9010ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:29.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.979+0000 7faa95e00640 1 --2- 172.21.15.27:0/195526695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa90075620 0x7faa9010ea70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009900 tx=0x7faa7c02a480 comp rx=0 tx=0).stop 2024-04-26T23:09:29.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.979+0000 7faa95e00640 1 -- 172.21.15.27:0/195526695 shutdown_connections 2024-04-26T23:09:29.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.979+0000 7faa95e00640 1 --2- 172.21.15.27:0/195526695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa90075620 0x7faa9010ea70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:29.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.979+0000 7faa95e00640 1 --2- 172.21.15.27:0/195526695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa9010efb0 0x7faa901113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:29.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.979+0000 7faa95e00640 1 --2- 172.21.15.27:0/195526695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa90074d00 0x7faa900750e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:29.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.979+0000 7faa95e00640 1 -- 172.21.15.27:0/195526695 >> 172.21.15.27:0/195526695 conn(0x7faa900fe240 msgr2=0x7faa90100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:29.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.980+0000 7faa95e00640 1 -- 172.21.15.27:0/195526695 shutdown_connections 2024-04-26T23:09:29.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.980+0000 7faa95e00640 1 -- 172.21.15.27:0/195526695 wait complete. 2024-04-26T23:09:29.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.981+0000 7faa95e00640 1 Processor -- start 2024-04-26T23:09:29.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.981+0000 7faa95e00640 1 -- start start 2024-04-26T23:09:29.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.981+0000 7faa95e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa90074d00 0x7faa9019b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:29.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.981+0000 7faa95e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa90075620 0x7faa9019b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:29.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.981+0000 7faa95e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa9010efb0 0x7faa9019f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:29.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.981+0000 7faa95e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faa90114c10 con 0x7faa90075620 2024-04-26T23:09:29.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.981+0000 7faa95e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7faa90114a90 con 0x7faa9010efb0 2024-04-26T23:09:29.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.981+0000 7faa95e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa90114d90 con 0x7faa90074d00 2024-04-26T23:09:29.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.981+0000 7faa94a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa90074d00 0x7faa9019b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:29.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.981+0000 7faa95400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa9010efb0 0x7faa9019f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:29.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.982+0000 7faa94a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa90074d00 0x7faa9019b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:47250/0 (socket says 172.21.15.27:47250) 2024-04-26T23:09:29.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.982+0000 7faa95400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa9010efb0 0x7faa9019f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:47746/0 (socket says 172.21.15.27:47746) 2024-04-26T23:09:29.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.982+0000 7faa94a00640 1 -- 172.21.15.27:0/3329320789 learned_addr learned my addr 172.21.15.27:0/3329320789 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:09:29.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.982+0000 7faa8fe00640 1 --2- 172.21.15.27:0/3329320789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa90075620 0x7faa9019b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:29.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.982+0000 7faa94a00640 1 -- 172.21.15.27:0/3329320789 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa9010efb0 msgr2=0x7faa9019f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:29.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.982+0000 7faa94a00640 1 --2- 172.21.15.27:0/3329320789 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa9010efb0 0x7faa9019f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:29.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.982+0000 7faa94a00640 1 -- 172.21.15.27:0/3329320789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa90075620 msgr2=0x7faa9019b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:29.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.982+0000 7faa94a00640 1 --2- 172.21.15.27:0/3329320789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa90075620 0x7faa9019b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:29.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.982+0000 7faa94a00640 1 -- 172.21.15.27:0/3329320789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa901a0090 con 0x7faa90074d00 2024-04-26T23:09:29.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.982+0000 7faa95400640 1 --2- 172.21.15.27:0/3329320789 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa9010efb0 0x7faa9019f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:09:29.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.982+0000 7faa94a00640 1 --2- 172.21.15.27:0/3329320789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa90074d00 0x7faa9019b0f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faa8400e9f0 tx=0x7faa8400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:29.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.983+0000 7faa8d600640 1 -- 172.21.15.27:0/3329320789 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8400cd80 con 0x7faa90074d00 2024-04-26T23:09:29.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.983+0000 7faa8d600640 1 -- 172.21.15.27:0/3329320789 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa840108e0 con 0x7faa90074d00 2024-04-26T23:09:29.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.983+0000 7faa95e00640 1 -- 172.21.15.27:0/3329320789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa901a0380 con 0x7faa90074d00 2024-04-26T23:09:29.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.983+0000 7faa8d600640 1 -- 172.21.15.27:0/3329320789 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84018980 con 0x7faa90074d00 2024-04-26T23:09:29.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.983+0000 7faa95e00640 1 -- 172.21.15.27:0/3329320789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa901a7c10 con 0x7faa90074d00 2024-04-26T23:09:29.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.984+0000 7faa95e00640 1 -- 172.21.15.27:0/3329320789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa900506c0 con 0x7faa90074d00 2024-04-26T23:09:29.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.986+0000 7faa8d600640 1 -- 172.21.15.27:0/3329320789 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faa84010430 con 0x7faa90074d00 2024-04-26T23:09:29.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.987+0000 7faa8d600640 1 --2- 172.21.15.27:0/3329320789 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7faa780a3e40 0x7faa780a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:29.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.987+0000 7faa8d600640 1 -- 172.21.15.27:0/3329320789 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9700+0+0 (secure 0 0 0) 0x7faa840d0f70 con 0x7faa90074d00 2024-04-26T23:09:29.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.987+0000 7faa8fe00640 1 --2- 172.21.15.27:0/3329320789 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7faa780a3e40 0x7faa780a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:29.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.988+0000 7faa8fe00640 1 --2- 172.21.15.27:0/3329320789 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7faa780a3e40 0x7faa780a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faa7c005ab0 tx=0x7faa7c005a20 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:29.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:29.991+0000 7faa8d600640 1 -- 172.21.15.27:0/3329320789 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faa8409cac0 con 0x7faa90074d00 2024-04-26T23:09:30.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.234+0000 7faa95e00640 1 -- 172.21.15.27:0/3329320789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa9019c3f0 con 0x7faa90074d00 2024-04-26T23:09:30.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.235+0000 7faa8d600640 1 -- 172.21.15.27:0/3329320789 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11850 (secure 0 0 0) 0x7faa840a0dc0 con 0x7faa90074d00 2024-04-26T23:09:30.236 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T23:09:30.236 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T22:55:25.265331+0000","modified":"2024-04-26T23:06:30.721960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24578,"mds_1":24577,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/1620677319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1620677319},{"type":"v1","addr":"172.21.15.94:6839","nonce":1620677319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.27:6841/1571355196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1571355196},{"type":"v1","addr":"172.21.15.27:6841","nonce":1571355196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24577":{"gid":24577,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.27:6839/3193530521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3193530521},{"type":"v1","addr":"172.21.15.27:6839","nonce":3193530521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/95952000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":95952000},{"type":"v1","addr":"172.21.15.80:6839","nonce":95952000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24578,"qdb_cluster":[24560,24577,24578]},"id":1}]} 2024-04-26T23:09:30.236 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 36 2024-04-26T23:09:30.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.237+0000 7faa95e00640 1 -- 172.21.15.27:0/3329320789 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7faa780a3e40 msgr2=0x7faa780a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:30.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.237+0000 7faa95e00640 1 --2- 172.21.15.27:0/3329320789 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7faa780a3e40 0x7faa780a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faa7c005ab0 tx=0x7faa7c005a20 comp rx=0 tx=0).stop 2024-04-26T23:09:30.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.237+0000 7faa95e00640 1 -- 172.21.15.27:0/3329320789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa90074d00 msgr2=0x7faa9019b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:30.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.237+0000 7faa95e00640 1 --2- 172.21.15.27:0/3329320789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa90074d00 0x7faa9019b0f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faa8400e9f0 tx=0x7faa8400eeb0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.237+0000 7faa95e00640 1 -- 172.21.15.27:0/3329320789 shutdown_connections 2024-04-26T23:09:30.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.237+0000 7faa95e00640 1 --2- 172.21.15.27:0/3329320789 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7faa780a3e40 0x7faa780a6300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.237+0000 7faa95e00640 1 --2- 172.21.15.27:0/3329320789 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa9010efb0 0x7faa9019f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.237+0000 7faa95e00640 1 --2- 172.21.15.27:0/3329320789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa90075620 0x7faa9019b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.237+0000 7faa95e00640 1 --2- 172.21.15.27:0/3329320789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa90074d00 0x7faa9019b0f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.237+0000 7faa95e00640 1 -- 172.21.15.27:0/3329320789 >> 172.21.15.27:0/3329320789 conn(0x7faa900fe240 msgr2=0x7faa900fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:30.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.237+0000 7faa95e00640 1 -- 172.21.15.27:0/3329320789 shutdown_connections 2024-04-26T23:09:30.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.238+0000 7faa95e00640 1 -- 172.21.15.27:0/3329320789 wait complete. 2024-04-26T23:09:30.253 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:29 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1141141128' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T23:09:30.256 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:09:30.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:29 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1141141128' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T23:09:30.288 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:29 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1141141128' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T23:09:30.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.467+0000 7f0323e00640 1 -- 172.21.15.27:0/2452816228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0324103910 msgr2=0x7f0324103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.467+0000 7f0323e00640 1 --2- 172.21.15.27:0/2452816228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0324103910 0x7f0324103cf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f030c009900 tx=0x7f030c02a480 comp rx=0 tx=0).stop 2024-04-26T23:09:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.467+0000 7f0323e00640 1 -- 172.21.15.27:0/2452816228 shutdown_connections 2024-04-26T23:09:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.467+0000 7f0323e00640 1 --2- 172.21.15.27:0/2452816228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324104230 0x7f032410c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.467+0000 7f0323e00640 1 --2- 172.21.15.27:0/2452816228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f032410cce0 0x7f032410f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.467+0000 7f0323e00640 1 --2- 172.21.15.27:0/2452816228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0324103910 0x7f0324103cf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.467+0000 7f0323e00640 1 -- 172.21.15.27:0/2452816228 >> 172.21.15.27:0/2452816228 conn(0x7f03240783e0 msgr2=0x7f03240ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.468+0000 7f0323e00640 1 -- 172.21.15.27:0/2452816228 shutdown_connections 2024-04-26T23:09:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.468+0000 7f0323e00640 1 -- 172.21.15.27:0/2452816228 wait complete. 2024-04-26T23:09:30.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.469+0000 7f0323e00640 1 Processor -- start 2024-04-26T23:09:30.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.469+0000 7f0323e00640 1 -- start start 2024-04-26T23:09:30.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.470+0000 7f0323e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324103910 0x7f032419b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:30.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.470+0000 7f0323e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0324104230 0x7f032419b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:30.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.470+0000 7f0323e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f032410cce0 0x7f032419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:30.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.470+0000 7f0323e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0324112990 con 0x7f0324104230 2024-04-26T23:09:30.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.470+0000 7f0323e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0324112810 con 0x7f0324103910 2024-04-26T23:09:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.470+0000 7f0323e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0324112b10 con 0x7f032410cce0 2024-04-26T23:09:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.470+0000 7f0322000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0324104230 0x7f032419b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.470+0000 7f0322000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0324104230 0x7f032419b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51374/0 (socket says 172.21.15.27:51374) 2024-04-26T23:09:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.470+0000 7f0322000640 1 -- 172.21.15.27:0/1019397532 learned_addr learned my addr 172.21.15.27:0/1019397532 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:09:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.470+0000 7f0322a00640 1 --2- 172.21.15.27:0/1019397532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324103910 0x7f032419b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.470+0000 7f0323400640 1 --2- 172.21.15.27:0/1019397532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f032410cce0 0x7f032419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.470+0000 7f0322000640 1 -- 172.21.15.27:0/1019397532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f032410cce0 msgr2=0x7f032419f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.470+0000 7f0322000640 1 --2- 172.21.15.27:0/1019397532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f032410cce0 0x7f032419f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.470+0000 7f0322000640 1 -- 172.21.15.27:0/1019397532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324103910 msgr2=0x7f032419b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.470+0000 7f0322000640 1 --2- 172.21.15.27:0/1019397532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324103910 0x7f032419b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.471+0000 7f0322000640 1 -- 172.21.15.27:0/1019397532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03241a0020 con 0x7f0324104230 2024-04-26T23:09:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.471+0000 7f0323400640 1 --2- 172.21.15.27:0/1019397532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f032410cce0 0x7f032419f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:09:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.471+0000 7f0322a00640 1 --2- 172.21.15.27:0/1019397532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324103910 0x7f032419b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:09:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.471+0000 7f0322000640 1 --2- 172.21.15.27:0/1019397532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0324104230 0x7f032419b570 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0314009d50 tx=0x7f031400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.471+0000 7f0328a00640 1 -- 172.21.15.27:0/1019397532 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314003b70 con 0x7f0324104230 2024-04-26T23:09:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.471+0000 7f0328a00640 1 -- 172.21.15.27:0/1019397532 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0314003d10 con 0x7f0324104230 2024-04-26T23:09:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.471+0000 7f0328a00640 1 -- 172.21.15.27:0/1019397532 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03140108a0 con 0x7f0324104230 2024-04-26T23:09:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.471+0000 7f0323e00640 1 -- 172.21.15.27:0/1019397532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03241a0310 con 0x7f0324104230 2024-04-26T23:09:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.471+0000 7f0323e00640 1 -- 172.21.15.27:0/1019397532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03241a7ba0 con 0x7f0324104230 2024-04-26T23:09:30.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.473+0000 7f0323e00640 1 -- 172.21.15.27:0/1019397532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03240506c0 con 0x7f0324104230 2024-04-26T23:09:30.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.475+0000 7f0328a00640 1 -- 172.21.15.27:0/1019397532 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0314004000 con 0x7f0324104230 2024-04-26T23:09:30.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.476+0000 7f0328a00640 1 --2- 172.21.15.27:0/1019397532 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f03080a3e40 0x7f03080a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:30.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.477+0000 7f0328a00640 1 -- 172.21.15.27:0/1019397532 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9700+0+0 (secure 0 0 0) 0x7f03140d1310 con 0x7f0324104230 2024-04-26T23:09:30.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.477+0000 7f0322a00640 1 --2- 172.21.15.27:0/1019397532 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f03080a3e40 0x7f03080a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:30.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.478+0000 7f0322a00640 1 --2- 172.21.15.27:0/1019397532 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f03080a3e40 0x7f03080a6300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f030c031440 tx=0x7f030c005b50 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:30.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.481+0000 7f0328a00640 1 -- 172.21.15.27:0/1019397532 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f031409d4b0 con 0x7f0324104230 2024-04-26T23:09:30.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.722+0000 7f0323e00640 1 -- 172.21.15.27:0/1019397532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f032419c380 con 0x7f0324104230 2024-04-26T23:09:30.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.723+0000 7f0328a00640 1 -- 172.21.15.27:0/1019397532 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11850 (secure 0 0 0) 0x7f031409cdb0 con 0x7f0324104230 2024-04-26T23:09:30.724 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T23:09:30.724 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3122709125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3122709125},{"type":"v1","addr":"172.21.15.27:6835","nonce":3122709125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/899705990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":899705990},{"type":"v1","addr":"172.21.15.80:6833","nonce":899705990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/828002248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":828002248},{"type":"v1","addr":"172.21.15.94:6833","nonce":828002248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1639451555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1639451555},{"type":"v1","addr":"172.21.15.27:6837","nonce":1639451555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3792288301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3792288301},{"type":"v1","addr":"172.21.15.80:6835","nonce":3792288301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3513669668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3513669668},{"type":"v1","addr":"172.21.15.94:6835","nonce":3513669668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T22:55:25.265331+0000","modified":"2024-04-26T23:06:30.721960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24578,"mds_1":24577,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/3963089995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3963089995},{"type":"v1","addr":"172.21.15.80:6837","nonce":3963089995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3367319222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3367319222},{"type":"v1","addr":"172.21.15.94:6837","nonce":3367319222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/1620677319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1620677319},{"type":"v1","addr":"172.21.15.94:6839","nonce":1620677319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.27:6841/1571355196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1571355196},{"type":"v1","addr":"172.21.15.27:6841","nonce":1571355196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24577":{"gid":24577,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.27:6839/3193530521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3193530521},{"type":"v1","addr":"172.21.15.27:6839","nonce":3193530521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/95952000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":95952000},{"type":"v1","addr":"172.21.15.80:6839","nonce":95952000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24578,"qdb_cluster":[24560,24577,24578]},"id":1}]} 2024-04-26T23:09:30.724 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 36 2024-04-26T23:09:30.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.725+0000 7f0323e00640 1 -- 172.21.15.27:0/1019397532 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f03080a3e40 msgr2=0x7f03080a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:30.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.725+0000 7f0323e00640 1 --2- 172.21.15.27:0/1019397532 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f03080a3e40 0x7f03080a6300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f030c031440 tx=0x7f030c005b50 comp rx=0 tx=0).stop 2024-04-26T23:09:30.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.725+0000 7f0323e00640 1 -- 172.21.15.27:0/1019397532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0324104230 msgr2=0x7f032419b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:30.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.725+0000 7f0323e00640 1 --2- 172.21.15.27:0/1019397532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0324104230 0x7f032419b570 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0314009d50 tx=0x7f031400c6a0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.726+0000 7f0323e00640 1 -- 172.21.15.27:0/1019397532 shutdown_connections 2024-04-26T23:09:30.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.726+0000 7f0323e00640 1 --2- 172.21.15.27:0/1019397532 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f03080a3e40 0x7f03080a6300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.726+0000 7f0323e00640 1 --2- 172.21.15.27:0/1019397532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f032410cce0 0x7f032419f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.726+0000 7f0323e00640 1 --2- 172.21.15.27:0/1019397532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0324104230 0x7f032419b570 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.726+0000 7f0323e00640 1 --2- 172.21.15.27:0/1019397532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324103910 0x7f032419b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.726+0000 7f0323e00640 1 -- 172.21.15.27:0/1019397532 >> 172.21.15.27:0/1019397532 conn(0x7f03240783e0 msgr2=0x7f0324101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:30.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.726+0000 7f0323e00640 1 -- 172.21.15.27:0/1019397532 shutdown_connections 2024-04-26T23:09:30.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.726+0000 7f0323e00640 1 -- 172.21.15.27:0/1019397532 wait complete. 2024-04-26T23:09:30.744 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:09:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.959+0000 7f2f9dc00640 1 -- 172.21.15.27:0/2334947780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f98074cd0 msgr2=0x7f2f980750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.959+0000 7f2f9dc00640 1 --2- 172.21.15.27:0/2334947780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f98074cd0 0x7f2f980750b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009900 tx=0x7f2f8802a480 comp rx=0 tx=0).stop 2024-04-26T23:09:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.959+0000 7f2f9dc00640 1 -- 172.21.15.27:0/2334947780 shutdown_connections 2024-04-26T23:09:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.959+0000 7f2f9dc00640 1 --2- 172.21.15.27:0/2334947780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f981138e0 0x7f2f98115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.959+0000 7f2f9dc00640 1 --2- 172.21.15.27:0/2334947780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f980755f0 0x7f2f981133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.959+0000 7f2f9dc00640 1 --2- 172.21.15.27:0/2334947780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f98074cd0 0x7f2f980750b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.959+0000 7f2f9dc00640 1 -- 172.21.15.27:0/2334947780 >> 172.21.15.27:0/2334947780 conn(0x7f2f980fe3c0 msgr2=0x7f2f981007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.959+0000 7f2f9dc00640 1 -- 172.21.15.27:0/2334947780 shutdown_connections 2024-04-26T23:09:30.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.959+0000 7f2f9dc00640 1 -- 172.21.15.27:0/2334947780 wait complete. 2024-04-26T23:09:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.960+0000 7f2f9dc00640 1 Processor -- start 2024-04-26T23:09:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.961+0000 7f2f9dc00640 1 -- start start 2024-04-26T23:09:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.961+0000 7f2f9dc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f98074cd0 0x7f2f9819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.961+0000 7f2f9dc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f980755f0 0x7f2f9819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.961+0000 7f2f9dc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f981138e0 0x7f2f981a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.961+0000 7f2f9dc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98119180 con 0x7f2f98074cd0 2024-04-26T23:09:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.961+0000 7f2f9dc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98119000 con 0x7f2f981138e0 2024-04-26T23:09:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.961+0000 7f2f9dc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98119300 con 0x7f2f980755f0 2024-04-26T23:09:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.961+0000 7f2f96a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f98074cd0 0x7f2f9819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.961+0000 7f2f96a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f98074cd0 0x7f2f9819f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51394/0 (socket says 172.21.15.27:51394) 2024-04-26T23:09:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.961+0000 7f2f96a00640 1 -- 172.21.15.27:0/175762157 learned_addr learned my addr 172.21.15.27:0/175762157 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:09:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.961+0000 7f2f96000640 1 --2- 172.21.15.27:0/175762157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f980755f0 0x7f2f9819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.961+0000 7f2f97400640 1 --2- 172.21.15.27:0/175762157 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f981138e0 0x7f2f981a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.962+0000 7f2f96a00640 1 -- 172.21.15.27:0/175762157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f980755f0 msgr2=0x7f2f9819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.962+0000 7f2f96a00640 1 --2- 172.21.15.27:0/175762157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f980755f0 0x7f2f9819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.962+0000 7f2f96a00640 1 -- 172.21.15.27:0/175762157 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f981138e0 msgr2=0x7f2f981a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.962+0000 7f2f96a00640 1 --2- 172.21.15.27:0/175762157 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f981138e0 0x7f2f981a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.962+0000 7f2f96a00640 1 -- 172.21.15.27:0/175762157 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f981a4730 con 0x7f2f98074cd0 2024-04-26T23:09:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.962+0000 7f2f96a00640 1 --2- 172.21.15.27:0/175762157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f98074cd0 0x7f2f9819f740 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2f88005940 tx=0x7f2f880040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.962+0000 7f2f8f400640 1 -- 172.21.15.27:0/175762157 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8803f030 con 0x7f2f98074cd0 2024-04-26T23:09:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.962+0000 7f2f8f400640 1 -- 172.21.15.27:0/175762157 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f880044c0 con 0x7f2f98074cd0 2024-04-26T23:09:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.962+0000 7f2f9dc00640 1 -- 172.21.15.27:0/175762157 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f981a49c0 con 0x7f2f98074cd0 2024-04-26T23:09:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.962+0000 7f2f8f400640 1 -- 172.21.15.27:0/175762157 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88035070 con 0x7f2f98074cd0 2024-04-26T23:09:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.963+0000 7f2f9dc00640 1 -- 172.21.15.27:0/175762157 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f981ac2b0 con 0x7f2f98074cd0 2024-04-26T23:09:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.964+0000 7f2f9dc00640 1 -- 172.21.15.27:0/175762157 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f980697e0 con 0x7f2f98074cd0 2024-04-26T23:09:30.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.966+0000 7f2f8f400640 1 -- 172.21.15.27:0/175762157 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2f88042030 con 0x7f2f98074cd0 2024-04-26T23:09:30.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.968+0000 7f2f8f400640 1 --2- 172.21.15.27:0/175762157 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f2f7c0a3d20 0x7f2f7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:30.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.968+0000 7f2f8f400640 1 -- 172.21.15.27:0/175762157 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9700+0+0 (secure 0 0 0) 0x7f2f880f0ff0 con 0x7f2f98074cd0 2024-04-26T23:09:30.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.968+0000 7f2f96000640 1 --2- 172.21.15.27:0/175762157 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f2f7c0a3d20 0x7f2f7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:30.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.969+0000 7f2f96000640 1 --2- 172.21.15.27:0/175762157 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f2f7c0a3d20 0x7f2f7c0a61e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2f98100b30 tx=0x7f2f84005f90 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:30.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:30.973+0000 7f2f8f400640 1 -- 172.21.15.27:0/175762157 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2f880bcba0 con 0x7f2f98074cd0 2024-04-26T23:09:31.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:30 smithi027 ceph-mon[28034]: pgmap v338: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:09:31.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:30 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/3329320789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:09:31.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:30 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1019397532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:09:31.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.171+0000 7f2f9dc00640 1 -- 172.21.15.27:0/175762157 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f98105d30 con 0x7f2f98074cd0 2024-04-26T23:09:31.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.173+0000 7f2f8f400640 1 -- 172.21.15.27:0/175762157 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+25528 (secure 0 0 0) 0x7f2f880c0ea0 con 0x7f2f98074cd0 2024-04-26T23:09:31.175 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T23:09:31.175 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":103,"fsid":"95ebdda6-041e-11ef-bc93-c7b262605968","created":"2024-04-26T22:45:52.404180+0000","modified":"2024-04-26T23:09:21.551256+0000","last_up_change":"2024-04-26T22:54:22.776979+0000","last_in_change":"2024-04-26T22:54:02.489429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:49:54.482679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:55:22.899174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:55:24.218295+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T22:55:28.394547+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6f4675-af17-4d78-87cd-9b1ccef44def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6803","nonce":4133324880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6805","nonce":4133324880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6809","nonce":4133324880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4133324880},{"type":"v1","addr":"172.21.15.27:6807","nonce":4133324880}]},"public_addr":"172.21.15.27:6803/4133324880","cluster_addr":"172.21.15.27:6805/4133324880","heartbeat_back_addr":"172.21.15.27:6809/4133324880","heartbeat_front_addr":"172.21.15.27:6807/4133324880","state":["exists","up"]},{"osd":1,"uuid":"e980f676-0a81-4253-8a39-db33f37ed79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6801","nonce":691528245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6803","nonce":691528245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6807","nonce":691528245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":691528245},{"type":"v1","addr":"172.21.15.80:6805","nonce":691528245}]},"public_addr":"172.21.15.80:6801/691528245","cluster_addr":"172.21.15.80:6803/691528245","heartbeat_back_addr":"172.21.15.80:6807/691528245","heartbeat_front_addr":"172.21.15.80:6805/691528245","state":["exists","up"]},{"osd":2,"uuid":"57ae3ef4-6f8d-4ee2-bcdb-8c49e6051ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6801","nonce":3315198187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6803","nonce":3315198187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6807","nonce":3315198187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3315198187},{"type":"v1","addr":"172.21.15.94:6805","nonce":3315198187}]},"public_addr":"172.21.15.94:6801/3315198187","cluster_addr":"172.21.15.94:6803/3315198187","heartbeat_back_addr":"172.21.15.94:6807/3315198187","heartbeat_front_addr":"172.21.15.94:6805/3315198187","state":["exists","up"]},{"osd":3,"uuid":"95f248ab-e3e8-425c-a0f8-c26cfc6f1be8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6811","nonce":506143849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6813","nonce":506143849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6817","nonce":506143849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":506143849},{"type":"v1","addr":"172.21.15.27:6815","nonce":506143849}]},"public_addr":"172.21.15.27:6811/506143849","cluster_addr":"172.21.15.27:6813/506143849","heartbeat_back_addr":"172.21.15.27:6817/506143849","heartbeat_front_addr":"172.21.15.27:6815/506143849","state":["exists","up"]},{"osd":4,"uuid":"1d4f0826-5221-41cc-a658-fc518acf4c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6809","nonce":1483108588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6811","nonce":1483108588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6815","nonce":1483108588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1483108588},{"type":"v1","addr":"172.21.15.80:6813","nonce":1483108588}]},"public_addr":"172.21.15.80:6809/1483108588","cluster_addr":"172.21.15.80:6811/1483108588","heartbeat_back_addr":"172.21.15.80:6815/1483108588","heartbeat_front_addr":"172.21.15.80:6813/1483108588","state":["exists","up"]},{"osd":5,"uuid":"7369cc9a-e487-46bb-8f93-42a025150669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6809","nonce":396894302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6811","nonce":396894302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6815","nonce":396894302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":396894302},{"type":"v1","addr":"172.21.15.94:6813","nonce":396894302}]},"public_addr":"172.21.15.94:6809/396894302","cluster_addr":"172.21.15.94:6811/396894302","heartbeat_back_addr":"172.21.15.94:6815/396894302","heartbeat_front_addr":"172.21.15.94:6813/396894302","state":["exists","up"]},{"osd":6,"uuid":"675deb21-5cf8-457d-acdb-188d3372dd8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6819","nonce":1730660655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6821","nonce":1730660655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6825","nonce":1730660655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1730660655},{"type":"v1","addr":"172.21.15.27:6823","nonce":1730660655}]},"public_addr":"172.21.15.27:6819/1730660655","cluster_addr":"172.21.15.27:6821/1730660655","heartbeat_back_addr":"172.21.15.27:6825/1730660655","heartbeat_front_addr":"172.21.15.27:6823/1730660655","state":["exists","up"]},{"osd":7,"uuid":"5e9aa483-e206-4ba2-b39b-9f7547da1459","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6817","nonce":534257127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6819","nonce":534257127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6823","nonce":534257127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":534257127},{"type":"v1","addr":"172.21.15.80:6821","nonce":534257127}]},"public_addr":"172.21.15.80:6817/534257127","cluster_addr":"172.21.15.80:6819/534257127","heartbeat_back_addr":"172.21.15.80:6823/534257127","heartbeat_front_addr":"172.21.15.80:6821/534257127","state":["exists","up"]},{"osd":8,"uuid":"effb8164-0b8e-425f-af66-624c2909eb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6817","nonce":484486649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6819","nonce":484486649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6823","nonce":484486649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":484486649},{"type":"v1","addr":"172.21.15.94:6821","nonce":484486649}]},"public_addr":"172.21.15.94:6817/484486649","cluster_addr":"172.21.15.94:6819/484486649","heartbeat_back_addr":"172.21.15.94:6823/484486649","heartbeat_front_addr":"172.21.15.94:6821/484486649","state":["exists","up"]},{"osd":9,"uuid":"6c52e6ee-8407-498e-a0fe-579588406f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6827","nonce":2196150976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6829","nonce":2196150976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6833","nonce":2196150976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2196150976},{"type":"v1","addr":"172.21.15.27:6831","nonce":2196150976}]},"public_addr":"172.21.15.27:6827/2196150976","cluster_addr":"172.21.15.27:6829/2196150976","heartbeat_back_addr":"172.21.15.27:6833/2196150976","heartbeat_front_addr":"172.21.15.27:6831/2196150976","state":["exists","up"]},{"osd":10,"uuid":"33af8d56-6def-4c77-9fb2-8d2837c18c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6825","nonce":3521249391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6827","nonce":3521249391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6831","nonce":3521249391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3521249391},{"type":"v1","addr":"172.21.15.80:6829","nonce":3521249391}]},"public_addr":"172.21.15.80:6825/3521249391","cluster_addr":"172.21.15.80:6827/3521249391","heartbeat_back_addr":"172.21.15.80:6831/3521249391","heartbeat_front_addr":"172.21.15.80:6829/3521249391","state":["exists","up"]},{"osd":11,"uuid":"ffee2993-b4e0-4309-9cd8-83ac32df51c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6825","nonce":2763583925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6827","nonce":2763583925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6831","nonce":2763583925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2763583925},{"type":"v1","addr":"172.21.15.94:6829","nonce":2763583925}]},"public_addr":"172.21.15.94:6825/2763583925","cluster_addr":"172.21.15.94:6827/2763583925","heartbeat_back_addr":"172.21.15.94:6831/2763583925","heartbeat_front_addr":"172.21.15.94:6829/2763583925","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:01.901331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:26.902108+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:49:51.505136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:26.650274+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:50:53.619067+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:19.248712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:51:57.022379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:23.686197+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:52:49.314550+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:26.852702+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:53:54.406498+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:54:20.836516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4},{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/4202829010":"2024-04-27T22:58:35.543340+0000","172.21.15.27:0/3814309846":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/2767985348":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/1524274936":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3451701535":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/542026788":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6801/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:0/3746496955":"2024-04-27T22:58:35.543340+0000","172.21.15.27:0/3136245138":"2024-04-27T22:47:04.292223+0000","172.21.15.27:6800/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/1199904816":"2024-04-27T22:58:35.543340+0000","172.21.15.27:6800/1521074971":"2024-04-27T22:46:28.107236+0000","172.21.15.27:6801/1661629783":"2024-04-27T22:47:04.292223+0000","172.21.15.27:0/3276968774":"2024-04-27T22:58:35.543340+0000","172.21.15.27:0/3449139270":"2024-04-27T22:58:35.543340+0000","172.21.15.27:6801/3636526714":"2024-04-27T22:58:35.543340+0000","172.21.15.27:6800/3636526714":"2024-04-27T22:58:35.543340+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:09:31.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.176+0000 7f2f9dc00640 1 -- 172.21.15.27:0/175762157 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f2f7c0a3d20 msgr2=0x7f2f7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:31.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.176+0000 7f2f9dc00640 1 --2- 172.21.15.27:0/175762157 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f2f7c0a3d20 0x7f2f7c0a61e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2f98100b30 tx=0x7f2f84005f90 comp rx=0 tx=0).stop 2024-04-26T23:09:31.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.176+0000 7f2f9dc00640 1 -- 172.21.15.27:0/175762157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f98074cd0 msgr2=0x7f2f9819f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:31.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.176+0000 7f2f9dc00640 1 --2- 172.21.15.27:0/175762157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f98074cd0 0x7f2f9819f740 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2f88005940 tx=0x7f2f880040c0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.176+0000 7f2f9dc00640 1 -- 172.21.15.27:0/175762157 shutdown_connections 2024-04-26T23:09:31.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.176+0000 7f2f9dc00640 1 --2- 172.21.15.27:0/175762157 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f2f7c0a3d20 0x7f2f7c0a61e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.176+0000 7f2f9dc00640 1 --2- 172.21.15.27:0/175762157 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f981138e0 0x7f2f981a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.176+0000 7f2f9dc00640 1 --2- 172.21.15.27:0/175762157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f980755f0 0x7f2f9819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.176+0000 7f2f9dc00640 1 --2- 172.21.15.27:0/175762157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f98074cd0 0x7f2f9819f740 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.176+0000 7f2f9dc00640 1 -- 172.21.15.27:0/175762157 >> 172.21.15.27:0/175762157 conn(0x7f2f980fe3c0 msgr2=0x7f2f980ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:31.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.177+0000 7f2f9dc00640 1 -- 172.21.15.27:0/175762157 shutdown_connections 2024-04-26T23:09:31.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.177+0000 7f2f9dc00640 1 -- 172.21.15.27:0/175762157 wait complete. 2024-04-26T23:09:31.194 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:09:31.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:30 smithi080 ceph-mon[32324]: pgmap v338: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:09:31.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:30 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/3329320789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:09:31.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:30 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1019397532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:09:31.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:30 smithi094 ceph-mon[32889]: pgmap v338: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T23:09:31.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:30 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/3329320789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:09:31.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:30 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1019397532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:09:31.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.392+0000 7f97c3200640 1 -- 172.21.15.27:0/2940102633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97bc1089e0 msgr2=0x7f97bc108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:31.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.392+0000 7f97c3200640 1 --2- 172.21.15.27:0/2940102633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97bc1089e0 0x7f97bc108dc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009860 tx=0x7f97ac02a340 comp rx=0 tx=0).stop 2024-04-26T23:09:31.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.392+0000 7f97c3200640 1 -- 172.21.15.27:0/2940102633 shutdown_connections 2024-04-26T23:09:31.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.392+0000 7f97c3200640 1 --2- 172.21.15.27:0/2940102633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc103390 0x7f97bc10f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.392+0000 7f97c3200640 1 --2- 172.21.15.27:0/2940102633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc1029d0 0x7f97bc102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.392+0000 7f97c3200640 1 --2- 172.21.15.27:0/2940102633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97bc1089e0 0x7f97bc108dc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.392+0000 7f97c3200640 1 -- 172.21.15.27:0/2940102633 >> 172.21.15.27:0/2940102633 conn(0x7f97bc0fe460 msgr2=0x7f97bc100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:31.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.393+0000 7f97c3200640 1 -- 172.21.15.27:0/2940102633 shutdown_connections 2024-04-26T23:09:31.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.393+0000 7f97c3200640 1 -- 172.21.15.27:0/2940102633 wait complete. 2024-04-26T23:09:31.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.394+0000 7f97c3200640 1 Processor -- start 2024-04-26T23:09:31.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.394+0000 7f97c3200640 1 -- start start 2024-04-26T23:09:31.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.394+0000 7f97c3200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97bc1029d0 0x7f97bc074c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:31.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.394+0000 7f97c3200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc103390 0x7f97bc075170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:31.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.394+0000 7f97c3200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc1089e0 0x7f97bc0756b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:31.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.394+0000 7f97c3200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc112ca0 con 0x7f97bc1089e0 2024-04-26T23:09:31.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.394+0000 7f97c3200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc112b20 con 0x7f97bc1029d0 2024-04-26T23:09:31.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.394+0000 7f97c3200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc112e20 con 0x7f97bc103390 2024-04-26T23:09:31.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.395+0000 7f97c1200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc1089e0 0x7f97bc0756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:31.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.395+0000 7f97c1200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc1089e0 0x7f97bc0756b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51416/0 (socket says 172.21.15.27:51416) 2024-04-26T23:09:31.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.395+0000 7f97bbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97bc1029d0 0x7f97bc074c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:31.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.395+0000 7f97c1200640 1 -- 172.21.15.27:0/1537503850 learned_addr learned my addr 172.21.15.27:0/1537503850 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:09:31.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.395+0000 7f97b3e00640 1 --2- 172.21.15.27:0/1537503850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc103390 0x7f97bc075170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:31.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.395+0000 7f97c1200640 1 -- 172.21.15.27:0/1537503850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc103390 msgr2=0x7f97bc075170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:31.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.395+0000 7f97c1200640 1 --2- 172.21.15.27:0/1537503850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc103390 0x7f97bc075170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.395+0000 7f97c1200640 1 -- 172.21.15.27:0/1537503850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97bc1029d0 msgr2=0x7f97bc074c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:31.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.395+0000 7f97c1200640 1 --2- 172.21.15.27:0/1537503850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97bc1029d0 0x7f97bc074c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.395+0000 7f97c1200640 1 -- 172.21.15.27:0/1537503850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc1a7680 con 0x7f97bc1089e0 2024-04-26T23:09:31.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.395+0000 7f97c1200640 1 --2- 172.21.15.27:0/1537503850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc1089e0 0x7f97bc0756b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f97b400eb10 tx=0x7f97b400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:31.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.396+0000 7f97b9600640 1 -- 172.21.15.27:0/1537503850 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b400ce20 con 0x7f97bc1089e0 2024-04-26T23:09:31.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.396+0000 7f97c3200640 1 -- 172.21.15.27:0/1537503850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97bc1a7970 con 0x7f97bc1089e0 2024-04-26T23:09:31.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.396+0000 7f97b9600640 1 -- 172.21.15.27:0/1537503850 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97b4005020 con 0x7f97bc1089e0 2024-04-26T23:09:31.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.396+0000 7f97b9600640 1 -- 172.21.15.27:0/1537503850 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4010690 con 0x7f97bc1089e0 2024-04-26T23:09:31.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.396+0000 7f97c3200640 1 -- 172.21.15.27:0/1537503850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97bc1a7f10 con 0x7f97bc1089e0 2024-04-26T23:09:31.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.397+0000 7f97c3200640 1 -- 172.21.15.27:0/1537503850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97bc0697e0 con 0x7f97bc1089e0 2024-04-26T23:09:31.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.399+0000 7f97b9600640 1 -- 172.21.15.27:0/1537503850 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f97b4005510 con 0x7f97bc1089e0 2024-04-26T23:09:31.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.400+0000 7f97b9600640 1 --2- 172.21.15.27:0/1537503850 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f97a00a3d20 0x7f97a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:31.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.401+0000 7f97b9600640 1 -- 172.21.15.27:0/1537503850 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9700+0+0 (secure 0 0 0) 0x7f97b40d12b0 con 0x7f97bc1089e0 2024-04-26T23:09:31.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.401+0000 7f97bbe00640 1 --2- 172.21.15.27:0/1537503850 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f97a00a3d20 0x7f97a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:31.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.402+0000 7f97bbe00640 1 --2- 172.21.15.27:0/1537503850 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f97a00a3d20 0x7f97a00a61e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f97ac02a930 tx=0x7f97ac030340 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:31.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.409+0000 7f97b9600640 1 -- 172.21.15.27:0/1537503850 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f97b409d4b0 con 0x7f97bc1089e0 2024-04-26T23:09:31.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.654+0000 7f97c3200640 1 -- 172.21.15.27:0/1537503850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f97bc10af40 con 0x7f97bc1089e0 2024-04-26T23:09:31.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.657+0000 7f97b9600640 1 -- 172.21.15.27:0/1537503850 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7f97b409cdb0 con 0x7f97bc1089e0 2024-04-26T23:09:31.658 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:09:31.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.662+0000 7f97c3200640 1 -- 172.21.15.27:0/1537503850 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f97a00a3d20 msgr2=0x7f97a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:31.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.662+0000 7f97c3200640 1 --2- 172.21.15.27:0/1537503850 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f97a00a3d20 0x7f97a00a61e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f97ac02a930 tx=0x7f97ac030340 comp rx=0 tx=0).stop 2024-04-26T23:09:31.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.662+0000 7f97c3200640 1 -- 172.21.15.27:0/1537503850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc1089e0 msgr2=0x7f97bc0756b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:31.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.662+0000 7f97c3200640 1 --2- 172.21.15.27:0/1537503850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc1089e0 0x7f97bc0756b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f97b400eb10 tx=0x7f97b400ef10 comp rx=0 tx=0).stop 2024-04-26T23:09:31.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.663+0000 7f97c3200640 1 -- 172.21.15.27:0/1537503850 shutdown_connections 2024-04-26T23:09:31.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.663+0000 7f97c3200640 1 --2- 172.21.15.27:0/1537503850 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f97a00a3d20 0x7f97a00a61e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.663+0000 7f97c3200640 1 --2- 172.21.15.27:0/1537503850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc1089e0 0x7f97bc0756b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.663+0000 7f97c3200640 1 --2- 172.21.15.27:0/1537503850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc103390 0x7f97bc075170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.663+0000 7f97c3200640 1 --2- 172.21.15.27:0/1537503850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97bc1029d0 0x7f97bc074c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.663+0000 7f97c3200640 1 -- 172.21.15.27:0/1537503850 >> 172.21.15.27:0/1537503850 conn(0x7f97bc0fe460 msgr2=0x7f97bc0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:31.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.663+0000 7f97c3200640 1 -- 172.21.15.27:0/1537503850 shutdown_connections 2024-04-26T23:09:31.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.663+0000 7f97c3200640 1 -- 172.21.15.27:0/1537503850 wait complete. 2024-04-26T23:09:31.692 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:09:31.865 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-a[69277]: ignoring --setuser ceph since I am not root 2024-04-26T23:09:31.865 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-a[69277]: ignoring --setgroup ceph since I am not root 2024-04-26T23:09:31.865 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-mds[69301]: -- 172.21.15.27:0/3895636436 <== mon.1 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563716dbfa40 con 0x563716d92800 2024-04-26T23:09:31.865 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-a[69277]: starting mds.a at 2024-04-26T23:09:31.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a[28021]: 2024-04-26T23:09:31.655+0000 7faca0c00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:09:31.866 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-g[68931]: ignoring --setuser ceph since I am not root 2024-04-26T23:09:31.866 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-g[68931]: ignoring --setgroup ceph since I am not root 2024-04-26T23:09:31.866 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-mds[68935]: -- 172.21.15.27:0/2158237674 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560c94369a40 con 0x560c9433cc00 2024-04-26T23:09:31.866 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-g[68931]: starting mds.g at 2024-04-26T23:09:31.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.895+0000 7f43cd000640 1 -- 172.21.15.27:0/819159435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43c81044c0 msgr2=0x7f43c810c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:31.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.895+0000 7f43cd000640 1 --2- 172.21.15.27:0/819159435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43c81044c0 0x7f43c810c9a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009860 tx=0x7f43b002a3d0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.896+0000 7f43cd000640 1 -- 172.21.15.27:0/819159435 shutdown_connections 2024-04-26T23:09:31.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.896+0000 7f43cd000640 1 --2- 172.21.15.27:0/819159435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43c81044c0 0x7f43c810c9a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.896+0000 7f43cd000640 1 --2- 172.21.15.27:0/819159435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43c810cee0 0x7f43c810f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.896+0000 7f43cd000640 1 --2- 172.21.15.27:0/819159435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8103ba0 0x7f43c8103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.896+0000 7f43cd000640 1 -- 172.21.15.27:0/819159435 >> 172.21.15.27:0/819159435 conn(0x7f43c80789e0 msgr2=0x7f43c8078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:31.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.896+0000 7f43cd000640 1 -- 172.21.15.27:0/819159435 shutdown_connections 2024-04-26T23:09:31.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.896+0000 7f43cd000640 1 -- 172.21.15.27:0/819159435 wait complete. 2024-04-26T23:09:31.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.897+0000 7f43cd000640 1 Processor -- start 2024-04-26T23:09:31.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.897+0000 7f43cd000640 1 -- start start 2024-04-26T23:09:31.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.897+0000 7f43cd000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8103ba0 0x7f43c819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:31.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.897+0000 7f43cd000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43c81044c0 0x7f43c819b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.897+0000 7f43cd000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43c810cee0 0x7f43c819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.897+0000 7f43cd000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43c8112be0 con 0x7f43c8103ba0 2024-04-26T23:09:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.897+0000 7f43cd000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f43c8112a60 con 0x7f43c81044c0 2024-04-26T23:09:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.897+0000 7f43cd000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f43c8112d60 con 0x7f43c810cee0 2024-04-26T23:09:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.897+0000 7f43c5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8103ba0 0x7f43c819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.897+0000 7f43c5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8103ba0 0x7f43c819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51480/0 (socket says 172.21.15.27:51480) 2024-04-26T23:09:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.897+0000 7f43c5e00640 1 -- 172.21.15.27:0/550235051 learned_addr learned my addr 172.21.15.27:0/550235051 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:09:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.897+0000 7f43c6800640 1 --2- 172.21.15.27:0/550235051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43c810cee0 0x7f43c819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.898+0000 7f43bfe00640 1 --2- 172.21.15.27:0/550235051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43c81044c0 0x7f43c819b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.898+0000 7f43c6800640 1 -- 172.21.15.27:0/550235051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43c81044c0 msgr2=0x7f43c819b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.898+0000 7f43c6800640 1 --2- 172.21.15.27:0/550235051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43c81044c0 0x7f43c819b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.898+0000 7f43c6800640 1 -- 172.21.15.27:0/550235051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8103ba0 msgr2=0x7f43c819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.898+0000 7f43c6800640 1 --2- 172.21.15.27:0/550235051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8103ba0 0x7f43c819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:31.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.898+0000 7f43c6800640 1 -- 172.21.15.27:0/550235051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c81a0330 con 0x7f43c810cee0 2024-04-26T23:09:31.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.898+0000 7f43c5e00640 1 --2- 172.21.15.27:0/550235051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8103ba0 0x7f43c819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:09:31.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.898+0000 7f43c6800640 1 --2- 172.21.15.27:0/550235051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43c810cee0 0x7f43c819fc00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f43c0005fd0 tx=0x7f43c000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:31.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.898+0000 7f43bea00640 1 -- 172.21.15.27:0/550235051 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c000cd80 con 0x7f43c810cee0 2024-04-26T23:09:31.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.899+0000 7f43bea00640 1 -- 172.21.15.27:0/550235051 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43c0005020 con 0x7f43c810cee0 2024-04-26T23:09:31.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.899+0000 7f43bea00640 1 -- 172.21.15.27:0/550235051 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0010690 con 0x7f43c810cee0 2024-04-26T23:09:31.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.899+0000 7f43cd000640 1 -- 172.21.15.27:0/550235051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c81a0620 con 0x7f43c810cee0 2024-04-26T23:09:31.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.899+0000 7f43cd000640 1 -- 172.21.15.27:0/550235051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c81a7eb0 con 0x7f43c810cee0 2024-04-26T23:09:31.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.900+0000 7f43cd000640 1 -- 172.21.15.27:0/550235051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c80697e0 con 0x7f43c810cee0 2024-04-26T23:09:31.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.902+0000 7f43bea00640 1 -- 172.21.15.27:0/550235051 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f43c00054d0 con 0x7f43c810cee0 2024-04-26T23:09:31.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.903+0000 7f43bea00640 1 --2- 172.21.15.27:0/550235051 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f43ac0a3da0 0x7f43ac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:31.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.903+0000 7f43bea00640 1 -- 172.21.15.27:0/550235051 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9870+0+0 (secure 0 0 0) 0x7f43c00d1290 con 0x7f43c810cee0 2024-04-26T23:09:31.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.904+0000 7f43c5e00640 1 --2- 172.21.15.27:0/550235051 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f43ac0a3da0 0x7f43ac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:31.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.904+0000 7f43c5e00640 1 --2- 172.21.15.27:0/550235051 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f43ac0a3da0 0x7f43ac0a6260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f43b80096c0 tx=0x7f43b8009260 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:31.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:31.907+0000 7f43bea00640 1 -- 172.21.15.27:0/550235051 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f43c009d390 con 0x7f43c810cee0 2024-04-26T23:09:31.986 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 23:09:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-c[63511]: ignoring --setuser ceph since I am not root 2024-04-26T23:09:31.986 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 23:09:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-c[63511]: ignoring --setgroup ceph since I am not root 2024-04-26T23:09:31.987 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 23:09:31 smithi094 ceph-mds[63515]: -- 172.21.15.94:0/743752 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561627177a40 con 0x56162714b000 2024-04-26T23:09:31.987 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 23:09:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-c[63511]: starting mds.c at 2024-04-26T23:09:31.987 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 23:09:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-i[63824]: ignoring --setuser ceph since I am not root 2024-04-26T23:09:31.987 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 23:09:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-i[63824]: ignoring --setgroup ceph since I am not root 2024-04-26T23:09:31.987 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 23:09:31 smithi094 ceph-mds[63828]: -- 172.21.15.94:0/3478256960 <== mon.1 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d02f501a40 con 0x55d02f4d4800 2024-04-26T23:09:31.987 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 23:09:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-i[63824]: starting mds.i at 2024-04-26T23:09:31.988 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 23:09:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-b[63021]: ignoring --setuser ceph since I am not root 2024-04-26T23:09:31.988 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 23:09:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-b[63021]: ignoring --setgroup ceph since I am not root 2024-04-26T23:09:31.988 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 23:09:31 smithi080 ceph-mds[63025]: -- 172.21.15.80:0/1238887057 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eebb7a3a40 con 0x55eebb777000 2024-04-26T23:09:31.988 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 23:09:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-b[63021]: starting mds.b at 2024-04-26T23:09:31.988 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 23:09:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-h[62709]: ignoring --setuser ceph since I am not root 2024-04-26T23:09:31.988 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 23:09:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-h[62709]: ignoring --setgroup ceph since I am not root 2024-04-26T23:09:31.988 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 23:09:31 smithi080 ceph-mds[62713]: -- 172.21.15.80:0/76032747 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55573507ba40 con 0x55573504ec00 2024-04-26T23:09:31.988 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 23:09:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-h[62709]: starting mds.h at 2024-04-26T23:09:32.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.145+0000 7f43cd000640 1 -- 172.21.15.27:0/550235051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f43c81a81e0 con 0x7f43c810cee0 2024-04-26T23:09:32.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:31 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/175762157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:09:32.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:31 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1537503850' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T23:09:32.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:31 smithi080 ceph-mon[32324]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T23:09:32.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:31 smithi080 ceph-mon[32324]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:09:32.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:31 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/1537503850' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T23:09:32.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:31 smithi080 ceph-mon[32324]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T23:09:32.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:31 smithi080 ceph-mon[32324]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T23:09:32.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:31 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T23:09:32.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:31 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/175762157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:09:32.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:31 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1537503850' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T23:09:32.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:31 smithi094 ceph-mon[32889]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T23:09:32.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:31 smithi094 ceph-mon[32889]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:09:32.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:31 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/1537503850' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T23:09:32.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:31 smithi094 ceph-mon[32889]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T23:09:32.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:31 smithi094 ceph-mon[32889]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T23:09:32.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:31 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T23:09:32.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/175762157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:09:32.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1537503850' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T23:09:32.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-mon[28034]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T23:09:32.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-mon[28034]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:09:32.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/1537503850' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T23:09:32.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-mon[28034]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T23:09:32.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-mon[28034]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T23:09:32.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:31 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T23:09:32.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.672+0000 7f43bea00640 1 -- 172.21.15.27:0/550235051 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7f43c009cc90 con 0x7f43c810cee0 2024-04-26T23:09:32.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.674+0000 7f43cd000640 1 -- 172.21.15.27:0/550235051 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f43ac0a3da0 msgr2=0x7f43ac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:32.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.674+0000 7f43cd000640 1 --2- 172.21.15.27:0/550235051 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f43ac0a3da0 0x7f43ac0a6260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f43b80096c0 tx=0x7f43b8009260 comp rx=0 tx=0).stop 2024-04-26T23:09:32.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.674+0000 7f43cd000640 1 -- 172.21.15.27:0/550235051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43c810cee0 msgr2=0x7f43c819fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:32.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.674+0000 7f43cd000640 1 --2- 172.21.15.27:0/550235051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43c810cee0 0x7f43c819fc00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f43c0005fd0 tx=0x7f43c000ee60 comp rx=0 tx=0).stop 2024-04-26T23:09:32.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.675+0000 7f43cd000640 1 -- 172.21.15.27:0/550235051 shutdown_connections 2024-04-26T23:09:32.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.675+0000 7f43cd000640 1 --2- 172.21.15.27:0/550235051 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7f43ac0a3da0 0x7f43ac0a6260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:32.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.675+0000 7f43cd000640 1 --2- 172.21.15.27:0/550235051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43c81044c0 0x7f43c819b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:32.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.675+0000 7f43cd000640 1 --2- 172.21.15.27:0/550235051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43c810cee0 0x7f43c819fc00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:32.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.675+0000 7f43cd000640 1 --2- 172.21.15.27:0/550235051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8103ba0 0x7f43c819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:32.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.675+0000 7f43cd000640 1 -- 172.21.15.27:0/550235051 >> 172.21.15.27:0/550235051 conn(0x7f43c80789e0 msgr2=0x7f43c8108e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:32.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.675+0000 7f43cd000640 1 -- 172.21.15.27:0/550235051 shutdown_connections 2024-04-26T23:09:32.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.675+0000 7f43cd000640 1 -- 172.21.15.27:0/550235051 wait complete. 2024-04-26T23:09:32.703 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:09:32.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.896+0000 7fb667e00640 1 -- 172.21.15.27:0/4196119255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb66810efa0 msgr2=0x7fb668111390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:32.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.896+0000 7fb667e00640 1 --2- 172.21.15.27:0/4196119255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb66810efa0 0x7fb668111390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb660009860 tx=0x7fb66002a3d0 comp rx=0 tx=0).stop 2024-04-26T23:09:32.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.896+0000 7fb667e00640 1 -- 172.21.15.27:0/4196119255 shutdown_connections 2024-04-26T23:09:32.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.896+0000 7fb667e00640 1 --2- 172.21.15.27:0/4196119255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb66810efa0 0x7fb668111390 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:32.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.896+0000 7fb667e00640 1 --2- 172.21.15.27:0/4196119255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb668075600 0x7fb66810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:32.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.896+0000 7fb667e00640 1 --2- 172.21.15.27:0/4196119255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb668074ce0 0x7fb6680750c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:32.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.896+0000 7fb667e00640 1 -- 172.21.15.27:0/4196119255 >> 172.21.15.27:0/4196119255 conn(0x7fb6680fe1d0 msgr2=0x7fb6681005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:32.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.897+0000 7fb667e00640 1 -- 172.21.15.27:0/4196119255 shutdown_connections 2024-04-26T23:09:32.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.897+0000 7fb667e00640 1 -- 172.21.15.27:0/4196119255 wait complete. 2024-04-26T23:09:32.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.897+0000 7fb667e00640 1 Processor -- start 2024-04-26T23:09:32.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.898+0000 7fb667e00640 1 -- start start 2024-04-26T23:09:32.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.898+0000 7fb667e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb668074ce0 0x7fb66819b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:32.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.898+0000 7fb667e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb668075600 0x7fb66819b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:32.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.898+0000 7fb667e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb66810efa0 0x7fb66819f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:32.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.898+0000 7fb667e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb668114bc0 con 0x7fb66810efa0 2024-04-26T23:09:32.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.898+0000 7fb667e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb668114a40 con 0x7fb668075600 2024-04-26T23:09:32.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.898+0000 7fb667e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb668114d40 con 0x7fb668074ce0 2024-04-26T23:09:32.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.898+0000 7fb666000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb668075600 0x7fb66819b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:32.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.898+0000 7fb666a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb668074ce0 0x7fb66819b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:32.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.898+0000 7fb666a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb668074ce0 0x7fb66819b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:47402/0 (socket says 172.21.15.27:47402) 2024-04-26T23:09:32.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.898+0000 7fb666a00640 1 -- 172.21.15.27:0/807973177 learned_addr learned my addr 172.21.15.27:0/807973177 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:09:32.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.898+0000 7fb667400640 1 --2- 172.21.15.27:0/807973177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb66810efa0 0x7fb66819f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:32.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.899+0000 7fb666a00640 1 -- 172.21.15.27:0/807973177 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb668075600 msgr2=0x7fb66819b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:32.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.899+0000 7fb666a00640 1 --2- 172.21.15.27:0/807973177 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb668075600 0x7fb66819b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:32.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.899+0000 7fb666a00640 1 -- 172.21.15.27:0/807973177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb66810efa0 msgr2=0x7fb66819f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:32.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.899+0000 7fb666a00640 1 --2- 172.21.15.27:0/807973177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb66810efa0 0x7fb66819f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:32.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.899+0000 7fb666a00640 1 -- 172.21.15.27:0/807973177 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6681a00e0 con 0x7fb668074ce0 2024-04-26T23:09:32.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.899+0000 7fb666a00640 1 --2- 172.21.15.27:0/807973177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb668074ce0 0x7fb66819b140 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb65800c420 tx=0x7fb65800c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:32.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.900+0000 7fb65f400640 1 -- 172.21.15.27:0/807973177 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658004b20 con 0x7fb668074ce0 2024-04-26T23:09:32.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.900+0000 7fb667e00640 1 -- 172.21.15.27:0/807973177 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6681a03d0 con 0x7fb668074ce0 2024-04-26T23:09:32.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.900+0000 7fb65f400640 1 -- 172.21.15.27:0/807973177 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb658003e20 con 0x7fb668074ce0 2024-04-26T23:09:32.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.900+0000 7fb667e00640 1 -- 172.21.15.27:0/807973177 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6681a7c60 con 0x7fb668074ce0 2024-04-26T23:09:32.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.900+0000 7fb65f400640 1 -- 172.21.15.27:0/807973177 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65801a020 con 0x7fb668074ce0 2024-04-26T23:09:32.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.901+0000 7fb667e00640 1 -- 172.21.15.27:0/807973177 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6680506c0 con 0x7fb668074ce0 2024-04-26T23:09:32.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.903+0000 7fb65f400640 1 -- 172.21.15.27:0/807973177 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb65801d020 con 0x7fb668074ce0 2024-04-26T23:09:32.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.904+0000 7fb65f400640 1 --2- 172.21.15.27:0/807973177 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fb64c0a3da0 0x7fb64c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:32.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.905+0000 7fb65f400640 1 -- 172.21.15.27:0/807973177 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9870+0+0 (secure 0 0 0) 0x7fb6580d0df0 con 0x7fb668074ce0 2024-04-26T23:09:32.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.905+0000 7fb666000640 1 --2- 172.21.15.27:0/807973177 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fb64c0a3da0 0x7fb64c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:32.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.905+0000 7fb666000640 1 --2- 172.21.15.27:0/807973177 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fb64c0a3da0 0x7fb64c0a6260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb668100860 tx=0x7fb650005f90 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:32.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:32.907+0000 7fb65f400640 1 -- 172.21.15.27:0/807973177 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb65809cef0 con 0x7fb668074ce0 2024-04-26T23:09:33.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:33.110+0000 7fb667e00640 1 -- 172.21.15.27:0/807973177 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb66810a120 con 0x7fb668074ce0 2024-04-26T23:09:33.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: pgmap v340: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T23:09:33.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/550235051' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T23:09:33.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T23:09:33.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T23:09:33.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T23:09:33.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: Cluster is now healthy 2024-04-26T23:09:33.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T23:09:33.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.94:6836/1441178071,v1:172.21.15.94:6838/1441178071] up:boot 2024-04-26T23:09:33.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.94:6837/984369057,v1:172.21.15.94:6839/984369057] up:boot 2024-04-26T23:09:33.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.80:6836/3304339799,v1:172.21.15.80:6838/3304339799] up:boot 2024-04-26T23:09:33.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.80:6837/1290477759,v1:172.21.15.80:6839/1290477759] up:boot 2024-04-26T23:09:33.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.27:6838/3130504128,v1:172.21.15.27:6839/3130504128] up:boot 2024-04-26T23:09:33.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: mds.? [v2:172.21.15.27:6840/764653759,v1:172.21.15.27:6841/764653759] up:boot 2024-04-26T23:09:33.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: fsmap 12 up:standby 2024-04-26T23:09:33.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T23:09:33.272 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T23:09:33.273 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T23:09:33.273 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T23:09:33.273 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T23:09:33.273 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:32 smithi080 ceph-mon[32324]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T23:09:33.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: pgmap v340: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T23:09:33.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/550235051' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T23:09:33.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T23:09:33.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T23:09:33.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T23:09:33.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: Cluster is now healthy 2024-04-26T23:09:33.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T23:09:33.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.94:6836/1441178071,v1:172.21.15.94:6838/1441178071] up:boot 2024-04-26T23:09:33.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.94:6837/984369057,v1:172.21.15.94:6839/984369057] up:boot 2024-04-26T23:09:33.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.80:6836/3304339799,v1:172.21.15.80:6838/3304339799] up:boot 2024-04-26T23:09:33.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.80:6837/1290477759,v1:172.21.15.80:6839/1290477759] up:boot 2024-04-26T23:09:33.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.27:6838/3130504128,v1:172.21.15.27:6839/3130504128] up:boot 2024-04-26T23:09:33.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: mds.? [v2:172.21.15.27:6840/764653759,v1:172.21.15.27:6841/764653759] up:boot 2024-04-26T23:09:33.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: fsmap 12 up:standby 2024-04-26T23:09:33.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T23:09:33.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T23:09:33.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T23:09:33.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T23:09:33.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T23:09:33.286 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:32 smithi094 ceph-mon[32889]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T23:09:33.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: pgmap v340: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T23:09:33.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/550235051' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T23:09:33.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T23:09:33.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T23:09:33.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T23:09:33.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: Cluster is now healthy 2024-04-26T23:09:33.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T23:09:33.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.94:6836/1441178071,v1:172.21.15.94:6838/1441178071] up:boot 2024-04-26T23:09:33.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.94:6837/984369057,v1:172.21.15.94:6839/984369057] up:boot 2024-04-26T23:09:33.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.80:6836/3304339799,v1:172.21.15.80:6838/3304339799] up:boot 2024-04-26T23:09:33.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.80:6837/1290477759,v1:172.21.15.80:6839/1290477759] up:boot 2024-04-26T23:09:33.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.27:6838/3130504128,v1:172.21.15.27:6839/3130504128] up:boot 2024-04-26T23:09:33.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: mds.? [v2:172.21.15.27:6840/764653759,v1:172.21.15.27:6841/764653759] up:boot 2024-04-26T23:09:33.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: fsmap 12 up:standby 2024-04-26T23:09:33.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T23:09:33.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T23:09:33.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T23:09:33.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T23:09:33.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T23:09:33.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:32 smithi027 ceph-mon[28034]: from='mgr.25391 172.21.15.27:0/2786712023' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T23:09:33.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:33.996+0000 7fb65f400640 1 -- 172.21.15.27:0/807973177 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) ==== 181+0+0 (secure 0 0 0) 0x7fb65809c7f0 con 0x7fb668074ce0 2024-04-26T23:09:34.001 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:09:34.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.003+0000 7fb667e00640 1 -- 172.21.15.27:0/807973177 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fb64c0a3da0 msgr2=0x7fb64c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:34.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.003+0000 7fb667e00640 1 --2- 172.21.15.27:0/807973177 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fb64c0a3da0 0x7fb64c0a6260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb668100860 tx=0x7fb650005f90 comp rx=0 tx=0).stop 2024-04-26T23:09:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.003+0000 7fb667e00640 1 -- 172.21.15.27:0/807973177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb668074ce0 msgr2=0x7fb66819b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.003+0000 7fb667e00640 1 --2- 172.21.15.27:0/807973177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb668074ce0 0x7fb66819b140 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb65800c420 tx=0x7fb65800c8e0 comp rx=0 tx=0).stop 2024-04-26T23:09:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.003+0000 7fb667e00640 1 -- 172.21.15.27:0/807973177 shutdown_connections 2024-04-26T23:09:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.003+0000 7fb667e00640 1 --2- 172.21.15.27:0/807973177 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fb64c0a3da0 0x7fb64c0a6260 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.003+0000 7fb667e00640 1 --2- 172.21.15.27:0/807973177 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb668075600 0x7fb66819b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.003+0000 7fb667e00640 1 --2- 172.21.15.27:0/807973177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb66810efa0 0x7fb66819f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:34.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.003+0000 7fb667e00640 1 --2- 172.21.15.27:0/807973177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb668074ce0 0x7fb66819b140 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:34.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.003+0000 7fb667e00640 1 -- 172.21.15.27:0/807973177 >> 172.21.15.27:0/807973177 conn(0x7fb6680fe1d0 msgr2=0x7fb6680ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:34.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.004+0000 7fb667e00640 1 -- 172.21.15.27:0/807973177 shutdown_connections 2024-04-26T23:09:34.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.004+0000 7fb667e00640 1 -- 172.21.15.27:0/807973177 wait complete. 2024-04-26T23:09:34.031 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:09:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.240+0000 7fd11be00640 1 -- 172.21.15.27:0/286994578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd11c10efa0 msgr2=0x7fd11c1113b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.240+0000 7fd11be00640 1 --2- 172.21.15.27:0/286994578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd11c10efa0 0x7fd11c1113b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd114009ca0 tx=0x7fd11402a950 comp rx=0 tx=0).stop 2024-04-26T23:09:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.240+0000 7fd11be00640 1 -- 172.21.15.27:0/286994578 shutdown_connections 2024-04-26T23:09:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.240+0000 7fd11be00640 1 --2- 172.21.15.27:0/286994578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd11c075620 0x7fd11c10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.240+0000 7fd11be00640 1 --2- 172.21.15.27:0/286994578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd11c10efa0 0x7fd11c1113b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.240+0000 7fd11be00640 1 --2- 172.21.15.27:0/286994578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd11c074d00 0x7fd11c0750e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.240+0000 7fd11be00640 1 -- 172.21.15.27:0/286994578 >> 172.21.15.27:0/286994578 conn(0x7fd11c0fe210 msgr2=0x7fd11c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.240+0000 7fd11be00640 1 -- 172.21.15.27:0/286994578 shutdown_connections 2024-04-26T23:09:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.240+0000 7fd11be00640 1 -- 172.21.15.27:0/286994578 wait complete. 2024-04-26T23:09:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.241+0000 7fd11be00640 1 Processor -- start 2024-04-26T23:09:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.242+0000 7fd11be00640 1 -- start start 2024-04-26T23:09:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.242+0000 7fd11be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd11c074d00 0x7fd11c19b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.242+0000 7fd11be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd11c075620 0x7fd11c19b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.242+0000 7fd11be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd11c10efa0 0x7fd11c19fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.242+0000 7fd11be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c114bb0 con 0x7fd11c074d00 2024-04-26T23:09:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.242+0000 7fd11be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c114a30 con 0x7fd11c10efa0 2024-04-26T23:09:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.242+0000 7fd11be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c114d30 con 0x7fd11c075620 2024-04-26T23:09:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.242+0000 7fd11aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd11c074d00 0x7fd11c19b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.242+0000 7fd11aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd11c074d00 0x7fd11c19b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51516/0 (socket says 172.21.15.27:51516) 2024-04-26T23:09:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.242+0000 7fd11aa00640 1 -- 172.21.15.27:0/2719270607 learned_addr learned my addr 172.21.15.27:0/2719270607 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:09:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.242+0000 7fd11a000640 1 --2- 172.21.15.27:0/2719270607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd11c075620 0x7fd11c19b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.243+0000 7fd11b400640 1 --2- 172.21.15.27:0/2719270607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd11c10efa0 0x7fd11c19fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.243+0000 7fd11aa00640 1 -- 172.21.15.27:0/2719270607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd11c075620 msgr2=0x7fd11c19b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.243+0000 7fd11aa00640 1 --2- 172.21.15.27:0/2719270607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd11c075620 0x7fd11c19b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.243+0000 7fd11aa00640 1 -- 172.21.15.27:0/2719270607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd11c10efa0 msgr2=0x7fd11c19fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.243+0000 7fd11aa00640 1 --2- 172.21.15.27:0/2719270607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd11c10efa0 0x7fd11c19fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.243+0000 7fd11aa00640 1 -- 172.21.15.27:0/2719270607 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11c1a0140 con 0x7fd11c074d00 2024-04-26T23:09:34.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.243+0000 7fd11a000640 1 --2- 172.21.15.27:0/2719270607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd11c075620 0x7fd11c19b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:09:34.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.243+0000 7fd11aa00640 1 --2- 172.21.15.27:0/2719270607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd11c074d00 0x7fd11c19b150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00ea60 tx=0x7fd10c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:34.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.243+0000 7fd113400640 1 -- 172.21.15.27:0/2719270607 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c00cde0 con 0x7fd11c074d00 2024-04-26T23:09:34.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.244+0000 7fd113400640 1 -- 172.21.15.27:0/2719270607 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd10c0035f0 con 0x7fd11c074d00 2024-04-26T23:09:34.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.244+0000 7fd113400640 1 -- 172.21.15.27:0/2719270607 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c010690 con 0x7fd11c074d00 2024-04-26T23:09:34.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.244+0000 7fd11be00640 1 -- 172.21.15.27:0/2719270607 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11c1a0430 con 0x7fd11c074d00 2024-04-26T23:09:34.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.244+0000 7fd11be00640 1 -- 172.21.15.27:0/2719270607 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd11c1a7cc0 con 0x7fd11c074d00 2024-04-26T23:09:34.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.245+0000 7fd11be00640 1 -- 172.21.15.27:0/2719270607 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd11c0506c0 con 0x7fd11c074d00 2024-04-26T23:09:34.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.248+0000 7fd113400640 1 -- 172.21.15.27:0/2719270607 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd10c0037a0 con 0x7fd11c074d00 2024-04-26T23:09:34.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.251+0000 7fd113400640 1 --2- 172.21.15.27:0/2719270607 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd1000a3da0 0x7fd1000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:34.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.251+0000 7fd113400640 1 -- 172.21.15.27:0/2719270607 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9308+0+0 (secure 0 0 0) 0x7fd10c0d0d80 con 0x7fd11c074d00 2024-04-26T23:09:34.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.251+0000 7fd11a000640 1 --2- 172.21.15.27:0/2719270607 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd1000a3da0 0x7fd1000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:34.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.252+0000 7fd11a000640 1 --2- 172.21.15.27:0/2719270607 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd1000a3da0 0x7fd1000a6260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd1080095f0 tx=0x7fd108005f90 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:34.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.258+0000 7fd113400640 1 -- 172.21.15.27:0/2719270607 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd10c09d0f0 con 0x7fd11c074d00 2024-04-26T23:09:34.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:34 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/807973177' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T23:09:34.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:34 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T23:09:34.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:34 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/807973177' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T23:09:34.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:34 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T23:09:34.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:34 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/807973177' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T23:09:34.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:34 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T23:09:34.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:34.472+0000 7fd11be00640 1 -- 172.21.15.27:0/2719270607 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd11c19c4a0 con 0x7fd11c074d00 2024-04-26T23:09:35.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.008+0000 7fd113400640 1 -- 172.21.15.27:0/2719270607 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) ==== 169+0+0 (secure 0 0 0) 0x7fd10c09c9f0 con 0x7fd11c074d00 2024-04-26T23:09:35.009 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-04-26T23:09:35.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.012+0000 7fd11be00640 1 -- 172.21.15.27:0/2719270607 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd1000a3da0 msgr2=0x7fd1000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:35.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.012+0000 7fd11be00640 1 --2- 172.21.15.27:0/2719270607 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd1000a3da0 0x7fd1000a6260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd1080095f0 tx=0x7fd108005f90 comp rx=0 tx=0).stop 2024-04-26T23:09:35.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.012+0000 7fd11be00640 1 -- 172.21.15.27:0/2719270607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd11c074d00 msgr2=0x7fd11c19b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:35.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.012+0000 7fd11be00640 1 --2- 172.21.15.27:0/2719270607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd11c074d00 0x7fd11c19b150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00ea60 tx=0x7fd10c00ef20 comp rx=0 tx=0).stop 2024-04-26T23:09:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.013+0000 7fd11be00640 1 -- 172.21.15.27:0/2719270607 shutdown_connections 2024-04-26T23:09:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.013+0000 7fd11be00640 1 --2- 172.21.15.27:0/2719270607 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7fd1000a3da0 0x7fd1000a6260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.013+0000 7fd11be00640 1 --2- 172.21.15.27:0/2719270607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd11c10efa0 0x7fd11c19fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.013+0000 7fd11be00640 1 --2- 172.21.15.27:0/2719270607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd11c075620 0x7fd11c19b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.013+0000 7fd11be00640 1 --2- 172.21.15.27:0/2719270607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd11c074d00 0x7fd11c19b150 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.013+0000 7fd11be00640 1 -- 172.21.15.27:0/2719270607 >> 172.21.15.27:0/2719270607 conn(0x7fd11c0fe210 msgr2=0x7fd11c0ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.013+0000 7fd11be00640 1 -- 172.21.15.27:0/2719270607 shutdown_connections 2024-04-26T23:09:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.013+0000 7fd11be00640 1 -- 172.21.15.27:0/2719270607 wait complete. 2024-04-26T23:09:35.042 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-26T23:09:35.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.251+0000 7ff343e00640 1 -- 172.21.15.27:0/1377487306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff344108fa0 msgr2=0x7ff344109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:35.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.251+0000 7ff343e00640 1 --2- 172.21.15.27:0/1377487306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff344108fa0 0x7ff344109380 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff334009900 tx=0x7ff33402a3f0 comp rx=0 tx=0).stop 2024-04-26T23:09:35.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.251+0000 7ff343e00640 1 -- 172.21.15.27:0/1377487306 shutdown_connections 2024-04-26T23:09:35.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.251+0000 7ff343e00640 1 --2- 172.21.15.27:0/1377487306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff34410e060 0x7ff344115c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:35.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.251+0000 7ff343e00640 1 --2- 172.21.15.27:0/1377487306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff344109950 0x7ff34410d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:35.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.251+0000 7ff343e00640 1 --2- 172.21.15.27:0/1377487306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff344108fa0 0x7ff344109380 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:35.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.251+0000 7ff343e00640 1 -- 172.21.15.27:0/1377487306 >> 172.21.15.27:0/1377487306 conn(0x7ff3440784b0 msgr2=0x7ff34407a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:35.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.251+0000 7ff343e00640 1 -- 172.21.15.27:0/1377487306 shutdown_connections 2024-04-26T23:09:35.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.251+0000 7ff343e00640 1 -- 172.21.15.27:0/1377487306 wait complete. 2024-04-26T23:09:35.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.252+0000 7ff343e00640 1 Processor -- start 2024-04-26T23:09:35.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.252+0000 7ff343e00640 1 -- start start 2024-04-26T23:09:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff343e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff344108fa0 0x7ff34419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff343e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff34410e060 0x7ff34419fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff343e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3441a3e10 0x7ff3441a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff343e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff344118fc0 con 0x7ff3441a3e10 2024-04-26T23:09:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff343e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff344118e40 con 0x7ff34410e060 2024-04-26T23:09:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff343e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff344119140 con 0x7ff344108fa0 2024-04-26T23:09:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff343400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3441a3e10 0x7ff3441a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff342000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff34410e060 0x7ff34419fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff342000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff34410e060 0x7ff34419fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.27:47902/0 (socket says 172.21.15.27:47902) 2024-04-26T23:09:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff342000640 1 -- 172.21.15.27:0/2487833363 learned_addr learned my addr 172.21.15.27:0/2487833363 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T23:09:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff343400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3441a3e10 0x7ff3441a42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51538/0 (socket says 172.21.15.27:51538) 2024-04-26T23:09:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff342a00640 1 --2- 172.21.15.27:0/2487833363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff344108fa0 0x7ff34419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff342000640 1 -- 172.21.15.27:0/2487833363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff344108fa0 msgr2=0x7ff34419f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:35.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff342000640 1 --2- 172.21.15.27:0/2487833363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff344108fa0 0x7ff34419f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:35.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff342000640 1 -- 172.21.15.27:0/2487833363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3441a3e10 msgr2=0x7ff3441a42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:35.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff342000640 1 --2- 172.21.15.27:0/2487833363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3441a3e10 0x7ff3441a42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:35.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff342000640 1 -- 172.21.15.27:0/2487833363 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3441a4900 con 0x7ff34410e060 2024-04-26T23:09:35.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.253+0000 7ff343400640 1 --2- 172.21.15.27:0/2487833363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3441a3e10 0x7ff3441a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:09:35.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.254+0000 7ff342000640 1 --2- 172.21.15.27:0/2487833363 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff34410e060 0x7ff34419fae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff33000e9f0 tx=0x7ff33000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:35.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.254+0000 7ff33b400640 1 -- 172.21.15.27:0/2487833363 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33000ccc0 con 0x7ff34410e060 2024-04-26T23:09:35.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.254+0000 7ff343e00640 1 -- 172.21.15.27:0/2487833363 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3441a4bf0 con 0x7ff34410e060 2024-04-26T23:09:35.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.254+0000 7ff33b400640 1 -- 172.21.15.27:0/2487833363 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3300054d0 con 0x7ff34410e060 2024-04-26T23:09:35.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.254+0000 7ff33b400640 1 -- 172.21.15.27:0/2487833363 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330010430 con 0x7ff34410e060 2024-04-26T23:09:35.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.254+0000 7ff343e00640 1 -- 172.21.15.27:0/2487833363 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3441ac3e0 con 0x7ff34410e060 2024-04-26T23:09:35.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.255+0000 7ff343e00640 1 -- 172.21.15.27:0/2487833363 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3440506c0 con 0x7ff34410e060 2024-04-26T23:09:35.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.257+0000 7ff33b400640 1 -- 172.21.15.27:0/2487833363 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff3300046c0 con 0x7ff34410e060 2024-04-26T23:09:35.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.259+0000 7ff33b400640 1 --2- 172.21.15.27:0/2487833363 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7ff3280a3d20 0x7ff3280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:09:35.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.259+0000 7ff33b400640 1 -- 172.21.15.27:0/2487833363 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8625+0+0 (secure 0 0 0) 0x7ff3300d0c20 con 0x7ff34410e060 2024-04-26T23:09:35.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.259+0000 7ff342a00640 1 --2- 172.21.15.27:0/2487833363 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7ff3280a3d20 0x7ff3280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:09:35.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.260+0000 7ff342a00640 1 --2- 172.21.15.27:0/2487833363 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7ff3280a3d20 0x7ff3280a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff334031440 tx=0x7ff334005b50 comp rx=0 tx=0).ready entity=mgr.25391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:09:35.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.261+0000 7ff33b400640 1 -- 172.21.15.27:0/2487833363 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff33009d1f0 con 0x7ff34410e060 2024-04-26T23:09:35.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:35 smithi080 ceph-mon[32324]: pgmap v341: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T23:09:35.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:35 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T23:09:35.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:35 smithi080 ceph-mon[32324]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T23:09:35.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:35 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2719270607' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T23:09:35.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:35 smithi094 ceph-mon[32889]: pgmap v341: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T23:09:35.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:35 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T23:09:35.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:35 smithi094 ceph-mon[32889]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T23:09:35.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:35 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2719270607' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T23:09:35.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:35 smithi027 ceph-mon[28034]: pgmap v341: 129 pgs: 129 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T23:09:35.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:35 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T23:09:35.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:35 smithi027 ceph-mon[28034]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T23:09:35.365 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:35 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2719270607' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T23:09:35.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:35.470+0000 7ff343e00640 1 -- 172.21.15.27:0/2487833363 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff3441a08a0 con 0x7ff34410e060 2024-04-26T23:09:36.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:36.036+0000 7ff33b400640 1 -- 172.21.15.27:0/2487833363 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v107) ==== 178+0+0 (secure 0 0 0) 0x7ff33009caf0 con 0x7ff34410e060 2024-04-26T23:09:36.038 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data_ec' removed 2024-04-26T23:09:36.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:36.042+0000 7ff343e00640 1 -- 172.21.15.27:0/2487833363 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7ff3280a3d20 msgr2=0x7ff3280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:36.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:36.042+0000 7ff343e00640 1 --2- 172.21.15.27:0/2487833363 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7ff3280a3d20 0x7ff3280a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff334031440 tx=0x7ff334005b50 comp rx=0 tx=0).stop 2024-04-26T23:09:36.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:36.042+0000 7ff343e00640 1 -- 172.21.15.27:0/2487833363 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff34410e060 msgr2=0x7ff34419fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:09:36.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:36.042+0000 7ff343e00640 1 --2- 172.21.15.27:0/2487833363 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff34410e060 0x7ff34419fae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff33000e9f0 tx=0x7ff33000edf0 comp rx=0 tx=0).stop 2024-04-26T23:09:36.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:36.042+0000 7ff343e00640 1 -- 172.21.15.27:0/2487833363 shutdown_connections 2024-04-26T23:09:36.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:36.043+0000 7ff343e00640 1 --2- 172.21.15.27:0/2487833363 >> [v2:172.21.15.27:6800/3058395889,v1:172.21.15.27:6801/3058395889] conn(0x7ff3280a3d20 0x7ff3280a61e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:36.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:36.043+0000 7ff343e00640 1 --2- 172.21.15.27:0/2487833363 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff34410e060 0x7ff34419fae0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:36.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:36.043+0000 7ff343e00640 1 --2- 172.21.15.27:0/2487833363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3441a3e10 0x7ff3441a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:36.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:36.043+0000 7ff343e00640 1 --2- 172.21.15.27:0/2487833363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff344108fa0 0x7ff34419f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:09:36.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:36.043+0000 7ff343e00640 1 -- 172.21.15.27:0/2487833363 >> 172.21.15.27:0/2487833363 conn(0x7ff3440784b0 msgr2=0x7ff344079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:09:36.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:36.043+0000 7ff343e00640 1 -- 172.21.15.27:0/2487833363 shutdown_connections 2024-04-26T23:09:36.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T23:09:36.043+0000 7ff343e00640 1 -- 172.21.15.27:0/2487833363 wait complete. 2024-04-26T23:09:36.069 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T23:09:36.138 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T23:09:36.165 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T23:09:36.195 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-26T23:09:36.195 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T23:09:36.257 DEBUG:teuthology.orchestra.run.smithi080:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T23:09:36.278 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T23:09:36.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:36 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2719270607' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T23:09:36.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:36 smithi094 ceph-mon[32889]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T23:09:36.284 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:36 smithi094 ceph-mon[32889]: from='client.? 172.21.15.27:0/2487833363' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T23:09:36.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:36 smithi094 ceph-mon[32889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T23:09:36.292 INFO:tasks.cephadm:Stopping all daemons... 2024-04-26T23:09:36.293 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-26T23:09:36.293 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mon.a 2024-04-26T23:09:36.353 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:36 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2719270607' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T23:09:36.353 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:36 smithi027 ceph-mon[28034]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T23:09:36.353 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:36 smithi027 ceph-mon[28034]: from='client.? 172.21.15.27:0/2487833363' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T23:09:36.353 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:36 smithi027 ceph-mon[28034]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T23:09:36.407 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:36 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2719270607' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T23:09:36.407 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:36 smithi080 ceph-mon[32324]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T23:09:36.408 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:36 smithi080 ceph-mon[32324]: from='client.? 172.21.15.27:0/2487833363' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T23:09:36.408 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:36 smithi080 ceph-mon[32324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T23:09:36.618 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:36 smithi027 systemd[1]: Stopping Ceph mon.a for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:09:37.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:36 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a[28021]: 2024-04-26T23:09:36.832+0000 7faca7a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:09:37.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:36 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a[28021]: 2024-04-26T23:09:36.832+0000 7faca7a00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-26T23:09:37.492 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:37 smithi027 podman[88758]: 2024-04-26 23:09:37.115343603 +0000 UTC m=+0.441927727 container stop a43cb86680a7d01387b3d36eef60300d0e3d9f320cd25e47a15be9876a9c39a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-26T23:09:37.492 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:37 smithi027 podman[88758]: 2024-04-26 23:09:37.117531662 +0000 UTC m=+0.444115768 container died a43cb86680a7d01387b3d36eef60300d0e3d9f320cd25e47a15be9876a9c39a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-04-26T23:09:37.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:37 smithi027 podman[88758]: 2024-04-26 23:09:37.665499767 +0000 UTC m=+0.992083939 container cleanup a43cb86680a7d01387b3d36eef60300d0e3d9f320cd25e47a15be9876a9c39a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T23:09:37.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:37 smithi027 bash[88758]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a 2024-04-26T23:09:38.364 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 23:09:38 smithi027 podman[88772]: 2024-04-26 23:09:38.047778297 +0000 UTC m=+0.909709018 container remove a43cb86680a7d01387b3d36eef60300d0e3d9f320cd25e47a15be9876a9c39a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T23:09:38.601 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mon.a.service' 2024-04-26T23:09:38.748 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:09:38.748 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-26T23:09:38.748 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-26T23:09:38.748 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mon.b 2024-04-26T23:09:39.078 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:38 smithi080 systemd[1]: Stopping Ceph mon.b for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:09:39.438 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:39 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-b[32320]: 2024-04-26T23:09:39.077+0000 7fbd0572e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:09:39.438 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:39 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-b[32320]: 2024-04-26T23:09:39.077+0000 7fbd0572e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-26T23:09:39.438 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:39 smithi080 podman[68962]: 2024-04-26 23:09:39.139500917 +0000 UTC m=+0.193779399 container died fe1b7809b5c131b406023f199bb443aa97a5337ebd1a926c67d61d7aeb9879fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T23:09:39.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:39 smithi080 podman[68962]: 2024-04-26 23:09:39.537255221 +0000 UTC m=+0.591533701 container cleanup fe1b7809b5c131b406023f199bb443aa97a5337ebd1a926c67d61d7aeb9879fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T23:09:39.772 INFO:journalctl@ceph.mon.b.smithi080.stdout:Apr 26 23:09:39 smithi080 bash[68962]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-b 2024-04-26T23:09:39.954 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mon.b.service' 2024-04-26T23:09:39.999 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:09:40.000 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-26T23:09:40.000 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-26T23:09:40.000 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mon.c 2024-04-26T23:09:40.285 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:40 smithi094 systemd[1]: Stopping Ceph mon.c for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:09:40.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:40 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-c[32870]: 2024-04-26T23:09:40.321+0000 7fc1b60a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:09:40.784 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:40 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-c[32870]: 2024-04-26T23:09:40.321+0000 7fc1b60a4700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-26T23:09:40.785 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:40 smithi094 podman[69946]: 2024-04-26 23:09:40.516882796 +0000 UTC m=+0.317686027 container died d9d25be2ae96ac3d9cfbd7c36002f8af5f8dfc687da0eaf188830ccb5897bdfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T23:09:41.192 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:40 smithi094 podman[69946]: 2024-04-26 23:09:40.906929448 +0000 UTC m=+0.707732682 container cleanup d9d25be2ae96ac3d9cfbd7c36002f8af5f8dfc687da0eaf188830ccb5897bdfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T23:09:41.192 INFO:journalctl@ceph.mon.c.smithi094.stdout:Apr 26 23:09:40 smithi094 bash[69946]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mon-c 2024-04-26T23:09:41.334 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mon.c.service' 2024-04-26T23:09:41.374 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:09:41.375 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-26T23:09:41.375 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-26T23:09:41.375 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mgr.x 2024-04-26T23:09:41.865 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 23:09:41 smithi027 systemd[1]: Stopping Ceph mgr.x for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:09:42.237 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 23:09:41 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T23:09:41.980+0000 7fc92ce00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:09:42.237 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 23:09:41 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x[28263]: 2024-04-26T23:09:41.980+0000 7fc92ce00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-26T23:09:42.237 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 23:09:42 smithi027 podman[88879]: 2024-04-26 23:09:42.087908067 +0000 UTC m=+0.268221246 container died a588d685001bb42ecf77c547ae172f4f9a8147f7dfb7cee8fe7317cf0b7f20a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-26T23:09:42.865 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 23:09:42 smithi027 podman[88879]: 2024-04-26 23:09:42.632457057 +0000 UTC m=+0.812770217 container cleanup a588d685001bb42ecf77c547ae172f4f9a8147f7dfb7cee8fe7317cf0b7f20a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5) 2024-04-26T23:09:42.865 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 23:09:42 smithi027 bash[88879]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x 2024-04-26T23:09:43.346 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 23:09:42 smithi027 podman[88893]: 2024-04-26 23:09:42.909336542 +0000 UTC m=+0.804442574 container remove a588d685001bb42ecf77c547ae172f4f9a8147f7dfb7cee8fe7317cf0b7f20a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T23:09:43.374 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-26T23:09:43.494 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:09:43.495 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-26T23:09:43.495 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-26T23:09:43.495 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mgr.y 2024-04-26T23:09:43.771 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 23:09:43 smithi080 systemd[1]: Stopping Ceph mgr.y for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:09:44.134 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 23:09:43 smithi080 podman[69082]: 2024-04-26 23:09:43.830244242 +0000 UTC m=+0.141991803 container died 6fd3b77644f81d3b2a8800e968c119f5c004d9484831e13f822847fcc97712f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T23:09:44.478 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 23:09:44 smithi080 podman[69082]: 2024-04-26 23:09:44.234750798 +0000 UTC m=+0.546498358 container cleanup 6fd3b77644f81d3b2a8800e968c119f5c004d9484831e13f822847fcc97712f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0) 2024-04-26T23:09:44.478 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Apr 26 23:09:44 smithi080 bash[69082]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-y 2024-04-26T23:09:44.604 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-26T23:09:44.637 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:09:44.638 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-26T23:09:44.638 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-26T23:09:44.638 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mgr.z 2024-04-26T23:09:44.984 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 23:09:44 smithi094 systemd[1]: Stopping Ceph mgr.z for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:09:45.285 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 23:09:44 smithi094 podman[70067]: 2024-04-26 23:09:44.984886893 +0000 UTC m=+0.152598700 container died ea2b3c353262876eabbeac395ae055f2898ce94f834871f72cb0eeeea4ca15a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T23:09:45.647 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 23:09:45 smithi094 podman[70067]: 2024-04-26 23:09:45.404764071 +0000 UTC m=+0.572475875 container cleanup ea2b3c353262876eabbeac395ae055f2898ce94f834871f72cb0eeeea4ca15a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T23:09:45.647 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 23:09:45 smithi094 bash[70067]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z 2024-04-26T23:09:45.880 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-26T23:09:45.900 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 23:09:45 smithi094 podman[70081]: 2024-04-26 23:09:45.648083817 +0000 UTC m=+0.662235213 container remove ea2b3c353262876eabbeac395ae055f2898ce94f834871f72cb0eeeea4ca15a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T23:09:45.900 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 23:09:45 smithi094 systemd[1]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-26T23:09:45.900 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 23:09:45 smithi094 systemd[1]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-04-26T23:09:45.900 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 23:09:45 smithi094 systemd[1]: Stopped Ceph mgr.z for 95ebdda6-041e-11ef-bc93-c7b262605968. 2024-04-26T23:09:45.900 INFO:journalctl@ceph.mgr.z.smithi094.stdout:Apr 26 23:09:45 smithi094 systemd[1]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mgr.z.service: Consumed 20.825s CPU time. 2024-04-26T23:09:45.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:09:45.913 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-26T23:09:45.913 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-26T23:09:45.913 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.0 2024-04-26T23:09:46.320 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 23:09:46 smithi027 systemd[1]: Stopping Ceph osd.0 for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:09:46.615 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 23:09:46 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-0[42699]: 2024-04-26T23:09:46.511+0000 7f58d8000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:09:46.615 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 23:09:46 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-0[42699]: 2024-04-26T23:09:46.511+0000 7f58d8000700 -1 osd.0 107 *** Got signal Terminated *** 2024-04-26T23:09:46.615 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 23:09:46 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-0[42699]: 2024-04-26T23:09:46.511+0000 7f58d8000700 -1 osd.0 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T23:09:52.511 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 23:09:52 smithi027 podman[89001]: 2024-04-26 23:09:52.145790762 +0000 UTC m=+5.777079103 container died c65a63d6aba5dd64122fb3c43a3d8e6f6187904454bcbd79b81a61dd56f71e30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T23:09:52.865 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 23:09:52 smithi027 podman[89001]: 2024-04-26 23:09:52.688274613 +0000 UTC m=+6.319562977 container cleanup c65a63d6aba5dd64122fb3c43a3d8e6f6187904454bcbd79b81a61dd56f71e30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T23:09:52.865 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 23:09:52 smithi027 bash[89001]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-0 2024-04-26T23:09:53.367 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 23:09:52 smithi027 podman[89030]: 2024-04-26 23:09:52.898072803 +0000 UTC m=+0.736286383 container remove c65a63d6aba5dd64122fb3c43a3d8e6f6187904454bcbd79b81a61dd56f71e30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-0, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-26T23:09:54.115 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 23:09:53 smithi027 podman[89123]: 2024-04-26 23:09:53.70947879 +0000 UTC m=+0.036502397 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:09:54.115 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 23:09:53 smithi027 podman[89123]: 2024-04-26 23:09:53.982544597 +0000 UTC m=+0.309568229 container create ce53c31925d158d7026523803d6fc9c65c82f6cea7edfbc68e2dd4f34d66e2e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T23:09:54.486 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 23:09:54 smithi027 podman[89123]: 2024-04-26 23:09:54.336726819 +0000 UTC m=+0.663750491 container init ce53c31925d158d7026523803d6fc9c65c82f6cea7edfbc68e2dd4f34d66e2e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=) 2024-04-26T23:09:54.486 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 23:09:54 smithi027 podman[89123]: 2024-04-26 23:09:54.349078735 +0000 UTC m=+0.676102374 container start ce53c31925d158d7026523803d6fc9c65c82f6cea7edfbc68e2dd4f34d66e2e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-04-26T23:09:54.486 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 23:09:54 smithi027 podman[89123]: 2024-04-26 23:09:54.485229946 +0000 UTC m=+0.812253631 container attach ce53c31925d158d7026523803d6fc9c65c82f6cea7edfbc68e2dd4f34d66e2e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-26T23:09:55.115 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 23:09:54 smithi027 podman[89123]: 2024-04-26 23:09:54.823443497 +0000 UTC m=+1.150467136 container died ce53c31925d158d7026523803d6fc9c65c82f6cea7edfbc68e2dd4f34d66e2e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T23:09:55.827 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.0.service' 2024-04-26T23:09:55.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:09:55.951 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-26T23:09:55.951 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-26T23:09:55.952 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.1 2024-04-26T23:09:56.271 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 23:09:56 smithi080 systemd[1]: Stopping Ceph osd.1 for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:09:56.771 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 23:09:56 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-1[36926]: 2024-04-26T23:09:56.275+0000 7f0816dc6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:09:56.771 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 23:09:56 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-1[36926]: 2024-04-26T23:09:56.275+0000 7f0816dc6700 -1 osd.1 107 *** Got signal Terminated *** 2024-04-26T23:09:56.771 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 23:09:56 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-1[36926]: 2024-04-26T23:09:56.275+0000 7f0816dc6700 -1 osd.1 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T23:10:02.021 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 23:10:01 smithi080 podman[69207]: 2024-04-26 23:10:01.770231535 +0000 UTC m=+5.624084251 container died f44ba4c2285e96976d01e18d7e15dc1c33ebdeeb1b8aa30d2b5a4a602cd94279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-04-26T23:10:02.407 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 23:10:02 smithi080 podman[69207]: 2024-04-26 23:10:02.173244309 +0000 UTC m=+6.027097040 container cleanup f44ba4c2285e96976d01e18d7e15dc1c33ebdeeb1b8aa30d2b5a4a602cd94279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T23:10:02.407 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 23:10:02 smithi080 bash[69207]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-1 2024-04-26T23:10:02.771 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 23:10:02 smithi080 podman[69226]: 2024-04-26 23:10:02.408022046 +0000 UTC m=+0.628284137 container remove f44ba4c2285e96976d01e18d7e15dc1c33ebdeeb1b8aa30d2b5a4a602cd94279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2) 2024-04-26T23:10:03.128 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 23:10:02 smithi080 podman[69317]: 2024-04-26 23:10:02.742446917 +0000 UTC m=+0.017600779 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:10:03.129 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 23:10:02 smithi080 podman[69317]: 2024-04-26 23:10:02.858986234 +0000 UTC m=+0.134140090 container create eb9b04d50539eff7ba594920deb5d9a8ec7ee2d6baf44fe2b62f84ed63f59a54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T23:10:03.399 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 23:10:03 smithi080 podman[69317]: 2024-04-26 23:10:03.128811415 +0000 UTC m=+0.403965271 container init eb9b04d50539eff7ba594920deb5d9a8ec7ee2d6baf44fe2b62f84ed63f59a54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T23:10:03.399 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 23:10:03 smithi080 podman[69317]: 2024-04-26 23:10:03.13319033 +0000 UTC m=+0.408344187 container start eb9b04d50539eff7ba594920deb5d9a8ec7ee2d6baf44fe2b62f84ed63f59a54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-26T23:10:03.399 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 23:10:03 smithi080 podman[69317]: 2024-04-26 23:10:03.237361508 +0000 UTC m=+0.512515393 container attach eb9b04d50539eff7ba594920deb5d9a8ec7ee2d6baf44fe2b62f84ed63f59a54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T23:10:03.766 INFO:journalctl@ceph.osd.1.smithi080.stdout:Apr 26 23:10:03 smithi080 podman[69317]: 2024-04-26 23:10:03.455679403 +0000 UTC m=+0.730833255 container died eb9b04d50539eff7ba594920deb5d9a8ec7ee2d6baf44fe2b62f84ed63f59a54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-26T23:10:04.068 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.1.service' 2024-04-26T23:10:04.105 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:10:04.105 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-26T23:10:04.105 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-26T23:10:04.105 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.2 2024-04-26T23:10:04.426 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 23:10:04 smithi094 systemd[1]: Stopping Ceph osd.2 for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:10:04.685 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 23:10:04 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-2[38042]: 2024-04-26T23:10:04.426+0000 7fa0a452b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:10:04.685 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 23:10:04 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-2[38042]: 2024-04-26T23:10:04.426+0000 7fa0a452b700 -1 osd.2 107 *** Got signal Terminated *** 2024-04-26T23:10:04.685 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 23:10:04 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-2[38042]: 2024-04-26T23:10:04.426+0000 7fa0a452b700 -1 osd.2 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T23:10:10.245 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 23:10:09 smithi094 podman[70191]: 2024-04-26 23:10:09.934517764 +0000 UTC m=+5.630911257 container died ba169acef7fee1d0a03fa077943134fd2e8837fff45fa5ab53ba600ede1512c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T23:10:10.535 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 23:10:10 smithi094 podman[70191]: 2024-04-26 23:10:10.336009857 +0000 UTC m=+6.032403350 container cleanup ba169acef7fee1d0a03fa077943134fd2e8837fff45fa5ab53ba600ede1512c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T23:10:10.535 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 23:10:10 smithi094 bash[70191]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-2 2024-04-26T23:10:10.960 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 23:10:10 smithi094 podman[70220]: 2024-04-26 23:10:10.604455783 +0000 UTC m=+0.667453367 container remove ba169acef7fee1d0a03fa077943134fd2e8837fff45fa5ab53ba600ede1512c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T23:10:11.241 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 23:10:10 smithi094 podman[70313]: 2024-04-26 23:10:10.863567786 +0000 UTC m=+0.017872759 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:10:11.241 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 23:10:10 smithi094 podman[70313]: 2024-04-26 23:10:10.980596472 +0000 UTC m=+0.134901488 container create 7b4cd6dc39d38dd120abc318c9aa97c3b8bcb56e1ff7ff8998de6d45120f45bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T23:10:11.241 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 23:10:11 smithi094 podman[70313]: 2024-04-26 23:10:11.24198856 +0000 UTC m=+0.396293526 container init 7b4cd6dc39d38dd120abc318c9aa97c3b8bcb56e1ff7ff8998de6d45120f45bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T23:10:11.534 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 23:10:11 smithi094 podman[70313]: 2024-04-26 23:10:11.245444876 +0000 UTC m=+0.399749842 container start 7b4cd6dc39d38dd120abc318c9aa97c3b8bcb56e1ff7ff8998de6d45120f45bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T23:10:11.535 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 23:10:11 smithi094 podman[70313]: 2024-04-26 23:10:11.34773149 +0000 UTC m=+0.502036457 container attach 7b4cd6dc39d38dd120abc318c9aa97c3b8bcb56e1ff7ff8998de6d45120f45bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-26T23:10:11.837 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 26 23:10:11 smithi094 podman[70313]: 2024-04-26 23:10:11.538330647 +0000 UTC m=+0.692635608 container died 7b4cd6dc39d38dd120abc318c9aa97c3b8bcb56e1ff7ff8998de6d45120f45bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T23:10:12.200 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.2.service' 2024-04-26T23:10:12.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:10:12.240 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-26T23:10:12.241 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-26T23:10:12.241 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.3 2024-04-26T23:10:12.615 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:12 smithi027 systemd[1]: Stopping Ceph osd.3 for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:10:12.784 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:12 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:12.457+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:13.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:13 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:13.089+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:13.366 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:12 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-3[49238]: 2024-04-26T23:10:12.865+0000 7f80b5200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:10:13.366 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:12 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-3[49238]: 2024-04-26T23:10:12.865+0000 7f80b5200700 -1 osd.3 107 *** Got signal Terminated *** 2024-04-26T23:10:13.366 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:12 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-3[49238]: 2024-04-26T23:10:12.865+0000 7f80b5200700 -1 osd.3 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T23:10:13.784 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:13 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:13.494+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:14.364 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:14 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:14.059+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:14.784 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:14 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:14.457+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:15.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:14 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:14.865+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:15.365 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:14 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-3[49238]: 2024-04-26T23:10:14.957+0000 7f80aba00700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:49.451424+0000 front 2024-04-26T23:09:49.451346+0000 (oldest deadline 2024-04-26T23:10:14.151509+0000) 2024-04-26T23:10:15.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:15 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:15.062+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:15.784 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:15 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:15.494+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:16.267 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:15 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-3[49238]: 2024-04-26T23:10:15.989+0000 7f80aba00700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:49.451424+0000 front 2024-04-26T23:09:49.451346+0000 (oldest deadline 2024-04-26T23:10:14.151509+0000) 2024-04-26T23:10:16.267 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:16 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:16.088+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:16.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:15 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:15.856+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:16.784 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:16 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:16.457+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:17.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:16 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:16.678+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:17.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:16 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:16.885+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:17.364 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:16 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-3[49238]: 2024-04-26T23:10:16.969+0000 7f80aba00700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:49.451424+0000 front 2024-04-26T23:09:49.451346+0000 (oldest deadline 2024-04-26T23:10:14.151509+0000) 2024-04-26T23:10:17.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:17 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:17.128+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:17.784 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:17 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:17.440+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:18.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:17 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:17.662+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:18.286 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:17 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:17.905+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:18.364 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:18 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:18.114+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:18.784 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:18 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:18.442+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:18.823 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:18 smithi027 podman[89228]: 2024-04-26 23:10:18.481630064 +0000 UTC m=+5.759579339 container died 288f76c469b9ba8de740bcc52e3f6db59a1f3e2d378f4bad3ec10954c220e3bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424) 2024-04-26T23:10:19.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:18 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:18.622+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:19.087 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:18 smithi027 podman[89228]: 2024-04-26 23:10:18.992683031 +0000 UTC m=+6.270632299 container cleanup 288f76c469b9ba8de740bcc52e3f6db59a1f3e2d378f4bad3ec10954c220e3bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T23:10:19.087 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:18 smithi027 bash[89228]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-3 2024-04-26T23:10:19.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:18 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:18.920+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:19.363 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:19 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:19.087+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:19.614 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:19 smithi027 podman[89246]: 2024-04-26 23:10:19.36334889 +0000 UTC m=+0.863446402 container remove 288f76c469b9ba8de740bcc52e3f6db59a1f3e2d378f4bad3ec10954c220e3bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-26T23:10:19.784 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:19 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:19.431+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:20.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:19 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:19.612+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:20.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:19 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:19.968+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:20.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:20 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:20.093+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:20.366 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:20 smithi027 podman[89334]: 2024-04-26 23:10:20.152926061 +0000 UTC m=+0.034298237 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:10:20.366 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:20 smithi027 podman[89334]: 2024-04-26 23:10:20.286818346 +0000 UTC m=+0.168190532 container create c5778ff628166e277d4618c326dbf6023c3e16f0b82e07e1cd8cec1959e8aa16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T23:10:20.785 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:20 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:20.458+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:20.865 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:20 smithi027 podman[89334]: 2024-04-26 23:10:20.575622336 +0000 UTC m=+0.456994564 container init c5778ff628166e277d4618c326dbf6023c3e16f0b82e07e1cd8cec1959e8aa16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T23:10:20.865 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:20 smithi027 podman[89334]: 2024-04-26 23:10:20.588848192 +0000 UTC m=+0.470220372 container start c5778ff628166e277d4618c326dbf6023c3e16f0b82e07e1cd8cec1959e8aa16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T23:10:20.865 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:20 smithi027 podman[89334]: 2024-04-26 23:10:20.672861637 +0000 UTC m=+0.554233911 container attach c5778ff628166e277d4618c326dbf6023c3e16f0b82e07e1cd8cec1959e8aa16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T23:10:21.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:20 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:20.653+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:21.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:20 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:20.988+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:21.365 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 23:10:21 smithi027 podman[89334]: 2024-04-26 23:10:21.070012167 +0000 UTC m=+0.951384340 container died c5778ff628166e277d4618c326dbf6023c3e16f0b82e07e1cd8cec1959e8aa16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD) 2024-04-26T23:10:21.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:21 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:21.083+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:21.784 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:21 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:21.415+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:21.944 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.3.service' 2024-04-26T23:10:22.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:21 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:21.605+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:22.063 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:10:22.063 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-26T23:10:22.063 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-26T23:10:22.063 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.4 2024-04-26T23:10:22.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:22 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:22.016+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:22.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:22 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:22.091+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:22.370 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:22 smithi080 systemd[1]: Stopping Ceph osd.4 for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:10:22.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:22 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:22.569+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:22.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:22 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:22.569+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:22.772 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:22 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4[43920]: 2024-04-26T23:10:22.370+0000 7f87afb60700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:10:22.772 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:22 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4[43920]: 2024-04-26T23:10:22.370+0000 7f87afb60700 -1 osd.4 107 *** Got signal Terminated *** 2024-04-26T23:10:22.772 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:22 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4[43920]: 2024-04-26T23:10:22.370+0000 7f87afb60700 -1 osd.4 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T23:10:22.784 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:22 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:22.408+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:23.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:23 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:23.022+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:23.521 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:23 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:23.194+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:23.614 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:23 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:23.121+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:23.784 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:23 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:23.370+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:24.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:23 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:23.521+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:24.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:23 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:23.521+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:24.325 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:24 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:24.050+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:24.364 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:24 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:24.107+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:24.501 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:24 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4[43920]: 2024-04-26T23:10:24.216+0000 7f87ac154700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.312188+0000 front 2024-04-26T23:10:00.312218+0000 (oldest deadline 2024-04-26T23:10:23.812330+0000) 2024-04-26T23:10:24.501 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:24 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:24.212+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:24.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:24 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:24.500+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:24.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:24 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:24.500+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:24.784 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:24 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:24.325+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:24.784 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:24 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:24.325+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.084822+0000 front 2024-04-26T23:09:58.084839+0000 (oldest deadline 2024-04-26T23:10:23.384877+0000) 2024-04-26T23:10:25.516 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:25 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4[43920]: 2024-04-26T23:10:25.172+0000 7f87ac154700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.312188+0000 front 2024-04-26T23:10:00.312218+0000 (oldest deadline 2024-04-26T23:10:23.812330+0000) 2024-04-26T23:10:25.517 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:25 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:25.248+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:25.534 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:25 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:25.076+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:25.534 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:25 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:25.317+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:25.534 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:25 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:25.317+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.084822+0000 front 2024-04-26T23:09:58.084839+0000 (oldest deadline 2024-04-26T23:10:23.384877+0000) 2024-04-26T23:10:25.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:25 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:25.147+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:25.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:25 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:25.516+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:25.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:25 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:25.516+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:26.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:25 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:25.707+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:26.521 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:26 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:26.245+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:26.521 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:26 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4[43920]: 2024-04-26T23:10:26.211+0000 7f87ac154700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.312188+0000 front 2024-04-26T23:10:00.312218+0000 (oldest deadline 2024-04-26T23:10:23.812330+0000) 2024-04-26T23:10:26.534 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:26 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:26.278+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:26.534 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:26 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:26.278+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.084822+0000 front 2024-04-26T23:09:58.084839+0000 (oldest deadline 2024-04-26T23:10:23.384877+0000) 2024-04-26T23:10:26.534 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:26 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:26.100+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:26.535 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:26 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:26.100+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:26.614 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:26 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:26.118+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:26.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:26 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:26.118+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:27.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:26 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:26.565+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:27.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:26 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:26.565+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:27.034 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:26 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:26.719+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:27.521 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:27 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:27.116+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:27.521 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:27 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:27.116+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:27.522 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:27 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4[43920]: 2024-04-26T23:10:27.165+0000 7f87ac154700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.312188+0000 front 2024-04-26T23:10:00.312218+0000 (oldest deadline 2024-04-26T23:10:23.812330+0000) 2024-04-26T23:10:27.522 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:27 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:27.294+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:27.534 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:27 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:27.270+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:27.534 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:27 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:27.270+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.084822+0000 front 2024-04-26T23:09:58.084839+0000 (oldest deadline 2024-04-26T23:10:23.384877+0000) 2024-04-26T23:10:27.535 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:27 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:27.075+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:27.535 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:27 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:27.075+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:27.865 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:27 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:27.520+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:28.021 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:27 smithi080 podman[69435]: 2024-04-26 23:10:27.729475545 +0000 UTC m=+5.479605324 container died aef85c85f29564b58e873cf559b783e6126e6bf403310ba9589b19061eaa91ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T23:10:28.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:27 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:27.601+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:28.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:27 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:27.601+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:28.030 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:27 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:27.705+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:28.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:28 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:28.029+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:28.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:28 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:28.029+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:28.285 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:28 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:28.271+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:28.285 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:28 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:28.271+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.084822+0000 front 2024-04-26T23:09:58.084839+0000 (oldest deadline 2024-04-26T23:10:23.384877+0000) 2024-04-26T23:10:28.339 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:28 smithi080 podman[69435]: 2024-04-26 23:10:28.121952837 +0000 UTC m=+5.872082610 container cleanup aef85c85f29564b58e873cf559b783e6126e6bf403310ba9589b19061eaa91ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T23:10:28.339 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:28 smithi080 bash[69435]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4 2024-04-26T23:10:28.364 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:28 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:28.104+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:28.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:28 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:28.104+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:28.627 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:28 smithi080 podman[69452]: 2024-04-26 23:10:28.40681489 +0000 UTC m=+0.673454594 container remove aef85c85f29564b58e873cf559b783e6126e6bf403310ba9589b19061eaa91ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-26T23:10:28.628 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:28 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:28.339+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:28.864 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:28 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:28.560+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:28.885 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:28 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:28.627+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:28.886 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:28 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:28.627+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:28.886 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:28 smithi080 podman[69541]: 2024-04-26 23:10:28.642445407 +0000 UTC m=+0.018339010 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:10:28.886 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:28 smithi080 podman[69541]: 2024-04-26 23:10:28.749438534 +0000 UTC m=+0.125332119 container create 4bfc2fff876862349223c0257e20235928ffe128843885abffa85b6dab601250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T23:10:28.983 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:28 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:28.726+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:29.271 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:28 smithi080 podman[69541]: 2024-04-26 23:10:28.985757281 +0000 UTC m=+0.361650860 container init 4bfc2fff876862349223c0257e20235928ffe128843885abffa85b6dab601250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-26T23:10:29.272 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:28 smithi080 podman[69541]: 2024-04-26 23:10:28.988908097 +0000 UTC m=+0.364801679 container start 4bfc2fff876862349223c0257e20235928ffe128843885abffa85b6dab601250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T23:10:29.272 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:29 smithi080 podman[69541]: 2024-04-26 23:10:29.085931171 +0000 UTC m=+0.461824764 container attach 4bfc2fff876862349223c0257e20235928ffe128843885abffa85b6dab601250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T23:10:29.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:28 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:28.982+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:29.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:28 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:28.982+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:29.539 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:29 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:29.129+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:29.539 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:29 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:29.129+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:29.589 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:29 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:29.342+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:29.589 INFO:journalctl@ceph.osd.4.smithi080.stdout:Apr 26 23:10:29 smithi080 podman[69541]: 2024-04-26 23:10:29.285937527 +0000 UTC m=+0.661831127 container died 4bfc2fff876862349223c0257e20235928ffe128843885abffa85b6dab601250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T23:10:29.732 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:29 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:29.313+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:29.732 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:29 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:29.313+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.084822+0000 front 2024-04-26T23:09:58.084839+0000 (oldest deadline 2024-04-26T23:10:23.384877+0000) 2024-04-26T23:10:29.864 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:29 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:29.538+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:29.890 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:29 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:29.620+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:29.890 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:29 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:29.620+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:29.915 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.4.service' 2024-04-26T23:10:29.954 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:10:29.954 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-26T23:10:29.954 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-26T23:10:29.955 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.5 2024-04-26T23:10:29.982 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:29 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:29.973+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:29.982 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:29 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:29.973+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:29.982 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:29 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:29.731+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:30.267 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:29 smithi094 systemd[1]: Stopping Ceph osd.5 for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:10:30.267 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:30 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:30.266+0000 7f6d7ea09700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:10:30.364 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:30 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:30.105+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:30.364 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:30 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:30.105+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:30.534 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:30 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:30.266+0000 7f6d7ea09700 -1 osd.5 107 *** Got signal Terminated *** 2024-04-26T23:10:30.534 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:30 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:30.266+0000 7f6d7ea09700 -1 osd.5 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T23:10:30.534 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:30 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:30.294+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:30.535 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:30 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:30.294+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.084822+0000 front 2024-04-26T23:09:58.084839+0000 (oldest deadline 2024-04-26T23:10:23.384877+0000) 2024-04-26T23:10:30.630 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:30 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:30.379+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:30.864 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:30 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:30.501+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:31.007 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:30 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:30.689+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:31.007 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:30 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:30.689+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:31.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:30 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:30.630+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:31.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:30 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:30.630+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:31.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:30 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:30.630+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:31.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:31.007+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:31.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:31.007+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:31.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:31.007+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:31.551 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:31 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:31.136+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:31.551 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:31 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:31.136+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:31.699 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:31.285+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:31.699 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:31.285+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.084822+0000 front 2024-04-26T23:09:58.084839+0000 (oldest deadline 2024-04-26T23:10:23.384877+0000) 2024-04-26T23:10:31.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:31.427+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:31.773 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:31.598+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:31.773 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:31.598+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:31.773 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:31 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:31.598+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:31.864 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:31 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:31.550+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:31.966 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:31.699+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:31.966 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:31.699+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:32.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:31.965+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:32.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:31.965+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:32.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:31 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:31.965+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:32.546 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:32 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:32.148+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:32.546 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:32 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:32.148+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:32.710 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:32 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:32.326+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:32.710 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:32 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:32.326+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.084822+0000 front 2024-04-26T23:09:58.084839+0000 (oldest deadline 2024-04-26T23:10:23.384877+0000) 2024-04-26T23:10:32.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:32 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:32.466+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:32.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:32 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:32.616+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:32.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:32 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:32.616+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:32.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:32 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:32.616+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:32.864 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:32 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:32.545+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:32.865 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:32 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:32.545+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:06.547019+0000 front 2024-04-26T23:10:06.547352+0000 (oldest deadline 2024-04-26T23:10:31.847298+0000) 2024-04-26T23:10:32.971 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:32 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:32.709+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:32.971 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:32 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:32.709+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:33.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:32 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:32.970+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:33.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:32 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:32.970+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:33.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:32 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:32.970+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:33.509 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:33 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:33.186+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:33.509 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:33 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:33.186+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:33.672 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:33 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:33.672+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:33.672 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:33 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:33.672+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:33.672 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:33 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:33.310+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:33.673 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:33 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:33.310+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.084822+0000 front 2024-04-26T23:09:58.084839+0000 (oldest deadline 2024-04-26T23:10:23.384877+0000) 2024-04-26T23:10:33.673 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:33 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:33.310+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.485831+0000 front 2024-04-26T23:10:07.485808+0000 (oldest deadline 2024-04-26T23:10:32.786007+0000) 2024-04-26T23:10:33.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:33 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:33.503+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:33.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:33 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:33.503+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:33.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:33 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:33.601+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:33.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:33 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:33.601+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:33.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:33 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:33.601+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:33.864 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:33 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:33.508+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:33.864 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:33 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:33.508+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:06.547019+0000 front 2024-04-26T23:10:06.547352+0000 (oldest deadline 2024-04-26T23:10:31.847298+0000) 2024-04-26T23:10:34.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:33 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:33.926+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:34.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:33 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:33.926+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:34.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:33 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:33.926+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:34.497 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:34.232+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:34.497 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:34.232+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:34.497 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:34.232+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:34.694 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:34 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:34.337+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:46.283676+0000 front 2024-04-26T23:09:46.283797+0000 (oldest deadline 2024-04-26T23:10:12.183659+0000) 2024-04-26T23:10:34.694 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:34 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:34.337+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.084822+0000 front 2024-04-26T23:09:58.084839+0000 (oldest deadline 2024-04-26T23:10:23.384877+0000) 2024-04-26T23:10:34.694 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:34 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5[44727]: 2024-04-26T23:10:34.337+0000 7f6d76ff5700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.485831+0000 front 2024-04-26T23:10:07.485808+0000 (oldest deadline 2024-04-26T23:10:32.786007+0000) 2024-04-26T23:10:34.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:34 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:34.509+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:34.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:34 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:34.509+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:34.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:34 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:34.587+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:34.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:34 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:34.587+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:34.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:34 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:34.587+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:34.864 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:34.496+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:34.865 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:34.496+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:06.547019+0000 front 2024-04-26T23:10:06.547352+0000 (oldest deadline 2024-04-26T23:10:31.847298+0000) 2024-04-26T23:10:35.034 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:34 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:34.694+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:35.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:34 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:34.694+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:35.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:34 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:34.934+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:35.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:34 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:34.934+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:35.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:34 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:34.934+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:35.540 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:35.232+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:35.540 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:35.232+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:35.540 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:35.232+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:35.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:35 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:35.471+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:35.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:35 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:35.471+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:35.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:35 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:35.594+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:35.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:35 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:35.594+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:35.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:35 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:35.594+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:35.864 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:35.539+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:35.865 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:35 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:35.539+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:06.547019+0000 front 2024-04-26T23:10:06.547352+0000 (oldest deadline 2024-04-26T23:10:31.847298+0000) 2024-04-26T23:10:35.901 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:35 smithi094 podman[70417]: 2024-04-26 23:10:35.688013176 +0000 UTC m=+5.544576038 container died 01742e580df4420bca7b129870cf2fcfb642b66f68cb3df62b8bf2da32ba5d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T23:10:35.901 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:35 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:35.678+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:35.901 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:35 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:35.678+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:36.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:35 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:35.900+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:36.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:35 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:35.900+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:36.286 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:35 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:35.900+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:36.286 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:36 smithi094 podman[70417]: 2024-04-26 23:10:36.092923002 +0000 UTC m=+5.949485861 container cleanup 01742e580df4420bca7b129870cf2fcfb642b66f68cb3df62b8bf2da32ba5d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T23:10:36.287 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:36 smithi094 bash[70417]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5 2024-04-26T23:10:36.550 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:36 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:36.261+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:36.551 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:36 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:36.261+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:36.551 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:36 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:36.261+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:36.679 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:36 smithi094 podman[70439]: 2024-04-26 23:10:36.336597686 +0000 UTC m=+0.653295081 container remove 01742e580df4420bca7b129870cf2fcfb642b66f68cb3df62b8bf2da32ba5d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T23:10:36.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:36 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:36.511+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:36.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:36 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:36.511+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:36.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:36 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:36.625+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:36.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:36 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:36.625+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:36.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:36 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:36.625+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:36.864 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:36 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:36.549+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:36.865 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:36 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:36.549+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:06.547019+0000 front 2024-04-26T23:10:06.547352+0000 (oldest deadline 2024-04-26T23:10:31.847298+0000) 2024-04-26T23:10:36.932 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:36 smithi094 podman[70537]: 2024-04-26 23:10:36.582992118 +0000 UTC m=+0.017648782 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:10:36.933 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:36 smithi094 podman[70537]: 2024-04-26 23:10:36.709167504 +0000 UTC m=+0.143824184 container create d4a6e6856612673ef9226b4e8497b861a084404ee720e5f225c7ba233ca0a61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T23:10:36.933 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:36 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:36.701+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:36.933 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:36 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:36.701+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:37.252 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:36 smithi094 podman[70537]: 2024-04-26 23:10:36.957803284 +0000 UTC m=+0.392459924 container init d4a6e6856612673ef9226b4e8497b861a084404ee720e5f225c7ba233ca0a61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T23:10:37.252 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:36 smithi094 podman[70537]: 2024-04-26 23:10:36.961069852 +0000 UTC m=+0.395726490 container start d4a6e6856612673ef9226b4e8497b861a084404ee720e5f225c7ba233ca0a61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T23:10:37.252 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:37 smithi094 podman[70537]: 2024-04-26 23:10:37.057560687 +0000 UTC m=+0.492217339 container attach d4a6e6856612673ef9226b4e8497b861a084404ee720e5f225c7ba233ca0a61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T23:10:37.252 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:36 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:36.932+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:37.252 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:36 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:36.932+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:37.252 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:36 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:36.932+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:37.503 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:37 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:37.227+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:37.503 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:37 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:37.227+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:37.503 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:37 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:37.227+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:37.534 INFO:journalctl@ceph.osd.5.smithi094.stdout:Apr 26 23:10:37 smithi094 podman[70537]: 2024-04-26 23:10:37.252082171 +0000 UTC m=+0.686738815 container died d4a6e6856612673ef9226b4e8497b861a084404ee720e5f225c7ba233ca0a61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-26T23:10:37.864 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:37 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:37.502+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:37.865 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:37 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:37.502+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:06.547019+0000 front 2024-04-26T23:10:06.547352+0000 (oldest deadline 2024-04-26T23:10:31.847298+0000) 2024-04-26T23:10:37.910 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:37 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:37.730+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:37.910 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:37 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:37.730+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:37.962 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.5.service' 2024-04-26T23:10:38.021 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:37 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:37.548+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:38.021 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:37 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:37.548+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:38.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:37 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:37.675+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:38.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:37 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:37.675+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:38.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:37 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:37.675+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:38.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:10:38.041 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-26T23:10:38.041 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-26T23:10:38.041 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.6 2024-04-26T23:10:38.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:37 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:37.909+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:38.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:37 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:37.909+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:38.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:37 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:37.909+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:38.548 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:38 smithi027 systemd[1]: Stopping Ceph osd.6 for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:10:38.548 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:38 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:38.274+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:38.548 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:38 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:38.275+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:38.548 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:38 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:38.275+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:38.865 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:38 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:38.546+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:38.865 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:38 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:38.546+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:06.547019+0000 front 2024-04-26T23:10:06.547352+0000 (oldest deadline 2024-04-26T23:10:31.847298+0000) 2024-04-26T23:10:38.865 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:38 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:38.670+0000 7f9829200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:10:38.865 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:38 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:38.670+0000 7f9829200700 -1 osd.6 107 *** Got signal Terminated *** 2024-04-26T23:10:38.866 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:38 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:38.670+0000 7f9829200700 -1 osd.6 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T23:10:39.021 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:38 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:38.529+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:39.021 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:38 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:38.529+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:39.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:38 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:38.702+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:39.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:38 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:38.702+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:39.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:38 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:38.702+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:39.034 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:38 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:38.683+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:39.034 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:38 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:38.683+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:39.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:38 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:38.872+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:39.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:38 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:38.872+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:39.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:38 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:38.872+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:39.500 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:39 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:39.499+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:39.614 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:39 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:39.318+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:39.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:39 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:39.318+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:39.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:39 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:39.318+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:39.615 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:39 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:39.518+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:39.615 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:39 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:39.518+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:06.547019+0000 front 2024-04-26T23:10:06.547352+0000 (oldest deadline 2024-04-26T23:10:31.847298+0000) 2024-04-26T23:10:39.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:39 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:39.499+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:39.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:39 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:39.662+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:39.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:39 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:39.662+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:39.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:39 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:39.662+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:40.034 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:39 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:39.697+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:40.034 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:39 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:39.697+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:40.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:39 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:39.878+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:40.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:39 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:39.878+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:40.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:39 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:39.878+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:40.615 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:40 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:40.487+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:40.615 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:40 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:40.487+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:06.547019+0000 front 2024-04-26T23:10:06.547352+0000 (oldest deadline 2024-04-26T23:10:31.847298+0000) 2024-04-26T23:10:40.615 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:40 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:40.487+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.051584+0000 front 2024-04-26T23:10:17.051622+0000 (oldest deadline 2024-04-26T23:10:39.950705+0000) 2024-04-26T23:10:40.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:40 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:40.290+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:40.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:40 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:40.290+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:40.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:40 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:40.290+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:40.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:40 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:40.471+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:40.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:40 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:40.471+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:40.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:40 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:40.637+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:40.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:40 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:40.637+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:40.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:40 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:40.637+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:40.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:40 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:40.637+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:41.034 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:40 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:40.697+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:41.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:40 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:40.697+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:41.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:40 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:40.865+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:41.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:40 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:40.865+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:41.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:40 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:40.865+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:41.614 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:41 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:41.467+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:41.615 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:41 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:41.467+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:06.547019+0000 front 2024-04-26T23:10:06.547352+0000 (oldest deadline 2024-04-26T23:10:31.847298+0000) 2024-04-26T23:10:41.615 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:41 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:41.467+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.051584+0000 front 2024-04-26T23:10:17.051622+0000 (oldest deadline 2024-04-26T23:10:39.950705+0000) 2024-04-26T23:10:41.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:41 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:41.261+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:41.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:41 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:41.261+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:41.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:41 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:41.261+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:41.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:41 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:41.480+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:41.774 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:41 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:41.480+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:41.774 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:41 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:41.480+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.521925+0000 front 2024-04-26T23:10:16.521760+0000 (oldest deadline 2024-04-26T23:10:40.621639+0000) 2024-04-26T23:10:41.774 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:41 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:41.599+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:41.774 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:41 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:41.599+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:41.774 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:41 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:41.599+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:41.774 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:41 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:41.599+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:42.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:41 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:41.685+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:42.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:41 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:41.685+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:42.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:41 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:41.907+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:42.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:41 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:41.907+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:42.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:41 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:41.907+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:42.517 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:42 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:42.226+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:42.517 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:42 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:42.226+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:42.517 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:42 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:42.226+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:42.517 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:42 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:42.226+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:42.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:42 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:42.500+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:42.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:42 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:42.500+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:42.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:42 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:42.500+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.521925+0000 front 2024-04-26T23:10:16.521760+0000 (oldest deadline 2024-04-26T23:10:40.621639+0000) 2024-04-26T23:10:42.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:42 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:42.570+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:42.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:42 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:42.570+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:42.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:42 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:42.570+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:42.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:42 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:42.571+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:42.864 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:42 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:42.516+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:42.865 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:42 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:42.516+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:06.547019+0000 front 2024-04-26T23:10:06.547352+0000 (oldest deadline 2024-04-26T23:10:31.847298+0000) 2024-04-26T23:10:42.865 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:42 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:42.516+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.051584+0000 front 2024-04-26T23:10:17.051622+0000 (oldest deadline 2024-04-26T23:10:39.950705+0000) 2024-04-26T23:10:43.034 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:42 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:42.660+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:43.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:42 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:42.660+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:43.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:42 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:42.660+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:43.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:42 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:42.879+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:43.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:42 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:42.879+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:43.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:42 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:42.879+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:43.554 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:43 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:43.200+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:43.554 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:43 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:43.200+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:43.554 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:43 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:43.200+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:43.554 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:43 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:43.200+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:43.865 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:43 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:43.553+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.645771+0000 front 2024-04-26T23:10:00.645695+0000 (oldest deadline 2024-04-26T23:10:26.545922+0000) 2024-04-26T23:10:43.865 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:43 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:43.553+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:06.547019+0000 front 2024-04-26T23:10:06.547352+0000 (oldest deadline 2024-04-26T23:10:31.847298+0000) 2024-04-26T23:10:43.865 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:43 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6[55750]: 2024-04-26T23:10:43.553+0000 7f9824a00700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.051584+0000 front 2024-04-26T23:10:17.051622+0000 (oldest deadline 2024-04-26T23:10:39.950705+0000) 2024-04-26T23:10:44.021 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:43 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:43.539+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:44.021 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:43 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:43.539+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:44.021 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:43 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:43.539+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.521925+0000 front 2024-04-26T23:10:16.521760+0000 (oldest deadline 2024-04-26T23:10:40.621639+0000) 2024-04-26T23:10:44.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:43 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:43.618+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:44.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:43 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:43.618+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:44.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:43 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:43.618+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:44.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:43 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:43.618+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:44.034 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:43 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:43.654+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:44.034 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:43 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:43.654+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:44.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:43 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:43.654+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:44.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:43 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:43.830+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:44.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:43 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:43.830+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:44.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:43 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:43.830+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:44.615 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:44 smithi027 podman[89437]: 2024-04-26 23:10:44.233732062 +0000 UTC m=+5.717138292 container died 5a5a214bd945bc5abd2ad864baecd8adc674ad3d5d9957ab4aea8a7310663b82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True) 2024-04-26T23:10:44.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:44 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:44.235+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:44.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:44 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:44.235+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:44.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:44 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:44.235+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:44.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:44 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:44.235+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:44.922 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:44 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:44.523+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:44.923 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:44 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:44.523+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:44.923 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:44 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:44.523+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.521925+0000 front 2024-04-26T23:10:16.521760+0000 (oldest deadline 2024-04-26T23:10:40.621639+0000) 2024-04-26T23:10:44.923 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:44 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:44.639+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:44.923 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:44 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:44.639+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:44.923 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:44 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:44.639+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:44.923 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:44 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:44.639+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:45.034 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:44 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:44.615+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:45.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:44 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:44.615+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:45.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:44 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:44.615+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:45.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:44 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:44.784+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:45.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:44 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:44.784+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:45.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:44 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:44.784+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:45.065 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:44 smithi027 podman[89437]: 2024-04-26 23:10:44.804712035 +0000 UTC m=+6.288118250 container cleanup 5a5a214bd945bc5abd2ad864baecd8adc674ad3d5d9957ab4aea8a7310663b82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T23:10:45.065 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:44 smithi027 bash[89437]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6 2024-04-26T23:10:45.364 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:45 smithi027 podman[89455]: 2024-04-26 23:10:45.064180531 +0000 UTC m=+0.815042737 container remove 5a5a214bd945bc5abd2ad864baecd8adc674ad3d5d9957ab4aea8a7310663b82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T23:10:45.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:45 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:45.203+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:45.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:45 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:45.203+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:45.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:45 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:45.203+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:45.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:45 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:45.203+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:45.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:45 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:45.643+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:45.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:45 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:45.643+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:45.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:45 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:45.643+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:45.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:45 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:45.643+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:45.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:45 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:45.475+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:45.772 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:45 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:45.475+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:45.772 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:45 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:45.475+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.521925+0000 front 2024-04-26T23:10:16.521760+0000 (oldest deadline 2024-04-26T23:10:40.621639+0000) 2024-04-26T23:10:46.006 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:45 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:45.580+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:46.007 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:45 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:45.580+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:46.007 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:45 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:45.580+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:46.007 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:45 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:45.817+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:46.007 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:45 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:45.817+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:46.007 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:45 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:45.817+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:46.191 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:45 smithi027 podman[89545]: 2024-04-26 23:10:45.843137975 +0000 UTC m=+0.037790597 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:10:46.191 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:45 smithi027 podman[89545]: 2024-04-26 23:10:45.973480746 +0000 UTC m=+0.168133413 container create b3c80ee3e6c4e23de74a7c713d17bd8f8267b14fffc92726a20b8071ced0e4b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T23:10:46.437 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:46 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:46.437+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:46.438 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:46 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:46.437+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:46.438 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:46 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:46.437+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.521925+0000 front 2024-04-26T23:10:16.521760+0000 (oldest deadline 2024-04-26T23:10:40.621639+0000) 2024-04-26T23:10:46.451 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:46 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:46.192+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:46.451 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:46 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:46.192+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:46.451 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:46 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:46.192+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:46.452 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:46 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:46.192+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:46.452 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:46 smithi027 podman[89545]: 2024-04-26 23:10:46.310485989 +0000 UTC m=+0.505138686 container init b3c80ee3e6c4e23de74a7c713d17bd8f8267b14fffc92726a20b8071ced0e4b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T23:10:46.452 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:46 smithi027 podman[89545]: 2024-04-26 23:10:46.322059684 +0000 UTC m=+0.516712306 container start b3c80ee3e6c4e23de74a7c713d17bd8f8267b14fffc92726a20b8071ced0e4b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T23:10:46.452 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:46 smithi027 podman[89545]: 2024-04-26 23:10:46.451049103 +0000 UTC m=+0.645701787 container attach b3c80ee3e6c4e23de74a7c713d17bd8f8267b14fffc92726a20b8071ced0e4b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T23:10:46.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:46 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:46.632+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:46.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:46 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:46.632+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:46.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:46 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:46.632+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:46.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:46 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:46.632+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:47.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:46 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:46.622+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:47.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:46 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:46.622+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:47.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:46 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:46.622+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:47.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:46 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:46.851+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:47.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:46 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:46.851+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:47.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:46 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:46.851+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:47.115 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 23:10:46 smithi027 podman[89545]: 2024-04-26 23:10:46.802979358 +0000 UTC m=+0.997631997 container died b3c80ee3e6c4e23de74a7c713d17bd8f8267b14fffc92726a20b8071ced0e4b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T23:10:47.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:47 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:47.154+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:47.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:47 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:47.154+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:47.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:47 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:47.154+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:47.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:47 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:47.154+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:47.673 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.6.service' 2024-04-26T23:10:47.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:47 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:47.482+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:47.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:47 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:47.482+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:47.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:47 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:47.482+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.521925+0000 front 2024-04-26T23:10:16.521760+0000 (oldest deadline 2024-04-26T23:10:40.621639+0000) 2024-04-26T23:10:47.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:47 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:47.629+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:47.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:47 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:47.629+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:47.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:47 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:47.629+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:47.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:47 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:47.629+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:47.781 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:10:47.781 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-26T23:10:47.781 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-26T23:10:47.781 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.7 2024-04-26T23:10:48.034 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:47 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:47.664+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:48.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:47 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:47.664+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:48.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:47 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:47.664+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:48.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:47 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:47.898+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:48.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:47 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:47.898+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:48.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:47 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:47.898+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:48.101 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:47 smithi080 systemd[1]: Stopping Ceph osd.7 for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:10:48.101 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:48 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:48.100+0000 7fe170838700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:10:48.101 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:48 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:48.100+0000 7fe170838700 -1 osd.7 107 *** Got signal Terminated *** 2024-04-26T23:10:48.101 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:48 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:48.100+0000 7fe170838700 -1 osd.7 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T23:10:48.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:48 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:48.121+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:48.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:48 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:48.121+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:48.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:48 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:48.121+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:48.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:48 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:48.121+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:49.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:48 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:48.650+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:49.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:48 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:48.650+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:49.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:48 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:48.650+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:49.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:48 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:48.650+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:49.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:48 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:48.650+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:10:49.022 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:48 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:48.525+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:49.022 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:48 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:48.525+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:49.022 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:48 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:48.525+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.521925+0000 front 2024-04-26T23:10:16.521760+0000 (oldest deadline 2024-04-26T23:10:40.621639+0000) 2024-04-26T23:10:49.034 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:48 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:48.677+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:49.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:48 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:48.677+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:49.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:48 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:48.677+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:49.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:48 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:48.870+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:49.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:48 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:48.870+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:49.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:48 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:48.870+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:49.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:49 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:49.078+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:49.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:49 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:49.078+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:49.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:49 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:49.078+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:49.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:49 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:49.078+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:50.021 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:49 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:49.569+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:50.022 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:49 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:49.569+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:50.022 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:49 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:49.569+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.521925+0000 front 2024-04-26T23:10:16.521760+0000 (oldest deadline 2024-04-26T23:10:40.621639+0000) 2024-04-26T23:10:50.022 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:49 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:49.569+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.922381+0000 front 2024-04-26T23:10:25.922448+0000 (oldest deadline 2024-04-26T23:10:48.822485+0000) 2024-04-26T23:10:50.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:49 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:49.689+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:50.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:49 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:49.689+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:50.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:49 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:49.689+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:50.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:49 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:49.689+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:50.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:49 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:49.689+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:10:50.034 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:49 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:49.726+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:50.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:49 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:49.726+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:50.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:49 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:49.726+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:50.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:49 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:49.879+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:50.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:49 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:49.879+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:50.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:49 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:49.879+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:50.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:49 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:49.879+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:10:50.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:50 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:50.051+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:50.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:50 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:50.051+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:50.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:50 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:50.051+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:50.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:50 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:50.051+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:51.021 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:50 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:50.590+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:51.021 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:50 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:50.590+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:51.022 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:50 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:50.590+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.521925+0000 front 2024-04-26T23:10:16.521760+0000 (oldest deadline 2024-04-26T23:10:40.621639+0000) 2024-04-26T23:10:51.022 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:50 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:50.590+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.922381+0000 front 2024-04-26T23:10:25.922448+0000 (oldest deadline 2024-04-26T23:10:48.822485+0000) 2024-04-26T23:10:51.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:50 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:50.653+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:51.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:50 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:50.653+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:51.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:50 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:50.653+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:51.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:50 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:50.653+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:51.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:50 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:50.653+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:10:51.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:50 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:50.761+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:51.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:50 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:50.761+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:51.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:50 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:50.761+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:51.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:50 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:50.885+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:51.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:50 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:50.885+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:51.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:50 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:50.885+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:51.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:50 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:50.885+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:10:51.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:51 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:51.039+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:51.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:51 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:51.039+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:51.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:51 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:51.039+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:51.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:51 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:51.039+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:51.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:51 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:51.039+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:10:52.021 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:51 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:51.592+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:52.022 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:51 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:51.592+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:52.022 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:51 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:51.592+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.521925+0000 front 2024-04-26T23:10:16.521760+0000 (oldest deadline 2024-04-26T23:10:40.621639+0000) 2024-04-26T23:10:52.022 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:51 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:51.592+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.922381+0000 front 2024-04-26T23:10:25.922448+0000 (oldest deadline 2024-04-26T23:10:48.822485+0000) 2024-04-26T23:10:52.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:51 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:51.633+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:52.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:51 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:51.633+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:52.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:51 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:51.633+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:52.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:51 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:51.633+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:52.023 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:51 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:51.633+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:10:52.034 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:51 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:51.738+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:52.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:51 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:51.738+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:52.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:51 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:51.738+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:52.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:51 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:51.848+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:52.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:51 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:51.848+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:52.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:51 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:51.848+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:52.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:51 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:51.848+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:10:52.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:52 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:52.004+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:52.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:52 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:52.004+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:52.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:52 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:52.004+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:52.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:52 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:52.004+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:52.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:52 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:52.004+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:10:53.021 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:52 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:52.561+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.717448+0000 front 2024-04-26T23:09:58.717277+0000 (oldest deadline 2024-04-26T23:10:22.817442+0000) 2024-04-26T23:10:53.021 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:52 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:52.561+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.319456+0000 front 2024-04-26T23:10:07.319497+0000 (oldest deadline 2024-04-26T23:10:32.520167+0000) 2024-04-26T23:10:53.022 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:52 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:52.561+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.521925+0000 front 2024-04-26T23:10:16.521760+0000 (oldest deadline 2024-04-26T23:10:40.621639+0000) 2024-04-26T23:10:53.022 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:52 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7[50476]: 2024-04-26T23:10:52.561+0000 7fe16ce2c700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.922381+0000 front 2024-04-26T23:10:25.922448+0000 (oldest deadline 2024-04-26T23:10:48.822485+0000) 2024-04-26T23:10:53.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:52 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:52.665+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:53.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:52 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:52.665+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:53.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:52 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:52.665+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:53.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:52 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:52.666+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:53.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:52 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:52.666+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:10:53.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:52 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:52.689+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:53.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:52 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:52.689+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:53.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:52 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:52.689+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:53.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:52 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:52.689+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.825770+0000 front 2024-04-26T23:10:26.825824+0000 (oldest deadline 2024-04-26T23:10:52.125692+0000) 2024-04-26T23:10:53.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:52 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:52.867+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:53.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:52 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:52.867+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:53.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:52 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:52.867+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:53.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:52 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:52.867+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:10:53.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:53 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:53.030+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:53.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:53 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:53.030+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:53.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:53 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:53.030+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:53.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:53 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:53.030+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:53.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:53 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:53.030+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:10:53.769 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:53 smithi080 podman[69660]: 2024-04-26 23:10:53.471575358 +0000 UTC m=+5.492274356 container died 0cbad678a9898b89dff5ee2eea89036f1a8eadbfe679b4edb18b9ce49e7d074b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T23:10:53.769 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:53 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:53.649+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:53.769 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:53 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:53.649+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:53.769 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:53 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:53.649+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:53.769 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:53 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:53.649+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:53.769 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:53 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:53.649+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:10:54.021 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:53 smithi080 podman[69660]: 2024-04-26 23:10:53.859463736 +0000 UTC m=+5.880162730 container cleanup 0cbad678a9898b89dff5ee2eea89036f1a8eadbfe679b4edb18b9ce49e7d074b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True) 2024-04-26T23:10:54.021 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:53 smithi080 bash[69660]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7 2024-04-26T23:10:54.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:53 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:53.709+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:54.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:53 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:53.709+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:54.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:53 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:53.709+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:54.035 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:53 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:53.709+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.825770+0000 front 2024-04-26T23:10:26.825824+0000 (oldest deadline 2024-04-26T23:10:52.125692+0000) 2024-04-26T23:10:54.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:53 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:53.917+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:54.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:53 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:53.917+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:54.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:53 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:53.917+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:54.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:53 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:53.917+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:10:54.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:54 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:54.059+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:54.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:54 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:54.059+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:54.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:54 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:54.059+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:54.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:54 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:54.059+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:54.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:54 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:54.059+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:10:54.468 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:54 smithi080 podman[69677]: 2024-04-26 23:10:54.145491121 +0000 UTC m=+0.671022054 container remove 0cbad678a9898b89dff5ee2eea89036f1a8eadbfe679b4edb18b9ce49e7d074b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-04-26T23:10:54.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:54 smithi080 podman[69769]: 2024-04-26 23:10:54.37215264 +0000 UTC m=+0.017241917 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:10:54.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:54 smithi080 podman[69769]: 2024-04-26 23:10:54.489054198 +0000 UTC m=+0.134143460 container create ca08bdf249be78bee0052df05d43948cec2be97404cfe67d0a5503b9dc3671d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T23:10:54.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:54 smithi080 podman[69769]: 2024-04-26 23:10:54.715700306 +0000 UTC m=+0.360789567 container init ca08bdf249be78bee0052df05d43948cec2be97404cfe67d0a5503b9dc3671d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T23:10:54.771 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:54 smithi080 podman[69769]: 2024-04-26 23:10:54.718719781 +0000 UTC m=+0.363809043 container start ca08bdf249be78bee0052df05d43948cec2be97404cfe67d0a5503b9dc3671d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-26T23:10:54.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:54 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:54.655+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:54.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:54 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:54.655+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:54.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:54 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:54.655+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:54.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:54 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:54.655+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:54.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:54 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:54.655+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:10:54.956 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:54 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:54.693+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:54.956 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:54 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:54.693+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:54.957 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:54 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:54.693+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:54.957 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:54 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:54.693+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.825770+0000 front 2024-04-26T23:10:26.825824+0000 (oldest deadline 2024-04-26T23:10:52.125692+0000) 2024-04-26T23:10:55.036 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:54 smithi080 podman[69769]: 2024-04-26 23:10:54.815753135 +0000 UTC m=+0.460842414 container attach ca08bdf249be78bee0052df05d43948cec2be97404cfe67d0a5503b9dc3671d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T23:10:55.036 INFO:journalctl@ceph.osd.7.smithi080.stdout:Apr 26 23:10:55 smithi080 podman[69769]: 2024-04-26 23:10:55.019019882 +0000 UTC m=+0.664109144 container died ca08bdf249be78bee0052df05d43948cec2be97404cfe67d0a5503b9dc3671d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T23:10:55.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:54 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:54.956+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:55.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:54 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:54.956+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:55.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:54 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:54.956+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:55.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:54 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:54.956+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:10:55.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:55 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:55.071+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:55.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:55 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:55.071+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:55.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:55 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:55.071+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:55.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:55 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:55.071+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:55.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:55 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:55.071+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:10:55.642 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.7.service' 2024-04-26T23:10:55.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:10:55.680 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-26T23:10:55.680 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-26T23:10:55.680 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.8 2024-04-26T23:10:55.921 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:55 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:55.651+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:55.921 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:55 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:55.651+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:55.921 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:55 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:55.651+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:55.921 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:55 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:55.651+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.825770+0000 front 2024-04-26T23:10:26.825824+0000 (oldest deadline 2024-04-26T23:10:52.125692+0000) 2024-04-26T23:10:55.921 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:55 smithi094 systemd[1]: Stopping Ceph osd.8 for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:10:56.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:55 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:55.686+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:56.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:55 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:55.686+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:56.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:55 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:55.686+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:56.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:55 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:55.686+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:56.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:55 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:55.686+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:10:56.284 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:56 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:56.006+0000 7f87b514b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:10:56.284 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:56 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:56.006+0000 7f87b514b700 -1 osd.8 107 *** Got signal Terminated *** 2024-04-26T23:10:56.285 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:56 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:56.006+0000 7f87b514b700 -1 osd.8 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T23:10:56.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:55 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:55.921+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:56.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:55 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:55.921+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:56.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:55 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:55.921+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:56.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:55 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:55.921+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:10:56.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:55 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:55.921+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:10:56.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:56 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:56.074+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:56.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:56 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:56.074+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:56.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:56 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:56.074+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:56.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:56 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:56.074+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:56.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:56 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:56.074+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:10:56.938 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:56 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:56.614+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:56.938 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:56 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:56.614+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:56.938 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:56 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:56.614+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:56.938 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:56 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:56.614+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.825770+0000 front 2024-04-26T23:10:26.825824+0000 (oldest deadline 2024-04-26T23:10:52.125692+0000) 2024-04-26T23:10:57.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:56 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:56.664+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:57.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:56 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:56.664+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:57.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:56 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:56.664+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:57.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:56 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:56.664+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:57.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:56 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:56.664+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:10:57.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:56 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:56.664+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:10:57.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:56 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:56.938+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:57.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:56 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:56.938+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:57.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:56 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:56.938+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:57.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:56 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:56.938+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:10:57.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:56 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:56.938+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:10:57.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:57 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:57.093+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:57.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:57 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:57.093+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:57.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:57 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:57.093+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:57.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:57 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:57.093+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:57.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:57 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:57.093+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:10:57.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:57 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:57.094+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:30.957285+0000 front 2024-04-26T23:10:30.957099+0000 (oldest deadline 2024-04-26T23:10:56.257243+0000) 2024-04-26T23:10:57.909 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:57 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:57.578+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:57.909 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:57 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:57.578+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:57.909 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:57 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:57.578+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:57.909 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:57 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:57.578+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.825770+0000 front 2024-04-26T23:10:26.825824+0000 (oldest deadline 2024-04-26T23:10:52.125692+0000) 2024-04-26T23:10:57.909 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:57 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:57.578+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.927143+0000 front 2024-04-26T23:10:34.927176+0000 (oldest deadline 2024-04-26T23:10:57.227212+0000) 2024-04-26T23:10:58.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:57 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:57.672+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:58.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:57 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:57.672+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:58.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:57 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:57.672+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:58.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:57 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:57.672+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:58.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:57 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:57.672+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:10:58.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:57 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:57.672+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:10:58.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:57 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:57.909+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:58.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:57 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:57.909+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:58.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:57 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:57.909+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:58.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:57 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:57.909+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:10:58.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:57 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:57.909+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:10:58.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:58 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:58.144+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:58.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:58 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:58.144+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:58.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:58 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:58.144+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:58.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:58 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:58.144+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:58.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:58 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:58.144+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:10:58.616 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:58 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:58.144+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:30.957285+0000 front 2024-04-26T23:10:30.957099+0000 (oldest deadline 2024-04-26T23:10:56.257243+0000) 2024-04-26T23:10:58.904 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:58 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:58.583+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:58.904 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:58 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:58.583+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:58.904 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:58 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:58.583+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:58.905 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:58 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:58.583+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.825770+0000 front 2024-04-26T23:10:26.825824+0000 (oldest deadline 2024-04-26T23:10:52.125692+0000) 2024-04-26T23:10:58.905 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:58 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:58.583+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.927143+0000 front 2024-04-26T23:10:34.927176+0000 (oldest deadline 2024-04-26T23:10:57.227212+0000) 2024-04-26T23:10:59.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:58 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:58.674+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:10:59.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:58 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:58.674+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:10:59.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:58 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:58.674+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:10:59.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:58 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:58.674+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:10:59.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:58 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:58.674+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:10:59.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:58 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:58.674+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:10:59.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:58 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:58.904+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:10:59.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:58 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:58.904+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:10:59.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:58 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:58.904+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:10:59.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:58 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:58.904+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:10:59.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:58 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:58.904+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:10:59.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:59 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:59.169+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:10:59.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:59 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:59.169+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:10:59.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:59 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:59.169+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:10:59.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:59 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:59.169+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:10:59.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:59 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:59.169+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:10:59.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:10:59 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:10:59.169+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:30.957285+0000 front 2024-04-26T23:10:30.957099+0000 (oldest deadline 2024-04-26T23:10:56.257243+0000) 2024-04-26T23:10:59.880 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:59 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:59.568+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:10:59.880 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:59 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:59.568+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:10:59.880 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:59 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:59.568+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:10:59.881 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:59 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:59.568+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.825770+0000 front 2024-04-26T23:10:26.825824+0000 (oldest deadline 2024-04-26T23:10:52.125692+0000) 2024-04-26T23:10:59.881 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:10:59 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:10:59.568+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.927143+0000 front 2024-04-26T23:10:34.927176+0000 (oldest deadline 2024-04-26T23:10:57.227212+0000) 2024-04-26T23:11:00.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:59 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:59.632+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:00.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:59 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:59.632+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:00.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:59 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:59.632+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:00.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:59 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:59.632+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:00.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:59 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:59.632+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:00.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:10:59 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:10:59.632+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:00.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:59 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:59.880+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:00.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:59 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:59.880+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:00.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:59 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:59.880+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:00.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:59 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:59.880+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:00.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:10:59 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:10:59.880+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:00.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:00 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:00.146+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:11:00.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:00 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:00.146+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:11:00.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:00 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:00.146+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:11:00.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:00 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:00.146+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:11:00.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:00 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:00.146+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:11:00.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:00 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:00.146+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:30.957285+0000 front 2024-04-26T23:10:30.957099+0000 (oldest deadline 2024-04-26T23:10:56.257243+0000) 2024-04-26T23:11:00.784 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:11:00 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:11:00.519+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:59.722935+0000 front 2024-04-26T23:09:59.722790+0000 (oldest deadline 2024-04-26T23:10:25.622865+0000) 2024-04-26T23:11:00.785 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:11:00 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:11:00.519+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:05.623185+0000 front 2024-04-26T23:10:05.623169+0000 (oldest deadline 2024-04-26T23:10:30.323365+0000) 2024-04-26T23:11:00.785 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:11:00 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:11:00.519+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:16.224729+0000 front 2024-04-26T23:10:16.224608+0000 (oldest deadline 2024-04-26T23:10:42.124588+0000) 2024-04-26T23:11:00.785 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:11:00 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:11:00.519+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.825770+0000 front 2024-04-26T23:10:26.825824+0000 (oldest deadline 2024-04-26T23:10:52.125692+0000) 2024-04-26T23:11:00.785 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:11:00 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8[51306]: 2024-04-26T23:11:00.519+0000 7f87ad737700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.927143+0000 front 2024-04-26T23:10:34.927176+0000 (oldest deadline 2024-04-26T23:10:57.227212+0000) 2024-04-26T23:11:01.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:00 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:00.648+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:01.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:00 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:00.648+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:01.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:00 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:00.648+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:01.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:00 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:00.648+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:01.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:00 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:00.648+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:01.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:00 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:00.648+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:01.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:00 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:00.900+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:01.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:00 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:00.900+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:01.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:00 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:00.900+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:01.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:00 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:00.900+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:01.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:00 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:00.900+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:01.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:01 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:01.195+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:11:01.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:01 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:01.195+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:11:01.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:01 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:01.195+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:11:01.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:01 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:01.195+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:11:01.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:01 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:01.195+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:11:01.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:01 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:01.195+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:30.957285+0000 front 2024-04-26T23:10:30.957099+0000 (oldest deadline 2024-04-26T23:10:56.257243+0000) 2024-04-26T23:11:01.824 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:11:01 smithi094 podman[70644]: 2024-04-26 23:11:01.567148591 +0000 UTC m=+5.693664698 container died 70399ea6c0be9a4ce6c0dd6cff8af8a8981abf64a68c3f215394da21ba0cfd86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T23:11:02.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:01 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:01.631+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:02.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:01 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:01.631+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:02.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:01 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:01.631+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:02.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:01 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:01.631+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:02.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:01 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:01.631+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:02.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:01 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:01.631+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:02.133 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:11:01 smithi094 podman[70644]: 2024-04-26 23:11:01.907178771 +0000 UTC m=+6.033694877 container cleanup 70399ea6c0be9a4ce6c0dd6cff8af8a8981abf64a68c3f215394da21ba0cfd86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T23:11:02.133 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:11:01 smithi094 bash[70644]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8 2024-04-26T23:11:02.133 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:01 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:01.858+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:02.133 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:01 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:01.858+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:02.133 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:01 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:01.858+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:02.133 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:01 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:01.858+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:02.133 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:01 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:01.858+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:02.534 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:11:02 smithi094 podman[70663]: 2024-04-26 23:11:02.133821552 +0000 UTC m=+0.568858249 container remove 70399ea6c0be9a4ce6c0dd6cff8af8a8981abf64a68c3f215394da21ba0cfd86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-26T23:11:02.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:02 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:02.191+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:11:02.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:02 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:02.191+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:11:02.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:02 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:02.191+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:11:02.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:02 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:02.191+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:11:02.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:02 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:02.191+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:11:02.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:02 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:02.191+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:30.957285+0000 front 2024-04-26T23:10:30.957099+0000 (oldest deadline 2024-04-26T23:10:56.257243+0000) 2024-04-26T23:11:02.820 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:11:02 smithi094 podman[70753]: 2024-04-26 23:11:02.466954127 +0000 UTC m=+0.018186799 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:11:02.820 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:11:02 smithi094 podman[70753]: 2024-04-26 23:11:02.56796566 +0000 UTC m=+0.119198309 container create ef3d27a0e1447d6c93504e08c39005da4e84656a1c7d8114bc6104523b71cce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True) 2024-04-26T23:11:02.820 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:11:02 smithi094 podman[70753]: 2024-04-26 23:11:02.820688792 +0000 UTC m=+0.371921438 container init ef3d27a0e1447d6c93504e08c39005da4e84656a1c7d8114bc6104523b71cce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424) 2024-04-26T23:11:03.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:02 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:02.659+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:03.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:02 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:02.659+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:03.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:02 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:02.659+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:03.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:02 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:02.659+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:03.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:02 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:02.659+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:03.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:02 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:02.659+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:03.114 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:02 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:02.884+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:03.114 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:02 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:02.884+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:03.114 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:02 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:02.884+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:03.114 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:02 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:02.884+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:03.115 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:02 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:02.884+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:03.115 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:11:02 smithi094 podman[70753]: 2024-04-26 23:11:02.824478272 +0000 UTC m=+0.375710922 container start ef3d27a0e1447d6c93504e08c39005da4e84656a1c7d8114bc6104523b71cce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T23:11:03.115 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:11:02 smithi094 podman[70753]: 2024-04-26 23:11:02.946056315 +0000 UTC m=+0.497288976 container attach ef3d27a0e1447d6c93504e08c39005da4e84656a1c7d8114bc6104523b71cce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T23:11:03.115 INFO:journalctl@ceph.osd.8.smithi094.stdout:Apr 26 23:11:03 smithi094 podman[70753]: 2024-04-26 23:11:03.114994183 +0000 UTC m=+0.666226836 container died ef3d27a0e1447d6c93504e08c39005da4e84656a1c7d8114bc6104523b71cce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-26T23:11:03.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:03 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:03.222+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:11:03.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:03 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:03.222+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:11:03.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:03 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:03.222+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:11:03.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:03 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:03.222+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:11:03.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:03 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:03.222+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:11:03.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:03 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:03.222+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:30.957285+0000 front 2024-04-26T23:10:30.957099+0000 (oldest deadline 2024-04-26T23:10:56.257243+0000) 2024-04-26T23:11:03.732 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.8.service' 2024-04-26T23:11:03.769 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:11:03.769 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-26T23:11:03.769 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-26T23:11:03.769 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.9 2024-04-26T23:11:04.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:03 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:03.702+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:04.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:03 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:03.702+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:04.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:03 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:03.702+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:04.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:03 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:03.702+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:04.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:03 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:03.702+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:04.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:03 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:03.702+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:04.034 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:03 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:03.883+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:04.034 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:03 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:03.883+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:04.034 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:03 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:03.883+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:04.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:03 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:03.883+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:04.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:03 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:03.883+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:04.114 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:03 smithi027 systemd[1]: Stopping Ceph osd.9 for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:11:04.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:04 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:04.201+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:11:04.616 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:04 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:04.201+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:11:04.616 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:04 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:04.201+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:11:04.616 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:04 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:04.201+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:11:04.616 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:04 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:04.201+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:11:04.616 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:04 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:04.201+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:30.957285+0000 front 2024-04-26T23:10:30.957099+0000 (oldest deadline 2024-04-26T23:10:56.257243+0000) 2024-04-26T23:11:04.616 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:04 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:04.423+0000 7f3fdbc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:11:04.616 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:04 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:04.423+0000 7f3fdbc00700 -1 osd.9 107 *** Got signal Terminated *** 2024-04-26T23:11:04.616 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:04 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:04.423+0000 7f3fdbc00700 -1 osd.9 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T23:11:05.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:04 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:04.734+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:05.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:04 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:04.734+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:05.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:04 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:04.734+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:05.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:04 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:04.734+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:05.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:04 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:04.734+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:05.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:04 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:04.734+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:05.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:04 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:04.866+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:05.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:04 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:04.866+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:05.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:04 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:04.866+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:05.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:04 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:04.866+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:05.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:04 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:04.866+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:05.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:05 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:05.224+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:11:05.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:05 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:05.224+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:11:05.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:05 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:05.224+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:11:05.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:05 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:05.224+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:11:05.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:05 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:05.224+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:11:05.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:05 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:05.224+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:30.957285+0000 front 2024-04-26T23:10:30.957099+0000 (oldest deadline 2024-04-26T23:10:56.257243+0000) 2024-04-26T23:11:06.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:05 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:05.710+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:06.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:05 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:05.710+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:06.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:05 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:05.710+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:06.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:05 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:05.710+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:06.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:05 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:05.710+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:06.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:05 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:05.710+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:06.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:05 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:05.710+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:42.865337+0000 front 2024-04-26T23:10:42.865463+0000 (oldest deadline 2024-04-26T23:11:05.164926+0000) 2024-04-26T23:11:06.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:05 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:05.854+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:06.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:05 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:05.854+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:06.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:05 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:05.854+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:06.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:05 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:05.854+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:06.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:05 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:05.854+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:06.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:05 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:05.854+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:06.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:06 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:06.234+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:11:06.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:06 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:06.234+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:11:06.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:06 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:06.234+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:11:06.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:06 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:06.234+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:11:06.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:06 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:06.234+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:11:06.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:06 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:06.234+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:30.957285+0000 front 2024-04-26T23:10:30.957099+0000 (oldest deadline 2024-04-26T23:10:56.257243+0000) 2024-04-26T23:11:07.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:06 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:06.714+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:07.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:06 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:06.714+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:07.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:06 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:06.714+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:07.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:06 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:06.714+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:07.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:06 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:06.714+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:07.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:06 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:06.714+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:07.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:06 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:06.714+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:42.865337+0000 front 2024-04-26T23:10:42.865463+0000 (oldest deadline 2024-04-26T23:11:05.164926+0000) 2024-04-26T23:11:07.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:06 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:06.814+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:07.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:06 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:06.814+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:07.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:06 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:06.814+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:07.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:06 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:06.814+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:07.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:06 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:06.814+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:07.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:06 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:06.814+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:07.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:07 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:07.232+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:11:07.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:07 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:07.232+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:11:07.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:07 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:07.232+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:11:07.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:07 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:07.232+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:11:07.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:07 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:07.232+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:11:07.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:07 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:07.232+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:30.957285+0000 front 2024-04-26T23:10:30.957099+0000 (oldest deadline 2024-04-26T23:10:56.257243+0000) 2024-04-26T23:11:08.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:07 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:07.722+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:08.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:07 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:07.722+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:08.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:07 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:07.722+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:08.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:07 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:07.722+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:08.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:07 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:07.722+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:08.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:07 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:07.722+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:08.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:07 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:07.722+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:42.865337+0000 front 2024-04-26T23:10:42.865463+0000 (oldest deadline 2024-04-26T23:11:05.164926+0000) 2024-04-26T23:11:08.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:07 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:07.785+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:08.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:07 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:07.785+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:08.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:07 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:07.785+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:08.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:07 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:07.785+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:08.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:07 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:07.785+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:08.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:07 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:07.785+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:08.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:08 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:08.242+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:11:08.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:08 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:08.242+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:11:08.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:08 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:08.242+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:11:08.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:08 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:08.242+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:11:08.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:08 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:08.242+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:11:08.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:08 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:08.242+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:30.957285+0000 front 2024-04-26T23:10:30.957099+0000 (oldest deadline 2024-04-26T23:10:56.257243+0000) 2024-04-26T23:11:09.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:08 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:08.732+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:09.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:08 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:08.732+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:09.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:08 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:08.732+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:09.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:08 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:08.732+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:09.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:08 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:08.732+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:09.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:08 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:08.732+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:09.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:08 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:08.732+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:42.865337+0000 front 2024-04-26T23:10:42.865463+0000 (oldest deadline 2024-04-26T23:11:05.164926+0000) 2024-04-26T23:11:09.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:08 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:08.809+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:09.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:08 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:08.809+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:09.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:08 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:08.809+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:09.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:08 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:08.809+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:09.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:08 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:08.809+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:09.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:08 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:08.809+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:09.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:09 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:09.265+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.946170+0000 front 2024-04-26T23:09:48.946267+0000 (oldest deadline 2024-04-26T23:10:12.444998+0000) 2024-04-26T23:11:09.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:09 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:09.265+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:00.549182+0000 front 2024-04-26T23:10:00.548969+0000 (oldest deadline 2024-04-26T23:10:25.249245+0000) 2024-04-26T23:11:09.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:09 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:09.265+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.052644+0000 front 2024-04-26T23:10:08.052242+0000 (oldest deadline 2024-04-26T23:10:33.352351+0000) 2024-04-26T23:11:09.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:09 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:09.265+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:13.354097+0000 front 2024-04-26T23:10:13.353911+0000 (oldest deadline 2024-04-26T23:10:41.554166+0000) 2024-04-26T23:11:09.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:09 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:09.265+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.856068+0000 front 2024-04-26T23:10:26.856363+0000 (oldest deadline 2024-04-26T23:10:50.956211+0000) 2024-04-26T23:11:09.616 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:09 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9[62320]: 2024-04-26T23:11:09.265+0000 7f3fd7400700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:30.957285+0000 front 2024-04-26T23:10:30.957099+0000 (oldest deadline 2024-04-26T23:10:56.257243+0000) 2024-04-26T23:11:10.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:09 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:09.684+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:10.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:09 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:09.684+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:10.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:09 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:09.684+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:10.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:09 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:09.684+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:10.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:09 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:09.684+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:10.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:09 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:09.684+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:10.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:09 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:09.684+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:42.865337+0000 front 2024-04-26T23:10:42.865463+0000 (oldest deadline 2024-04-26T23:11:05.164926+0000) 2024-04-26T23:11:10.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:09 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:09.796+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:10.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:09 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:09.796+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:10.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:09 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:09.796+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:10.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:09 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:09.796+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:10.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:09 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:09.796+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:10.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:09 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:09.796+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:10.365 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:10 smithi027 podman[89664]: 2024-04-26 23:11:10.076013465 +0000 UTC m=+5.814026710 container died a80701b2cb2677d4c2b10ecfeac89915ff3dbe5f88c64b884b8454a0d9a73dee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T23:11:10.865 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:10 smithi027 podman[89664]: 2024-04-26 23:11:10.549223632 +0000 UTC m=+6.287236914 container cleanup a80701b2cb2677d4c2b10ecfeac89915ff3dbe5f88c64b884b8454a0d9a73dee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T23:11:10.865 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:10 smithi027 bash[89664]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9 2024-04-26T23:11:11.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:10 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:10.658+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:11.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:10 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:10.658+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:11.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:10 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:10.658+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:11.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:10 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:10.658+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:11.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:10 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:10.658+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:11.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:10 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:10.658+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:11.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:10 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:10.658+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:42.865337+0000 front 2024-04-26T23:10:42.865463+0000 (oldest deadline 2024-04-26T23:11:05.164926+0000) 2024-04-26T23:11:11.034 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:10 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:10.755+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:11.034 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:10 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:10.755+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:11.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:10 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:10.755+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:11.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:10 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:10.755+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:11.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:10 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:10.755+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:11.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:10 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:10.755+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:11.364 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:10 smithi027 podman[89680]: 2024-04-26 23:11:10.891702434 +0000 UTC m=+0.802760279 container remove a80701b2cb2677d4c2b10ecfeac89915ff3dbe5f88c64b884b8454a0d9a73dee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-26T23:11:11.865 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:11 smithi027 podman[89770]: 2024-04-26 23:11:11.42016999 +0000 UTC m=+0.039426179 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:11:11.866 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:11 smithi027 podman[89770]: 2024-04-26 23:11:11.679946922 +0000 UTC m=+0.299203117 container create c17e01d304cff1c8fa01b298a23c07ee0c11982c127059348ba9e926a4c321c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T23:11:12.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:11 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:11.647+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:12.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:11 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:11.647+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:12.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:11 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:11.647+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:12.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:11 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:11.647+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:12.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:11 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:11.647+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:12.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:11 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:11.647+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:12.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:11 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:11.647+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:42.865337+0000 front 2024-04-26T23:10:42.865463+0000 (oldest deadline 2024-04-26T23:11:05.164926+0000) 2024-04-26T23:11:12.034 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:11 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:11.735+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:12.034 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:11 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:11.735+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:12.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:11 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:11.735+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:12.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:11 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:11.735+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:12.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:11 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:11.735+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:12.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:11 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:11.735+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:12.153 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:11 smithi027 podman[89770]: 2024-04-26 23:11:11.988222017 +0000 UTC m=+0.607478235 container init c17e01d304cff1c8fa01b298a23c07ee0c11982c127059348ba9e926a4c321c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-26T23:11:12.153 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:12 smithi027 podman[89770]: 2024-04-26 23:11:12.000368372 +0000 UTC m=+0.619624633 container start c17e01d304cff1c8fa01b298a23c07ee0c11982c127059348ba9e926a4c321c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T23:11:12.153 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:12 smithi027 podman[89770]: 2024-04-26 23:11:12.153553924 +0000 UTC m=+0.772810125 container attach c17e01d304cff1c8fa01b298a23c07ee0c11982c127059348ba9e926a4c321c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-26T23:11:12.865 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 23:11:12 smithi027 podman[89770]: 2024-04-26 23:11:12.475180881 +0000 UTC m=+1.094437098 container died c17e01d304cff1c8fa01b298a23c07ee0c11982c127059348ba9e926a4c321c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T23:11:13.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:12 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:12.614+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:13.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:12 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:12.614+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:13.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:12 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:12.614+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:13.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:12 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:12.614+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:13.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:12 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:12.614+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:13.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:12 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:12.614+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:13.022 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:12 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:12.614+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:42.865337+0000 front 2024-04-26T23:10:42.865463+0000 (oldest deadline 2024-04-26T23:11:05.164926+0000) 2024-04-26T23:11:13.034 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:12 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:12.748+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:13.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:12 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:12.748+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:13.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:12 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:12.748+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:13.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:12 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:12.748+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:13.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:12 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:12.748+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:13.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:12 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:12.748+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:13.298 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.9.service' 2024-04-26T23:11:13.402 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:11:13.402 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-26T23:11:13.402 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-26T23:11:13.402 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.10 2024-04-26T23:11:13.679 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:13 smithi080 systemd[1]: Stopping Ceph osd.10 for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:11:13.680 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:13 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:13.573+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:13.680 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:13 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:13.573+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:13.680 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:13 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:13.573+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:13.680 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:13 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:13.573+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:13.680 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:13 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:13.573+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:13.680 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:13 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:13.573+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:13.680 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:13 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:13.573+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:42.865337+0000 front 2024-04-26T23:10:42.865463+0000 (oldest deadline 2024-04-26T23:11:05.164926+0000) 2024-04-26T23:11:14.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:13 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:13.679+0000 7fc1ace8d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:11:14.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:13 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:13.679+0000 7fc1ace8d700 -1 osd.10 107 *** Got signal Terminated *** 2024-04-26T23:11:14.021 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:13 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:13.679+0000 7fc1ace8d700 -1 osd.10 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T23:11:14.034 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:13 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:13.779+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:14.034 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:13 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:13.779+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:14.034 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:13 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:13.779+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:14.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:13 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:13.779+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:14.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:13 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:13.779+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:14.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:13 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:13.779+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:14.922 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:14 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:14.554+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:14.923 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:14 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:14.554+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:14.923 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:14 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:14.554+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:14.923 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:14 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:14.554+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:14.923 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:14 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:14.554+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:14.923 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:14 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:14.554+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:14.923 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:14 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:14.554+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:42.865337+0000 front 2024-04-26T23:10:42.865463+0000 (oldest deadline 2024-04-26T23:11:05.164926+0000) 2024-04-26T23:11:15.034 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:14 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:14.743+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:15.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:14 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:14.743+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:15.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:14 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:14.743+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:15.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:14 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:14.743+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:15.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:14 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:14.743+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:15.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:14 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:14.743+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:15.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:15 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:15.506+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:15.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:15 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:15.506+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:15.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:15 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:15.506+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:15.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:15 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:15.506+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:15.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:15 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:15.506+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:15.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:15 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:15.506+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:15.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:15 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:15.506+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:42.865337+0000 front 2024-04-26T23:10:42.865463+0000 (oldest deadline 2024-04-26T23:11:05.164926+0000) 2024-04-26T23:11:16.034 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:15 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:15.784+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:16.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:15 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:15.784+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:16.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:15 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:15.784+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:16.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:15 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:15.784+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:16.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:15 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:15.784+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:16.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:15 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:15.784+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:16.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:16 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:16.471+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:16.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:16 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:16.471+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:16.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:16 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:16.471+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:16.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:16 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:16.471+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:16.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:16 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:16.471+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:16.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:16 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:16.471+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:16.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:16 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:16.471+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:42.865337+0000 front 2024-04-26T23:10:42.865463+0000 (oldest deadline 2024-04-26T23:11:05.164926+0000) 2024-04-26T23:11:17.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:16 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:16.834+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:17.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:16 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:16.834+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:17.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:16 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:16.834+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:17.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:16 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:16.834+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:17.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:16 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:16.834+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:17.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:16 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:16.834+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:17.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:16 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:16.834+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-04-26T23:10:50.245409+0000 front 2024-04-26T23:10:50.245318+0000 (oldest deadline 2024-04-26T23:11:16.145209+0000) 2024-04-26T23:11:17.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:17 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:17.471+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:17.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:17 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:17.471+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:17.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:17 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:17.471+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:17.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:17 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:17.471+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:17.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:17 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:17.471+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:17.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:17 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:17.471+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:17.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:17 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:17.471+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:42.865337+0000 front 2024-04-26T23:10:42.865463+0000 (oldest deadline 2024-04-26T23:11:05.164926+0000) 2024-04-26T23:11:18.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:17 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:17.879+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:18.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:17 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:17.879+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:18.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:17 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:17.879+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:18.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:17 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:17.879+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:18.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:17 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:17.879+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:18.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:17 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:17.879+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:18.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:17 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:17.879+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-04-26T23:10:50.245409+0000 front 2024-04-26T23:10:50.245318+0000 (oldest deadline 2024-04-26T23:11:16.145209+0000) 2024-04-26T23:11:18.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:18 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:18.493+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:51.355915+0000 front 2024-04-26T23:09:51.356146+0000 (oldest deadline 2024-04-26T23:10:16.655870+0000) 2024-04-26T23:11:18.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:18 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:18.493+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:09:58.356736+0000 front 2024-04-26T23:09:58.356796+0000 (oldest deadline 2024-04-26T23:10:22.456886+0000) 2024-04-26T23:11:18.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:18 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:18.493+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:08.159109+0000 front 2024-04-26T23:10:08.158976+0000 (oldest deadline 2024-04-26T23:10:30.458894+0000) 2024-04-26T23:11:18.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:18 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:18.493+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T23:10:17.460800+0000 front 2024-04-26T23:10:17.460640+0000 (oldest deadline 2024-04-26T23:10:40.360444+0000) 2024-04-26T23:11:18.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:18 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:18.493+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:26.662023+0000 front 2024-04-26T23:10:26.662072+0000 (oldest deadline 2024-04-26T23:10:47.762117+0000) 2024-04-26T23:11:18.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:18 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:18.493+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:34.763708+0000 front 2024-04-26T23:10:34.763833+0000 (oldest deadline 2024-04-26T23:10:56.463557+0000) 2024-04-26T23:11:18.772 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:18 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-26T23:11:18.493+0000 7fc1a9481700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:42.865337+0000 front 2024-04-26T23:10:42.865463+0000 (oldest deadline 2024-04-26T23:11:05.164926+0000) 2024-04-26T23:11:19.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:18 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:18.840+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:19.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:18 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:18.840+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:19.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:18 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:18.840+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:19.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:18 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:18.840+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:19.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:18 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:18.840+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:19.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:18 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:18.840+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:19.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:18 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:18.840+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-04-26T23:10:50.245409+0000 front 2024-04-26T23:10:50.245318+0000 (oldest deadline 2024-04-26T23:11:16.145209+0000) 2024-04-26T23:11:19.521 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:19 smithi080 podman[69874]: 2024-04-26 23:11:19.251595501 +0000 UTC m=+5.662804297 container died c88caadc1b0b8697782d13f31bf1353f9b992f971e76688add23bbfe92f11867 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T23:11:19.873 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:19 smithi080 podman[69874]: 2024-04-26 23:11:19.621813492 +0000 UTC m=+6.033022297 container cleanup c88caadc1b0b8697782d13f31bf1353f9b992f971e76688add23bbfe92f11867 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-26T23:11:19.873 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:19 smithi080 bash[69874]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10 2024-04-26T23:11:20.182 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:19 smithi080 podman[69903]: 2024-04-26 23:11:19.873316798 +0000 UTC m=+0.624970353 container remove c88caadc1b0b8697782d13f31bf1353f9b992f971e76688add23bbfe92f11867 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T23:11:20.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:19 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:19.835+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:20.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:19 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:19.835+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:20.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:19 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:19.835+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:20.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:19 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:19.835+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:20.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:19 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:19.835+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:20.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:19 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:19.835+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:20.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:19 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:19.835+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-04-26T23:10:50.245409+0000 front 2024-04-26T23:10:50.245318+0000 (oldest deadline 2024-04-26T23:11:16.145209+0000) 2024-04-26T23:11:20.460 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:20 smithi080 podman[69994]: 2024-04-26 23:11:20.182876069 +0000 UTC m=+0.101184030 container create 203e606957abb0d894fe5cb92915581b80c34c1e213c41b05aaebd0f6213ee50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T23:11:20.460 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:20 smithi080 podman[69994]: 2024-04-26 23:11:20.099539402 +0000 UTC m=+0.017847338 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:11:20.460 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:20 smithi080 podman[69994]: 2024-04-26 23:11:20.385354012 +0000 UTC m=+0.303661931 container init 203e606957abb0d894fe5cb92915581b80c34c1e213c41b05aaebd0f6213ee50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T23:11:20.460 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:20 smithi080 podman[69994]: 2024-04-26 23:11:20.388946698 +0000 UTC m=+0.307254618 container start 203e606957abb0d894fe5cb92915581b80c34c1e213c41b05aaebd0f6213ee50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T23:11:20.460 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:20 smithi080 podman[69994]: 2024-04-26 23:11:20.460292283 +0000 UTC m=+0.378600220 container attach 203e606957abb0d894fe5cb92915581b80c34c1e213c41b05aaebd0f6213ee50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T23:11:20.771 INFO:journalctl@ceph.osd.10.smithi080.stdout:Apr 26 23:11:20 smithi080 podman[69994]: 2024-04-26 23:11:20.680416057 +0000 UTC m=+0.598723976 container died 203e606957abb0d894fe5cb92915581b80c34c1e213c41b05aaebd0f6213ee50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-04-26T23:11:21.281 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.10.service' 2024-04-26T23:11:21.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:20 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:20.794+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:21.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:20 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:20.794+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:21.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:20 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:20.794+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:21.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:20 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:20.794+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:21.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:20 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:20.794+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:21.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:20 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:20.794+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:21.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:20 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:20.794+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-04-26T23:10:50.245409+0000 front 2024-04-26T23:10:50.245318+0000 (oldest deadline 2024-04-26T23:11:16.145209+0000) 2024-04-26T23:11:21.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:11:21.320 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-26T23:11:21.320 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-26T23:11:21.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.11 2024-04-26T23:11:21.621 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:21 smithi094 systemd[1]: Stopping Ceph osd.11 for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:11:22.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:21 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:21.620+0000 7efde1eec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:11:22.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:21 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:21.620+0000 7efde1eec700 -1 osd.11 107 *** Got signal Terminated *** 2024-04-26T23:11:22.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:21 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:21.620+0000 7efde1eec700 -1 osd.11 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T23:11:22.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:21 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:21.810+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:22.035 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:21 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:21.810+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:22.036 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:21 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:21.810+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:22.036 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:21 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:21.810+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:22.036 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:21 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:21.810+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:22.036 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:21 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:21.810+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:22.036 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:21 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:21.810+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-04-26T23:10:50.245409+0000 front 2024-04-26T23:10:50.245318+0000 (oldest deadline 2024-04-26T23:11:16.145209+0000) 2024-04-26T23:11:23.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:22 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:22.831+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:23.286 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:22 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:22.831+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:23.286 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:22 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:22.831+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:23.286 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:22 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:22.831+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:23.286 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:22 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:22.831+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:23.286 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:22 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:22.831+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:23.286 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:22 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:22.831+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-04-26T23:10:50.245409+0000 front 2024-04-26T23:10:50.245318+0000 (oldest deadline 2024-04-26T23:11:16.145209+0000) 2024-04-26T23:11:23.286 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:22 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:22.831+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6820 osd.8 since back 2024-04-26T23:10:56.145642+0000 front 2024-04-26T23:10:56.145627+0000 (oldest deadline 2024-04-26T23:11:22.045730+0000) 2024-04-26T23:11:24.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:23 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:23.812+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:24.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:23 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:23.812+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:24.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:23 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:23.812+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:24.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:23 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:23.812+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:24.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:23 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:23.812+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:24.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:23 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:23.812+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:24.286 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:23 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:23.812+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-04-26T23:10:50.245409+0000 front 2024-04-26T23:10:50.245318+0000 (oldest deadline 2024-04-26T23:11:16.145209+0000) 2024-04-26T23:11:24.286 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:23 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:23.812+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6820 osd.8 since back 2024-04-26T23:10:56.145642+0000 front 2024-04-26T23:10:56.145627+0000 (oldest deadline 2024-04-26T23:11:22.045730+0000) 2024-04-26T23:11:25.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:24 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:24.790+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:25.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:24 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:24.790+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:25.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:24 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:24.790+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:25.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:24 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:24.790+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:25.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:24 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:24.790+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:25.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:24 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:24.790+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:25.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:24 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:24.790+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-04-26T23:10:50.245409+0000 front 2024-04-26T23:10:50.245318+0000 (oldest deadline 2024-04-26T23:11:16.145209+0000) 2024-04-26T23:11:25.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:24 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:24.790+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6820 osd.8 since back 2024-04-26T23:10:56.145642+0000 front 2024-04-26T23:10:56.145627+0000 (oldest deadline 2024-04-26T23:11:22.045730+0000) 2024-04-26T23:11:26.284 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:25 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:25.836+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T23:09:48.834729+0000 front 2024-04-26T23:09:48.834719+0000 (oldest deadline 2024-04-26T23:10:14.734411+0000) 2024-04-26T23:11:26.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:25 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:25.836+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-04-26T23:10:01.136251+0000 front 2024-04-26T23:10:01.136187+0000 (oldest deadline 2024-04-26T23:10:25.236197+0000) 2024-04-26T23:11:26.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:25 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:25.836+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6804 osd.2 since back 2024-04-26T23:10:07.437533+0000 front 2024-04-26T23:10:07.437505+0000 (oldest deadline 2024-04-26T23:10:30.937622+0000) 2024-04-26T23:11:26.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:25 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:25.836+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-04-26T23:10:25.440438+0000 front 2024-04-26T23:10:25.440371+0000 (oldest deadline 2024-04-26T23:10:48.940470+0000) 2024-04-26T23:11:26.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:25 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:25.836+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6812 osd.5 since back 2024-04-26T23:10:31.141918+0000 front 2024-04-26T23:10:31.141946+0000 (oldest deadline 2024-04-26T23:10:55.842006+0000) 2024-04-26T23:11:26.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:25 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:25.836+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T23:10:40.844707+0000 front 2024-04-26T23:10:40.844512+0000 (oldest deadline 2024-04-26T23:11:04.944234+0000) 2024-04-26T23:11:26.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:25 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:25.836+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-04-26T23:10:50.245409+0000 front 2024-04-26T23:10:50.245318+0000 (oldest deadline 2024-04-26T23:11:16.145209+0000) 2024-04-26T23:11:26.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:25 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11[58211]: 2024-04-26T23:11:25.836+0000 7efdda4d8700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.94:6820 osd.8 since back 2024-04-26T23:10:56.145642+0000 front 2024-04-26T23:10:56.145627+0000 (oldest deadline 2024-04-26T23:11:22.045730+0000) 2024-04-26T23:11:27.423 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:27 smithi094 podman[70872]: 2024-04-26 23:11:27.103500521 +0000 UTC m=+5.587205408 container died c1c7bc0edf8ef198fbe96ae863bc51fcaae8aac00a73bae406272fd397481553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T23:11:27.785 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:27 smithi094 podman[70872]: 2024-04-26 23:11:27.422442033 +0000 UTC m=+5.906146926 container cleanup c1c7bc0edf8ef198fbe96ae863bc51fcaae8aac00a73bae406272fd397481553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-26T23:11:27.785 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:27 smithi094 bash[70872]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11 2024-04-26T23:11:27.785 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:27 smithi094 podman[70889]: 2024-04-26 23:11:27.599405816 +0000 UTC m=+0.491169189 container remove c1c7bc0edf8ef198fbe96ae863bc51fcaae8aac00a73bae406272fd397481553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T23:11:28.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:27 smithi094 podman[70979]: 2024-04-26 23:11:27.901471709 +0000 UTC m=+0.017676133 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:11:28.285 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:28 smithi094 podman[70979]: 2024-04-26 23:11:28.042080064 +0000 UTC m=+0.158284472 container create a5b6188d32d18b89ccfd98f56c88281846304c693b79e03a1f0e058363084300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T23:11:28.575 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:28 smithi094 podman[70979]: 2024-04-26 23:11:28.28656123 +0000 UTC m=+0.402765652 container init a5b6188d32d18b89ccfd98f56c88281846304c693b79e03a1f0e058363084300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T23:11:28.575 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:28 smithi094 podman[70979]: 2024-04-26 23:11:28.289799064 +0000 UTC m=+0.406003479 container start a5b6188d32d18b89ccfd98f56c88281846304c693b79e03a1f0e058363084300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T23:11:28.575 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:28 smithi094 podman[70979]: 2024-04-26 23:11:28.370137319 +0000 UTC m=+0.486341743 container attach a5b6188d32d18b89ccfd98f56c88281846304c693b79e03a1f0e058363084300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T23:11:28.855 INFO:journalctl@ceph.osd.11.smithi094.stdout:Apr 26 23:11:28 smithi094 podman[70979]: 2024-04-26 23:11:28.575892561 +0000 UTC m=+0.692096982 container died a5b6188d32d18b89ccfd98f56c88281846304c693b79e03a1f0e058363084300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T23:11:29.141 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@osd.11.service' 2024-04-26T23:11:29.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:11:29.180 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-26T23:11:29.180 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-26T23:11:29.181 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.a 2024-04-26T23:11:29.614 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 23:11:29 smithi027 systemd[1]: Stopping Ceph mds.a for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:11:30.115 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 23:11:29 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-a[69277]: 2024-04-26T23:11:29.782+0000 7fe1b5600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:11:30.115 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 23:11:29 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-a[69277]: 2024-04-26T23:11:29.782+0000 7fe1b5600700 -1 mds.a *** got signal Terminated *** 2024-04-26T23:11:32.115 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 23:11:31 smithi027 podman[89886]: 2024-04-26 23:11:31.824296611 +0000 UTC m=+2.176217171 container died 3666b592592601eb68fa40700dfd31e75910f064767c479d8ee0aa014c996b97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T23:11:32.399 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 23:11:32 smithi027 podman[89886]: 2024-04-26 23:11:32.375886149 +0000 UTC m=+2.727806715 container cleanup 3666b592592601eb68fa40700dfd31e75910f064767c479d8ee0aa014c996b97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T23:11:32.399 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 23:11:32 smithi027 bash[89886]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-a 2024-04-26T23:11:33.114 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 23:11:32 smithi027 podman[89900]: 2024-04-26 23:11:32.651904243 +0000 UTC m=+0.809009730 container remove 3666b592592601eb68fa40700dfd31e75910f064767c479d8ee0aa014c996b97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T23:11:33.204 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.a.service' 2024-04-26T23:11:33.324 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:11:33.324 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-26T23:11:33.324 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-26T23:11:33.324 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.d 2024-04-26T23:11:33.865 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 23:11:33 smithi027 systemd[1]: Stopping Ceph mds.d for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:11:34.365 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 23:11:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-d[68287]: 2024-04-26T23:11:34.003+0000 7f7dd7600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:11:34.365 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 23:11:34 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-d[68287]: 2024-04-26T23:11:34.004+0000 7f7dd7600700 -1 mds.d *** got signal Terminated *** 2024-04-26T23:11:36.043 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 23:11:35 smithi027 podman[90007]: 2024-04-26 23:11:35.726948037 +0000 UTC m=+1.919490061 container died abcc565af192780581507ea15f3919480fab12162f49dd51e1280f834b6a7f7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T23:11:36.365 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 23:11:36 smithi027 podman[90007]: 2024-04-26 23:11:36.187831427 +0000 UTC m=+2.380373441 container cleanup abcc565af192780581507ea15f3919480fab12162f49dd51e1280f834b6a7f7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424) 2024-04-26T23:11:36.365 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 23:11:36 smithi027 bash[90007]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-d 2024-04-26T23:11:36.864 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 23:11:36 smithi027 podman[90021]: 2024-04-26 23:11:36.455942331 +0000 UTC m=+0.709708830 container remove abcc565af192780581507ea15f3919480fab12162f49dd51e1280f834b6a7f7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-26T23:11:37.033 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.d.service' 2024-04-26T23:11:37.153 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:11:37.153 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-26T23:11:37.153 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-26T23:11:37.153 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.g 2024-04-26T23:11:37.614 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 23:11:37 smithi027 systemd[1]: Stopping Ceph mds.g for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:11:38.114 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 23:11:37 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-g[68931]: 2024-04-26T23:11:37.782+0000 7fe86f800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:11:38.115 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 23:11:37 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-g[68931]: 2024-04-26T23:11:37.782+0000 7fe86f800700 -1 mds.g *** got signal Terminated *** 2024-04-26T23:11:40.114 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 23:11:39 smithi027 podman[90126]: 2024-04-26 23:11:39.818052022 +0000 UTC m=+2.189613105 container died d8edcbaf187c27600259c228685cb4eb05679f9e1fceefd2e98cb6a7e438e998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-26T23:11:40.518 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 23:11:40 smithi027 podman[90126]: 2024-04-26 23:11:40.279965301 +0000 UTC m=+2.651526390 container cleanup d8edcbaf187c27600259c228685cb4eb05679f9e1fceefd2e98cb6a7e438e998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-g, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-26T23:11:40.518 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 23:11:40 smithi027 bash[90126]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-g 2024-04-26T23:11:40.865 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 23:11:40 smithi027 podman[90140]: 2024-04-26 23:11:40.517985631 +0000 UTC m=+0.676133001 container remove d8edcbaf187c27600259c228685cb4eb05679f9e1fceefd2e98cb6a7e438e998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T23:11:41.063 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.g.service' 2024-04-26T23:11:41.179 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:11:41.179 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-26T23:11:41.179 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-26T23:11:41.180 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.j 2024-04-26T23:11:41.614 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 23:11:41 smithi027 systemd[1]: Stopping Ceph mds.j for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:11:42.114 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 23:11:41 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-j[67910]: 2024-04-26T23:11:41.827+0000 7f2a24c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:11:42.115 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 23:11:41 smithi027 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-j[67910]: 2024-04-26T23:11:41.827+0000 7f2a24c00700 -1 mds.j *** got signal Terminated *** 2024-04-26T23:11:43.364 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 23:11:43 smithi027 podman[90245]: 2024-04-26 23:11:43.145037633 +0000 UTC m=+1.475681297 container died 03423de96aae6e19beb0db2b8722345e047482e8168932ed2c53c9ccf4ee08ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T23:11:43.821 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 23:11:43 smithi027 podman[90245]: 2024-04-26 23:11:43.553812601 +0000 UTC m=+1.884456296 container cleanup 03423de96aae6e19beb0db2b8722345e047482e8168932ed2c53c9ccf4ee08ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T23:11:43.821 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 23:11:43 smithi027 bash[90245]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-j 2024-04-26T23:11:44.114 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 23:11:43 smithi027 podman[90268]: 2024-04-26 23:11:43.821387977 +0000 UTC m=+0.657951703 container remove 03423de96aae6e19beb0db2b8722345e047482e8168932ed2c53c9ccf4ee08ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T23:11:44.365 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.j.service' 2024-04-26T23:11:44.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:11:44.484 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-26T23:11:44.485 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-26T23:11:44.485 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.b 2024-04-26T23:11:44.771 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 23:11:44 smithi080 systemd[1]: Stopping Ceph mds.b for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:11:45.271 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 23:11:44 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-b[63021]: 2024-04-26T23:11:44.806+0000 7f6491e4f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:11:45.271 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 23:11:44 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-b[63021]: 2024-04-26T23:11:44.806+0000 7f6491e4f700 -1 mds.b *** got signal Terminated *** 2024-04-26T23:11:48.021 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 23:11:47 smithi080 podman[70099]: 2024-04-26 23:11:47.729478374 +0000 UTC m=+3.032570719 container died 4f14a8823937eb46ddc14daae622a3492431a000e1d61ff5fab8e6ef37155d16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-26T23:11:48.365 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 23:11:48 smithi080 podman[70099]: 2024-04-26 23:11:48.114526648 +0000 UTC m=+3.417618988 container cleanup 4f14a8823937eb46ddc14daae622a3492431a000e1d61ff5fab8e6ef37155d16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T23:11:48.365 INFO:journalctl@ceph.mds.b.smithi080.stdout:Apr 26 23:11:48 smithi080 bash[70099]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-b 2024-04-26T23:11:48.502 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.b.service' 2024-04-26T23:11:48.538 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:11:48.538 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-26T23:11:48.538 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-26T23:11:48.538 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.e 2024-04-26T23:11:49.021 INFO:journalctl@ceph.mds.e.smithi080.stdout:Apr 26 23:11:48 smithi080 systemd[1]: Stopping Ceph mds.e for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:11:49.021 INFO:journalctl@ceph.mds.e.smithi080.stdout:Apr 26 23:11:48 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-e[62389]: 2024-04-26T23:11:48.859+0000 7f377ad10700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:11:49.021 INFO:journalctl@ceph.mds.e.smithi080.stdout:Apr 26 23:11:48 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-e[62389]: 2024-04-26T23:11:48.859+0000 7f377ad10700 -1 mds.e *** got signal Terminated *** 2024-04-26T23:11:51.484 INFO:journalctl@ceph.mds.e.smithi080.stdout:Apr 26 23:11:51 smithi080 podman[70231]: 2024-04-26 23:11:51.165101255 +0000 UTC m=+2.407191520 container died cb6cb4468c2c475b34d90ab5446dfa1835c1e16e4b038d6077ff11589319e634 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T23:11:51.484 INFO:journalctl@ceph.mds.e.smithi080.stdout:Apr 26 23:11:51 smithi080 podman[70231]: 2024-04-26 23:11:51.484791422 +0000 UTC m=+2.726881696 container cleanup cb6cb4468c2c475b34d90ab5446dfa1835c1e16e4b038d6077ff11589319e634 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True) 2024-04-26T23:11:51.771 INFO:journalctl@ceph.mds.e.smithi080.stdout:Apr 26 23:11:51 smithi080 bash[70231]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-e 2024-04-26T23:11:51.771 INFO:journalctl@ceph.mds.e.smithi080.stdout:Apr 26 23:11:51 smithi080 podman[70245]: 2024-04-26 23:11:51.669527411 +0000 UTC m=+0.506061737 container remove cb6cb4468c2c475b34d90ab5446dfa1835c1e16e4b038d6077ff11589319e634 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-04-26T23:11:51.903 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.e.service' 2024-04-26T23:11:51.934 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:11:51.935 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-26T23:11:51.935 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-26T23:11:51.935 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.h 2024-04-26T23:11:52.246 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 23:11:52 smithi080 systemd[1]: Stopping Ceph mds.h for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:11:52.521 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 23:11:52 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-h[62709]: 2024-04-26T23:11:52.246+0000 7fbc3af4b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:11:52.521 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 23:11:52 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-h[62709]: 2024-04-26T23:11:52.246+0000 7fbc3af4b700 -1 mds.h *** got signal Terminated *** 2024-04-26T23:11:56.021 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 23:11:55 smithi080 podman[70353]: 2024-04-26 23:11:55.73008674 +0000 UTC m=+3.576273468 container died 52d0925938a37cbfc30f910f269413b4ba60e9403c2f22b09116aceb5075d658 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-26T23:11:56.356 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 23:11:56 smithi080 podman[70353]: 2024-04-26 23:11:56.105893784 +0000 UTC m=+3.952080500 container cleanup 52d0925938a37cbfc30f910f269413b4ba60e9403c2f22b09116aceb5075d658 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T23:11:56.356 INFO:journalctl@ceph.mds.h.smithi080.stdout:Apr 26 23:11:56 smithi080 bash[70353]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-h 2024-04-26T23:11:56.482 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.h.service' 2024-04-26T23:11:56.513 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:11:56.513 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-26T23:11:56.513 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-26T23:11:56.513 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.k 2024-04-26T23:11:56.773 INFO:journalctl@ceph.mds.k.smithi080.stdout:Apr 26 23:11:56 smithi080 systemd[1]: Stopping Ceph mds.k for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:11:57.271 INFO:journalctl@ceph.mds.k.smithi080.stdout:Apr 26 23:11:56 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-k[62070]: 2024-04-26T23:11:56.825+0000 7fd125891700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:11:57.271 INFO:journalctl@ceph.mds.k.smithi080.stdout:Apr 26 23:11:56 smithi080 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-k[62070]: 2024-04-26T23:11:56.825+0000 7fd125891700 -1 mds.k *** got signal Terminated *** 2024-04-26T23:11:59.260 INFO:journalctl@ceph.mds.k.smithi080.stdout:Apr 26 23:11:59 smithi080 podman[70475]: 2024-04-26 23:11:59.004840279 +0000 UTC m=+2.278470004 container died 7012fecda154b31978e2fa53777cca52d14620043bd7dba9abcd8f5eb5c0fcaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T23:11:59.511 INFO:journalctl@ceph.mds.k.smithi080.stdout:Apr 26 23:11:59 smithi080 podman[70475]: 2024-04-26 23:11:59.342862609 +0000 UTC m=+2.616492325 container cleanup 7012fecda154b31978e2fa53777cca52d14620043bd7dba9abcd8f5eb5c0fcaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424) 2024-04-26T23:11:59.511 INFO:journalctl@ceph.mds.k.smithi080.stdout:Apr 26 23:11:59 smithi080 bash[70475]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-k 2024-04-26T23:11:59.511 INFO:journalctl@ceph.mds.k.smithi080.stdout:Apr 26 23:11:59 smithi080 podman[70489]: 2024-04-26 23:11:59.510618367 +0000 UTC m=+0.509485955 container remove 7012fecda154b31978e2fa53777cca52d14620043bd7dba9abcd8f5eb5c0fcaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T23:11:59.720 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.k.service' 2024-04-26T23:11:59.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:11:59.751 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-26T23:11:59.751 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-26T23:11:59.751 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.c 2024-04-26T23:12:00.034 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 23:11:59 smithi094 systemd[1]: Stopping Ceph mds.c for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:12:00.299 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 23:12:00 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-c[63511]: 2024-04-26T23:12:00.040+0000 7f39e0011700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:12:00.299 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 23:12:00 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-c[63511]: 2024-04-26T23:12:00.040+0000 7f39e0011700 -1 mds.c *** got signal Terminated *** 2024-04-26T23:12:04.035 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 23:12:03 smithi094 podman[71084]: 2024-04-26 23:12:03.719089544 +0000 UTC m=+3.783534828 container died fa1e2683a695780eba2cfa1522e5735e4898950d6ba91b8ef9bf02d8337f83fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T23:12:04.358 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 23:12:04 smithi094 podman[71084]: 2024-04-26 23:12:04.141133996 +0000 UTC m=+4.205579280 container cleanup fa1e2683a695780eba2cfa1522e5735e4898950d6ba91b8ef9bf02d8337f83fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True) 2024-04-26T23:12:04.358 INFO:journalctl@ceph.mds.c.smithi094.stdout:Apr 26 23:12:04 smithi094 bash[71084]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-c 2024-04-26T23:12:04.481 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.c.service' 2024-04-26T23:12:04.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:12:04.515 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-26T23:12:04.515 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-26T23:12:04.515 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.f 2024-04-26T23:12:04.785 INFO:journalctl@ceph.mds.f.smithi094.stdout:Apr 26 23:12:04 smithi094 systemd[1]: Stopping Ceph mds.f for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:12:05.284 INFO:journalctl@ceph.mds.f.smithi094.stdout:Apr 26 23:12:04 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-f[63203]: 2024-04-26T23:12:04.844+0000 7f7ebbdcd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:12:05.284 INFO:journalctl@ceph.mds.f.smithi094.stdout:Apr 26 23:12:04 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-f[63203]: 2024-04-26T23:12:04.844+0000 7f7ebbdcd700 -1 mds.f *** got signal Terminated *** 2024-04-26T23:12:06.395 INFO:journalctl@ceph.mds.f.smithi094.stdout:Apr 26 23:12:06 smithi094 podman[71217]: 2024-04-26 23:12:06.115116097 +0000 UTC m=+1.379868639 container died 885bd0a555de9a32fc8c768ec439f5b034c5e72e22e6b194faf09cb138a25668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T23:12:06.737 INFO:journalctl@ceph.mds.f.smithi094.stdout:Apr 26 23:12:06 smithi094 podman[71217]: 2024-04-26 23:12:06.478651342 +0000 UTC m=+1.743403883 container cleanup 885bd0a555de9a32fc8c768ec439f5b034c5e72e22e6b194faf09cb138a25668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T23:12:06.737 INFO:journalctl@ceph.mds.f.smithi094.stdout:Apr 26 23:12:06 smithi094 bash[71217]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-f 2024-04-26T23:12:06.863 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.f.service' 2024-04-26T23:12:06.896 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:12:06.896 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-26T23:12:06.896 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-26T23:12:06.897 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.i 2024-04-26T23:12:07.034 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 23:12:06 smithi094 systemd[1]: Stopping Ceph mds.i for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:12:07.534 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 23:12:07 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-i[63824]: 2024-04-26T23:12:07.223+0000 7f0ad8f8e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:12:07.535 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 23:12:07 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-i[63824]: 2024-04-26T23:12:07.223+0000 7f0ad8f8e700 -1 mds.i *** got signal Terminated *** 2024-04-26T23:12:11.525 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 23:12:11 smithi094 podman[71339]: 2024-04-26 23:12:11.230426406 +0000 UTC m=+4.116277803 container died d19201f63fdd1ad6ba3d24e8bb94eec8c0318fde9527ed5cb0044f96b5b895b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T23:12:11.784 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 23:12:11 smithi094 podman[71339]: 2024-04-26 23:12:11.624498688 +0000 UTC m=+4.510350086 container cleanup d19201f63fdd1ad6ba3d24e8bb94eec8c0318fde9527ed5cb0044f96b5b895b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T23:12:11.785 INFO:journalctl@ceph.mds.i.smithi094.stdout:Apr 26 23:12:11 smithi094 bash[71339]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-i 2024-04-26T23:12:12.042 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.i.service' 2024-04-26T23:12:12.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:12:12.073 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-26T23:12:12.073 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-26T23:12:12.073 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.l 2024-04-26T23:12:12.394 INFO:journalctl@ceph.mds.l.smithi094.stdout:Apr 26 23:12:12 smithi094 systemd[1]: Stopping Ceph mds.l for 95ebdda6-041e-11ef-bc93-c7b262605968... 2024-04-26T23:12:12.784 INFO:journalctl@ceph.mds.l.smithi094.stdout:Apr 26 23:12:12 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-l[62881]: 2024-04-26T23:12:12.394+0000 7feb097a2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:12:12.784 INFO:journalctl@ceph.mds.l.smithi094.stdout:Apr 26 23:12:12 smithi094 ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-l[62881]: 2024-04-26T23:12:12.394+0000 7feb097a2700 -1 mds.l *** got signal Terminated *** 2024-04-26T23:12:16.362 INFO:journalctl@ceph.mds.l.smithi094.stdout:Apr 26 23:12:16 smithi094 podman[71461]: 2024-04-26 23:12:16.062963276 +0000 UTC m=+3.781086604 container died c369057cfe5b86b5227fd45dcef707ebdce8c257dbd4c026e9a2db8fbde23eb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-26T23:12:16.687 INFO:journalctl@ceph.mds.l.smithi094.stdout:Apr 26 23:12:16 smithi094 podman[71461]: 2024-04-26 23:12:16.47009176 +0000 UTC m=+4.188215075 container cleanup c369057cfe5b86b5227fd45dcef707ebdce8c257dbd4c026e9a2db8fbde23eb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T23:12:16.687 INFO:journalctl@ceph.mds.l.smithi094.stdout:Apr 26 23:12:16 smithi094 bash[71461]: ceph-95ebdda6-041e-11ef-bc93-c7b262605968-mds-l 2024-04-26T23:12:16.818 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95ebdda6-041e-11ef-bc93-c7b262605968@mds.l.service' 2024-04-26T23:12:16.847 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T23:12:16.848 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-26T23:12:16.848 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T23:12:17.195 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T23:12:22.574 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T23:12:22.762 INFO:teuthology.orchestra.run.smithi080.stdout:Deleting cluster with fsid: 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T23:12:25.228 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T23:12:25.417 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T23:12:27.735 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T23:12:27.832 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T23:12:27.864 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T23:12:27.889 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-26T23:12:27.891 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675270/remote/smithi027/crash 2024-04-26T23:12:27.892 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/crash -- . 2024-04-26T23:12:27.983 INFO:teuthology.orchestra.run.smithi027.stderr:tar: /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T23:12:27.983 INFO:teuthology.orchestra.run.smithi027.stderr:tar: Error is not recoverable: exiting now 2024-04-26T23:12:27.988 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675270/remote/smithi080/crash 2024-04-26T23:12:27.989 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/crash -- . 2024-04-26T23:12:28.040 INFO:teuthology.orchestra.run.smithi080.stderr:tar: /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T23:12:28.040 INFO:teuthology.orchestra.run.smithi080.stderr:tar: Error is not recoverable: exiting now 2024-04-26T23:12:28.042 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675270/remote/smithi094/crash 2024-04-26T23:12:28.042 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/crash -- . 2024-04-26T23:12:28.077 INFO:teuthology.orchestra.run.smithi094.stderr:tar: /var/lib/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T23:12:28.077 INFO:teuthology.orchestra.run.smithi094.stderr:tar: Error is not recoverable: exiting now 2024-04-26T23:12:28.078 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-26T23:12:28.078 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-26T23:12:28.190 INFO:tasks.cephadm:Compressing logs... 2024-04-26T23:12:28.190 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T23:12:28.195 DEBUG:teuthology.orchestra.run.smithi080:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T23:12:28.197 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T23:12:28.224 INFO:teuthology.orchestra.run.smithi094.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.log 2024-04-26T23:12:28.225 INFO:teuthology.orchestra.run.smithi094.stderr:: No such file or directory 2024-04-26T23:12:28.225 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-26T23:12:28.225 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-26T23:12:28.225 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T23:12:28.225 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-26T23:12:28.225 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-26T23:12:28.225 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-26T23:12:28.225 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-26T23:12:28.225 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-26T23:12:28.225 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.5% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T23:12:28.225 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T23:12:28.229 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T23:12:28.230 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-26T23:12:28.230 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T23:12:28.230 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-26T23:12:28.230 INFO:teuthology.orchestra.run.smithi094.stderr: 94.4% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-26T23:12:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T23:12:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.log 2024-04-26T23:12:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T23:12:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-26T23:12:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-26T23:12:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-26T23:12:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-26T23:12:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-26T23:12:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-26T23:12:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mgr.y.log: 87.5% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T23:12:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-26T23:12:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T23:12:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.h.log: 90.4% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T23:12:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-26T23:12:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.6% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T23:12:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T23:12:28.237 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.11.log: 92.0% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-26T23:12:28.237 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T23:12:28.237 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose --/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.8.log: /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-26T23:12:28.238 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.e.log: 94.5% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-26T23:12:28.238 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-26T23:12:28.238 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-volume.log: 92.0% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-26T23:12:28.238 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-26T23:12:28.247 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T23:12:28.249 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T23:12:28.249 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T23:12:28.249 INFO:teuthology.orchestra.run.smithi080.stderr: 91.0% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T23:12:28.251 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T23:12:28.252 INFO:teuthology.orchestra.run.smithi094.stderr: 94.4% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-26T23:12:28.253 INFO:teuthology.orchestra.run.smithi027.stderr:find: ‘/var/log/rbd-target-api’gzip: No such file or directory -5 2024-04-26T23:12:28.253 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.log 2024-04-26T23:12:28.254 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T23:12:28.255 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T23:12:28.255 INFO:teuthology.orchestra.run.smithi080.stderr: 94.4% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-26T23:12:28.256 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-26T23:12:28.256 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-26T23:12:28.258 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-26T23:12:28.259 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.j.log: 87.4%gzip -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T23:12:28.259 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-26T23:12:28.260 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-26T23:12:28.261 INFO:teuthology.orchestra.run.smithi027.stderr: 90.2% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T23:12:28.262 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-26T23:12:28.264 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-26T23:12:28.264 INFO:teuthology.orchestra.run.smithi027.stderr: 94.5% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-26T23:12:28.265 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T23:12:28.267 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-26T23:12:28.269 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T23:12:28.269 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T23:12:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-26T23:12:28.276 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-26T23:12:28.282 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T23:12:28.287 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-26T23:12:28.291 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 94.2% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-26T23:12:28.297 INFO:teuthology.orchestra.run.smithi027.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-26T23:12:28.301 INFO:teuthology.orchestra.run.smithi027.stderr: 90.9% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T23:12:28.308 INFO:teuthology.orchestra.run.smithi027.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T23:12:28.689 INFO:teuthology.orchestra.run.smithi027.stderr: 90.2% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-26T23:12:28.727 INFO:teuthology.orchestra.run.smithi094.stderr: 91.9% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-26T23:12:28.750 INFO:teuthology.orchestra.run.smithi080.stderr: 91.8% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-26T23:12:29.666 INFO:teuthology.orchestra.run.smithi027.stderr: 90.3% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-26T23:12:30.416 INFO:teuthology.orchestra.run.smithi080.stderr: 95.2% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-26T23:12:30.811 INFO:teuthology.orchestra.run.smithi094.stderr: 95.4% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-26T23:12:30.955 INFO:teuthology.orchestra.run.smithi094.stderr: 94.8% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-26T23:12:35.586 INFO:teuthology.orchestra.run.smithi080.stderr: 92.0% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-26T23:12:38.539 INFO:teuthology.orchestra.run.smithi094.stderr: 91.9% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-26T23:12:39.045 INFO:teuthology.orchestra.run.smithi094.stderr: 91.7% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-26T23:12:40.223 INFO:teuthology.orchestra.run.smithi080.stderr: 91.8% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-26T23:12:40.301 INFO:teuthology.orchestra.run.smithi027.stderr: 91.6% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-26T23:12:40.861 INFO:teuthology.orchestra.run.smithi094.stderr: 91.7% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-26T23:12:42.199 INFO:teuthology.orchestra.run.smithi027.stderr: 91.6% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-26T23:12:42.419 INFO:teuthology.orchestra.run.smithi080.stderr: 91.5% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-26T23:12:43.858 INFO:teuthology.orchestra.run.smithi080.stderr: 91.5% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-26T23:12:44.608 INFO:teuthology.orchestra.run.smithi094.stderr: 91.6% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-26T23:12:44.610 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-26T23:12:44.610 INFO:teuthology.orchestra.run.smithi094.stderr:real 0m16.398s 2024-04-26T23:12:44.610 INFO:teuthology.orchestra.run.smithi094.stderr:user 0m54.217s 2024-04-26T23:12:44.610 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m1.716s 2024-04-26T23:12:45.335 INFO:teuthology.orchestra.run.smithi027.stderr: 91.4% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-26T23:12:47.741 INFO:teuthology.orchestra.run.smithi027.stderr: 91.4% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-26T23:12:49.481 INFO:teuthology.orchestra.run.smithi080.stderr: 93.5% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-26T23:12:49.483 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-26T23:12:49.484 INFO:teuthology.orchestra.run.smithi080.stderr:real 0m21.272s 2024-04-26T23:12:49.484 INFO:teuthology.orchestra.run.smithi080.stderr:user 1m10.863s 2024-04-26T23:12:49.484 INFO:teuthology.orchestra.run.smithi080.stderr:sys 0m2.215s 2024-04-26T23:12:54.834 INFO:teuthology.orchestra.run.smithi027.stderr: 93.3% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-26T23:12:59.223 INFO:teuthology.orchestra.run.smithi027.stderr: 93.4% -- replaced with /var/log/ceph/95ebdda6-041e-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-26T23:12:59.228 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-04-26T23:12:59.228 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m31.000s 2024-04-26T23:12:59.228 INFO:teuthology.orchestra.run.smithi027.stderr:user 1m54.135s 2024-04-26T23:12:59.228 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m7.792s 2024-04-26T23:12:59.229 INFO:tasks.cephadm:Archiving logs... 2024-04-26T23:12:59.230 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675270/remote/smithi027/log 2024-04-26T23:12:59.231 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T23:13:15.061 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675270/remote/smithi080/log 2024-04-26T23:13:15.062 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T23:13:25.995 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675270/remote/smithi094/log 2024-04-26T23:13:25.996 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T23:13:34.671 INFO:tasks.cephadm:Removing cluster... 2024-04-26T23:13:34.671 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 --force 2024-04-26T23:13:35.004 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T23:13:35.951 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 --force 2024-04-26T23:13:36.144 INFO:teuthology.orchestra.run.smithi080.stdout:Deleting cluster with fsid: 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T23:13:36.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 95ebdda6-041e-11ef-bc93-c7b262605968 --force 2024-04-26T23:13:36.686 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: 95ebdda6-041e-11ef-bc93-c7b262605968 2024-04-26T23:13:37.021 INFO:tasks.cephadm:Removing cephadm ... 2024-04-26T23:13:37.021 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T23:13:37.083 DEBUG:teuthology.orchestra.run.smithi080:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T23:13:37.101 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T23:13:37.116 INFO:tasks.cephadm:Teardown complete 2024-04-26T23:13:37.116 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T23:13:37.128 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T23:13:37.128 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T23:13:37.134 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T23:13:37.143 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T23:13:37.291 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T23:13:37.302 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T23:13:37.302 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T23:13:37.311 DEBUG:teuthology.orchestra.run.smithi080:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T23:13:37.313 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T23:13:37.327 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-04-26T23:13:37.327 INFO:teuthology.orchestra.run.smithi080.stderr:bash: line 1: ntpq: command not found 2024-04-26T23:13:37.331 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T23:13:37.331 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-04-26T23:13:37.331 INFO:teuthology.orchestra.run.smithi094.stdout:^* hv01.front.sepia.ceph.com 3 8 377 240 +1455us[+1443us] +/- 42ms 2024-04-26T23:13:37.331 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 45 +384us[ +384us] +/- 79ms 2024-04-26T23:13:37.331 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 -2936us[-2936us] +/- 91ms 2024-04-26T23:13:37.331 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:13:37.332 INFO:teuthology.orchestra.run.smithi080.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T23:13:37.332 INFO:teuthology.orchestra.run.smithi080.stdout:=============================================================================== 2024-04-26T23:13:37.332 INFO:teuthology.orchestra.run.smithi080.stdout:^* hv01.front.sepia.ceph.com 3 7 377 108 +1286us[+1264us] +/- 44ms 2024-04-26T23:13:37.333 INFO:teuthology.orchestra.run.smithi080.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 48 +236us[ +236us] +/- 79ms 2024-04-26T23:13:37.333 INFO:teuthology.orchestra.run.smithi080.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 -3062us[-3062us] +/- 91ms 2024-04-26T23:13:37.333 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:13:37.359 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-04-26T23:13:37.368 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T23:13:37.368 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-04-26T23:13:37.368 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv01.front.sepia.ceph.com 3 6 377 45 +1408us[+1386us] +/- 45ms 2024-04-26T23:13:37.368 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv02.front.sepia.ceph.com 3 8 377 245 +425us[ +395us] +/- 76ms 2024-04-26T23:13:37.368 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 113 -2889us[-2916us] +/- 90ms 2024-04-26T23:13:37.368 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:13:37.370 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T23:13:37.381 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T23:13:37.381 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T23:13:37.447 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T23:13:37.672 DEBUG:teuthology.orchestra.run.smithi080:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T23:13:37.744 DEBUG:teuthology.orchestra.run.smithi094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T23:13:37.809 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T23:13:37.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:13:37.909 DEBUG:teuthology.orchestra.run.smithi080:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T23:13:37.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:13:37.941 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T23:13:37.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:13:37.968 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T23:13:37.980 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T23:13:38.022 INFO:teuthology.task.internal:Duration was 2186.808365 seconds 2024-04-26T23:13:38.022 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T23:13:38.073 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T23:13:38.073 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T23:13:38.081 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T23:13:38.084 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T23:13:38.124 INFO:teuthology.orchestra.run.smithi080.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T23:13:38.126 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T23:13:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T23:13:38.708 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T23:13:38.708 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T23:13:38.708 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T23:13:38.774 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi080.front.sepia.ceph.com 2024-04-26T23:13:38.774 DEBUG:teuthology.orchestra.run.smithi080:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T23:13:38.802 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-04-26T23:13:38.802 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T23:13:38.830 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T23:13:38.830 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T23:13:38.835 DEBUG:teuthology.orchestra.run.smithi080:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T23:13:38.845 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T23:13:38.899 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T23:13:38.899 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T23:13:38.903 DEBUG:teuthology.orchestra.run.smithi080:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T23:13:38.911 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T23:13:39.463 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T23:13:39.476 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T23:13:39.476 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T23:13:39.560 DEBUG:teuthology.orchestra.run.smithi080:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T23:13:39.585 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T23:13:39.609 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T23:13:39.621 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T23:13:39.629 DEBUG:teuthology.orchestra.run.smithi080:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T23:13:39.631 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T23:13:39.653 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern = core 2024-04-26T23:13:39.672 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-04-26T23:13:39.720 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2024-04-26T23:13:39.759 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T23:13:39.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:13:39.806 DEBUG:teuthology.orchestra.run.smithi080:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T23:13:39.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:13:39.821 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T23:13:39.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:13:39.835 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T23:13:39.847 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T23:13:39.848 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675270/remote/smithi027 2024-04-26T23:13:39.848 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T23:13:39.958 DEBUG:teuthology.misc:Transferring archived files from smithi080:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675270/remote/smithi080 2024-04-26T23:13:39.959 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T23:13:39.996 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675270/remote/smithi094 2024-04-26T23:13:39.997 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T23:13:40.031 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T23:13:40.031 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T23:13:40.039 DEBUG:teuthology.orchestra.run.smithi080:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T23:13:40.043 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T23:13:40.092 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T23:13:40.106 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T23:13:40.107 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T23:13:40.131 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T23:13:40.131 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T23:13:40.139 DEBUG:teuthology.orchestra.run.smithi080:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T23:13:40.141 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T23:13:40.154 INFO:teuthology.orchestra.run.smithi080.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 23:13 /home/ubuntu/cephtest 2024-04-26T23:13:40.155 INFO:teuthology.orchestra.run.smithi094.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 23:13 /home/ubuntu/cephtest 2024-04-26T23:13:40.192 INFO:teuthology.orchestra.run.smithi027.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 23:13 /home/ubuntu/cephtest 2024-04-26T23:13:40.195 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T23:13:40.208 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T23:13:40.294 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/fsstress}} duration: 2186.8083646297455 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-04-26T23:13:40.294 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T23:13:40.372 INFO:teuthology.run:pass